Math

Ask a Math Question, Get an Answer ASAP!

Have your own Math question?

8 Math Tutors are Online Now

characters left:

Not a Math Question?

Question

What is a p-value and 'statistical significance'? and if a finding is statistically significant, why is it also necessary to consider practical significance?

Submitted: 88 days and 23 hours ago.

Category: Math

Value: $9

Status: AWAITING CUSTOMER ACTION

Subject: gard

Already Tried:
not undertanding the text. too complex, i need it simple so I can get more complex after I have basic understanding. Use plenty of examples

Posted by Susan Athena 88 days and 22 hours ago.

Answer

Hi. Thank you for your question.

Your answer is below.

Kind regards,
Susan

p value refers to the probability of getting a false positive result (Type I error) due to variations in sampling.

Let's say that I am testing a new drug to see if patients recover more quickly when given the new drug as opposed to an old one. My signficance test shows a quicker mean recovery time for the patients taking the new drug, with a p value of 0.02. This means that, if I ran the same experiment 100 times with a different set of patients sampled in the exact same way, I would expect 98 of those experiments to show the same result (a quicker recovery for the experimental group) and 2 to show a different result (the same or longer recovery time).

Statistical significance is simply the comparison of the p value to a predetermined cutoff score that we will consider sufficient evidence. For most studies, a cutoff level of either 0.05 or 0.01 is used. In my example, if the cutoff of .05 is used, then we'd consider the result to be statistically significant, meaning that we have sufficient evidence to reject the null hypothesis and assert that patients taking the new drug have faster recovery times. However, if we had decided on the 0.01 level (a more rigorous test), then we would not be able to assert statistical significance, and we would not reject the null hypothesis.

Even if the finding is statistically significant, the result may not matter. In my example, all the significant result shows is that the mean times are different. However, if the difference is, say, 10.5 days compared with 10.4 days, then the result may not be very meaningful. Even though the groups differ, the degree of difference really does not matter.

88 days and 16 hours ago.

Reply

Still too complex for me

Accepted Answer

Dear XXXXXXX,

Let me see if I can break it down for you.

When you do a statistical test, you are trying to see if there is a good reason to think that your null hypothesis is false. So if you were testing to see if two groups (say men and women) have different means (say different mean salaries), then your null hypothesis would be that their salaries are equal. You get a sample and you look to see how different those sample are in order to make your decision.

Suppose, though, that the average income for all men and the average income for all women is the same. Exactly the same. No difference. Even then, if you selected 10 men and 10 women at random, you wouldn't expect their sample averages to be the same. In fact, even if their average salary of all men and women is exactly the same, every time you draw a sample, the sample means will be slightly different.

So suppose that the average for men is $50,000 per year. I might take a sample of 10 men and get a sample average of $49000. See? It varies because of the randomness in sampling.

So maybe you take a sample of 10 men and 10 women and the men's sample average is 51,000, and the women's sample is 48,000. The question is this: Is the difference between these sample means due to randomness? Or is there really a difference between the average salary for all men and all women.

That's why we do the statistical test.

The p-value is this: If it is true that the true average salary for all men and all women really is the same, what's the probability that the samples that we drew would be as different as they are.

So, in the above example, the p-value would tell us the probability that there would be a $3000 difference in our sample averages if the true averages were really the same. So, a very small p-value means that if men and women really do average the same salaries, then we have observed a very rare event. And hence, we conclude that men and women probably don't have the same salaries.

On the other hand, if the p-value is big, then we don't have much evidence against the null hypothesis.

Make sense?

The practical significance thing is important for this reason. If you take a really really really large sample, then you will be able to detect even the slightest difference between two groups. So suppose the actual average salary for all men is $50,000.00. And suppose that the actual average salary for all women is $49,999.99. That is, they differ by one penny.

If you take a large enough sample, then you will be able to reject your null hypothesis that the salaries are the same. But the question we have to ask is, "who cares if men make, on average 1 penny more than women per year?"

That's why we have to be concerned about practical significance.

-----

Make sense? Feel free to ask questions. I'm happy to answer any follow up questions.

Accepts:
Expert:	Greg
Pos. Feedback:	99.6 %
Answered:	8/25/2009

Graduate Student

I teach math and statistics at a college level as a graduate teaching assistant

Disclaimer: Information in questions, answers, and other posts on this site ("Posts") comes from individual users, not JustAnswer; JustAnswer is not responsible for Posts. Posts are for general information, are not intended to substitute for informed professional advice (medical, legal, veterinary, financial, etc.), or to establish a professional-client relationship. The site and services are provided "as is" with no warranty or representations by JustAnswer regarding the qualifications of Experts. To see what credentials have been verified by a third-party service, please click on the "Verified" symbol in some Experts' profiles. JustAnswer is not intended or designed for EMERGENCY questions which should be directed immediately by telephone or in-person to qualified professionals.