Confessions of a PhD Student (2): “I don’t understand statistical studies”

Most of the articles I have read for my literature review are quantitative. I am used to dealing with percentages, means, correlations and chi-square tests. I know that the result is significant when p is smaller than 0.05. I can handle these basics.

But I have a problem… When statistics get more sophisticated and terms like multiple regression, effect size, post hoc analysis and MANCOVA start appearing, I get lost.

Recently I found a meta-analysis by Bernard and colleagues (2009), and I read the following:

In addition, the linear association between treatment strength and effect size was significant (βRegression[1, 73] = .09, p = .01, QRegression = 6.66, p = .01).

Ok… So… What does that mean? Seriously, what does that mean? It’s a single sentence, but I find it so hard to understand!

Here is what I know:

  • There is a p, smaller than .01… Whatever they did, it was significant, important.
  • Treatment strength is one of the variables they are considering. They are talking about an association (linear… how many types of associations are there, anyway?) between one variable and an effect size.

Here is what I do not know:

  • Is the effect size another variable?
  • What is  QRegression?
  • What is βRegression?
  • If they are talking about one association, why is there a βRegression and a QRegression?
  • Why does βRegression have numbers between brackets and QRegression does not?
  • What are those numbers between the brackets (1, 73)?

And that is just one sentence! Now try reading a whole paragraph!

We found that the overall unadjusted average effect size of 0.10 was significantly different from zero, z(73) = 3.52, p < .001, and significantly heterogeneous, QT(73) = 209.86, p < .001. We then looked to see whether the variability might be explained by methodological quality. The scores on the methodological quality scale for these achievement data ranged from 6 to 14 and produced the frequency distribution of scale categories shown in Table 3. The scale categories significantly explained effect size, QB(8) = 30.30, p < .001 and, treated as an ordinal scale in regression, significantly predicted effect size (βRegression[1, 73] =.08, p < .001, QRegression = 23.50, p < .001). Based on this analysis, we decided to classify the scale into three larger categories of methodological quality.

Do you understand that? Ok. Maybe I am being unfair. I am not providing you the context of the study. In my case, reading the whole study didn’t help much. I still don’t get it.

When I am reading a very quantitative study, like a meta-analysis, I usually skip the numerical babble and jump down directly to the end. I know that in the discussion and conclusions, the author will translate all those numbers and statistical terms into words that I can actually understand.

However, I am aware of the problems of doing this. Since I don’t understand half of the results, I have to trust in the correctness of the conclusions. Maybe there was a methodological problem. Maybe one of the assumptions of the studied variables was not met, making the results untrustworthy. Maybe the author stated something without having enough evidence to support it. Maybe. But I wouldn’t know.

I am a PhD student, and I don’t understand statistical studies.

Ok, I said it.

Funny enough, it is not easy to find training about this. I have taken several statistic courses… But they were not as specialized as what I need now. Fortunately, I still have two more years to figure this out and learn…

– Brenda Padilla

Reference

Bernard, R. M., Abrami, P. C., Borokhovski, E., Wade, C. A., Tamim, R. M., Surkes, M. A. & Bethel, E. C. (2009). A meta-analysis of three types of interaction treatments in distance education. Review of Educational Research, 79(3), 1243-1289.

Advertisements
Leave a comment

9 Comments

  1. dear bdra
    I feel sorry for you.
    If you have taken two courses in statististics, it should have been enough fıor you are looking for.
    It seems those two courses you have failed not passed. Then please take two more courses in statistics.
    I feel sorry for your university

    Reply
  2. Brenda Padilla

     /  April 24, 2011

    Dear Muvaffak,

    The courses I took focused on descriptive statistics. As I wrote, I do understand descriptive statistics.

    The courses I took didn’t include topics like multiple regression, effect size, MANCOVA, etc. These topics are what I do not understand. I still have not taken a course that will help me learn this.

    Brenda

    PS- No need to feel sorry. Knowing my current limitations helps me seek the right training options for me. A PhD is a journey of learning.

    Reply
  3. Zbigniew Koziel

     /  April 24, 2011

    Hi Brenda,

    I don’t feel sory for you because I am also learning statistics and it is not easy!!! BTW I also took a course and I can see and perfectly understand that during a course you just cannot learn everything because the whole statistics is huge and people spending years on statistics.

    I am very far from critisizing any research(er), anybody who comes and put comments for the others. However, I must say that it is pointless if someone comes here to critisise you not providing any portion of the help, this is just waist of time.

    I’ve read the paper and I also had problem to understand it but I am learning stats by myself and it is not my strong side. I’d be happy if you could share with explanbation if you find any.

    Best regards,

    Reply
  4. OK No comment is needed.

    Reply
  5. It’s definitely normal to not understand everything you encounter as a PhD student. However, one thing that will help in solving problems is a change in vocabulary you use to describe it all. Write up here: http://www.opensourcephds.com/2010/06/how-to-solve-really-hard-problems.html

    Reply
  6. fd

     /  August 5, 2013

    Hi Brenda

    I just wanted to know how did you go with solving this for you?

    Best wishes for you

    Reply
  7. Deniz

     /  November 4, 2013

    Hi Brenda,

    I don’t have a phd and in fact, I’m just a second year engineering student but I can make some sense of that paragraph thanks to my girlfriend (psychology major, science graduate) and this website.

    http://www.statsoft.com/Textbook/Elementary-Statistics-Concepts/button/1
    This website will help you understand statistics. explore the left sidebar 🙂

    Have fun learning.

    Deniz

    PS: Now, I feel sorry for Muvaffak, and I’m going to address him in his own language:

    Abicim, sence gerek varmi boyle gereksiz bisekilde yorum yapmaya. Sadece kendini assaliyosun baskalarini ezmeye calisarak. Yoruma gerek yok demektense ozur dilem deseydin senin icin uzulmezdim. Sadece adina ve yazdigin yorumlara bakarak senin dunya goruslerin, dinin, yasam tarzin hakkinda cok soyleyebilirim. Ve bu yuzden senin icin uzgunum. Umarim bi gun aklin basina gelir.

    Reply
  8. Brenda Padilla

     /  November 4, 2013

    Thank you!

    I have tried to find a suitable course, but I haven’t been able to… Most courses I find are either really basic or really advanced. I need an intermediate course.

    My plan is to get involved in a research project that requires the type of statistical analysis I want to learn. I think that having a mentor will help. 🙂

    Reply
  9. Deniz

     /  November 5, 2013

    I agree. If you can find a research position, you can learn to use the statistics. However, you can study them yourself as well. There is nothing in a university that you cannot possibly learn by yourself if you are willing to learn. It seems to me that you are willing to learn statistics so there is no reason for you to not to do so 🙂 There are plenty of sources to use on the internet.

    When you take a university course, the professor provides you the knowledge and the ways to challenge it whilst you could still do all that yourself, for free. I am not necessarily saying that university education is not necessary or helpful. Universities establishe a way of judging candidates’ knowledge for certain tasks and jobs by providing accreditation and the label. However, when you are learning things for your own interest, you do not have to take university courses.

    If you can’t find a research position, you can make your own research. You can make surveys for free and evaluate the statistics by using your knowledge you acquired from the internet (for free). Of course, if you have time and the will power to learn.

    I hope I don’t cause you to drop out of school 🙂

    Take care and have fun learning.

    Oh and in the future if you decide to self conduct a survey, do let me know about it. I am interested in many things. For what its worth, I may be helpful 🙂

    Reply

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: