Checking the source: how to read a scientific paper

Wouldn’t it be great if scientific papers were written in such a way that anyone could easily access and understand them? Instead of having arguments about contentious issues in which people end up quoting random internet sites or celebrity commentators, we could actually refer to the original scientific data.

Well, unfortunately, there is not much chance that scientific papers are going to change – despite claims that their style contributes to few of them ever being read. And there are also concerns that those who take it upon themselves to turn them into plain English either over-simplify or over-hype the findings. 

So, to improve things, we are going to have to bite the bullet and learn how to better read research papers.

The good news is that there are lots and lots of guides out there on how to tackle an academic work. The bad news is that most of them are more useful in learning how to write than read. So, here’s a simple guide for getting all the best information from these sometimes daunting sources.{%recommended 518%}

1. Where to start

The first thing to know is where to start. The section at the top called the Abstract, right?

Wrong! Leave the Abstract until the very end. It is probably the most dense and impenetrable part of most papers. Go straight to the Introduction, which is generally written in pretty smooth prose and gives you the big picture feel for what the study was actually all about.

2. The end before the beginning

Next, jump ahead to the section headed Discussion, or Conclusion, which will tell you what the paper found – again in a fairly easy-to-read style. 

Most scientific papers follow a structure known as IMRAD (which stands for “Introduction, Method, Results, and Discussion”), although a recent comment piece in the journal Science suggested that for most readers the style was better described as OFRBDDR (“Optimism, Fear, Regret, Bafflement, Distraction, Determination, and Rage”).

3. Making sense of the results

If you need more information, you need to put your thinking cap on now, and turn to the Results section, which is often cluttered with graphs full of error bars and statistics. This is not for the faint-hearted and can seem at times like something produced by a German Enigma machine. But fortunately, with a bit of help you won’t need several years and Benedict Cumberbatch (sorry, I mean Alan Turing) to decipher it.

Here are a few of the key terms explained: 

• Error Bars. These are those funny-looking sideways Hs on top of graphs. They represent the variability of data, and are used to show the uncertainty (or error) in any reported measurement. 

• CI or Confidence Intervals are a measure of uncertainty. Say you sample a particular population of people and find the mean number out of every 100 who read Cosmos magazine is 22. To extrapolate this across a wider group, you apply a formula and crunch some sums and get a figure such as a 95 Confidence Interval of between -2.3 and +3.3. This means you are 95% certain that the true mean of the wider population will lie between 22% minus 2.3, and 22% plus 3.3. But for a higher 99 CI, the spread would have to be larger, say -3.5 and +4.2. 
  
  Therefore, you are trading a high CI rating for a wider spread of values. In reading most studies, though, don’t get too lost in the detail unless you need to, and accept the mean figure given (after boning up on the difference between mean, median and mode).
• N is an important one to pay attention to. It refers to the sample size of a study, and a very small sample size can cloud the difference between correlation and causation. For instance, Andrew Wakefield’s discredited study on autism being supposedly caused by the MMR vaccine only had a sample size of 12 children (fewer than the original number of authors on the paper!). This was far too small to really get a good result, even ignoring all the other faults in the withdrawn paper.

4. What to skip

The Methodology section can be worth a read if you still need to know more, or have questions about how data was obtained, but you can often skip it and still get all the information you need from the Introduction, Discussion and Results sections.

5. The validity of the research

The next skill you need to develop is your bullshit detector radar. There are roughly 2.5 million new scientific papers published in English each year, but not all of them are of the same standard. 

In recent years there has been a rise in so-called predatory journals – basically, rubbish publications that are the equivalent of back-of-a-truck sales. If you cross the editors’ palms with silver you will get published, and a so-called peer review panel will make sure you at least spelled your title right. Maybe.

The rise of these journals means that there is a lot of very dodgy research getting published. One way to make sure the journal you are reading is bona fide is to look at its impact factor. This is a measure of how many times, on average, papers within it have been referenced (or cited) in other published papers. 

There is some debate among scientists about the value of impact factors, but, in general, they are assumed to be proxies for journal integrity. The higher the number, the better the publication.

Another good indicator of the quality of a journal paper is to look at the references used and see how many are by the article’s own author (this is called self-citation, and is generally not a good sign), how many are relevant to the topic by title, or are from journals with their own high impact ratings. And, of course, if the article you are reading is cited in lots of other articles, that is a positive indication.

6. Hone your skills

The more papers you read, the better you’ll get at spotting quality and rubbish, and also at understanding the “exemplary ongoing usage of codifying linguistic determinants” – otherwise known as jargon.