How science works: hypotheses, evidence and peer review

Science begins with an observation or a question. A hypothesis is a proposed explanation, and a useful one makes predictions that could turn out to be false. The philosopher Karl Popper made this idea of falsifiability famous: a claim that no result could ever contradict is not a scientific one, however attractive it is.

In everyday speech a theory means a guess. In science it means something much stronger: a well-tested explanation that ties many observations together, such as the theory of evolution by natural selection, the germ theory of disease or the theory of general relativity.

To test whether a treatment works, a researcher needs a comparison. In a randomised controlled trial, people are assigned by chance to receive the treatment or a placebo, so that the two groups are alike apart from the treatment. Blinding, in which neither the participants nor the researchers who assess them know who received which, stops expectations from colouring the results.

Many questions cannot be tested this way, for ethical or practical reasons, and have to be studied by observation, which is where the dangers of confounding arise. Ice cream sales and drownings both rise in summer, but ice cream does not cause drowning. Heat explains both. Careful studies try to measure and adjust for such hidden factors, and good scientists say plainly when they cannot.

A small study can easily mislead, because chance plays a larger part in a small sample, so how many people or samples were studied is always worth checking. A p-value measures how surprising the data would be if there were no real effect. It does not tell you the probability that the claim is true, and a result can be statistically significant and still tiny in practice. The size of the effect, and the range of uncertainty around it, usually tell you more.

Before most journals publish a paper, other experts in the field review it and may ask for changes or reject it. Peer review catches many errors, but it does not guarantee that the work is right, and it cannot find fraud easily. A preprint, a paper posted online before review on sites such as arXiv, has not yet been through it.

The real test is replication: whether other teams, using their own samples, get the same result. In some fields, notably parts of psychology and biomedicine, attempts to repeat famous findings have often failed, which led to reforms such as registering a study's plan in advance, sharing data, and publishing negative results.

Scientists speak in degrees of confidence, and that is often mistaken for weakness. When a scientific body says something is very likely, it has weighed a large body of independent evidence, and it is giving a calibrated statement, not hedging. A scientific consensus, such as on climate change or on vaccine safety, is the position that careful reviews of the evidence have reached, and it changes when new evidence demands it.

A few questions help with almost any report on a new study:

• Was it done on people, or on mice or cells? Results in animals often do not carry over.
• How big was the sample, and who was in it?
• Does the headline say causes when the study only found an association?
• Is it one study or a review of many? Does it agree with earlier work?
• Who paid for it, and does the press release claim more than the paper?
• Are the words may, could and might in the paper turned into will and does in the headline?

• Bayes' theorem in plain words: how evidence should change your mind

  Bayes theorem is a rule for updating a belief when new evidence arrives. A worked medical-test example shows why rare events and positive tests surprise us.

• How to read the news critically: a field guide

  Who is speaking, where does the claim come from, and what is missing? A practical guide to reading the news with care, with examples from India and beyond.

• Cognitive biases that shape how we read the news

  Confirmation bias, the availability heuristic, framing and more: how our minds distort what we read, and what helps. A guide for readers of the news.