David Spiegelhalter talks (up to 5') about small and large sample sizes .. and number of deaths must be large in in medical trials to get a confident result! Also Randomised Control Trials (the 'gold' standard) allows
- fair comparison - no cheating
- differences just due to chance OR drug is effective
- Statistical significance (16') eg 95% confidence level, confidence interval,
- Nigel Hawkes absolute and relative risks (25'), benefits expressed as a % ... but risks or disbenefits are often expressed as numbers eg 1 in 2000 - which sound small. This is called 'mis-matched framing'.
eg randomised-controlled trials
.... However, this still left the problem that if you wanted to rig (bias) the results to prove a treatment worked, you could preferentially give it to patients who were less sick, or give the older treatment to patients who were more sick. The solution to this, first used by the Medical Research Council in the 1940s for their study of whooping cough vaccines, is to randomly choose which patient is going to get the new treatment, and which is going to get the control (placebo) treatment.
Controlled trials with random allocation to the two groups became known as a randomised-controlled trials or RCTs. By randomising, not only do you end up with a balance of sicker and healthier patients in the two groups, you also end up with a balance between things you don't know about which may also have an impact on the patient's health and therefore the outcome of the treatment. Then — because, in theory, the only difference between the two groups is whether or not they received the treatment being tested — we can assume that any differences in outcome we observe are most likely due to the treatment and nothing else.
The randomised-controlled trial is the gold standard of clinical research and is now universally used to evaluate new treatments.