sample sizes in medical trials - how do you know whats good for you?

reposted from: http://plus.maths.org/podcasts/PlusPodcastFeb10.mp3 (a 30 minute podcast).

David Spiegelhalter talks (up to 5') about small and large sample sizes .. and number of deaths must be large in in medical trials to get a confident result! Also Randomised Control Trials (the 'gold' standard) allows

• fair comparison - no cheating
• differences just due to chance OR drug is effective

Also

• Statistical significance (16') eg 95% confidence level, confidence interval, 
• Nigel Hawkes absolute and relative risks (25'), benefits expressed as a % ... but risks or disbenefits are often expressed as numbers eg 1 in 2000 - which sound small. This is called 'mis-matched framing'.

Also read "Evaluating a medical treatment - how do you know it works?"

eg randomised-controlled trials

.... However, this still left the problem that if you wanted to rig (bias) the results to prove a treatment worked, you could preferentially give it to patients who were less sick, or give the older treatment to patients who were more sick. The solution to this, first used by the Medical Research Council in the 1940s for their study of whooping cough vaccines, is to randomly choose which patient is going to get the new treatment, and which is going to get the control (placebo) treatment.

Controlled trials with random allocation to the two groups became known as a randomised-controlled trials or RCTs. By randomising, not only do you end up with a balance of sicker and healthier patients in the two groups, you also end up with a balance between things you don't know about which may also have an impact on the patient's health and therefore the outcome of the treatment. Then — because, in theory, the only difference between the two groups is whether or not they received the treatment being tested — we can assume that any differences in outcome we observe are most likely due to the treatment and nothing else.

The randomised-controlled trial is the gold standard of clinical research and is now universally used to evaluate new treatments.

Statins discussed at Royal College of Pathologists lecture at Cheltenham Science Festival

At Cheltenham Science Festival on Saturday 12th June 2010 'Heart Attack' was presented by the Royal College of Pathologists. After the lecture one speaker told me that a 1% reduction in cholesterol levels using statins reduces risk of heart attacks by 2%. Statins can reduce total cholesterol levels by up to 25% therefore statins can reduce risk of heart attacks by up to 50%. The chance of muscle problems is 1 in 10,000 whilst the chance of other more serious side effects is very very small. The 'safe' UK level is 5 millimoles (mm) cholesterol. The Chinese typically have 2-3 mm cholesterol levels. Humans only need 1.5 mm cholesterol for normal health. A reduction of 25% cholesterol from 5 mm to 3.75mm could give a 50% reduction in the chances of a heart attack. NHS finance was the reason why the 'safe' 5mm level was not lower, based around cost of life saved over 10 year economic considerations. Cholesterol levels are of two types - good cholesterol (High Density Lipoproteins) and bad cholesterol (Low density Lipoproteins). Statins have been considered to be added to tap water he said (I didn't check his name).

The next lecture I attended was 'Worth the Risk' given by David Spiegelhalter the Winton Professor of the Public Understanding of Risk based in the Statistical Laboratory in the University of Cambridge. He blogs at Understanding Uncertainty. He explained the idea of the 'Micromort' (see plusmaths for a dynamic explanation) as a better way of explaining risk than 1 in a million chance. David  Spiegelhalter  said that surveys had found that 25% of the population consider a 1 in 10 risk to be LESS risky than a 1 in 100 risk! He mentioned that he was personally considering taking statins. After the lecture I asked him why he was not already taking statins. He said being 'healthy' meant that he couldn't get statins on the NHS and whilst recognising the benefits of statins (can reduce chance of heart attack by 40%) he was not good at taking pills.


A video has gone on Youtube called Professor Risk as part of a series made for the Cambridge 800 years anniversary. To be honest it's a bit lightweight when it comes to risk, and all the subtle bits got cut. We filmed a whole lot more including my GP taking my blood, discussing statins and screening for prostate cancer. All on the cutting-room floor. Good point: Stephen Fry does the intro voiceover. Bad point: me in my jim-jams. (source)


A brilliant way of explaining in differant ways the risks /rewards of Statins
example:

Eating & exercise

Eating & exercise 1 in 6 children were obese in 2002

Obesity prevalence among adults: by sex, England

The prevalence of obesity in England has increased markedly among both adults and children since the mid 1990s. In 2002 it was similar for both sexes; the rate for boys and girls was 17 per cent and for adults was 23 per cent. In 1995 the equivalent figures were 10 per cent for boys and 12 per cent for girls, 15 per cent for men and 18 per cent for women.

There is no evidence that the average calorific intake or consumption of foods rich in fat and added sugar has increased in the UK since the mid 1980s. Men aged 19 to 64 in 2000/01 reported a daily energy intake of approximately 2,323 kcal (a reduction of 6 per cent since 1986/87). Women in the same age groups reported 1,642 kcal, a reduction of 3 per cent.

Reductions over the same period were also observed in the contribution of total fat to total energy intake (from 38 to 34 per cent in men and from 39 to 34 per cent in women) and saturated fat (from 15 to 13 per cent in men and from 17 to 13 per cent in women).

Percentage of adults who meet the physical activity recommendations: by sex and age, 2003, England

In 2003 the percentage of adults meeting the recommendations for physical activity in England declined with age for both sexes. Men were more active than women in every age group and their activity levels declined steadily with age. For women, activity levels remained the same until the 45 to 54 age group, and then declined.

Since the early 1990s there has been a steady increase in the use of cars and a decrease in walking and cycling to school or to work in GB. Among children aged five to ten, the proportion who walked to school fell from 61 per cent in 1992–94 to 52 per cent in 2002–03, mirroring the equivalent 10 percentage point rise in the proportion of school journeys by car, from 30 per cent to 40 per cent.

Among adolescents aged 11 to 16, the proportion of journeys to school by car increased from 16 to 23 per cent over the same period, reflecting the combined decrease in journeys on foot or by bicycle.

For adults aged 17 and over, the proportion of journeys to work where the main mode of travel was by car rose from 66 per cent in 1989–91 to 71 per cent in 2002–03. During the same interval journeys that were mainly on foot fell from 13 to 10 per cent.

Sources: Health Survey for England 1994–2003, Department of Health
Welsh Health Survey 2003/04, National Assembly for Wales

Notes:
View the latest HSE data on obesity.

Notes & definitions

Published on 17 January 2006 at 9:30 am

The Tiger that Isn't - how medical results can mislead doctors

Reading "The Tiger that Isn't" by Michael Blastland & Andrew Dilnot (Amazon). Chapter on Risk pg 119-120 tells of Gerd Gigerenzer work on misleading statistics in medical diagnosis. This work really shocked me at the judgment errors that doctors can make when interpreting medical diagnoses.

Gigerenzer asked a group of physicians to tell him the chance of a patient truly having a condition (breast cancer) when a test (a mammogram) that was 90 per cent accurate at spotting those that had it, and 93 percent at spotting those that did not, came back positive. The condition affected 0.8% of women between 40-50 years. These figures are expressed as conditional probabilities.

Of the 24 doctors to whom he gave this information, just 2 worked out correctly the chance of the patient really having the condition.

Actually more than 9 out 10 positive tests under these assumptions are false positives and the patient is in the clear.

However when expressed as natural frequencies most doctors got the correct result. (Fig 1).

Source: British Medical Journal, 2003
Two ways of representing the same statistical information

Conditional probabilities

The probability that a woman has breast cancer is 0.8%. If she has breast cancer, the probability that a mammogram will show a positive result is 90%. If a woman does not have breast cancer the probability of a positive result is 7%. Take, for example, a woman who has a positive result. What is the probability that she actually has breast cancer?

Natural frequencies

Eight out of every 1000 women have breast cancer. Of these eight women with breast cancer seven will have a positive result (one will have a false negative) on mammography. Of the 992 women who do not have breast cancer some 70 will still have a positive (false positive) mammogram. Take, for example, a sample of women who have positive mammograms. How many of these women actually have breast cancer?

Using Conditional probabilities the answer is not clear. Using natural frequencies things are a lot clearer. Of the 77 positives (7 true positives +70 false positives) only 1 in 11 (9%) will actually have breast cancer. Only one of the cases of breast cancer was missed - the one false negative in a 1000 women tested.

Also from BMJ, 2003:-

The science fiction writer H G Wells predicted that in modern technological societies statistical thinking will one day be as necessary for efficient citizenship as the ability to read and write. How far have we got, a hundred or so years later? A glance at the literature shows a shocking lack of statistical understanding of the outcomes of modern technologies, from standard screening tests for HIV infection to DNA evidence. For instance, doctors with an average of 14 years of professional experience were asked to imagine using the Haemoccult test to screen for colorectal cancer. The prevalence of cancer was 0.3%, the sensitivity of the test was 50%, and the false positive rate was 3%. The doctors were asked: what is the probability that someone who tests positive actually has colorectal cancer? The correct answer is about 5%. However, the doctors' answers ranged from 1% to 99%, with about half of them estimating the probability as 50% (the sensitivity) or 47% (sensitivity minus false positive rate). If patients knew about this degree of variability and statistical innumeracy they would be justly alarmed.

More...

Relative risks

Women aged over 50 years are told that undergoing mammography screening reduces their risk of dying from breast cancer by 25%. Women in high risk groups are told that bilateral prophylactic mastectomy reduces their risk of dying from breast cancer by 80%.⁸ These numbers are relative risk reductions. The confusion produced by relative risks has received more attention in the medical literature than that of single event or conditional probabilities.^{9 10} Nevertheless, few patients realise that the impressive 25% figure means an absolute risk reduction of only one in 1000: of 1000 women who do not undergo mammography about four will die from breast cancer within 10 years, whereas out of 1000 women who do three will die.¹¹ Similarly, the 80% figure for prophylactic mastectomy refers to an absolute risk reduction of four in 100: five in 100 women in the high risk group who do not undergo prophylactic mastectomy will die of breast cancer, compared with one in 100 women who have had a mastectomy. One reason why most women misunderstand relative risks is that they think that the number relates to women like themselves who take part in screening or who are in a high risk group. But relative risks relate to a different class of women: to women who die of breast cancer without having been screened.

Summary points

• The inability to understand statistical information is not a mental deficiency of doctors or patients but is largely due to the poor presentation of the information
• Poorly presented statistical information may cause erroneous communication of risks, with serious consequences
• Single event probabilities, conditional probabilities (such as sensitivity and specificity), and relative risks are confusing because they make it difficult to understand what class of events a probability or percentage refers to
• For each confusing representation there is at least one alternative, such as natural frequency statements, which always specify a reference class and therefore avoid confusion, fostering insight
• Simple representations of risk can help professionals and patients move from innumeracy to insight and make consultations more time efficient
• Instruction in efficient communication of statistical information should be part of medical curriculums and doctors' continuing education