crabsallover highlights, key points, comments / links.
We love numbers at Behind the Headlines – statistics are our stock-in-trade.
As a fun experiment we decided to take a look at an average week of health news, break it all down, stick in a spreadsheet and see what patterns emerged. And we ended up with a number of surprising results.
Methods
We included the following newspapers in our analysis:
- Daily Express and Sunday Express
- The Guardian and The Observer
- The Independent and The Independent on Sunday
- Daily Mail and The Mail on Sunday
- Daily Mirror and Sunday Mirror
- The Sun – but not The Sun on Sunday (we forgot to update our newspaper order after the News of the World closed down)
- The Daily Telegraph and The Sunday Telegraph
- The Times and The Sunday Times
Our unscientific study looked at all the printed news from Monday November 26 to Sunday December 2.
For the purpose of the analysis, we used the following categories to classify each individual news story:
- Case reports – stories that report on, normally unusual, individual cases such as a Chinese girl who got a screwdriver stuck up her nose
- ‘Straight up’ news items – topical occurrences in healthcare or medicine, policy announcements and political debate around a certain issue – an ongoing news item for the week in question was the debate on minimum alcohol pricing
- Charity, support group, or 'think tank' reports – stories generated by an announcement, report or press release from one of these types of organisations – such as the story about migraine sufferers being discriminated against at work
- Surveys and commercial press releases – for example, a survey from a smartphone app manufacturer found that three-quarters of people ‘can’t bear’ to talk to other people before 8am
- Peer-reviewed research – a story reporting on new evidence published in a peer-reviewed journal
- Non peer-reviewed research – a story reporting on evidence that has not been published in a peer-reviewed journal, such as a presentation at a conference, or a press release published by a university or research centre
- Research – source unknown – this was used to classify news stories where it was unclear whether the evidence had or had not been published in a peer-reviewed journal
Types of stories not included in our analysis were:
- Health and lifestyle features – such as, how to lose a dress size by Christmas, or seven ways to tackle sleepless nights
- Opinion-driven stories – such as, neurologist calls for more dementia research funding
- Celebrity-driven stories – such as, soap star shares her seven-year battle with depression secret
Stories by type
In total there were 197 stories, of which there were:
- 85 'straight' news stories (43.18%)
- 41 case reports (20.83%)
- 38 peer reviewed research stories (19.30%)
- 15 charity, support group or think tank report stories (7.62%)
- 8 Research of unknown origin (4.06%)
- 6 Survey or press release stories (3.05%)
- 4 Non-peer reviewed research (2.03%)
A few points to consider
Case reports – too many?
It is unsurprising that topical news stories are the most popular type of stories found in newspapers. That is their job after all. But what we did find surprising is the high amount of case report type of studies – accounting for just over one in five of all stories.
While the 'human interest' element of these stories is undoubtedly newsworthy, focusing so much on case reports runs the risk of giving the reader a distorted view of the significance of specific health risks.
These types of stories hit the headlines as they are rare and unusual – not because they pose a threat to public health.
For example, a big new story of that week was the tragic case of a woman who died after eating a poisonous mushroom.
But the fact that around 50 people a year die after accidentally overdosing on painkillers each year in England and Wales goes largely unreported. As Dr Ben Goldacre notes in his book Bad Pharma, almost every MDMA-related (wiki = ecstasy) death gets a mention in a newspaper while only one in every 265 deaths due to paracetamol poisoning gets a mention.
Pure laziness
Even just eight stories where the papers did not bother to mention the source (the research-unknown stories) is eight too many. It is laziness or sloppiness on their part and should not be accepted by either editors or readers. It prevents anyone from delving into the facts behind the headlines and coming to their own judgement about it.
Surprising lack of survey stories
We were surprised at how few PR-driven survey or commercial stories there were during the week. This could be due to the fact that this was a ‘big news’ week, with the publication of the Leveson Report into press practices and standards, as well as the ongoing fallout over the Jimmy Savile scandal.
It is our experience that newspapers will make more use of these pre-packaged PR-driven stories (so-called ‘churnalism’) during slower news periods, such as during the summer (the ‘silly season’).
Output by individual papers
In terms of each individual newspapers’ output of news stories, the results were:
- The Daily Telegraph and The Sunday Telegraph 41 (20.83%)
- Daily Mail and The Mail on Sunday 37 (18.80%)
- Daily Mirror and Sunday Mirror 30 (15.24%)
- The Sun (Monday to Saturday) 26 (13.21%)
- Daily Express and Sunday Express 20 (10.16%)
- The Independent 16 (8.13%)
- The Guardian and The Observer 14 (7.11%)
- The Times and The Sunday Times 13 (6.60%)
Points to consider
Never mind the quality, feel the width?
While we can see that the Telegraph and Mail occupy the top two spots, the figures do not really do full justice to the extent of their health coverage.
While the Mirror may be in third spot, most of its stories were small one or two page paragraph items. While in contrast, the Mail and the Telegraph stories often occupied a third or a half of a page.
Behind the Headlines often appears to be critical of these papers, but the depth and amount of their coverage means these papers are more likely to slip up. However, they are not the only ones to make mistakes as The Independent, which puts out relatively few health stories, has been prone to several howlers in 2012.
The wrong Times?
The low score for The Times was also puzzling. It could be due to the fact that they were running a daily campaign into reducing stillbirths (which for the purposes of the study we considered to be opinion-driven stories), so they saw this as meeting their health news ‘quota’. Or it could just have been a pure statistical blip.
Papers publishing the most peer-reviewed research
Individual newspapers that published the most stories based on peer-reviewed evidence over the course of the week were:
- The Daily Mail and The Mail on Sunday 8 (21.06%)
- The Daily Telegraph and The Sunday Telegraph 7 (18.42%)
- The Daily Express and Sunday Express 6 (15.79%)
- The Independent 6 (15.79%)
- Daily Mirror and Sunday Mirror 5 (13.16%)
- The Sun (Monday to Saturday) 3 (7.90%)
- The Guardian and The Observer 2 (5.26%)
- The Times and The Sunday Times 1 (2.63%)
Points to consider
The Daily Mail: 'champion of evidence-based medicine'?
The Mail catches a lot of flak sometimes from Behind the Headlines. But there is no denying that when it comes to championing new evidence-based medical research, the Mail is way ahead of the pack. The reporting is often of good quality and when we do find an error it is more often than not an overblown headline somewhat misinterpreting the story beneath it.
The Mail's near-obsession with peer-reviewed research contrasts starkly with The Guardian, which only managed a measly two stories based on peer-reviewed research.
The top stories of the week
Top news stories
- Minimum alcohol pricing – 18 mentions
- Whooping cough rates worse for decade – 7 mentions
- New NICE guidelines on cycling and walking – 5 mentions
- Assaults on NHS staff are up – 4 mentions
- New HIV stats published – 4 mentions
- NHS to pay for music lessons – 4 mentions
- 52,000 denied NHS operations – 3 mentions
- GPs paid to treat illegal immigrants – 3 mentions
- MMR vaccine rates at record high – 3 mentions
- MP call to end free prescriptions for preventable diseases – 3 mentions
- Review of the Liverpool Care Pathway – 3 mentions
Most mentioned peer-reviewed studies
- Grapefruit drug overdose warnings – 6 mentions
- Pollution autism link – 6 mentions
- Syrup linked to diabetes epidemic – 4 mentions
- Too much exercise can damage heart – 2 mentions
- Fat test for babies – 2 mentions
- Bacon cancer link dismissed – 1 mention
- Extra sleep helps beat chronic pain – 1 mention
- Statins and exercise cut heart disease rates – 1 mention
The results of our experiment are unsurprising here. Millions of people take prescription drugs in this country and millions of people eat grapefruit. So the decision to cover a study looking at prescription drug-grapefruit interactions is pretty much a ‘no-brainer’ in terms of editors thinking about their potential audience.
Top case reports
- Woman dies after eating mushrooms – 4 mentions
- OAP waits 12 hours for lift home – 3 mentions
- Parkinson's drug turned man into sex addict – 3 mentions
- UK woman dies of rabies – 3 mentions
- Boy with 10-ounce cyst – 2 mentions
- Ruptured stomach – 2 mentions
- Young woman has both breasts removed – 2 mentions
We find that case reports involving tragic deaths are more likely to get coverage than case reports that, arguably, are more helpful in terms of providing useful public health information (such as The Guardian’s stories about dangers of unproven cancer treatments). Each of these stories is shocking, but as we mention above, sometimes case report stories may serve to inflate the popular image of the potential risks in everyday life.
Health news day by day
The total news stories broken down by days of the week are:
- Monday November 26 – 28 (14.22%)
- Tuesday November 27 – 36 (18.23%)
- Wednesday November 28 – 40 (20.32%)
- Thursday November 29 – 37 (18.80%)
- Friday November 30 – 23 (11.68%)
- Saturday December 1 – 18 (9.11%)
- Sunday December 2 – 15 (7.63%)
Tuesday to Thursday proved surprisingly packed with health news, for which can see no obvious explanation.
Some final points
It is important to stress a few final points about the information we have presented:
- This, in no way, should be taken as a systematic review of a week’s news. That would entail using a searchable electronic media archive (such as the LexisNexus news database), and ideally, a panel of researchers. We just used one trusty team member armed with nothing more than a pile of newspapers and a pair of scissors.
- We forgot to order the Sun on Sunday after the News of the World folded, which may mean our results are unrepresentative (for example, the Sun on Sunday may have decided to break with tradition and run a special evidence-based medicine supplement on the Sunday December 2).
- Our classification of specific stories into categories, such as case reports or peer-reviewed research, was entirely subjective.
Conclusion
So what have we learned from this fun exercise?
- The Daily Mail is probably the best source for news on peer-reviewed research (but you may want to check their reporting against the actual study – or at least the abstract)
- Case reports may help sell newspapers, but they may not contribute much to the public good
- Left-of-centre newspapers, despite their caring image, might not really care too much about health
- We're probably busiest in the middle of the week
- Health journalists probably don't work weekend shifts
- That we need to subscribe to The Sun on Sunday
Edited by NHS Choices. Follow Behind the Headlines on twitter.
No comments:
Post a Comment