crabsallover highlights, key points, comments / links.
Mail Online bias!
Mail Online bias!
Going on holiday really is good for your health... and the benefits last for months,' declares Mail Online, the website of the Daily Mail and Mail on Sunday.
Despite being listed in the "health" section of the website, the news is based on a report by Nuffield Health and Kuoni Travel Ltd. To more sceptical readers, the report may appear to be nothing more than an elaborate piece of marketing material.
If you were being hypercritical, this collaboration also arguably represents a financial conflict of interest so big you could see it from space.
The "healthy holiday" report bears little resemblance to a rigorous scientific study and has not undergone the peer review process, where independent experts scrutinise the methods and findings of a study. If the "research" had been subjected to peer review, it would almost certainly have been dismissed out of hand.
This small experiment involved just 12 people – half of whom were sent on exotic holidays, while half stayed at home – and tells us very little about the effect of holidays on our physical and mental health.
While its broad conclusion that holidays are generally good seems commonsense, we cannot read too much into this experiment because of a long list of methodological limitations.
Even if we do take the report's findings at face value, its results are less than impressive. In some cases, people actually experienced increased stress levels during their holiday and some gained weight.
What is depressing is Mail Online's willingness to take the report at face value, and its failure to inform readers about the extensive limitations of this "experiment".
Where did the story come from?
The experiment was carried out by staff from Nuffield Health (which runs gyms and hospitals) in collaboration with Kuoni Travel Ltd (a holiday company). While no funding source was explicitly stated, it appears to have been funded by one or both of the collaborating organisations.
The experiment was not published in a peer-reviewed journal and so has not undergone the scrutiny of independent experts in health or medicine. It was instead released as a brochure on the Nuffield Health website.
There is a substantial conflict of interest in a report like this, as both Nuffield Health and Kuoni stand to gain commercially from conclusions that support their respective core businesses of health and holidays.
While the Mail Online's reporting of the findings was accurate, what is worrying is what they failed to report – the significant conflicts of interest, the lack of a peer review process, and problems with the extremely small sample size. Although the story makes for a great feelgood headline, it could mislead many readers.
What kind of research was this?
This wasn't research in the way we would usually expect to see it: peer-reviewed, published in a journal and with clear methods. It is probably best to use the language of the authors and call it an "experiment", which is less scientifically rigorous.
As it has not been published in a peer-reviewed journal, this experiment has not been appraised by experts in the field. This crucial stage ensures that a study's conclusions are justified by the study design and findings, and allows for flaws in the research to be pointed out.
Without such a peer review process, the authors' conclusions may be mistaken and remain unchallenged.
What did the research involve?
The experiment recruited six couples and subjected them to a battery of clinical and psychological tests before sending three couples on a free holiday, while three couples stayed at home. It was unclear whether the stay-at-home controls had the equivalent time off work or if they continued working while the others jetted off.
Two weeks after the holidaymakers returned, more clinical and psychological tests were performed and participants wore heart monitors for several days. The Nuffield Health staff then reported differences in health and wellbeing measures between the couples who went on holiday and those who didn't.
Three holiday destinations involving different activities were selected to see if the impact of the type of holiday made any difference to health and wellbeing measures. One couple were sent to Thailand for an activity holiday, another couple were sent to Peru to volunteer, and the other couple went to the Maldives for a relaxing "fly and flop" holiday.
The six stay-at-home controls (three couples) were sought to match the lifestyles, age group, physical activity, and alcohol and caffeine intake of those sent on holiday. Controls underwent the same battery of physical and mental health assessments as the holidaymakers.
No randomisation or allocation concealment was reported in the study when assigning the couples to either the travellers group or those who stayed at home.
Statistical testing is also not reported, in order to compare the differences between the travellers and those who stayed at home. This is not a good approach, as it means that any reported differences may be due to chance.
What were the basic results?
The Nuffield Health brochure reported that having a vacation improved the holidaymakers' ability to recover from stress by 29%, while the ability of those who stayed at home deteriorated by 71%. Holidaymakers' sleep quality improved by 34 points, whereas stay-at-homers' sleep worsened by 27 points. Blood pressure was reduced in the holidaymakers by 6%, compared with an increase of 2% in those who didn't go on holiday.
Other reported improvements for holidaymakers included decreases in blood sugar levels, improved body shape, and improved energy and mood.
How did the researchers interpret the results?
The Nuffield report, called 'Revealed: how holidays help you live longer', stated that "taking the right kind of holiday for you may lower your stress levels, improve your resilience to stress, and therefore improve your mental and physical health".
Conclusion
This small experiment involving just 12 people (six couples) tells us very little about the effect of having a holiday on physical or mental health. While its conclusions seem commonsense, we cannot read much into this experiment for the following reasons:
Small study sample
Only 12 people took part in this study. Basing conclusions on the experiences of so few people is risky and unreliable. Studies on larger groups of people may reach different conclusions. Similarly, it is not clear how representative the 12 people in the study were in relation to the general UK population, as physical and mental health can vary with age, ethnicity and social background.
No statistical testing
There was no statistical testing performed in this experiment. This is a huge limitation. It means that we don't know whether any observed differences between the holidaymakers and those who stayed at home were actually likely to be real, or whether they are simply chance findings.
No allocation concealment
It is unclear whether the people who stayed at home knew they were taking part in an experiment on the effect of going on holiday. The knowledge that they weren't lucky enough to be sent on a free holiday, and were instead in the stay-at-home group, may have adversely affected their short-term health and physical measures.
Conflict of interest
Both parties in this report have financial interests in promoting the advice that holidays are good for your health, and that leading a healthy lifestyle helps you live longer. This may have biased the experiment design and reporting of the findings.
Not peer-reviewed
As discussed above, without being published in a peer-reviewed journal and appraised by other leading experts, the authors are free to report and conclude what they wish. The peer review and publication process can add an extra layer of reliability and believability to research findings that are absent in this report.
The definite claim of the report's title – that holidays "help you live longer" – is completely unsubstantiated, based on the evidence the authors present.
The bottom line is that this experiment contributes very little to scientific research, but does reinforce the commonsense view that a holiday is generally a good thing, whether you are whisked away to a luxury hotel in exotic climes or a seaside chalet in Skeggy.
Links to the headlines
Going on holiday really is good for your health... and the benefits last for months. Mail Online. Published January 29 2013
Further reading
Kuoni & Nuffield Health. The Holiday Health Report 2013 – How Holidays Make You Live Longer (PDF 3.4MB). Published online January 29 2013
