Book review: ‘Deadly Medicines and Organised Crime: How Big Pharma Has Corrupted Healthcare’ by Peter Gøtzsche

I have long been concerned about the conduct of drug companies. I worry about pervasive conflict of interest in the generation, synthesis and dissemination of the evidence that guides my clinical practice. So, when I was asked to review Peter Gøtzsche’s book on these topics, I was excited.

It is hard to imagine someone better qualified to write this book.

A biology and chemistry graduate, Peter Gøtzsche began his career as a sale rep for Astra. In 1977, he took responsibility for setting up a medical department at Astra-Syntex and saw from the inside how trials can be abused to build a case for particular products – in his case Naproxen, a non steroidal anti-inflammatory drug (NSAID). In 1978 he started medical school, whilst still working for the company, only leaving Astra-Syntex on qualification six years later.

Gøtzsche’s PhD focussed on bias in randomised controlled trials of NSAIDs for rheumatoid arthritis, generating important results regarding the impact of funding on outcome. He went on to help found the Cochrane Collaboration and to found the Nordic Cochrane Centre. His academic career has focussed on bias, trials and evidence synthesis.

Before I review Deadly Medicines and Organised Crime, I wish to state unequivocally that I agree with Gøtzsche on many fundamental points.

I think that industry funded research is inefficient and systematically favours the products of the sponsor. I am appalled by pervasive hidden trials data and publication bias [pdf]. As receipt of information from industry appears to be harmful, I think industry should play no role in medical education. The current model is clearly broken and some of the solutions Gøtzsche suggests – e.g. requiring independent trials before licensing medicines – are good ones.

Given the importance of raising these issues, particularly with prescribers, and the obvious expertise Gøtzsche had to draw on, it is a real shame this is not a better book.

The sentences are long and often muddled. Chapter three, in particular, is almost unreadable. The superlatives and hyperbole are pervasive, tiresome and, in places, a little offensive.

‘Drug reps are advised to work with key opinion leaders and turn them into ‘product champions’, and also to find younger people who can have their profile raised so that they also become key opinion leaders. A bit like Hitler-Jugend, so they can go out and terrorise common sense among those who are not yet members of the Party.’ (pg 82)

‘the results generated huge ‘controversy’, with innumerable letters and papers written by seemingly independent whore doctors who were hired guns for the company.’ (pg 100)

‘Whore doctors’, bizarrely, is indexed.

Gøtzsche tries to cram in too many anecdotes and ideas, many of them not clearly related to the preceding sentences. Chapter 21, the book’s manifesto for change, attempts to take on income inequality, the financial crisis and the organisation of US healthcare, in addition to the regulation of health research.

Given Gøtzsche’s background in evidence synthesis, a surprising feature of the book is the frequency with which strong assertions are backed up with stories, reference to a single trial or, occasionally, simply left unreferenced.

‘bipolar disorder…is mainly iatrogenic, caused by SSRIs [selective serotonin reuptake inhibitors] and ADHD [attention deficit hyperactivity disorder] drugs…’ (unreferenced claim, pg 230)

Often I found claims unconvincing but did not know enough to be sure. Occasionally I did know a little of the literature and could be more confident I was being given an incomplete picture.

For example, studying the association between antidepressants and falls is difficult. In observational studies, it is hard to adjust for clinicians favouring particular medications in frail patients they feel at high risk of falls.

‘a carefully controlled cohort study of depressed people over 65 years of age showed that SSRIs more often lead to falls than older antidepressants or if the depression was left untreated.’ (pg 198)

Whilst I am unfamiliar with the study cited by Gøtzsche to support his claim, I know that case only methods, which theoretically deal with this form of confounding better, find SSRIs and older antidepressants increase the risk of falls to similar extent. From a founder of the Cochrane Collaboration, I would have expected a more rounded account of the literature.

To be fair, there are some fascinating ideas in this book. The section on active placebos (pg 49) is great; the extent of the publication bias in trials of granisetron for post operative nausea and vomiting (pg 96) startling; some of the leaked industry documents truly frightening (for example, the table on page 239); and I really like the idea that scientists involved in important trials should conduct the analysis whilst still blinded to the treatment allocation, write two versions of the paper and only unblind the data when all co-authors have approved the manuscripts(pg 266). The reference lists are also likely to be useful to those taking more than a passing interest in these issues.

If you haven’t read much on this topic, you should.  You should then take action. I would recommend reading Ben Goldacre’s Bad Pharma or Jerome Kassirer’s On the Take. Peter Gøtzsche has made many important contributions to this field. Sadly, this book is not one of them.

This review was originally posted on the PLOS Speaking of Medicine Blog.

 

 

Sensible voices needed – deadline 28 August

Since the foundation of the NHS, people have been fretting that its universality might lead to it being ‘abused’ by visitors from other countries.

In his 1952 book In Place of Fear, Nye Bevan argued that such comments were ‘ill-informed and some…deliberately mischievous’. Bevan noted that a charging programme would require ‘British citizens to carry means of identification everywhere to prove that they are not visitors’. He concluded that, given the limited burden overseas visitors were then placing upon the NHS, ‘What began as an attempt to keep the Health Service for ourselves would end by being a nuisance to everybody. Happily, this is one of those occasions when generosity and convenience march together.’

Recent years have seen intense pressure from certain sections of the media to restrict the access that migrants have to NHS services. In the 2004, the then Labour Government responded by introducing a charging regime for certain groups of ‘overseas visitor’ seeking to access NHS secondary care. They acknowledged (see response to Q210) that they did this in the absence of any evidence of the extent of the ‘abuse’ that the regulations were intended to address. Unsurprisingly, in the years since the regulations were introduced, they have led to vulnerable individuals coming to harm. The following accounts come from a set of case studies published by the Refugee Council…

‘E, a young woman from China, was turned away several times by her local NHS trust, who told her that unless she could pay them several thousand pounds upfront, they would not support her through the birth of her baby. She gave birth at home, with no medical care, and then both she and her baby had to be admitted to hospital with serious health problems relating to the traumatic birth. Once discharged, the hospital continued to send E bills, which frightened her so much she fled her home. The whereabouts of her and her child are not known.’

‘G, is an Arab man whose nationality is disputed. He suffers from bowel cancer, and was admitted in an emergency because of uncontrolled bleeding. The clinicians in A&E scheduled him for an operation as soon as the bleeding stopped. However, once the hospital discovered G was a refused asylum seeker, he was given a bill for many thousands of pounds, and his operation was cancelled. He was discharged from hospital and told to come back “when his condition deteriorates”.’

A new Department of Health consultation seeks to restrict migrants’ access to NHS primary care, a proposal that will certainly increase the risk that vulnerable individuals will come to harm. The only serious research on overseas visitors use of NHS primary care, conducted by Imperial College London, concluded that, given the ‘the limited financial burden that Overseas Visitors appear to be having on primary medical services’, the bureaucracy required to operate a charging regimen would be hard to defend even on cost effectiveness grounds.

I would encourage healthcare workers to submit responses to the consultation – humane and rational voices need to be heard in this debate. The deadline for responses is 28 August.

I have copied sections of my response below, which you could adapt if you make a submission. You may find this website, produced by a coalition of concerned NGOs, helpful too.

Question 1:    Are there any other principles you think we should take into consideration?  

Yes, several. The main additional principle that should be considered is this – policies that are likely to result in individuals coming to harm should only be considered if there is clear evidence that they are needed and if there are no alternatives.

There are a growing number of case studies describing individuals whose health has been placed at risk or who have come to harm as a result of the existing NHS charging regulations [1]. Extending charging into primary care can only exacerbate these problems.

For refused asylum seekers and undocumented migrants from outside the EEA, both the threat of charging and perceived links between the NHS and immigration services [2] are likely to deter patients from seeking help when they are unwell.   

In your first principle, you have failed to understand that patients present with symptoms not diagnoses. Very often life threatening conditions present insidiously. Without access to routine investigations in both primary and secondary care, these conditions will be diagnosed late, leading to unnecessary suffering, onward spread of communicable disease and preventable mortality.

Making diagnostic services freely available might partially address this problem. However, this places healthcare professionals in the difficult position of being unable to treat patients with whom they have formed a therapeutic relationship (and to whom they therefore have a duty of care). It is also unreasonable to subject people to invasive diagnostics, x-ray irradiation, etc, if they may not be able to access treatment once a diagnosis is achieved.

That this is being discussed in the absence of evidence that ‘overseas visitors’ present a significant burden to the NHS speaks to the fact that ideology and cheap populism not evidence are driving this policy agenda. I am yet to meet a ‘health tourist’ in an NHS hospital.

The principle change proposed by this consultation is extending NHS charging into primary care. The only good evidence concerning use of primary care by ‘overseas visitors’ is the 2006 impact assessment conducted in Newham by academics from Imperial College London [3]. That survey found that charging for NHS primary care was unlikely be cost effective with the costs of operating a charging regime likely to be greater than any monies recovered from overseas visitors. 

Question 2:   Do you have any evidence of how our proposals may impact disproportionately on any of the protected characteristic groups?

It seems highly likely that individuals from ethnic minorities will be disproportionately affected by the proposed changes. Under the existing charging regime, we have not only seen those not entitled to care coming to harm [1]. We have also seen individuals from ethnic minorities who are entitled to free NHS treatment but have a poor understanding of or ability to express their rights coming to harm [1].

In the UK, there are already appalling racial inequalities in health outcomes. For example, Black African women are nearly four times more likely than White women to suffer a maternal death [4]. A significant number of these deaths are in women newly arrived in the UK who accessed antenatal care late. Given most women access antenatal care via their GP, restricting access to NHS primary care will exacerbate this problem.

Question 3:  Do you have any views on how to improve the ordinary residence qualification?

I believe that if individuals are in the UK, they should be able to access healthcare and other essential services and to contribute towards these services by working and paying tax. If they are not lawfully in the UK, that is a matter for the immigration services. We should not be using denial of essential services – destitution and avoidable sickness – as a means to force individuals from the country. Such policies are inhumane and also unlikely to succeed, given most migrants are young and healthy with little need for healthcare or state support.

Question 5:  Do you agree with the principle of exempting those with a long term relationship with the UK (evidenced by National Insurance contributions)?  How long should this have been for?  Are there any relevant circumstances under which this simple rule will lead to the unfair exclusion of any groups?    

Entitlement to NHS services has never been based on tax contributions. Women who do not work outside the home, the disabled, children and others who have never paid National Insurance contributions can and should be able to freely access NHS care. The NHS should remain free at the point of delivery to all individuals in the UK. The policing of our borders is the responsibility of immigration officials not healthcare workers.

Question 6: Do you support the principle that all temporary non-EEA migrants, and any dependants who accompany them, should make a direct contribution to the costs of their healthcare? 

No. If in the UK, they should contribute in taxation (VAT, income tax, etc) as others do and should be entitled to healthcare and other essential services. It is my understanding that such groups – largely young, healthy and in work – are already net contributors to the public purse.

Question 25: How can charges for primary care services best be applied to those who need to pay in the future?  What are the challenges for implementing a system of charging in primary care and how can these be overcome?

Introducing charging for NHS primary healthcare would be a disaster. This is not the first time a Government has proposed such a change. When this was last proposed, dozens of healthcare workers took time to point out the many problems with the idea [5, 6]. These include…

– humanitarian concerns: the likelihood that vulnerable individuals will come to harm.

– cost effectiveness: primary care is cheap, prevents future need for expensive hospital care, and what evidence there is suggests that the costs of operating a charging regimen would be greater than any monies recovered [3].

– public health: loss of control over communicable diseases, with individuals with communicable disease unable to access diagnosis and treatment, and no opportunity to identify and vaccinate unvaccinated people residing in our communities.

It is not at all clear to me whether the issues raised in that consultation have been considered in these new proposals, which is unfortunate as healthcare workers are busy people and should not be required to make the same points again in multiple Government consultations.

Question 26: Do you agree with the proposal to establish a legal gateway for information sharing to administer the charging regime?  What safeguards would be needed in such a gateway?

Unless the sharing of information is necessary to prevent serious harm, it is a basic tenet of medical ethics that data on health and usage of healthcare services will be treated as confidential. It would be totally inappropriate under any circumstances for any patient identifiable data to be shared with the immigration services and others not directly involved in clinical care. In Sweden, Medecins Sans Frontieres found that even the perception of links between the health and immigration services deterred vulnerable migrants from accessing healthcare [2]. 

1. First do no harm. London: Refugee Council, 2006. Available from http://www.refugeecouncil.org.uk/policy_research/research.

2. Experiences of Gömda in Sweden: Exclusion from health care for immigrants living without legal status. Stockholm: MSF Sweden, 2005. Available from http://www.rosengrenska.org/pdfs/ReportGomdaSwedenEn.pdf.

3. The identification and charging of Overseas Visitors at NHS services in Newham: a Consultation. London: Imperial College London, 2006. Available from http://www.lho.org.uk/viewResource.aspx?id=11948.

4. Saving Mothers’ Lives. London: CMACE, 2011. Available from http://www.rcog.org.uk/news/cmace-release-saving-mothers%E2%80%99-lives-report-%E2%80%93-reviewing-maternal-deaths-2006-2008.

5. Proposals to Exclude Overseas Visitors from Eligibility to Free NHS Primary Medical Services. A summary of submissions to a Department of Health consultation whose findings were never published. London: Global Health Advocacy Project, 2008. Available from https://www.researchgate.net/profile/Tom_Yates/contributions/?ev=prf_act.

6. Four Years Later: Charging Vulnerable Migrants for NHS Primary Medical Services. Students and junior doctors reveal the findings of an unpublished Department of Health consultation. London: Global Health Advocacy Project, 2009. Available from https://www.researchgate.net/profile/Tom_Yates/contributions/?ev=prf_act.

In both clinical practice and guideline development, doctors need to get better at saying “I don’t know”

Most people have symptoms from time to time. Strange twinges and rumblings, odd pains here and there, are the norm. Most of what we feel cannot be explained and is not an early warning of terrible things to come.

The role of the doctor is, therefore, to diagnose and treat a limited number of diseases, to manage symptoms and to offer reassurance where no concerning pathology can be identified.

I think it is okay to say “I don’t know what is causing this but I don’t think it is anything concerning.” You can offer symptomatic treatment if needed and explain the kinds of things you would be worried about. You can ‘safety net’ by telling people to come back if their symptoms develop or fail to settle.

For a while, I worked with someone who, whilst an excellent doctor, rarely ended a consultation without giving a diagnosis. Some quickly racked up a long medical history. Patients loved it – what a clever doctor. And the lives of those of us who took a more honest approach were made harder. We appeared inadequate in comparison.

As a profession, we need to move towards a place where we say “I don’t know” more often. However, inability to admit doubt is much more harmful in doctors developing clinical guidelines.

The recent adult HIV treatment guidelines from the International Antiviral Society USA are radical, recommending clinicians consider treatment regardless of the CD4 count. There are real problems with the panel’s conflicts of interest, the focus of exchange of letters between myself (with Joseph Sonnabend) and the authors in JAMA. The letter we would have liked JAMA to publish in response to the authors’ letter is copied below. However, that is not the focus of this blog. The IAS-USA guidelines are also an excellent example of inability to admit doubt.

Joseph, now retired, was an HIV physician in New York in the first decades of the HIV epidemic. He looked after a friend of mine. It was a bad time. A lot of young adults died.

When antiretroviral medications came on stream in the 1990s, there was much discussion about when best to start treatment. I have seen letters Joseph wrote at the time arguing that a randomised controlled trial, comparing outcomes in patients initiated at different CD4 counts, was needed. It is amazing that we are still waiting for a definitive answer.

The SMART and Haiti CIPRA trials, which began recruitment in 2002 and 2005 respectively, provided some data. START and ANRS 12136 TEMPRANO, the first trials to randomise patients to initiation at versus above a CD4 count of 350 cells per ul, are ongoing.

It is deeply unfortunate that, before the relavent trials have reported, IAS-USA have overinterpreted the observational data and come to such strong conclusions on initiation above CD4 350.

Interpreting observational evidence is difficult where outcomes might be confounded by variables that are hard to measure – for example, how frail patients appear to be. I am training as an epidemiologist. I think observational studies are important. However, there are good examples of observational data proving very misleading. Whenever possible, findings should be tested in randomised controlled trials (or in other studies incorporating an element of randomisation).

Why have these RCTs been so long coming? I suspect it is, in part, to do with clinicians’ difficulty in saying “we don’t know” and, in part, because we, in medicine, are too deferential. In the medical hierarchy, the junior is expected to defer to their senior and seniors are expected to make decisions. A doctor willing to make strong conclusions from data, like the doctor always willing to provide a diagnosis, appears clever and decisive – traits we value.

But in many fields of medicine, expert opinion is a curse. Unfounded consensus makes it harder to organise the trials that will give us the conclusive answers we need; harder to persuade patients to enrol into these trials.

When this expert opinion comes from individuals who benefit from their relationships with companies set to profit from the recommendations, then we have an additional problem. Sadly, this is all too common.

Where doubt exists in clinical care and an RCT is possible, this must be the priority. Other evidence is not as reliable and places patients at risk. We must challenge unsupported expert opinion and the presence on guideline panels of individuals with significant conflicts of interest.

We need to diagnose less and to admit the limits of our knowledge. Honesty is just as important as intelligence or decisiveness.

This blog was initially published by Quackdown.

 

Treatment of Human Immunodeficiency Virus in Adults—Reply

Tom A. Yates MBBS BSc MSc, Joseph Sonnabend MBBCh FRCP Ed

We appreciate Dr Thompson and colleagues’ reply [1] to our letter. [2] In our opinion, summarising the observational evidence is reasonable. However, to then recommend clinicians offer ‘ART to all patients regardless of CD4 cell count’ [3], goes beyond the available evidence. We imagine clinicians paying heed to this recommendation would be discouraged from enrolling patients into the START trial. [4]

The authors point out that their guidelines [3] cite HPTN 052 [5] in support of their strong (Grade A1a) recommendation to initiate antiretrovirals in people with CD4 <500 cells/ul. However, the control group in this RCT started treatment at CD4 <250 cells/ul. Therefore, the study will overestimate the benefits of initiating above 350 cells/ul.

The additional detail provided [1] on the committee’s arrangements to deal with conflicts of interest is essential. It allows readers to attempt an assessment of the extent of the bias they might introduce.

We understand that, particularly in fields such as HIV medicine, it is difficult to identify senior figures who have no associations with industry. It may be that a balance must be struck between experience and independence. However, the IAS-USA threshold as currently drawn is not defensible. For example, a clinician who 12 months previously undertook industry sponsored promotional or marketing activities cannot be considered independent, particularly if there are no restrictions on such lucrative roles in the near future.

If the IAS-USA panel is the most independent HIV guideline panel, that is an indictment of the field. We must demand better.

1. Yates and Sonnabend. Treatment of Human Immunodeficiency Virus in Adults. JAMA 2012; 308(15): 1522-1523.

2. Thompson et al. Treatment of Human Immunodeficiency Virus in Adults—Reply. JAMA 2012; 308(15): 1522-1523.

3. Thompson et al. Antiretroviral Treatment of Adult HIV Infection 2012. Recommendations of the International Antiviral Society–USA Panel. JAMA 2012; 308(4): 387-402.

4. The START Study website. Available at http://thestartstudy.org/index.html. Accessed September 1, 2012.

5. Cohen et al. Prevention of HIV-1 infection with early antiretroviral therapy. N Engl J Med 2011; 37 365(6): 493-505.

Why Panorama was wrong about ‘health tourism’

In my opinion a recent Panorama programme, “Britain’s Secret Health Tourists,” didn’t do justice to a complex issue. With a Department of Health consultation looming on the issue, it’s important doctors understand why. I’ve attempted to summarise the evidence on this issue elsewhere, so here I’ll stick to the key points that I think Panorama got wrong, conflated, or forgot to mention.

The key scoop presented by the Panorama team was that middle men were, for a fee, arranging GP registrations for foreign nationals. The problems with presenting this as a scoop are, firstly, that GPs are entitled to register whoever they wish so long as they don’t discriminate and, secondly, that the exploitation of vulnerable migrants is sadly not news.

It was argued that once you are registered with a GP, the NHS is your oyster. As evidence, we saw three undercover patients accessing blood tests or imaging. Hospitals were challenged and issued press releases promising to do better next time. A retired overseas visitors officer told us things were very bad.

And here is the first key issue that seems to have been lost in all the breathiness. Whether it happens in primary or secondary care, assessing eligibility, billing patients, and recovering debts carries a cost. People’s immigration status changes—visas expire, asylum appeals are lost or won—so this is not a once only activity.

When Sally Hargreaves and colleagues at Imperial College looked at this problem in Newham recently, it was concluded…

“In light of the broad scope of the organisational and procedural changes required for the effective implementation of the primary care proposals [charging for NHS primary care] in Newham, and the limited financial burden that Overseas Visitors appear to be having on primary medical services in the Borough, we conclude that the current proposals to streamline charging procedures at primary medical services with those in place at hospitals should be reconsidered.”

The 2001 census found Newham to be the most ethnically diverse part of the UK.

Assessing eligibility is also an inconvenience for everyone involved, including frontline staff and eligible patients. This was recognised by Nye Bevan, whose views were wrongly assumed by a Migration Watch spokesman interviewed for the programme.

“One of the consequences of the universality of the British Health Service is the free treatment of foreign visitors. This has given rise to a great deal of criticism, most of it ill-informed and some of it deliberately mischievous…The fact is, of course, that visitors to Britain subscribe to the national revenues as soon as they start consuming certain commodities…They make no direct contribution to the cost of the Health Service any more than does a British citizen. However, there are a number of more potent reasons why it would be unwise as well as mean to withhold the free service from the visitor to Britain. How do we distinguish a visitor from anybody else? Are British citizens to carry means of identification everywhere to prove that they are not visitors? For if the sheep are to be separated from the goats both must be classified. What began as an attempt to keep the Health Service for ourselves would end by being a nuisance to everybody. Happily, this is one of those occasions when generosity and convenience march together…” (In Place of Fear, 1952)

The second problem for me was that Panorama did not put the figures in context. Figures at a national level always sound dramatic. However, the £40 million allegedly written off over the last four years by hospitals in England and Wales is approximately 0.01% of the NHS budget over the same period. With no breakdown of who these people are, it is difficult to conclude anything from this.

However, the programme’s worst omission was any serious consideration of the harms that have resulted from the NHS charging regime that has been operating in England’s hospitals since 2004. Adam Hundt, a lawyer who specialises in defending vulnerable migrants who have fallen foul of these regulations, attempted to discuss one of his cases when he was interviewed. However, from the clip shown, Declan Lawn, the presenter, didn’t seem particularly interested.

When (on Twitter) I challenged Declan about this omission, he pointed to a section of the programme in which he interviewed Daljit, an undocumented migrant who obtained NHS treatment for injuries sustained whilst working in the construction industry. However, whilst Daljit, like many undocumented migrants, was homeless and destitute, his problems were not a consequence of denial of healthcare. When I pointed this out, Declan then tweeted, “no one says care should be denied. Just how we deal with it.”

The truth is that care is routinely denied to people like Daljit and that vulnerable people are coming to harm. In 2006, the Refugee Council published a set of case studies that make for shocking reading. The people described in these cases studies include individuals entitled to NHS care but unaware of or unable to communicate their rights. The rules remain largely unchanged and I continue to hear similar stories.

There is little evidence that health tourism is a significant burden on the NHS. I know this both from personal experience, having trained as a doctor in and around London, and because I have reviewed the published evidence. The impression created by the programme that England is liberal in its approach was misleading. Several European countries operate more liberal rules. For example, in Wales, refused asylum seekers have full access to NHS treatment whilst awaiting removal from the UK.

Perhaps it optimistic to hope that one day the Panorama theme will play, the credits will roll and the voiceover will say “health tourism is complex and has been overhyped—in tonight’s programme we will calmly review the evidence.”

However, if journalists won’t, healthcare professionals must make the effort to familiarise themselves with the literature. The expected Department of Health consultation is our chance to make the case for more humane and rational regulations. It is important that rational voices be heard.

This article was first posted under a different title on the BMJ Blog.

Epidemiology Cartoons

There are some great cartoons about epidemiology and population health out there. Here are some of my favourites…

1. ‘Correlation’ by xkcd

As any epidemiologist will tell you, correlation (or association) does not always mean causation! For example, having grey hair is associated with higher mortality but most readers will realise there is an alternative explanation for this association. Austin Bradford-Hill published a famous list of things that might suggest an association is causal in 1965.

2. ‘Correlation or Causation’ by Bloomberg is fun and deals with the same issue.

3. I like this South African cartoon about the importance of keeping things in perspective. I don’t know who drew it.

4. The cartoonist Zapiro has brilliantly chronicled the recent history of AIDS in South Africa. This cartoon attacking the late Health Minister, Manto Tshabalala-Msimang, is, I think, particularly good. For those unfamiliar with the back-story, there is a brief account under the cartoon and a longer account here.

5. I’ve just started a PhD and was pointed towards this visual representation of the process, which I thought elegant. Keeping things in perspective, again!

If you know of other good cartoons on epidemiology or population health, I’d love to hear from you.

Enjoy!

Is it time to start evaluating social policy using randomised trials?

In February, I went to see Rachel Glennerster speak in London at a meeting put on by organisations including the Alliance for Useful Evidence. I was impressed.

Glennerster is the director of the Abdul Latif Jameel Poverty Action Lab  (J-PAL) at MIT – the outfit behind hit book Poor Economics. J-PAL is at the forefront of a movement promoting the wider use of randomised evaluations, akin to those used in medicine, in the development of social policy. Esther Duflo’s TED talk, gives a good idea of what they are about (I note the evidence regarding the educational benefits of deworming has recently been challenged).

As Duncan Green of Oxfam puts it, “[J-PAL] are unashamedly, indeed belligerently, micro, small-is-beautiful technocrats…they are incrementalist and disagree with the structural focus of the ‘political economy’ approach or sweeping calls for revolution – they argue that positive changes can be achieved with less pain, often on a massive scale, even in hostile political environments, simply by minor tweaks to policies and institutions…And they aren’t scared to follow the logic of their own arguments, even when it takes them in some surprising directions…it’s hard to pigeon-hole the work as particularly right or left wing.”

The strongest arguments for this approach are the often counter-intuitive conclusions reached and the savings Governments might make by not investing in useless interventions.

Glennerster described a study demonstrating the dramatic impact of ‘commitment devices’ on fertiliser use in rural Kenya. A small but time limited discount on fertiliser offered at the time of harvest, when money was relatively plentiful, resulted in much higher uptake than a larger discount offered at other times in the year. “Present bias” (procrastination or prioritising today over tomorrow) could be overcome by forcing a decision at the point it would least impact on people’s immediate quality of life. Glennerster argued present bias was phenomenon observed in other contexts. In Indian slums, she said, people have been observed to pay others to look after their money for them, aware they might fritter it away on cups of tea or other sources of immediate pleasure. Glennerster described how, in The Philippines, commitment devices have been demonstrated to increase savings and reduce cigarette consumption.

One of Glennerster’s main arguments was that flexibility in study design – particularly choosing the best and least disruptive method of randomisation – allowed randomised evaluation of interventions in a broad range of circumstances. She outlined a range of options:

“Randomise phase in” (in epidemiology we call this a ‘stepped wedge’ design): if an intervention is being adopted but there is not capacity to roll it out everywhere at the same time, you can randomise the order in which it is introduced by clinic, school, region, etc. In medicine, this was first used to evaluate the impact of Hepatitis B vaccination on chronic infection and the risk of liver cancer in later life. Glennerster has used the design to demonstrate microcredit wasn’t as good as observational data had suggested.

“Randomisation at the cut off”: sometimes there are groups of people who definitely need an intervention (e.g. the poor) and groups who definitely don’t (e.g. the rich). Usually there is a group of people in whom the intervention is of unknown benefit. It is possible to gain information about the intervention by randomising the threshold at which the intervention is offered. Whilst you only gain information about borderline cases from this method, it is useful information. An iterative process can then be followed to test whether benefit can be demonstrated at the next threshold.

“Randomise intensity of intervention”: For example, you could randomise areas to receive 5 vs 10 evening class teachers. This allows assessment of local equilibrium effects – for example, whilst education might benefit individuals, you may be concerned that a larger pool of educated workers could drive down wages for skilled labour.

“Encouragement design”: If something has already been widely implemented but uptake is patchy, you may be able to assess its effect by randomising people to receive encouragement to access the intervention or no encouragement. Given encouragement may not be particularly effective and given many people in the control arm may access the intervention anyway, these studies may need to be big to demonstrate an effect.

An audience member wondered whether, in a time of cuts and austerity, information could be gained from examining the effect of the withdrawal of services.

The audience was mostly made up of British civil servants who were interested in the politics of randomised evaluation – how they could persuade their minister to adopt such an approach. There are obvious stumbling blocks, including the need in many fields for long term data to assess impact. Glennerster was realistic that politicians may not be willing to put their flagship policies to the test but thought politicians could often be persuaded to test details of their policies or to test the best means of implementing them. It works if they are allowed to set the questions!

Glennerster pointed out that the evaluation of PROGRESA, a conditional cash transfer programme in Mexico and perhaps the most famous example of randomised evaluation in social policy, was instigated by a Government who knew they were going to lose the next election. It was a way to safeguard their programme. They knew the next Government would find it hard to stop the trial once it was started and were confident the evaluation would show benefit, again making it hard for the next Government to drop the programme. Randomisation can be politically advantageous.

Many see evaluation as a global public good and feel funding for evaluation shouldn’t have to come out of programme budgets. A number of organisations including 3ie, the World Bank and the French Government have created central pots of money that those implementing programmes can apply to should they wish to evaluate what they are doing. These funds should, over time, prove cost saving.

I disagreed with only one point made at the talk. An audience member argued that in global health the move from funding health systems to funding disease specific programmes represented progress towards evidence based policy making. There is a danger here that we systematically prioritise interventions whose benefits are easy to measure or whose harms are difficult to measure. We shouldn’t go about evaluations thoughtlessly and we must compare like with like. However, this is not an argument against randomisation or rigorous evaluation if sensibly conducted.

Much of the most exciting work in this field is happening in low income countries – particularly at the interface between health and microeconomics. See, for example, this study looking at the impact on HIV transmission of cash transfers to young women in Malawi – a massively exciting result.

Over recent years, policy making in the UK has occured in something of an evidence vacuum – recent health service reforms being a case in point. Audience members pointed out, however, that, even here, there remain islands of reason. These include the Education Endowment Fund and the interesting work of the Behavioural Insights Team at the Cabinet Office.

So hope remains. Long live sensible policy making! Long live the randomisation revolution!

This blog was originally posted on the Speaking of Medicine blog.

Industry Sponsored Editorial Assistance

The September 2011 edition of the Quarterly Journal of Medicine contained two review articles which dealt with the use of new oral anticoagulants in patients with atrial fibrillation.

The first was by Prof Richard Hobbs and Isabelle Leach, who works for a reputable sounding organisation called Chameleon Communications. The funding and conflict of interest statements read as follows.

‘F.D.R.H. received no funding for this work. Bayer AG and Johnson & Johnson Pharmaceutical Research & Development, LLC, through funding of the professional medical writing services provided by IL. The sponsors were not involved in writing or editing the material. The authors take full responsibility for all content…F.D.R.H. has received occasional speaker fees or sponsorship from a variety of pharmaceutical companies, some with interests in AF including Boehringer Ingelheim, Pfizer, and Bayer. I.L. is an employee of Chameleon Communications International which received funding from the sponsors for her time on this manuscript.’

Bayer, who funded the editorial assistance, make Rivaroxaban, one of the drugs discussed in the article. Many of the companies from which Prof Hobbs has received money also make new oral anticoagulants. Chameleon Communications has been involved in preparing several other manuscripts in other journals on new anticoagulants over recent months.

The second, by Prof Joerg Kreuzer, was that edition’s ‘Editor’s Choice’ article. Prof Kreuzer’s funding and conflict of interest statement read as follows.

‘This work was supported by Boehringer Ingelheim. The author was fully responsible for all content and editorial decisions. The author received no financial support or other compensation related to the development of the paper. Editorial support was funded by Boehringer Ingelheim…Conflict of interest: None declared… The authors would like to thank Rebecca Gardner of PAREXEL, UK, for editorial assistance in the preparation of this article.’

Boehringer Ingelheim, make Dabigatran etexilate, a drug promoted in the article’s conclusion.

Concerned about these clear conflicts of interests, I wrote a letter to QJM arguing that the involvement of industry in the funding of review articles generates bias, probably above and beyond the mere choice of topics on which they focus. I requested answers to these four questions –

 a)  What is Isabelle Leach’s expertise in this field?

b) Could Chameleon Communications or PAREXEL highlight any published work with which they had assisted that recommends not prescribing a drug manufactured by the sponsor of the work?

c) Would the sponsors, PAREXEL or Chameleon Communications be willing to make public the agreements or contracts between their organizations pertaining to these QJM papers?

and d) Whether the editors had attempted to source an independent review on this topic and, if not, why not?

I concluded –

‘How long are we going to put up with a situation where our practice is informed by biased evidence summarized for us by people who have financial relationships with companies set to profit from alterations to our practice? These flagrant conflicts of interest would not be tolerated in other industries. We need to get our house in order.’

My letter prompted a note from the editor and a response from Prof Hobbs.

Both claimed that conflict of interest was a ‘complex’ area and defended their conduct by claiming all conflicts of interests had been declared and that the articles had been through peer review – claims I had not disputed. None of my questions were answered.

Prof Hobbs noted ‘Dr Yates…does not state what material he considers inaccurate or imbalanced’ and asked ‘Surely Dr Yates does not think that the paper should not have mentioned the emerging evidence base for these novel agents in addition to reviewing the data and associated usage of warfarin and aspirin?’ He asserted ‘The paper underwent independent review by the Journal and the authors responded to feedback. If the paper had not been assessed as a balanced review it would not have been accepted for publication.’

The editor, Prof Michael Bannon, declined to print my response to Prof Hobbs letter stating ‘The editorial decision is that this issue is now closed and no further correspondence will be considered regarding this.’

Happily, the internet offers me the opportunity to conclude a debate I believe to have been prematurely closed. This is the letter I submitted to QJM –

 

‘I thank the editors and Prof Hobbs for taking the time to respond to my letter on industry supported review articles [1, 2] but note the questions I raised in my letter are yet to be answered.

Both the editors [1] and Prof Hobbs [2] state the peer review process and the declaration of conflicts of interest as factors that protect readers from any bias. I am aware of no body of research that supports this position.

Research on the impact of disclosure of conflict of interest is contradictory [3, 4]. Recent work highlighting systematic differences between industry funded and independent review and comment articles on the rosiglitazone controversy suggest peer review offers no defence against funder bias finding its way into peer reviewed literature [5].

In answer to Prof Hobbs’ question [2], I believe it is important that clinicans are able to access review articles on this topic. However, they should be written by authors who have no financial relationship with the companies who make the products under discussion.

I am no expert on anticoagulation so am unable to critique his paper. Most other readers will be in the same position and thus unable to tell whether his clear conflicts of interest have affected the way in which he has presented the evidence. For this reason, journals should not be publishing reviews by authors with clear conflicts of interest and clinicians should not allow such articles to influence their practice.

Tom A. Yates

1. Notes from editor in response to: Yates TA. Questionable summaries of questionable evidence. QJM 2011; doi:10.1093/qjmed/hcr246.
2. Hobbs R. Challenges of anticoagulant therapy in patients with atrial fibrillation in clinical practice. QJM 2011; doi:10.1093/qjmed/hcr253.
3. Cain DM, Detsky AS. Everyone’s a Little Bit Biased (Even Physicians). JAMA 2008; 299: 2893-95.
4. Lacasse JR, Leo J. Knowledge of ghostwriting and financial conflicts-of-interest reduces the perceived credibility of biomedical research. BMC Res Notes 2011; 4(1): 27.
5. Wang AT, McCoy CP, Murad MH, Montori VM. Association between industry affiliation and position on cardiovascular risk with rosiglitazone: cross sectional systematic review. BMJ 2010; 340: c1344.’

 

Amy Wang’s review of articles commenting on the Rosiglitazone controversy, is well worth a read. After reading it, it is difficult to argue peer review offers any defence against bias creeping into the literature.

QJM, sadly, are not alone in printing review articles by authors with clear conflicts of interest. Our profession is up to its eye-balls in conflict of interest and other journals have even lower standards. 

Happily, there are groups fighting to reassert medicine’s independence from industry. It’s high time we got behind them.

Separate analysis of independent and industry supported studies would be informative

I am very glad the Cochrane Collaboration exists. However, it is important to consider how its activities might be harmful and to take every effort to mitigate this risk.

I periodically re-read Joel Lexchin’s review of outcome in industry supported vs non industry supported studies to remind myself of the powerful and pervasive impact that conflict of interest has on the evidence base. Given most randomised controlled trials are industry funded, there is a risk that Cochrane Reviews lend this biased evidence base legitimacy.

Of course, much of what the collaboration already does limits the chances of this bias being fed through into the findings of their reviews.

The wrong choice of comparator drug or outcome shouldn’t be fed through if inclusion criteria are robust.

Similarly, good inclusion criteria should limit the effects of poor study design. However, in Lexchin’s review, study quality did not explain the bias seen.

Publication bias is more problematic. Freedom of information requests have been needed in recent years to gain access to unpublished negative studies and get a true picture of treatment effect. Not all Cochrane reviewers go to these lengths.

I support recent calls on the Cochrane blog and in the literature, for meta-analysts to report the funding sources and author conflicts of interest of included studies.

However, I propose the Cochrane Collaboration go further. If all meta-analyses included a separate analysis of both independent and industry funded studies, the most perverse distortions of the evidence base might be visible.

Perhaps more importantly, routinely demanding such analyses from Cochrane reviewers would make it easy for the collaboration to assess periodically how effective their procedures are at preventing funder bias from being fed through into the conclusions of their reviews.

This blog was originally posted at BMJ.com.

Is stoicism an important and neglected confounder?

Most doctors and nurses will have a deep well of patient stories – examples of great fortitude and its converse. It is clear to any clinician that some patients either feel their symptoms (or report them) more than others.

This is well recognised in parts of the medical literature with, for example, the distinction made between irritable bowel syndrome patients and ‘non-patients’ – people with the same symptoms as those who seek care but who never present to services.

I wonder, however, whether stoicism has been neglected in the epidemiological literature. It seems likely to be related to many exposures and many diseases. Association with exposure and with disease are two of the key features of a confounding variable.

Stoicism is particularly likely to be associated with diseases where diagnosis relies on self-reported symptoms. Many diseases fall into this category.

I feel stoicism is likely to be related to exposures such as cycling to work every day, ambient household temperature, and so on. However, it’s not hard to imagine that many health behaviours might be influenced by how stoical people are.

Confounding is well illustrated using a classic example, coffee and cancer. People who drink coffee are more likely to get cancer. They are also more likely to be smokers. The well-established association between smoking and cancer largely explains the association between coffee and cancer. Smoking, in this example, is a confounder.

Epidemiologists try to control for confounders to avoid misleading results.

A Pubmed search for ’stoic*’ AND ‘confound*‘reveals 24 papers, only three of which are even vaguely relevant. None are epidemiological and I can access only one. By failing to control for stoicism in epidemiological studies, do we risk throwing up reams of spurious associations – warmer home environment is associated with fatigue, winter cycling is associated with less chronic low back pain, etc? Will this lead us to spend money trialling useless interventions?

In some cases it may be possible to adjust – person by person – for reported symptoms at another time point. However, to control for confounders, you usually need to be able to measure them.

It is entertaining to think how one might measure stoicism. For example: ‘When you last had a sore throat, did you (a) get on with it (b) have a day off work or (c) go to see your GP?’ We might apply more objective tests – time to first complaint when the waiting room is kept at ten degrees, perhaps? We could devise a scale.

A version of this blog was originally posted at BMJ.com.

Lies, damn lies, and epidemiology

“Dear sir, I have completely failed to understand a simple criticism of our work, please tell everyone, yours, BBCnews“ 

Tweet by @bengoldacre, 4 November 2011

The misuse of epidemiology is everywhere. Lets look at two prominent culprits.

”Three fold variation’ in UK bowel cancer death rates’ splashed a recent BBC News headline.

Had someone not thought to check the numbers we’d have all gone away grumbling about postcode lotteries. And we’d all have been wrong. What was going on?

All things vary. If a coin is flipped several times, chance dictates it will not land heads exactly half the time. But the more times you flip the coin, the closer to a half you get.

In the year in question, there were only six deaths from bowel cancer in the Shetland Islands, twelve in Antrim, fourteen in Watford, and so on. Even with a uniform rate of bowel cancer deaths across the UK, might there only be two deaths in Shetland the following year? Would that merit a headline or could it just be statistical noise?

Thankfully, as reported by Ben Goldacre, someone properly analysed the data. The variation in death rates seen was actually less than would be expected by chance. Embarrassingly, the BBC failed to understand the criticism.

Whilst the poor quality of science journalism has been well documented by Ben and others over the years, ineptitude is less concerning than the deliberate misuse of data.

The debate surrounding the recent UN High Level Meeting on Non Communicable Disease (NCD) was mired in epidemiological naughtiness. Among the most prominent culprits were the NCD Alliance, a grouping of NGOs backed by the pharmaceutical industry, who unfortunately – given their funding – emerged as the leading advocacy organisation.

“NCDs were responsible for 63% of all global deaths in 2008. This is not just a statistic, it is the deaths of 36 million people. With the incidence of NCDs predicted to rise by 17% over the next ten years worldwide, we must work together to ensure it is world leaders who attend the Summit and agree to a concrete set of commitments that will result in sustained action.”

Professor Jean Claude Mbanya, President of the International Diabetes Federation, quoted in a press release (pdf) from the NCD Alliance.

Given immortality is not currently achievable and that we must all die of something, presenting mortality data as counts is thoroughly misleading. Non age standardised counts fail to capture the difference between a death from a respiratory tract infection aged five and a death from ischaemic heart disease aged ninety five.

With huge expertise within the alliance, it is hard to believe that their presentation of data in this manner was anything other than a deliberate choice to spin the data and inflate the contribution of NCDs. Whilst counts are easier to understand than Disability Adjusted Life Years lost or age adjusted rates, the alliance could have presented data on numbers of premature deaths – less misleading and totally comprehensible.

I don’t dispute the assertion that NCDs are emerging as a major problem in low and middle income countries, and I’m gutted the summit achieved so little. However, the data is compelling when presented honestly. It doesn’t need to be spun.

It is very easy to misrepresent epidemiological data and for the unscrupulous to spin data for the press. Given the volume of coverage epidemiological research generates, groups such as the Science Media Centre have their work cut out.

I don’t know what the solution is. Ben Goldacre’s suggestion that we need more science editors and fewer science reporters seems sensible. I’d be interested in people’s thoughts.

A version of this blog was originally posted at BMJ.com.