An update on the Cochrane HPV vaccines review following a critical assessment published in the BMJ Evidence-Based Medicine Journal


Following publication of the Cochrane Systematic Review of HPV vaccines in May this year, the Time For Action group produced an open letter to the authors of the paper highlighting their concerns. At the end of July, a very critical assessment of this Systematic Review of HPV vaccines was published in the BMJ Evidence-Based Medicine journal. This assessment was authored by a team of scientists from the Nordic Cochrane Centre in Copenhagen and the Centre for Evidence Based Medicine at the University of Oxford in the UK. Far from the science being settled over HPV vaccines, it appears the debate is opening up.

On the 27th May 2018 the Time for Action group produced an open letter to the authors of the Cochrane Review on HPV vaccines in response to the paper recently published to a media fanfare proclaiming the vaccines safe and effective. A copy of the open letter was first sent to the corresponding author of the paper.

The response received was as follows:

I received end May your Time for Action letter. As suggested by Dr David Tovey, Editor-in-Chief of the Cochrane Library, I let you know that comments or questions related to “Arbyn M, Xu L, Simoens C, Martin-Hirsch PP. Prophylactic vaccination against human papillomaviruses to prevent cervical cancer and its precursors. Cochrane Database Syst Rev 2018; 5: CD009069”, can be submitted as a letter to the Editor following the foreseen procedures.  These procedures can be found at the website of the Cochrane Library ( Comments related to the press conference organised by the Cochrane Collaboration on 8 May 2018 in London can be addressed to (Head of Knowledge Translation, Cochrane Central Executive).”

As suggested, we have submitted our comments to the Cochrane Library, but have not yet received a response.

This week Lars Jorgenson, Peter Gotzsche and Tom Jefferson (from the Nordic Cochrane Centre in Copenhagen and the Centre for Evidence Based Medicine at the University of Oxford in the UK) published their assessment of the Cochrane Systematic Review, titled The Cochrane HPV vaccine review was incomplete and ignored important evidence of bias

Their conclusion:

Part of the Cochrane Collaboration’s motto is ‘trusted evidence’. We do not find the Cochrane HPV vaccine review to be ‘trusted evidence’, as it was influenced by reporting bias and biased trial designs. We believe that the Cochrane review does not meet the standards for Cochrane reviews or the needs of the citizens or healthcare providers that consult Cochrane reviews to make ‘informed decisions’, which also is part of Cochrane’s motto.”

The main points raised in the assessment are:

  • The Cochrane Review missed nearly half of all the eligible trials
  • In all of the trials that Cochrane included none used a placebo, all used an active comparator (the aluminium adjuvant or another vaccine)
  • Composite surrogate outcomes for cervical cancer were used (results were grouped together to include lesions that would often clear without intervention) and there was no description of women in the HPV vaccinated groups that did develop cervical cancers
  • Assessment of serious and systemic adverse events was incomplete and there was inadequate explanation of these, inadequate explanation of deaths that occurred in vaccinated women and lack of follow-up of unpublished data on SAEs
  • No assessment of HPV vaccine-related safety signals was done
  • All trials included were funded by the pharmaceutical companies and authors were considered to have conflicts of interest
  • The press release included people were considered to have conflicts of interest and it presented no balance

We are aware that Ministers in the Department of Health are quoting the Cochrane Systematic Review as evidence of safety and efficacy  – Time for Action have already requested a response from ministers and health agencies to this new article and we will continue to hold them to account.



Arbyn MXu LSimoens CMartin-Hirsch PPLProphylactic vaccination against human papillomaviruses to prevent cervical cancer and its precursorsCochrane Database of Systematic Reviews 2018, Issue 5. Art. No.: CD009069. DOI: 10.1002/14651858.CD009069.pub3

Jørgensen LGøtzsche PCJefferson T.  The Cochrane HPV vaccine review was incomplete and ignored important evidence of bias. 



The initial open letter to the authors of the Cochrane Systematic Review of HPV vaccines:



contact: Marc Arbyn, Unit of Cancer Epidemiology, Belgian Cancer Centre, Sciensano, Juliette Wytsmanstreet 14, Brussels, B-1050, Belgium,

Whilst we appreciate that the HPV vaccination has been given to millions of girls without apparent adverse reactions and that it may well be of health benefit to them in the future, we write as parents of girls who have developed severe, life-limiting illness within close proximity of having the HPV vaccination.  Some of our daughters have been ill for a long time, in some cases for 9+ years, whilst others have become ill more recently – girls affected have had/are having their education compromised, those older are not employable, some have been housebound for years.

We would like to raise some questions on the content of your recently published review “Prophylactic vaccination against human papillomaviruses to prevent cervical cancer and its precursors”,   and how the study was reported in the UK media We hope that you will take the time to respond.


As a group, we first became interested in the progress of this review when we saw the published protocol in December 2013 and read the comments by Catherine Riva et al which appeared on the Feedback section of the protocol in December 2014.

Of particular interest were the following comments:

Riva et al, 16 December 2014

  1. This Cochrane review is important in order to examine the validity and trustworthiness of the design of the clinical trials with regard to the choice of outcomes as well as the rigour with which these trials were conducted. Consequently, the reviewers will need to address certain problems and limitations in the design and conduct of the studies:
  • The documents we have obtained from the FDA indicate that there were changes in the protocol during the course of the trials and therefore during the approval process. These changes necessarily had a major impact on the quality of the reporting and redefinition of certain sub-groups in at least three instances.
  • The minutes from meetings of the Vaccines and Related Biological Products Advisory Committee (VRBPAC) show that the decision to fast track the research led the American regulatory officials to choose outcomes that would allow them to evaluate only the specific effectiveness of the vaccination on lesions associated with HPV 16 and 18 and not its effectiveness on all HPV-associated lesions.
  • The criteria required to satisfy a fast track procedure were not fulfilled but the fast tracking had an impact on the choice of outcome.
  • The entries regarding the trials on indicate that their primary and secondary outcomes were not registered prospectively. We believe that this Cochrane review should raise these issues with FDA officials and scientific journals which have published results from the Phase III trials because they have broken their own rules of proper scientific conduct.
  1. This Cochrane review is important for thoroughly evaluating a potentially increased risk of the subsequent development of precancerous lesions in women who already have HPV infections targeted by the vaccination at the time they are vaccinated. This risk has not been sufficiently examined although existing evidence indicates the need for a thorough and careful examination of the possibility. This evidence includes:
  • Results submitted to VRBPAC in June, 2006
  • The Australian study done by Brotherton et al. This Cochrane review is important to calculate and report the risk of subsequent development of precancerous lesions in women who already have HPV infections targeted by the vaccination at the time they are vaccinated, and the ways in which it must be communicated to vaccinated women and to vaccinated girls and their legal guardians.

 We note that, in the final version of the Protocol, the authors, via Jo Morrison, the Co-ordinating Editor, Cochrane Gynaecological Cancer Group, state: “We thank Catherine Riva and colleagues for their helpful suggestions and comments, many of which we plan to address in the full review, since they have commented on the protocol only.” One of Ms Riva’s comments highlighted “changes in the protocol during the course of the trials and therefore the approval process” which would have “had a major impact on the quality of the reporting and redefinition of certain sub-groups in at least three instances”.

We cannot see any reference to the issues raised by Ms Riva to:

  • changes being made to protocols during the course of trials and
  • the retrospective registration of clinical trials as detailed above,

as we also consider these to be extremely important matters.  Is this something that the authors intend to address in future revisions?

The authors agree that they have not looked in detail at the impact of the vaccination on all HPV associated lesions and this will be addressed in future revisions. We consider this to be extremely important as in the UK many of the girls vaccinated in the catch-up programme or who are now being vaccinated at University or High Street pharmacies will have been sexually active prior to vaccination – and therefore could be at risk of prior infection.  Could the authors please advise of the timescale to report on this critical matter?


This systematic review includes 26 trials. The recently published “Index of the human papillomavirus (HPV) vaccine industry clinical study programmes and non-industry funded studies: a necessary basis to address reporting bias in a systematic review”

included 206 trials of which “Two thirds of the indexed studies were randomized clinical trials (136/206, 66%) and about half were phase III studies (89/206, 43%).”  Could the authors please explain why their review focused on such a small number of trials?


Number of doses:

Protection against precancerous lesions and persistent infection was also strong (RR ≤ 0.15) when fewer than three doses were received” Would the authors agree that earlier vaccinees were unnecessarily given three doses? And this is indicative of the fact that insufficient testing was done prior to the deployment of the vaccine?

We also note your references to missing data as follows:

“The efficacy of HPV vaccines was generally lower when any high-grade squamous lesions, irrespective of HPV infection type, was considered compared to efficacy for HPV16/18-associated lesions.” 

“This Cochrane review primarily used efficacy data extractable from peer-reviewed published reports. Since, in principle all trials evaluated at baseline all enrolled women for presence of HPV genotypes and in addition cytology, and HPV serology, more efficacy data are available which would fit the defined analyses groups included in our Cochrane review. However, often only a restricted series of results were reported limiting the number of studies in each of the analyses (varying from one to eight), and gaps of non-reported outcomes (Figure 6 and Figure 7). Indeed, only the endpoints CIN2+ related to HPV16/18 (Analysis 2.2) and persistent infection of HPV16/18 at six months(Analysis 5.4) in women being negative for HPV16/18 DNA at enrolment have as many as eight trials in one forest plot. Originally, we planned requesting data from data owners, to fill in gaps with available unpublished data. However, due to constraints in time and other resources this was not possible. We do not believe that this has undermined the importance of our review.”  Could the authors please confirm that the missing data has now been requested so that it can be included in updated versions of this review?  Could the authors explain what were the time constraints, given that the protocol was first published in 2013?

We note the age range:

“Assessment of the variation of vaccine efficacy by age group in more detail than the broad distinction younger or older than 25 years could not be done for most studies by lack of reported age-specific data. However, for the bivalent vaccine, an analysis by five-year age group could be performed.” Could the authors please explain why the trials selected for reporting efficacy only included participants vaccinated at age 15 or over, since the vaccination is most often given to girls at a younger age?

We note references to efficacy in women who have already been exposed to HPV as follows:

Table 1. High-grade cervical lesions in women who were hrHPV negative at baseline.” This table contains some variable statistics on efficacy, particularly:

22% efficacy CIN2+ associated with HPV 6/11/16/18 at least 1 dose, age 24-45

-4% efficacy CIN2+ irrespective of HPV types, at least 1 dose age 24-45

This vaccine is marketed in the High Street as being “suitable for men and women aged 12 to 44 inclusive” (Ref

Would the authors agree that the issue of efficacy in women who may already have been exposed to HPV at the time of vaccination needs to be addressed as a matter of urgency?

Would the authors explain why it was not seen as important to address this issue in this review, particularly given the time that it has taken for this review to reach publication?

Dr Diane Harper completed a 10-year review of the HPV vaccination in 2017.

  • Duration of antibody response is critical for clinical prevention of HPV infection. Cervarix has high anti-HPV16 and HPV18 antibody titers for at least 9.4 years [15]in longitudinal follow-up studies; Gardasil has plateaued anti-HPV16 titers well above natural infection titers for at least 9 years, but anti-HPV18 titers that are not different from natural infection titers as early as 24 months after vaccination [16].”
  • Nearly 20% of Gardasil9 recipients had a loss of detectable anti-HPV18 titers after 24 months. In Gardasil recipients, after 1.5 years over 10% of women had no detectable anti-HPV18 titers, after 3 years over 20% of women lost detectable titers, and after 5 years nearly 35% of women lost detectable titers [18]

Could the authors comment on why this review by Dr Harper was not referenced? And why findings on efficacy differ from the findings of this review?


Vaccine safety was evaluated over a period of 6 months to 7 years in 23 studies in this Review.

Only a small number of studies were used to assess safety:

The Tables showing safety outcomes are:

Analysis 7.5 Overall systemic events and general symptoms. Compares placebo (6102) to vaccinated (6224) in 18,191women in 8 studies only.

Analysis 7.6 Serious Adverse Events. Compares placebo (605) to vaccinated (611) in 6,978 women in 21 studies only.

Analysis 7.7 Deaths. Compares placebo (11) to vaccinated (13) in 71,452 in 23 studies.

Could the authors explain why 23 studies were used for looking at deaths, 21 small studies were used for looking at SAEs and 8 larger studies were used for looking at systemic events?

Could the authors explain why the same trials and the same number of subjects were not used for all of the reporting on safety?

What was the reasoning behind choosing some studies to assess safety and not others?

Risk factors:

Particular effort was undertaken to assess severe adverse effects in order to inform health professionals, stakeholders, adolescent girls and women, not only about the potential beneficial effects of HPV vaccines but also about possible harms.”

While the Review looks to balance the efficacy and safety in clinical trials. What the review doesn’t do is inform stakeholders and make clear the absolute risk of cervical cancer or mention the presence of co-factors in those taking part that would make the subjects more susceptible to develop cervical cancer – for example smoking, contraception, parity, diet etc. Would the authors agree that these risk factors are important to consider when making a decision to vaccinate?

Design of systematic review to identify long term adverse events/pregnancy outcomes:

“We did not find an increased risk of serious adverse effects. Although the number of deaths is low overall, there were more deaths among women older than 25 years who received the vaccine. The deaths reported in the studies have been judged not to be related to the vaccine. Increased risk of adverse pregnancy outcomes after HPV vaccination cannot be excluded, although the risk of miscarriage and termination are similar between trial arms. Long-term of follow-up is needed to monitor the impact on cervical cancer, occurrence of rare harms and pregnancy outcomes.”Could the authors comment on what long-term follow-up is being done of trial participants who reported SAEs and when they will update this review with that data?

Lack of detail around systemic events:

Systemic events with general mild symptoms were similarly frequent in vaccinated recipients and placebo or control vaccine recipients (RR 1.02, 95% CI 0.98 to 1.07; participants = 18,191; studies = 8; I2 = 72%; moderate-quality evidence; Analysis 7.5). The risk of serious adverse effects was similar in those vaccinated and those who received placebo or control vaccine (RR 0.98, 95% CI 0.92 to 1.05; participants = 71,597; studies = 23; I2 = 6%)) high-quality evidence). There was little or no difference between the different vaccines (P = 0.19; I2 = 39.7%, Analysis 7.6). Restriction to data extracted only from publications in peer-reviewed journals yielded very similar results: RR 1.01, 95% CI 0.95 to 1.06; 71,452 participants; studies = 22, I2 = 0%, Figure 10), with very minor differences between the vaccine types (P = 0.83, I2 = 0%).”  There is very little detail on the side effects reported – can the authors confirm how long the reported symptoms lasted?

Inability to detect rare events:

“All estimates of adverse effects in our review were restricted to those reported from randomised trials and therefore could not detect rare events, for which post-marketing surveillance, pharmacovigilance activities and linkage studies, joining vaccine and morbidity registries, are needed.”  Would the authors agree that, given this statement, regulatory authorities should not rely on this review to make assertions around the absolute safety of the HPV vaccination?

Reference to studies that would not detect autoimmune autonomic disorders:

The review references the Donegan study (MHRA, 2012) and the Arnheim-Dahlstrom study (2013). Would the authors agree that neither of these studies would identify a signal for autoimmune autonomic disorders, which are currently of concern to parents and patients?

Unbalanced referencing:

“The committee concluded that the risk-benefit profile of prophylactic HPV vaccines remains favourable and expressed its concerns about unjustified claims of harm, which lack biological and epidemiological evidence, and which may affect the confidence of the public (Larson 2011).” We are genuinely shocked that this review has referenced Heidi Larson, an anthropologist, from the London School of Hygiene and Tropical Medicine yet has not referenced medical doctors who have published on suspected side-effects of the HPV vaccinations. Could the authors please say why they have not referenced peer-reviewed, published articles/case reports, by Dr Jill Schofield (Colorado), Dr Svetlana Blitshteyn (New York), Dr Rebecca Chandler (Uppsala), Professor Jeanne Hendrickson (Yale) and various authors from Japan?  (references provided at end of letter).

No epidemiology done to detect signal for POTS/CRPS:

“Evidence on rare potential harms, such as autoimmune disorders, are difficult to capture in randomised controlled trials (RCTs). The findings of this review should be seen in the context of surveillance studies which have been conducted globally since the licensing of the vaccines and have demonstrated a consistently good safety profile in population usage as reviewed by the Global Advisory Committee on Vaccine Safety (GACVS) of the WHO on multiple occasions.” – Could the authors refer us to any epidemiological study that has been done to investigate POTS or CRPS, for which a signal was raised in 2015 by the Danish health authorities?

Missing evidence on long-term pregnancy outcomes:

“We did not find conclusive evidence of increases in the risk of congenital anomalies and adverse pregnancy outcomes in vaccinated women who became pregnant throughout the trials. However, more evidence is needed to determine long-term outcomes in pregnant women who received the vaccine”. Do the authors agree that in the context of the current UK review of side-effects of Valproate and Primodos, that it is critical that this evidence is provided?  When do the authors expect to include long-term evidence in future updates to this review?


We think parents considering vaccinating their daughters or sons should pay particular attention to the following statements from the review:


“The purpose of prophylactic vaccination against HPV is to reduce the incidence of cervical cancer. However, this outcome could not be assessed in our review, since trials conducted were not powered and included insufficient follow-up time to demonstrate this endpoint.”

“Defining invasive cancer as an outcome of the trials was considered as an unethical and unfeasible endpoint and would require extremely expensive and lengthy observation periods and postpone the availability of vaccines for decades.”

“The studies were not designed to evaluate cervical cancer and the duration of the studies was too short to determine the effects of human papillomaviruses (HPV) vaccination on cervical cancer outcomes.”

“We planned to distinguish adverse effects occurring in the period between zero to four weeks and more than four weeks after administration of vaccines. However, since this timing of observation of adverse events was not documented uniformly in the trials reports, this distinction could not be implemented in the review.”

Woman aged 45, Mexico: “This woman had normal cytology but was HPV-18 DNA-positive at study entry (May 2006). At the next scheduled cytology testing at Month 12 (April 2007), the cytology finding was atypical squamous cells cannot exclude highgrade squamous intraepithelial lesion. She was diagnosed with metastatic cervical cancer in May 2007 (approximately 7 months after receiving the third dose of vaccine or control) and died in July 2008”


“Long-term surveillance and registry-based research (linking of vaccination databases with screening, cyto-histopathology, cancer registries and biobanks; and linking with morbidity, mortality and birth/maternity registries) are needed to establish vaccine efficacy and safety over time.”

The comparison of the risks of adverse events was compromised by the use of different products administered to participants in the control group, varying from adjuvant (often aluminium hydroxide or other aluminium compound) or an alternative vaccine (often Hepatitis A or Hepatitis B). Therefore, the pooled risks of adverse effects associated with HPV vaccines and the assumed risks for control groups must be interpreted cautiously.”

“Although the trials were large and no safety concerns were established, vaccine safety requires evaluation in surveillance studies after introduction of vaccination programmes


We would point out that this review has already been referenced by the media in the UK as a justification for deploying the vaccination to all teenage boys, however the review says:

Studies with male participants or special target groups such as immunocompromised patients were not included.”

Given the points made above, particularly that “the comparison of the risk of adverse events were compromised” and “the assumed risk for control groups must be interpreted cautiously”, please could the authors explain why a press release was issued that led to media headlines implying that the vaccine is completely safe and effective and also led to Professor Margaret Stanley saying on BBC Radio 4 that “there are no serious side-effects” – and thus leading parents into a false sense of security when considering the vaccination for their daughters, or indeed their sons?

Our view is that the questions on efficacy and safety have not been answered by this Systematic Review and that a lot of relevant information was omitted, leaving more questions than answers.

We would appreciate your response.

On behalf of

UK Families Affected by the HPV Vaccination 



Index of the human papillomavirus (HPV) vaccine industry clinical study programmes and non-industry funded studies: a necessary basis to address reporting bias in a systematic review, Jørgensen et al. Systematic Reviews(2018)

Diane M. Harper, Leslie R. DeMar, Gynecologic Oncology, Volume 147, Issue 2, November 2017, Pages 489