Estimating the effect of the 2005 change in BCG policy in England: A retrospective cohort study
Sam Abbott, Hannah Christensen, Nicky Welton, Ellen Brooks-Pollock
Source:vignettes/paper.Rmd
paper.RmdAuthors:
Sam Abbott, Bristol Medical School: Population Health Sciences, University of Bristol, Bristol, UK
Hannah Christensen, Bristol Medical School: Population Health Sciences, University of Bristol, Bristol, UK
Nicky Welton, Bristol Medical School: Population Health Sciences, University of Bristol, Bristol, UK
Ellen Brooks-Pollock, Bristol Medical School: Population Health Sciences, University of Bristol, Bristol, UK
Correspondence to: Sam Abbott, Bristol Medical School: Population Health Sciences, University of Bristol, Bristol BS8 2BN, UK; sam.abbott@bristol.ac.uk; 01173310185
WORD COUNT: Title 16 (20), Abstract 250 (250), Paper 3279 (3500)
PAGEBREAK
ABSTRACT
Background
In 2005 in England, universal Bacillus Calmette–Guérin (BCG) vaccination of school-age children was replaced by targeted BCG vaccination of high-risk neonates.
Aim
Estimate the impact of the 2005 change in BCG policy on tuberculosis incidence rates in England.
Methods
We conducted an observational study by combining notifications from the Enhanced Tuberculosis Surveillance system, with demographic data from the Labour Force Survey to construct retrospective cohorts relevant to both the universal, and targeted vaccination between Jan 1, 2000 and Dec 31, 2010. We then estimated incidence rates over a 5 year follow-up period and used regression modelling to estimate the impact of the change in policy on TB.
Results bottom of paper
Conclusions
Withdrawing universal vaccination at school-age and targeting vaccination towards high-risk neonates was associated with reduced incidence of TB. This was largely driven by reductions in the non-UK born with cases increasing in the UK born.
Keywords:
BCG, surveillance, vaccination policy, neonatal, school-age
Key Messages
There is little existing literature on the impact of withdrawing universal school-age BCG vaccination and introducing high-risk neonatal BCG vaccination on TB incidence rates in the populations directly affected by the vaccination programmes.
There was strong evidence that the change in policy was associated with a decrease in TB incidence rates in non-UK born neonates and school-age children. In the UK born individuals, there was some evidence that the change in policy was associated with an increase in TB incidence rates in those relevant to the universal school-age scheme, with little evidence of a decrease in incidence rates in those relevant to the high-risk neonatal vaccination scheme.
Overall the change in vaccination policy was associated with preventing TB cases, mainly in the non-UK born.
These results provide an important evaluation of the direct effects of both withdrawing and implementing a BCG vaccination programme in a low incidence, high income, country and are relevant to several other countries that have made similar changes to their vaccination programmes.
PAGEBREAK
INTRODUCTION
In 2005 England changed its Bacillus Calmette–Guérin (BCG) vaccination policy against tuberculosis (TB) from a universal programme aimed at 13 and 14 year olds to a targeted programme aimed at high-risk neonates. High risk babies are identified by local TB incidence and by the parents’ and grandparents’ country of origin. The change in policy was motivated by evidence of reduced TB transmission[1–3], and high effectiveness of the BCG vaccine in children[4–6], and variable effectiveness in adults[7]. Little work has been done to evaluate the impact of this change in vaccination policy.
Globally, several countries with low TB incidence have moved from universal vaccination, either of those at school-age or neonates, to targeted vaccination of neonates considered at high-risk of TB[7].In Sweden, which discontinued universal vaccination of neonates in favour of targeted vaccination of those at high risk, incidence rates in Swedish-born children increased slightly after the change in policy[8]. In France, which also switched from universal vaccination of children to targeted vaccination of those at high-risk, a study found that targeted vaccination may have reduced coverage in those most at risk[9].
The number of TB notifications in England increased from 6929 in 2004 to 8280 in 2011 but has since declined to 5137 in 2017[1]. A recent study found that this reduction may be linked to improved TB interventions[10]. Directly linking trends in TB incidence to transmission is complex because after an initial infection an individual may either develop active disease, or enter a latent stage which then may later develop into active disease. Incidence in children is a proxy of TB transmission, because any active TB disease in this population is attributable to recent transmission. Using this approach it is thought that TB transmission has been falling in England for the last 5 years, a notion supported by strain typing[1]. However, this does not take into account the change in BCG policy, which is likely to have reduced incidence rates in children.
Although the long term effects of BCG vaccination such as reducing the reactivation of latent cases and decreasing on-wards transmission are not readily detectable over short time scales the direct effects of vaccination on incidence rates can be estimated in vaccinated populations, when compared to comparable unvaccinated populations[11]. Here, We aimed to estimate the impact of the 2005 change in BCG policy on incidence rates, in both the UK and non-UK born populations, directly affected by it.
PAGEBREAK
METHODS
Data sources
Data on all notifications from the Enhanced Tuberculosis Surveillance (ETS) system from Jan 1, 2000 to Dec 31, 2015 were obtained from Public Health England (PHE). The ETS is maintained by PHE, and contains demographic, clinical, and microbiological data on all notified cases in England. A descriptive analysis of TB epidemiology in England is published each year, which fully details data collection and cleaning[1]. TB is highly heterogeneous in England with the majority of cases occurring in urban, non-UK born, populations. The yearly PHE report contains more descriptive detail[1].
We obtained yearly population estimates from the April to June Labour Force Survey (LFS) for 2000-2015. The LFS is a household study of the employment circumstances of the UK population, and provides the official measures of employment and unemployment in the UK. It is also used to study population demographics such as: ethnicity, country of birth, and age. Reporting practices have changed with time so the appropriate variables for age, country of origin, country of birth, and survey weight were extracted from each yearly extract, standardised, and combined into a single data-set.
Constructing Retrospective cohorts
We constructed retrospective cohorts of TB cases and individuals using the ETS and the LFS. TB cases were extracted from the ETS based on date of birth and date of TB notification.
Cohort 1: individuals aged 14 between 2000 and 2004, who were notified with TB whilst aged between 14 and 19 years old.
Comparison cohort 1: individuals aged 14 between 2005 and 2010, who were notified with TB whilst aged between 14 and 19 years old.
Cohort 2: individuals born between 2005 and 2010, who were notified with TB whilst aged 0-5 years old
Comparison cohort 2: individuals born between 2000 and 2004, who were notified with TB whilst aged 0-5 years old
Cohorts were stratified by vaccination programme using age criteria and then stratified further by whether the scheme was in place during the time period they entered the study. Each cohort was further stratified by UK birth status, with both non-UK born and UK born cases assumed to have been exposed to England’s vaccination policy. Corresponding population cohorts were calculated using the LFS population estimates, resulting in 8 population level cohorts, each with 5 years of follow up (Table 1).
| Cohort | Vaccination programme | Eligible for the programme* | Birth status | Age at study entry | Year of study entry |
|---|---|---|---|---|---|
| Cohort 1 | Universal | Yes | UK born | 14 | 2000-2004 |
| Comparison cohort 1 | Universal | No | UK born | 14 | 2005-2010 |
| Cohort 1 | Universal | Yes | Non-UK born | 14 | 2000-2004 |
| Comparison cohort 1 | Universal | No | Non-UK born | 14 | 2005-2010 |
| Comparison cohort 2 | Targeted | No | UK born | Birth | 2000-2004 |
| Cohort 2 | Targeted | Yes | UK born | Birth | 2005-2010 |
| Comparison cohort 2 | Targeted | No | Non-UK born | Birth | 2000-2004 |
| Cohort 2 | Targeted | Yes | Non-UK born | Birth | 2005-2010 |
| * Eligible signifies that the cohort fit the criteria for the programme and entered the study during the time period it was in operation not that the cohort was vaccinated by the programme. |
Statistical methods overview
We estimated incidence rates (with 95% confidence intervals) by year, age and place of birth as (number of cases) divided by (number of individuals of corresponding age). UK birth status was incomplete, with some evidence of a missing not at random mechanism. We imputed the missing data using a gradient boosting method (GBM; see supplementary information). We then used descriptive analysis to describe the observed trends in age-specific incidence rates over the study period, comparing incidence rates in the study populations relevant to both vaccination programmes before and after the change in BCG policy.
We calculated Incidence Rate Ratios (IRRs) with 95% credible intervals (CrIs) for the change in incidence rates associated with the change in BCG vaccination policy (modelled as a binary breakpoint at the start of 2005) for both the UK born and non-UK born populations that were relevant to the universal programme, and for the targeted programme using a range of models. We considered the following covariates: age[1,7], incidence rates in both the UK born and non-UK born who were not in the age group of interest[1], and year of study entry (as a random intercept). We evaluated a range of models using a statistically rigorous criterion, that accounted for model fit and complexity, for model selection.
Statistical modelling details
We first investigated a univariable Poisson model, followed by combinations of covariates (supplementary table S1). We also investigated a Negative Binomial model adjusting for the same covariates as in the best fitting Poisson model. The models were estimated with a Bayesian approach using Markov Chain Monte Carlo (MCMC), with default weakly informative priors (see supplementary information). Model fit, penalised by model complexity, was assessed using the leave one out cross validation information criterion (LOOIC) and its standard error[12]. Models were ranked by goodness of fit, using their LOOIC, with a smaller LOOIC indicating a better fit to the data after adjusting for the complexity of the model. No formal threshold for a change in the LOOIC was used, with changes in the LOOIC being evaluated in the context of their standard error. The inclusion of the change in policy in the best fitting model was tested by refitting the model excluding the change in policy and estimating the improvment in the LOOIC. Once the best fitting model had been identified we estimated the number of cases prevented, from 2005 until 2015, for each vaccination programme in the study population relevant to that programme (see supplementary information).
Implementation
R 3.5.2 was used for all analysis[13]. Missing data imputation using a GBM was implemented using the h2o package[14]. Incidence rates, with 95% confidence intervals, were calculated using the epiR package[15]. The brms package[16], and STAN[17], was used to perform MCMC. Models were run until convergence (4 chains with a burn in of 10,000, and 10,000 sampled iterations each), with convergence being assessed using trace plots and the R hat diagnostic[17]. All numeric confounders were centered and scaled by their standard deviation, and age was adjusted for using single year of age categories.
Ethical statement
Ethical approval was not required for this study which used anonymized secondary data sources only.
PAGEBREAK
RESULTS
Descriptive analysis
During the study period there were 114,820 notifications of TB in England, of which 93% (106765/114820) had their birth status recorded. Of notifications with a known birth status 27% (29096/106765) were UK born, while among notification with an imputed birth status, 33% (2634/8055) were UK born. Trends in incidence rates varied by age group and UK birth status (see supplementary information). During the study period, there were 1729 UK born cases and 2797 non-UK born cases in individuals relevant to the universal schools scheme, and 1431 UK born cases and 238 non-UK born cases relevant to the targeted neonatal scheme, who fit our age criteria. Univariable evidence for differences between mean incidence rates before and after the change in BCG policy in the UK born was weak. In the non-UK born incidence rates were lower after the change in BCG policy in both the cohort relevant to the universal school-age scheme and the cohort relevant to the targeted neonatal scheme (Figure 1).
Figure 1: Mean incidence rates per 100,000, with 95% confidence intervals for each retrospective cohort (see table 1 for cohort definitions), stratified by the vaccination policy and UK birth status for England, using surveillance data from 2000 to 2015. The top and bottom panels are on different scales in order to highlight trends in incidence rates over time.
Adjusted estimates of the effects of the change in policy on school-age children
In the UK born cohort relevant to universal vaccination there was some evidence, across all models that adjusted for age, that ending the scheme was associated with a modest increase in TB rates (Supplementary Table S2). Using the LOOIC goodness of fit criteria the best fitting model was found to be a Negative Binomial model that adjusted for the change in policy, age, and incidence rates in the UK born (Table 2). In this model there was some evidence of an assocation between the change in policy and an increase in incidence rates in those at school-age who were UK born, with an IRR of 1.08 (95%CrI 0.97, 1.19). Dropping the change in policy from the model resulted in a small decrease in the LOOIC (0.52 (SE 2.63)) but the change was too small, with too large a standard error, to conclusively state that the excluding the change in policy from the model improved the quality of model fit. We found that it was important to adjust for UK born incidence rates, otherwise the impact from the change in BCG vaccination policy was over-estimated.
For the comparable non-UK born cohort who were relevant to the universal vaccination there was evidence, in the best fitting model, that ending the scheme was associated with a decrease in incidence rates (IRR: 0.74 (95%CrI 0.61, 0.88)). The best fitting model was a Negative Binomial model which adjusted for the change in policy, age, incidence rates in the non-UK born, and year of eligibility as a random effect (Table 2). We found omitting change in policy from the model resulted in poorer model fit (LOOIC increase of 3.02 (SE 3.52)), suggesting that the policy change was an important factor explaining changes in incidence rates, after adjusting for other covariates. All models that adjusted for incidence rates in the UK born or non-UK born estimated similar IRRs (Supplementary Table S3).
| Variable | IRR (95% CrI) UK born* | IRR (95% CrI) Non-UK born* |
|---|---|---|
| Policy change† | ||
| Pre-change | Reference | Reference |
| Post-change | 1.08 (0.97, 1.19) | 0.74 (0.61, 0.88) |
| Age | ||
| 14 | Reference | Reference |
| 15 | 1.18 (0.98, 1.42) | 1.03 (0.87, 1.22) |
| 16 | 1.24 (1.03, 1.50) | 1.25 (1.07, 1.47) |
| 17 | 1.59 (1.33, 1.91) | 1.40 (1.19, 1.63) |
| 18 | 1.92 (1.60, 2.30) | 1.47 (1.26, 1.73) |
| 19 | 1.80 (1.49, 2.17) | 1.47 (1.24, 1.73) |
| UK born incidence rate (per standard deviation) | 1.08 (1.03, 1.14) | - |
| Non-UK born incidence rate (per standard deviation) | - | 1.11 (1.03, 1.19) |
| Year of study eligibility, group level | - | |
| Intercept (standard deviation) | - | 1.13 (1.05, 1.26) |
| Year of study eligibility, individual level | - | |
| 2000 | - | 1.10 (0.96, 1.29) |
| 2001 | - | 1.06 (0.93, 1.24) |
| 2002 | - | 1.07 (0.94, 1.25) |
| 2003 | - | 0.90 (0.76, 1.03) |
| 2004 | - | 0.89 (0.75, 1.02) |
| 2005 | - | 0.98 (0.85, 1.12) |
| 2006 | - | 1.13 (0.99, 1.33) |
| 2007 | - | 1.04 (0.91, 1.20) |
| 2008 | - | 0.96 (0.83, 1.09) |
| 2009 | - | 0.95 (0.81, 1.08) |
| 2010 | - | 0.96 (0.82, 1.11) |
| * Incidence Rate Ratio (95% credible interval) † There was an improvement in the LOOIC score of 0.52 (SE 2.63) from dropping the change in policy from the model in the UK born cohort and a -3.02 (SE 3.52) improvement in the non-UK born cohort.LOOIC - Leave one out cross validation information criterion; SE - Standard error |
Adjusted estimates of the effect of the change in policy in those relevant to the targeted neonatal programme
For the UK born cohort relevant to the targeted neonatal vaccination programme the evidence of an association, across all models, was mixed and credible intervals were wide compared to models for the UK born cohort relevant to the universal school-age vaccination programme (Supplementary Table S4). The best fitting model was a Poisson model which adjusted for the change in policy, age, UK born incidence rates, and year of study entry with a random effect (Table 3). In this model, there was weak evidence of an association between the change in BCG policy and an decrease in incidence rates in UK born neonates, with an IRR of 0.96 (95%CrI 0.82, 1.14). There was weak evidence to suggest that dropping the change in policy from this model improved the quality of the fit, with an improvement in the LOOIC score of 0.92 (SE 1.07). This suggests that the change in policy was not an important factor for explaining incidence rates, after adjusting for covariates. Models which also adjusted for non-UK born incidence rates estimated that the change in policy was associated with no change in incidence rates in the relevant cohort of neonates.
For the comparable non-UK born cohort who were relevant to the targeted neonatal vaccination programme there was evidence, across all models, that change in policy was associated with a large decrease in incidence rates (IRR: 0.62 (95%CrI 0.44, 0.88)) (Table 3 in the best fitting model). The best fitting model was a Negative Binomial model that adjusted for the change in policy, age, and non-UK born incidence rates (Table 3). All models which at least adjusted for age estimated comparable effects of the change in policy (Supplementary Table S5).
| Variable | IRR (95% CrI) UK born* | IRR (95% CrI) Non-UK born* |
|---|---|---|
| Policy change† | ||
| Pre-change | Reference | Reference |
| Post-change | 0.96 (0.82, 1.14) | 0.62 (0.44, 0.88) |
| Age | ||
| 0 | Reference | Reference |
| 1 | 1.39 (1.20, 1.61) | 0.49 (0.30, 0.83) |
| 2 | 1.24 (1.06, 1.44) | 0.49 (0.30, 0.80) |
| 3 | 1.21 (1.03, 1.41) | 0.42 (0.26, 0.68) |
| 4 | 0.90 (0.76, 1.06) | 0.41 (0.25, 0.66) |
| 5 | 0.89 (0.75, 1.06) | 0.27 (0.16, 0.45) |
| UK born incidence rate (per standard deviation) | 1.12 (1.06, 1.18) | - |
| Non-UK born incidence rate (per standard deviation) | - | 1.25 (1.04, 1.51) |
| Year of study eligibility, group level | - | |
| Intercept (standard deviation) | 1.13 (1.04, 1.26) | - |
| Year of study eligibility, individual level | - | |
| 2000 | 0.83 (0.68, 0.99) | - |
| 2001 | 0.93 (0.79, 1.07) | - |
| 2002 | 1.08 (0.95, 1.28) | - |
| 2003 | 1.07 (0.93, 1.26) | - |
| 2004 | 1.12 (0.97, 1.32) | - |
| 2005 | 1.02 (0.89, 1.17) | - |
| 2006 | 1.02 (0.89, 1.17) | - |
| 2007 | 0.97 (0.83, 1.11) | - |
| 2008 | 1.01 (0.88, 1.15) | - |
| 2009 | 1.01 (0.88, 1.16) | - |
| 2010 | 0.98 (0.85, 1.13) | - |
| * Incidence Rate Ratio (95% credible interval) † There was an improvement in the LOOIC score of 0.92 (SE 1.07) from dropping the change in policy from the model in the UK born cohort and a -3.45 (SE 4.63) improvement in the non-UK born cohort.LOOIC - Leave one out cross validation information criterion; SE - Standard error |
Magnitude of the estimated impact of the change in BCG policy
We estimate that the change in vaccination policy was associated with preventing 385 (95%CrI -105, 881) cases from 2005 until the end of the study period in the directly impacted populations after 5 years of follow up (Table 4). The majority of the cases prevented were in the non-UK born, with cases increasing slightly overall in the UK born. This was due to cases increasing in the UK born at school-age, and decreasing in UK born neonates, although both these estimates had large credible intervals.
| Vaccination Programme | Birth Status | Cases Prevented (95% CrI*) | Notified Cases |
|---|---|---|---|
| Universal school-age (vaccination at 14) | All | -291 (24, -571) | 2364 |
| UK born | 76 (188, -26) | 969 | |
| Non-UK born | -367 (-165, -546) | 1395 | |
| Targeted high-risk neonates (0) | All | 94 (-81, 310) | 906 |
| UK born | 30 (-95, 173) | 800 | |
| Non-UK born | 65 (14, 137) | 106 | |
| Change in Policy† | All | 385 (-105, 881) | 3270 |
| UK born | -46 (-284, 199) | 1769 | |
| Non-UK born | 431 (179, 682) | 1501 | |
| *95% CrI: 95% credible interval, † Estimated total number of cases prevented due to the change in vaccination policy in 2005 |
PAGEBREAK
DISCUSSION
In the non-UK born we found evidence of an association between the change in BCG policy and a decrease in TB incidence rates in both those at school-age and neonates, after 5 years of follow up. We found some evidence that the change in BCG policy was associated with a modest increase in incidence rates in the UK born population who were relevant to the universal school-age scheme and weaker evidence of a small decrease in incidence rates in the UK born population relevant to the targeted neonatal scheme. Overall, we found that the change in policy was associated with preventing 385 (95%CrI -105, 881) cases in the study population, from 2005 until the end of the study period, with the majority of the cases prevented in the non-UK born.
We were unable to estimate the impact of the change in BCG policy after 5 years post vaccination, so both our estimates of the positive and negative consequences are likely to be underestimates of the ongoing impact. Tuberculosis is a complex disease and the BCG vaccine is known to offer imperfect protection, which has been shown to vary both spatially and with time since vaccination[18,19]. By focusing on the impact of the change in policy on the directly affected populations within a short period of time, and by employing a multi-model approach we have limited the potential impact of these issues. Our study was based on a routine observational data set (ETS), and a repeated survey (LFS) both of which may have introduced bias. Whilst the LFS is a robust data source, widely used in academic studies[20–22], it is susceptible to sampling errors particularly in the young, and in the old, which may have biased the estimated incidence rates. As the ETS is routine surveillance system some level of missing data is inevitable. However, UK birth status is relatively complete (93% (106765/114820)) and we imputed missing values using an approach which accounted for missing not at random mechanisms captured by variables included in the imputation model. We were unable to adjust for known demographic risk factors for TB, notably socio-economic status[23,24], and ethnicity[23–25]. However, this confounding is likely to be mitigated by our use of multiple cohorts and our adjustment for incidence rates in the UK born and non-UK born. Finally, we have assumed that the effect we have estimated for the change in BCG policy is due to the changes in BCG vaccination policy as well as other associated changes in TB control policy, after adjusting for hypothesised confounders. However, there may have been additional policy changes which we have not accounted for.
Whilst little work has been done to assess the impact of the 2005 change in BCG vaccination several other studies have estimated the impact of changing BCG vaccination policy, although typically only from universal vaccination of neonates to targeted vaccination of high-risk neonates. A previous study in Sweden found that incidence rates in Swedish-born children increased after high-risk neonatal vaccination was implemented in place of a universal neonatal program, this corresponds with our finding that introducing neonatal vaccination had little impact on incidence rates in UK born neonates. Theoretical approaches have indicated that targeted vaccination of those at high-risk may be optimal in low incidence settings[26]. Our study extends this work by also considering the age of those given BCG vaccination, although we were unable to estimate the impact of a universal neonatal scheme as this has never been implemented nationally in England. It has previously been shown that targeted vaccination programmes may not reach those considered most at risk[27], our findings may support this view as we observed only a small decrease in incidence rates in UK born neonates after the introduction of the targeted neonatal vaccination programme. Alternatively, the effectiveness of the BCG in neonates, in England, may be lower than previously thought as we only observed a small decrease in incidence rates, whilst a previous study estimated BCG coverage at 68% (95%CI 65%, 71%) amongst those eligible for the targeted neonatal vaccination programme[28].
This study indicates that the change in England’s BCG vaccination policy was associated with a modest increase in incidence in the UK born that were relevant to the school-age vaccination programme, and with a small reduction in incidence in the UK born that were relevant to the high-risk neonatal vaccination programme, although both these estimates had wide credible intervals. We found stronger evidence of an association between the change in policy and a decrease in incidence rates in the non-UK born populations relevant to both programmes. This suggests that the change of vaccination policy to target high-risk neonates may have resulted in an increased focus on high-risk non-UK born individuals who may not have been the direct targets of the vaccination programme. Further validation is required using alternative study designs, but this result should be considered when vaccination policy changes are being considered. Our results should be interpreted carefully, especially in the non-UK born, as we could not fully rule out the impact of other TB control measures that may have been changed at the same time as vaccination policy. The severity of TB disease is known to differ across age groups with children having a higher incidence of TB meningitis, which can be severe, compared to other age groups[1]. This variation should also be considered when evaluating these results.
It is well established that interventions against infectious diseases, such as TB, should be evaluated not only for their direct effects but also for future indirect effects via ongoing transmission. Statistical approaches such as those used in this paper are not appropriate for capturing these future indirect effects, and instead dynamic disease models should be used. In addition, this study could not evaluate the impact of the neonatal programme on the high-risk population it targets, due to a lack of reliable data. Improved coverage data for the BCG programme is required to more fully evaluate its ongoing impact. As only 5 years of follow up data were available - and BCG vaccination has been shown to provide long lasting protection in the UK - repeating this study once more data is available may alter the findings[29]. For this reason this study has been implemented with reproducibility in mind - please see the code for details. Finally, the results from this study could be combined with estimates of the impact of TB disease, stratified by age, to give an estimate of the overall impact of the change in policy that accounts for the severity of disease.
Acknowledgements
The authors thank the TB section at Public Health England (PHE) for maintaining the Enhanced Tuberculosis Surveillance (ETS) system; all the healthcare workers involved in data collection for the ETS.
Contributors
SA, HC, and EBP conceived and designed the work. NJW provided guidance on the statistical methods used. SA undertook the analysis with advice from all other authors. All authors contributed to the interpretation of the data. SA wrote the first draft of the paper and all authors contributed to subsequent drafts. All authors approve the work for publication and agree to be accountable for the work.
Funding
SEA, HC, EBP and NJW are funded by the National Institute for Health Research Health Protection Research Unit (NIHR HPRU) in Evaluation of Interventions at University of Bristol in partnership with Public Health England (PHE). The views expressed are those of the author(s) and not necessarily those of the NHS, the NIHR, the Department of Health or Public Health England.
Conflicts of interest
HC reports receiving honoraria from Sanofi Pasteur, and consultancy fees from AstraZeneca, GSK and IMS Health, all paid to her employer.
Accessibility of data and programming code The code used to clean the data used in this paper can be found at: DOI:10.5281/zenodo.2551555 The code for this analysis, interim results, and final results can be found at: DOI:10.5281/zenodo.2583056
Results copy to top of paper
In the non-UK born, we found evidence for an association between a reduction in incidence rates and the change in BCG policy (school-age incidence rate ratio (IRR): 0.74 (95% credible interval (CrI) 0.61,0.88), neonatal IRR: 0.62 (95%CrI 0.44,0.88)). We found some evidence that the change in policy was associated with an increase in incidence rates in the UK born school-age population (IRR: 1.08 (95%CrI 0.97,1.19)) and weaker evidence of an association with a reduction in incidence rates in UK born neonates (IRR: 0.96 (95%CrI 0.82,1.14)). Overall, we found that the change in policy was associated with directly preventing 385 (95%CrI -105,881) cases.
PAGEBREAK
REFERENCES
1 PHE. Tuberculosis in England 2016 Report (presenting data to end of 2015). Public Heal Engl 2016;Version 1.:173.
2 Fine P. Stopping routine vaccination for tuberculosis in schools. BMJ 2005;331:647–8. doi:10.1136/bmj.331.7518.647
3 Teo SSS. Does BCG have a role in tuberculosis control and prevention in the United Kingdom? Arch Dis Child 2006;91:529–31. doi:10.1136/adc.2005.085043
4 Rodrigues LC, Diwan VK, Wheeler JG. Protective effect of BCG against tuberculous meningitis and miliary tuberculosis: a meta-analysis. Int J Epidemiol 1993;22:1154–8.http://www.ncbi.nlm.nih.gov/pubmed/8144299
5 Colditz GA, Brewer TF, Berkey CS et al. Efficacy of BCG Vaccine in the Prevention of Tuberculosis. JAMA 1994;271:698. doi:10.1001/jama.1994.03510330076038
6 Roy a, Eisenhut M, Harris RJ et al. Effect of BCG vaccination against Mycobacterium tuberculosis infection in children: systematic review and meta-analysis. Bmj 2014;349:g4643–3. doi:10.1136/bmj.g4643
7 Zwerling A, Behr MA, Verma A et al. The BCG World Atlas: a database of global BCG vaccination policies and practices. PLoS Med 2011;8:e1001012. doi:10.1371/journal.pmed.1001012
8 Romanus V, Svensson Å, Hallander HO. The impact of changing BCG coverage on tuberculosis incidence in Swedish-born children between 1969 and 1989. Tuber Lung Dis 1992;73:150–61. doi:10.1016/0962-8479(92)90149-E
9 Guthmann JP, Antoine D, Fonteneau L et al. Assessing BCG vaccination coverage and incidence of paediatric tuberculosis following two major changes in BCG vaccination policy in France. Eurosurveillance 2011;16:1–6.
10 Thomas HL, Harris RJ, Muzyamba MC et al. Reduction in tuberculosis incidence in the UK from 2011 to 2015: a population-based study. Thorax Published Online First: 2018. doi:10.1136/THORAXJNL-2017-211074
11 Parikh SR, Andrews NJ, Beebeejaun K et al. Effectiveness and impact of a reduced infant schedule of 4CMenB vaccine against group B meningococcal disease in England: a national observational cohort study. Lancet 2016;388:2775–82. doi:10.1016/S0140-6736(16)31921-3
12 Vehtari A, Gelman A, Gabry J. Practical Bayesian model evaluation using leave-one-out cross-validation and WAIC. Stat Comput 2016;1–20. doi:10.1007/s11222-016-9696-4
13 R Core Team. R: A Language and Environment for Statistical Computing. 2016.https://www.r-project.org/
14 H2O.ai. R Interface for H2O. 2018. https://github.com/h2oai/h2o-3
15 Stevenson M, Nunes T, Heuer C et al. epiR: Tools for the Analysis of Epidemiological Data. 2017.https://cran.r-project.org/package=epiR
16 Bürkner P-C. {brms}: An {R} Package for Bayesian Multilevel Models using Stan. J Stat Softw
17 Stan Development Team. {RStan}: the {R} interface to {Stan}. 2016.http://mc-stan.org/
18 Mangtani P, Abubakar I, Ariti C et al. Protection by BCG Vaccine Against Tuberculosis: A Systematic Review of Randomized Controlled Trials. Clin Infect Dis 2014;58:470–80. doi:10.1093/cid/cit790
19 Abubakar I, Pimpin L, Ariti C et al. Systematic review and meta-analysis of the current evidence on the duration of protection by bacillus Calmette-Guérin vaccination against tuberculosis. Health Technol Assess (Rockv) 2013;17:1–372, v–vi. doi:10.3310/hta17370
20 French CE, Antoine D, Gelb D et al. Tuberculosis in non-UK-born persons, England and Wales, 2001-2003. Int J Tuberc Lung Dis 2007;11:577–84.
21 Davies R, Jones M, Lloyd-Williams H. Age and Work-Related Health: Insights from the UK Labour Force Survey. Br J Ind Relations 2016;54:136–59. doi:10.1111/bjir.12059
22 Lindley J. The over-education of UK immigrants and minority ethnic groups: Evidence from the Labour Force Survey. Econ Educ Rev 2009;28:80–9. doi:10.1016/j.econedurev.2007.11.003
23 Parslow R, El-Shimy NA, Cundall DB et al. Tuberculosis, deprivation, and ethnicity in Leeds, UK, 1982-1997. Arch Dis Child 2001;84:109–13. doi:10.1136/adc.84.2.109
24 Bhatti N, Law MR, Morris JK et al. Increasing incidence of tuberculosis in England and Wales: a study of the likely causes. BMJ 1995;310:967–9. doi:10.1136/bmj.310.6985.967
25 Abubakar I, Laundy MT, French CE et al. Epidemiology and treatment outcome of childhood tuberculosis in England and Wales: 1999-2006. Arch Dis Child 2008;93:1017–21. doi:10.1136/adc.2008.139543
26 Manissero D, Lopalco PL, Levy-Bruhl D et al. Assessing the impact of different BCG vaccination strategies on severe childhood TB in low-intermediate prevalence settings. Vaccine 2008;26:2253–9. doi:10.1016/j.vaccine.2008.02.038
27 Feiring B, Laake I, Molden T et al. Do selective immunisation against tuberculosis and hepatitis B reach the targeted populations? A nationwide register-based study evaluating the recommendations in the Norwegian Childhood Immunisation Programme. Vaccine 2016;34:2015–20. doi:10.1016/j.vaccine.2016.02.060
28 Nguipdop-Djomo P, Mangtani P, Pedrazzoli D et al. Uptake of neonatal BCG vaccination in England: Performance of the current policy recommendations. Thorax 2014;69:87–9. doi:10.1136/thoraxjnl-2013-203931
29 Nguipdop-Djomo P, Heldal E, Rodrigues LC et al. Duration of BCG protection against tuberculosis and change in effectiveness with time since vaccination in Norway: a retrospective population-based cohort study. Lancet Infect Dis 2016;16:219–26. doi:10.1016/S1473-3099(15)00400-4
Online supplementary appendix: Estimating the effect of the 2005 change in BCG policy in England: A retrospective cohort study
Sam Abbott, Hannah Christensen, Nicky Welton, Ellen Brooks-Pollock
Model Definitions
| Model | Description |
|---|---|
| Model 1 | Poisson model adjusting for no fixed effects. |
| Model 2 | Poisson model adjusting with fixed effects for the change in policy. |
| Model 3 | Poisson model adjusting with fixed effects for the change in policy and incidence rates in the UK born. |
| Model 4 | Poisson model adjusting with fixed effects for the change in policy and incidence rates in the non-UK born. |
| Model 5 | Poisson model adjusting with fixed effects for the change in policy and incidence rates in the UK born and non-UK born populations. |
| Model 6 | Poisson model adjusting with fixed effects for the change in policy and age. |
| Model 7 | Poisson model adjusting with fixed effects for the change in policy, age, and incidence rates in the UK born. |
| Model 7 (Negative Binomial) | Negative binomial model adjusting with fixed effects for the change in policy, age, and incidence rates in the UK born. |
| Model 8 | Poisson model adjusting with fixed effects for the change in policy, age, and incidence rates in the non-UK born. |
| Model 8 (Negative Binomial) | Negative binomial model adjusting with fixed effects for the change in policy, age, and incidence rates in the non-UK born. |
| Model 9 | Poisson model adjusting with fixed effects for the change in policy, age, and incidence rates in the UK born and non-UK born populations. |
| Model 10 | Poisson model with a random intercept for year of study entry, adjusting for no fixed effects. |
| Model 11 | Poisson model with a random intercept for year of study entry, adjusting with fixed effects for the change in policy. |
| Model 12 | Poisson model with a random intercept for year of study entry, adjusting with fixed effects for the change in policy and incidence rates in the UK born. |
| Model 13 | Poisson model with a random intercept for year of study entry, adjusting with fixed effects for the change in policy and incidence rates in the non-UK born. |
| Model 14 | Poisson model with a random intercept for year of study entry, adjusting with fixed effects for the change in policy and incidence rates in the UK born and non-UK born populations. |
| Model 15 | Poisson model with a random intercept for year of study entry, adjusting with fixed effects for the change in policy and age. |
| Model 16 | Poisson model with a random intercept for year of study entry, adjusting with fixed effects for the change in policy, age, and incidence rates in the UK born. |
| Model 16 (Negative Binomial) | Negative binomial model with a random intercept for year of study entry, adjusting with fixed effects for the change in policy, age, and incidence rates in the UK born. |
| Model 17 | Poisson model with a random intercept for year of study entry, adjusting with fixed effects for the change in policy, age, and incidence rates in the non-UK born. |
| Model 17 (Negative Binomial) | Negative binomial model with a random intercept for year of study entry, adjusting with fixed effects for the change in policy, age, and incidence rates in the non-UK born. |
| Model 18 | Poisson model with a random intercept for year of study entry, adjusting with fixed effects for the change in policy, age, and incidence rates in the UK born and non-UK born populations. |
PAGEBREAK
Imputation of UK birth status
As we were imputing a single variable we reformulated the imputation as a categorical prediction problem. This allowed us to use techniques from machine learning to improve the quality of our imputation, whilst also validating it using metrics supported by theory. We included year of notification, sex, age, Public Health England Centre (PHEC), occupation, ethnic group, Index of Multiple Deprivation (2010) categorised into five groups for England (IMD rank), and risk factor count (risk factors considered; drug use, homelessness, alcohol misuse/abuse and prison). However, we could not account for a possible missing not at random mechanism not captured by these covariates. To train the model we first split the data with complete UK birth status into a training set (80%), a calibration set (5%) and a test set (15%). We then fit a gradient boosted machine with the 10000 trees, early stopping (at a precision of \(1e-5\), with 10 stopping rounds), a learning rate of 0.1, and a learn rate annealing of 0.99. Gradient boosted machines are a tree based method that can incorporate complex non-linear relationships and interactions. Much like a random forest model they work by ensembling a group of trees, but unlike a random forest model each tree is additive aiming to reduce the residual loss from previous trees. Once the model had been fit to the training set we performed platt scaling using the calibration data set. Our fitted imputation model had a Logloss of 0.28 on the test set, with an AUC of 0.93, both of which indicate a robust out of bag performance. We found that ethnic group was the most important variable for predicting UK birth status, followed by age and PHEC.
Using the fitted model we predicted the birth status for notifications where this was missing, using the F1 optimal threshold as our probability cut-off. It is common to impute missing values multiple times, to account for within- and between imputation variability. However, we considered this unnecessary for our analysis as the amount of missing data was small, our analysis considered only aggregate counts, our model metrics indicated a robust level of performance out of bag and any unaccounted for uncertainty would be outweighed by the uncertainty in our population denominator[10]. We found that cases with imputed birth status had a similar proportion of UK born to non-UK born cases as in the complete data (Supplementary Table S6).
| Status | Birth Status | Proportion of Cases (%) | Cases |
|---|---|---|---|
| Complete | 106765 | ||
| UK Born | 27.3 | 29096 | |
| Non-UK Born | 72.7 | 77669 | |
| Imputed | 8055 | ||
| UK Born | 32.7 | 2634 | |
| Non-UK Born | 67.3 | 5421 |
Inclusion of imputed values for UK birth status should reduce bias caused by any missing not at random mechanism captured by predictors included in the model. Graphical evaluation of UK birth status indicated that missingness has reduced over time, indicating a missing not at random mechanism. If only the complete case data then incidence rates would have reduced over the study period due to this mechanism, this may have biased our estimate of the impact of the change in policy.
PAGEBREAK
Prior choice
Default weakly informative priors were used based on those provided by the brms package. For the population-level effects this was an improper flat prior over the reals. For both the standard deviations of group level effects and the group level intercepts this was a half student-t prior with 3 degrees of freedom and a scale parameter that depended on the standard deviation of the response after applying the link function.
Estimating the magnitude of the estimated impact of the change in BCG policy
We estimated the magnitude of the estimated impact from the change in BCG policy by applying the IRR estimates from the best fitting model for each cohort to the observed number of notifications from 2005 until 2015 in our study population. For the cohorts relevant to the universal school-age vaccination scheme we estimated the number of prevented cases by first aggregating cases (\(C_O\)) and then using the following equation,
\[ C^i_P = C_O * (1 - I^i),\ \text{Where}\ i = e,\ l,\ u.\]
Where \(C^i_P\) is the predicted number of cases prevented using the mean (\(e\)), 2.5% bound (\(l\)) and 97.5% bound (\(u\)) of the IRR estimate \(I^i\). For the cohorts relevant to the targeted high-risk neonatal scheme we used a related equation, adjusting for the fact that the populations were exposed to the scheme and we therefore had to first estimate the number of cases that would have been observed had the scheme not been implemented. After simplification this results in the following equation,
\[ C^i_P = C_0\left(\frac{1}{I^i } - 1\right),\ \text{Where}\ i = e,\ l,\ u.\]
PAGEBREAK
Descriptive analysis of age-specific incidence rates
From 2000 until 2012 incidence rates in the UK born remained relatively stable but have since fallen year on year. In comparison incidence rates in the non-UK born increased from 2000 until 2005, since when they have also decreased year on year. In 14-19 year old’s, who were UK born, incidence rates remained relatively stable throughout the study period, except for the period between 2006 to 2009 in which they increased year on year. This trend was not observed in the non-UK born population aged 14-19, where incidence rates reached a peak in 2003, since when they have consistently declined. In those aged 0-5, who were UK born, incidence rates also increased year on year after the change in BCG policy, until 2008 since when they have declined. This does not match with the observed trend in incidence rates in the non-UK born population, aged 0-5, in which incidence rates declined steeply between 2005 and 2006, since when they have remained relatively stable (Supplementary Figure S1; Supplementary Table S7; Supplementary Table S8).
Supplementary Figure S1: Incidence rates per 100,000 for UK born population and non-UK born population, aged 0-5 and therefore directly affected by the targeted neonatal vaccination programme, and aged 14-19 and therefore directly affected by the universal school-age scheme.
PAGEBREAK
Incidence estimates for all cases, those aged 0-5 and those aged 14-19
| Year eligible for vaccination | Age group-All cases* | Age group-0-5* | Age group-14-19* |
|---|---|---|---|
| 2000 | 4.86 (4.66, 5.07) | 3.12 (2.58, 3.73) | 4.43 (3.77, 5.16) |
| 2001 | 4.81 (4.61, 5.02) | 4.06 (3.44, 4.76) | 4.65 (3.98, 5.40) |
| 2002 | 4.58 (4.39, 4.78) | 3.45 (2.88, 4.10) | 4.71 (4.05, 5.46) |
| 2003 | 4.23 (4.04, 4.42) | 2.76 (2.26, 3.34) | 3.67 (3.09, 4.33) |
| 2004 | 4.36 (4.17, 4.56) | 4.19 (3.56, 4.91) | 4.00 (3.40, 4.68) |
| 2005 | 4.40 (4.21, 4.60) | 3.72 (3.13, 4.40) | 4.61 (3.96, 5.34) |
| 2006 | 4.36 (4.17, 4.56) | 3.29 (2.74, 3.93) | 4.17 (3.56, 4.86) |
| 2007 | 4.42 (4.23, 4.62) | 4.57 (3.91, 5.30) | 4.07 (3.46, 4.75) |
| 2008 | 4.56 (4.37, 4.76) | 4.56 (3.91, 5.28) | 5.31 (4.61, 6.09) |
| 2009 | 4.63 (4.43, 4.83) | 4.05 (3.45, 4.73) | 5.25 (4.55, 6.02) |
| 2010 | 4.28 (4.09, 4.47) | 3.33 (2.80, 3.94) | 4.00 (3.40, 4.68) |
| 2011 | 4.51 (4.32, 4.71) | 3.20 (2.69, 3.79) | 4.32 (3.69, 5.03) |
| 2012 | 4.59 (4.39, 4.78) | 4.10 (3.51, 4.76) | 4.67 (4.00, 5.41) |
| 2013 | 4.15 (3.97, 4.34) | 2.60 (2.15, 3.13) | 4.36 (3.72, 5.09) |
| 2014 | 3.92 (3.74, 4.10) | 2.27 (1.85, 2.76) | 4.41 (3.76, 5.14) |
| 2015 | 3.46 (3.30, 3.63) | 2.16 (1.75, 2.64) | 4.07 (3.44, 4.78) |
| * Incidence rate per 100,000, with 95% confidence intervals |
| Year eligible for vaccination | Age group-All cases* | Age group-0-5* | Age group-14-19* |
|---|---|---|---|
| 2000 | 92.98 (90.10, 95.92) | 40.45 (28.56, 55.88) | 103.14 (89.60, 118.19) |
| 2001 | 92.95 (90.12, 95.84) | 26.36 (16.95, 39.47) | 122.40 (108.32, 137.85) |
| 2002 | 102.07 (99.18, 105.03) | 47.63 (34.62, 64.16) | 127.03 (112.59, 142.83) |
| 2003 | 99.65 (96.85, 102.50) | 24.81 (16.59, 35.94) | 127.53 (113.57, 142.75) |
| 2004 | 103.66 (100.82, 106.56) | 34.58 (24.13, 48.25) | 119.66 (106.18, 134.41) |
| 2005 | 110.48 (107.64, 113.37) | 42.83 (30.99, 57.91) | 127.26 (113.69, 142.04) |
| 2006 | 102.91 (100.28, 105.59) | 17.32 (11.13, 25.93) | 116.54 (103.91, 130.31) |
| 2007 | 93.26 (90.85, 95.71) | 12.69 (7.74, 19.87) | 92.99 (82.40, 104.58) |
| 2008 | 91.52 (89.19, 93.90) | 17.59 (11.66, 25.67) | 87.92 (77.84, 98.97) |
| 2009 | 92.47 (90.17, 94.82) | 17.69 (11.92, 25.44) | 84.34 (74.71, 94.90) |
| 2010 | 86.41 (84.21, 88.67) | 9.07 (5.11, 15.16) | 75.00 (66.19, 84.68) |
| 2011 | 88.88 (86.70, 91.10) | 16.65 (10.82, 24.70) | 74.41 (65.59, 84.12) |
| 2012 | 83.60 (81.51, 85.73) | 15.05 (9.97, 21.96) | 74.12 (65.45, 83.65) |
| 2013 | 72.03 (70.13, 73.97) | 10.80 (6.36, 17.41) | 63.04 (55.16, 71.77) |
| 2014 | 61.01 (59.29, 62.78) | 8.82 (5.19, 14.22) | 46.31 (39.90, 53.49) |
| 2015 | 52.18 (50.62, 53.77) | 10.69 (6.62, 16.54) | 37.55 (31.97, 43.83) |
| * Incidence rate per 100,000, with 95% confidence intervals |
PAGEBREAK
Direct effects of the change in policy on the UK born cohorts - results from all models
| Model | IRR (CrI 95%)* | Policy Change | Age | UK born rates | Non-UK born rates | Year of study entry | DoF** | LPD† | LOOIC (se)†† |
|---|---|---|---|---|---|---|---|---|---|
| Model 7 (Negative Binomial) | 1.08 (0.97, 1.19) | Yes | Yes | Yes | No | No | 9 | -211 | 439 (10) |
| Model 7 | 1.08 (1.00, 1.17) | Yes | Yes | Yes | No | No | 8 | -211 | 443 (14) |
| Model 9 | 1.12 (1.01, 1.25) | Yes | Yes | Yes | Yes | No | 9 | -210 | 445 (14) |
| Model 16 | 1.08 (0.97, 1.21) | Yes | Yes | Yes | No | Yes | 20 | -207 | 445 (14) |
| Model 18 | 1.12 (0.97, 1.28) | Yes | Yes | Yes | Yes | Yes | 21 | -207 | 447 (15) |
| Model 8 | 1.16 (1.04, 1.29) | Yes | Yes | No | Yes | No | 8 | -213 | 449 (17) |
| Model 6 | 1.06 (0.98, 1.15) | Yes | Yes | No | No | No | 7 | -215 | 452 (17) |
| Model 17 | 1.15 (1.00, 1.32) | Yes | Yes | No | Yes | Yes | 20 | -209 | 452 (17) |
| Model 15 | 1.06 (0.94, 1.20) | Yes | Yes | No | No | Yes | 19 | -209 | 453 (17) |
| Model 1 | 0.00 (0.00, 0.00) | No | No | No | No | No | 1 | -254 | 513 (26) |
| Model 2 | 1.06 (0.98, 1.14) | Yes | No | No | No | No | 2 | -252 | 515 (25) |
| Model 4 | 1.00 (0.90, 1.10) | Yes | No | No | Yes | No | 3 | -251 | 516 (25) |
| Model 3 | 1.06 (0.98, 1.15) | Yes | No | Yes | No | No | 3 | -252 | 518 (26) |
| Model 5 | 0.98 (0.89, 1.09) | Yes | No | Yes | Yes | No | 4 | -249 | 518 (24) |
| Model 13 | 0.94 (0.78, 1.12) | Yes | No | No | Yes | Yes | 15 | -237 | 518 (27) |
| Model 10 | 0.00 (0.00, 0.00) | No | No | No | No | Yes | 13 | -244 | 521 (28) |
| Model 11 | 1.06 (0.94, 1.20) | Yes | No | No | No | Yes | 14 | -244 | 522 (28) |
| Model 14 | 0.93 (0.78, 1.11) | Yes | No | Yes | Yes | Yes | 16 | -236 | 522 (27) |
| Model 12 | 1.06 (0.93, 1.20) | Yes | No | Yes | No | Yes | 15 | -243 | 526 (28) |
| * Incidence Rate Ratio, with 95% credible intervals, ** Degrees of Freedom, † Computed log pointwise predictive density, †† Leave one out information criterion, with standard error |
| Model | IRR (CrI 95%)* | Policy Change | Age | UK born rates | Non-UK born rates | Year of study entry | DoF** | LPD† | LOOIC (se)†† |
|---|---|---|---|---|---|---|---|---|---|
| Model 16 | 0.96 (0.82, 1.14) | Yes | Yes | Yes | No | Yes | 20 | -192 | 415 (12) |
| Model 16 (Negative Binomial) | 0.96 (0.82, 1.13) | Yes | Yes | Yes | No | Yes | 21 | -196 | 415 (10) |
| Model 18 | 0.99 (0.82, 1.18) | Yes | Yes | Yes | Yes | Yes | 21 | -192 | 417 (13) |
| Model 7 | 0.96 (0.88, 1.05) | Yes | Yes | Yes | No | No | 8 | -200 | 420 (15) |
| Model 9 | 1.00 (0.89, 1.12) | Yes | Yes | Yes | Yes | No | 9 | -200 | 422 (15) |
| Model 8 | 1.02 (0.91, 1.15) | Yes | Yes | No | Yes | No | 8 | -203 | 427 (16) |
| Model 6 | 0.95 (0.87, 1.03) | Yes | Yes | No | No | No | 7 | -204 | 428 (16) |
| Model 15 | 0.95 (0.83, 1.09) | Yes | Yes | No | No | Yes | 19 | -198 | 428 (14) |
| Model 17 | 1.02 (0.87, 1.20) | Yes | Yes | No | Yes | Yes | 20 | -198 | 429 (14) |
| Model 14 | 1.10 (0.92, 1.33) | Yes | No | Yes | Yes | Yes | 16 | -206 | 442 (16) |
| Model 5 | 1.08 (0.97, 1.21) | Yes | No | Yes | Yes | No | 4 | -216 | 445 (18) |
| Model 12 | 0.98 (0.83, 1.15) | Yes | No | Yes | No | Yes | 15 | -209 | 448 (17) |
| Model 4 | 1.12 (1.00, 1.24) | Yes | No | No | Yes | No | 3 | -219 | 449 (18) |
| Model 3 | 0.97 (0.89, 1.06) | Yes | No | Yes | No | No | 3 | -219 | 450 (19) |
| Model 13 | 1.14 (0.97, 1.35) | Yes | No | No | Yes | Yes | 15 | -211 | 452 (16) |
| Model 1 | 0.00 (0.00, 0.00) | No | No | No | No | No | 1 | -229 | 462 (21) |
| Model 2 | 0.95 (0.87, 1.03) | Yes | No | No | No | No | 2 | -228 | 463 (20) |
| Model 10 | 0.00 (0.00, 0.00) | No | No | No | No | Yes | 13 | -220 | 466 (19) |
| Model 11 | 0.95 (0.83, 1.09) | Yes | No | No | No | Yes | 14 | -219 | 467 (19) |
| * Incidence Rate Ratio, with 95% credible intervals, ** Degrees of Freedom, † Computed log pointwise predictive density, †† Leave one out information criterion, with standard error |
PAGEBREAK
Direct effects of the change in policy on the non-UK born cohorts - results from all models
| Model | IRR (CI 95%)* | Policy Change | Age | UK born rates | Non-UK born rates | Year of study entry | DoF** | LPD† | LOOIC (se)†† |
|---|---|---|---|---|---|---|---|---|---|
| Model 17 (Negative Binomial) | 0.74 (0.61, 0.88) | Yes | Yes | No | Yes | Yes | 21 | -228 | 483 (10) |
| Model 17 | 0.74 (0.62, 0.87) | Yes | Yes | No | Yes | Yes | 20 | -223 | 492 (16) |
| Model 18 | 0.73 (0.61, 0.87) | Yes | Yes | Yes | Yes | Yes | 21 | -222 | 493 (16) |
| Model 15 | 0.64 (0.53, 0.78) | Yes | Yes | No | No | Yes | 19 | -224 | 496 (18) |
| Model 16 | 0.65 (0.54, 0.78) | Yes | Yes | Yes | No | Yes | 20 | -223 | 496 (17) |
| Model 8 | 0.79 (0.73, 0.86) | Yes | Yes | No | Yes | No | 8 | -239 | 507 (20) |
| Model 9 | 0.79 (0.72, 0.86) | Yes | Yes | Yes | Yes | No | 9 | -238 | 511 (20) |
| Model 11 | 0.64 (0.52, 0.78) | Yes | No | No | No | Yes | 14 | -241 | 522 (22) |
| Model 10 | 0.00 (0.00, 0.00) | No | No | No | No | Yes | 13 | -241 | 523 (22) |
| Model 12 | 0.64 (0.53, 0.79) | Yes | No | Yes | No | Yes | 15 | -241 | 525 (22) |
| Model 13 | 0.64 (0.52, 0.79) | Yes | No | No | Yes | Yes | 15 | -241 | 526 (23) |
| Model 14 | 0.64 (0.52, 0.79) | Yes | No | Yes | Yes | Yes | 16 | -241 | 530 (23) |
| Model 7 | 0.66 (0.62, 0.70) | Yes | Yes | Yes | No | No | 8 | -248 | 532 (23) |
| Model 6 | 0.65 (0.61, 0.69) | Yes | Yes | No | No | No | 7 | -253 | 539 (27) |
| Model 4 | 0.70 (0.65, 0.76) | Yes | No | No | Yes | No | 3 | -270 | 556 (31) |
| Model 5 | 0.70 (0.64, 0.76) | Yes | No | Yes | Yes | No | 4 | -270 | 559 (31) |
| Model 2 | 0.65 (0.61, 0.69) | Yes | No | No | No | No | 2 | -275 | 561 (33) |
| Model 3 | 0.65 (0.61, 0.69) | Yes | No | Yes | No | No | 3 | -273 | 561 (32) |
| Model 1 | 0.00 (0.00, 0.00) | No | No | No | No | No | 1 | -341 | 692 (51) |
| * Incidence Rate Ratio, with 95% credible intervals, ** Degrees of Freedom, † Computed log pointwise predictive density, †† Leave one out information criterion, with standard error |
| Model | IRR (CI 95%)* | Policy Change | Age | UK born rates | Non-UK born rates | Year of study entry | DoF** | LPD† | LOOIC (se)†† |
|---|---|---|---|---|---|---|---|---|---|
| Model 8 (Negative Binomial) | 0.62 (0.44, 0.88) | Yes | Yes | No | Yes | No | 9 | -138 | 293 (15) |
| Model 8 | 0.64 (0.47, 0.86) | Yes | Yes | No | Yes | No | 8 | -137 | 295 (18) |
| Model 9 | 0.62 (0.45, 0.85) | Yes | Yes | Yes | Yes | No | 9 | -137 | 297 (18) |
| Model 6 | 0.47 (0.38, 0.58) | Yes | Yes | No | No | No | 7 | -139 | 298 (19) |
| Model 7 | 0.48 (0.39, 0.60) | Yes | Yes | Yes | No | No | 8 | -139 | 298 (19) |
| Model 17 | 0.63 (0.44, 0.89) | Yes | Yes | No | Yes | Yes | 20 | -135 | 298 (18) |
| Model 18 | 0.61 (0.42, 0.87) | Yes | Yes | Yes | Yes | Yes | 21 | -135 | 300 (18) |
| Model 15 | 0.47 (0.35, 0.62) | Yes | Yes | No | No | Yes | 19 | -136 | 301 (20) |
| Model 16 | 0.48 (0.36, 0.63) | Yes | Yes | Yes | No | Yes | 20 | -136 | 301 (19) |
| Model 4 | 0.82 (0.61, 1.10) | Yes | No | No | Yes | No | 3 | -147 | 304 (17) |
| Model 5 | 0.78 (0.58, 1.06) | Yes | No | Yes | Yes | No | 4 | -147 | 306 (18) |
| Model 13 | 0.83 (0.59, 1.16) | Yes | No | No | Yes | Yes | 15 | -145 | 308 (18) |
| Model 14 | 0.78 (0.55, 1.12) | Yes | No | Yes | Yes | Yes | 16 | -144 | 310 (19) |
| Model 3 | 0.52 (0.42, 0.64) | Yes | No | Yes | No | No | 3 | -152 | 314 (22) |
| Model 12 | 0.51 (0.38, 0.69) | Yes | No | Yes | No | Yes | 15 | -148 | 317 (23) |
| Model 2 | 0.49 (0.40, 0.61) | Yes | No | No | No | No | 2 | -156 | 319 (22) |
| Model 11 | 0.49 (0.37, 0.65) | Yes | No | No | No | Yes | 14 | -152 | 322 (23) |
| Model 10 | 0.00 (0.00, 0.00) | No | No | No | No | Yes | 13 | -150 | 330 (25) |
| Model 1 | 0.00 (0.00, 0.00) | No | No | No | No | No | 1 | -171 | 346 (27) |
| * Incidence Rate Ratio, with 95% credible intervals, ** Degrees of Freedom, † Computed log pointwise predictive density, †† Leave one out information criterion, with standard error |