Executive summary
* A four-page paper was widely reported with headlines such as "Gas stove pollution causes 12.7% of childhood asthma, study finds."
* The paper was a modelling exercise, which meant it could only produce estimates. The headlines claimed causation.
* The model used one number from a 2013 paper as its base. The researchers sought and found data since this date, but then ignored it.
* The one number chosen by the researchers was for current asthma. Had they chosen lifetime asthma, or all asthma, or wheeze (the main symptom of asthma), there would have been nothing to report.
* The one number was based on 10 studies, which dated back to 1977. Eight of these found no association between gas cooking and childhood asthma.
* The one number was claimed to be based on three North American and seven European studies, which were pooled together. There were in fact five North American and five European studies, which were pooled together. Since we have no idea how five and five became three and seven, the combined number can’t be dissected.
* The study was funded by an organisation that wants a carbon free world. There was an anti-gas/pro-electric motive from the outset. This was borne out in the methodology and the selection of one number that found something, ignoring vastly more numbers that found nothing.
Introduction
Most academic papers published go largely unnoticed. A few gain global attention. This week’s paper was one of those. It was featured from the UK Daily Mail “One in EIGHT childhood asthma cases are linked to gas stoves, study warns” (Ref 1) to the US Washington Post “Gas stove pollution causes 12.7% of childhood asthma, study finds” (Ref 2).
The paper was called “Population Attributable Fraction of Gas Stoves and Childhood Asthma in the United States” and it was by Gruenwald et al (Ref 3).
The lead author works for RMI (Rocky Mountain Institute) and RMI was one of two funders of the report. The other funder was the US National Cancer Institute. RMI wants a carbon free world (Ref 4). There’s upfront motive for an attack on gas cookers.
The first sentence of the abstract of the paper stated, "Indoor gas stove use for cooking is associated with an increased risk of current asthma among children and is prevalent in 35% of households in the United States (US)." This paper did not, therefore, discover an association between gas stoves and childhood asthma. It stated that this association existed and then proceeded to model numbers to estimate the proportion of childhood asthma attributable to gas stove use.
The word “model(s)” appeared only once in the paper. The word “estimate(s)” appeared five times. This was a modelling exercise which could only estimate. This fact was lost in the global reporting. As we always state, models are only as valid as their assumptions. This one assumed that gas stoves and childhood asthma were associated from the outset.
The study
The paper was only four pages long. It relied heavily on another paper, which was 14 pages long (Ref 5). This 14-page paper, published in 2013 by Lin et al, was a meta-analysis of other papers examining the association between gas cooking, asthma and wheeze. Wheeze is the main symptom of asthma. A meta-analysis, remember, is a pooling together of all papers on a topic. The Lin et al paper found and pooled 41 studies, which were relevant to this topic.
The method of the four-page modelling paper was as follows:
Step 1) Gruenwald et al used the Lin et al paper as a base. They looked for peer-reviewed papers published since that 2013 meta-analysis. They found 27 further papers. They stated, “none reported new associations between gas stove use and childhood asthma specifically in North America or Europe” and then gave no further mention to these 27 papers. They didn’t even report the references for them. They didn’t add them to the 2013 meta-analysis.
I used one of their search terms “gas cooking and children”, to find those words in the title or abstract of any paper. I limited results to the last 10 years. Nine papers were returned, including the Lin et al 2013 paper, so I knew I had captured the right period. (Their other search terms would have led to all 27 papers – I just needed to find one smoking gun).
The first paper after Lin et al was by Esplugues et al (Ref 6). This was a Spanish study, which found “Exposure to gas cooking in homes was not associated with respiratory outcomes. Odds Ratios (OR) were close to 1 and not statistically significant.” (My emphasis). That paper should have been included in meta-analysis and likely several more should also have been included. The whole point of meta-analysis is that you pool everything. Adding just this paper to the pool would have moderated the finding of the previous pool. This was a major flaw.
Step 2) They thus took one number (one association) from the Lin et al paper as their base. The main part of this Monday note is going to scrutinise that number. We’ll just finish the methodology first…
The number used was 1.34 (95% confidence interval of 1.12–1.57). This was claimed to be the risk ratio for current asthma in children in Europe and North America. This figure pooled together studies that looked at gas cooking vs electric, or gas cooking yes or no. The figure 1.34 can be presented as – there was a 34% higher risk of asthma in children in households with gas cooking.
Step 3) They estimated the proportion of children under 18 years old exposed to gas stoves in the US. That number was 35%, as you can see from the first sentence of the abstract quoted in the introduction above.
Their baseline number was based on Europe and North America. (We will soon see that there were no findings for North America.) However, they proceed to model the US alone. They used these two numbers (34% and 35%) to estimate something called the population attributable fraction (PAF). The PAF is defined as the fraction of all cases of a particular condition in a population that is attributable to a specific exposure. In this case, the PAF being estimated is the proportion of asthma in the US population, which is attributable to gas cooking. There is a formula for estimating PAF (Ref 7).
We have seen from the headlines in the opening paragraph that the PAF was calculated as 12.7% (US Washington Post). This was rounded to “one in eight” by the UK Mail.
And that’s it. The methodology in summary was 1a) find a risk ratio for an association between gas cooking and asthma in children from 2013 (34%). 1b) Ignore all evidence from the past 10 years. 2) Find the proportion of children exposed to gas cooking (35%). 3) Pop these numbers into a formula and out pops the result that 12.7% of childhood asthma in the US is attributable to gas cooking. We’re now going to take apart the first number of 34%, as everything found fails if that fails. (Notwithstanding that we have already found issues).
The base number
The Lin et al paper reported several different results. These were grouped by current asthma, lifetime asthma or all asthma and subgrouped by age of children studied, study region, proportion of gas cooking and year of study publication.
Gruenwald et al chose to take the data for current asthma only and for North America and Europe combined. Had they chosen lifetime asthma (asthma at any time in life), there would have been no findings for North America or Europe to report. Had they chosen all asthma (current and lifetime), there would have been no findings for North America to report. Had they chosen wheeze (current, or at any time in life, or both), there would have been no findings for North America or Europe to report.
While the Lin et al research found 41 studies to pool, many of these were focused on just one region or just one age group or just current asthma etc. This meant that Lin et al reported that only three studies provided data for the overall finding for North America for current asthma and only seven studies provided data for the overall finding for Europe for current asthma. Three and 7 are a long way from 41. But even these numbers were wrong.
I used Lin et al’s supplementary material to extract the 10 studies for North America and Europe and put them in a table below. There are 11 rows in the table below – the 10 studies and then Lin et al did a study of their own at the same time as their 2013 meta-analysis. This is not uncommon.
The Lin et al paper (rightly) reported that North America included USA and Canada. From the table, you can see that there are five studies from North America and not three. Germany, England, Scotland, Netherlands and Russia count as Europe and thus there are five European studies – six including the one Lin et al did.
In the table, look at the “Result – effect size” column. I’ve included the confidence interval (that’s the 0.17-3.46 kind of number in brackets). If this range includes 1.0, the result is not significant. It could have happened by chance. Notice immediately that most results are not significant. I’ve put in red the results that are not significant. Nine out of 11 results are not significant. This number is even higher (11 out of 13) if we include sub results on boys/girls and wheezing/not wheezing.
Some studies adjusted for lots of factors; others adjusted for nothing. Half didn’t even adjust for smoking. Only two adjusted for pets. This was despite one of the studies concluding “An estimated 32% of new asthma cases could be attributed to pets” (Ref 19). Despite this lack of adjustment, most findings were not significant.
The most significant finding, Spengler (Russia, 2004), also examined wheezing and found nothing of significance for wheezing. Hence, we’re being asked to believe that there was an association with a diagnosis of asthma and gas cooking, but not the main sign of asthma and gas cooking.
Teams of researchers have been studying a possible association between gas cooking and asthma in children since 1977 and have mostly found nothing. Only when the results were pooled together by Lin et al in 2013 was something found.
Using the wrong numbers
I simply don’t know how Lin et al claimed that there were three North American studies and seven European studies. We’ll have to proceed using their numbers for three and seven. Gruenwald et al used the Lin et al numbers without checking if they were correct.
The odds ratio found for the 3 North American studies was 1.36, (95% CI = 0.76–2.43). I’ve included the confidence interval to show that it includes 1.0, which is the line of no effect. There was thus no association found between gas cooking and current childhood asthma in North America. The odds ratio found for the 7 European studies was 1.34, (95% CI = 1.13–1.60). Gruenwald et al reportedthat the odds ratio for current asthma in North America and Europe combined (10 studies) was 1.34. The 95% Confidence Interval (CI) was reported to be 1.12–1.57.
The combined result (1.34) was so similar to the Europe result alone (1.34) that we can surmise that the North American studies had no impact on the combined result. This makes sense, as we know that the North America findings were not significant.
There were data for North America. The title of the paper focused on the US (not even North America). The PAF was estimated for North America. The rationale given for including Europe was “We combined effect sizes for North America and Europe given the similarities in housing characteristics and gas-stove usage patterns across these geographies.”
I hypothesise that the rationale for including Europe was that there would have been no baseline finding if this hadn’t been done.
Let’s summarise the errors. The Gruenwald et al paper was a modelling exercise. It modelled an estimate. The media headlines reported the estimate as fact. One other 2013 paper alone was used as the base for the model. Papers since 2013 were sought, but then ignored. Had all asthma been reviewed, nothing would have been found. Had wheeze been reviewed, nothing would have been found. Had North America alone been reviewed, nothing would have been found. Had US alone been reviewed, nothing would have been found. The vast majority of all research investigating an association between gas cooking and asthma in children, since 1977, has found nothing.
By combining Europe and North America in a model to make a claim about the US alone, one number could be used (1.34, 34%). We can’t trace how this number came about because five North American and five European studies were inexplicably pooled as three North American and seven European studies.
This was a house of cards reported as a structure of steel. The funding suggests that there was an agenda. The manipulations confirm this.
References
Ref 1: https://www.dailymail.co.uk/health/article-11615727/One-EIGHT-childhood-asthma-cases-linked-gas-stoves-study-warns.html
Ref 2: https://www.washingtonpost.com/politics/2023/01/06/gas-stove-pollution-causes-127-childhood-asthma-study-finds/
Ref 3: Gruenwald et al. Population Attributable Fraction of Gas Stoves and Childhood Asthma in the United States. International Journal of Environmental Research and Public Health. Dec 2022. https://www.mdpi.com/1660-4601/20/1/75
Ref 4: https://rmi.org/
Ref 5: Lin et al. Meta-analysis of the effects of indoor nitrogen dioxide and gas cooking on asthma and wheeze in children. International Journal of Epidemiology. 2013. https://academic.oup.com/ije/article/42/6/1724/737113
Ref 6: Esplugues et al. Prenatal exposure to cooking gas and respiratory health in infants is modified by tobacco smoke exposure and diet in the INMA birth cohort study. Environ Health. 2013 Dec. https://pubmed.ncbi.nlm.nih.gov/24289253/
Ref 7: Mansournia and Altman. Population attributable fraction. BMJ. 2018. https://www.bmj.com/content/360/bmj.k757
Ref 8: Behrens et al. Symptoms of asthma and the home environment. The ISAAC I and III cross-sectional surveys in Munster, Germany. Int Arch Allergy Immunol. 2005.
Ref 9: Carlsten et al. Combined exposure to dog and indoor pollution: incident asthma in a high-risk birth cohort. Eur Respir J. 2011.
Ref 10: Dekker et al. Childhood asthma and the indoor environment. Chest. 1991.
Ref 11: Diette et al. Home indoor pollutant exposures among inner-city children with and without asthma. Environ Health Perspect. 2007.
Ref 12: Hessel et al. The epidemiology of childhood asthma in Red Deer and Medicine Hat, Alberta. Can Respir J. 2001.
Ref 13: McConnell et al. Indoor risk factors for asthma in a prospective study of adolescents. Epidemiology. 2002.
Ref 14: Melia et al. Association between gas cooking and respiratory disease in children. Br Med J. 1977.
Ref 15: Spengler et al. Housing characteristics and children’s respiratory health in the Russian Federation. Am J Public Health. 2004
Ref 16: Tavernier et al. Endotoxin exposure in asthmatic children and matched healthy controls: results of IPEADAM study. Indoor Air. 2005.
Ref 17: Willers et al. Gas cooking, kitchen ventilation, and asthma, allergic symptoms and sensitization in young children – the PIAMA study. Allergy. 2006.
Ref 18: Lin et al. Gas cooking, respiratory and allergic outcomes in the PIAMA birth cohort study. Occup Environ Med. 2013.
Ref 19: McConnell et al. Indoor risk factors for asthma in a prospective study of adolescents. Epidemiology. 2002.
SOURCE: Zoë Harcombe – Read entire story here.