Lockdowns Didn't "Prematurely Age" Teen Girl's Brains
Why the new viral study is extremely misleading
Lockdowns have been one of the most contentious aspects of the COVID-19 pandemic since they were first enacted. Some people argue that every intervention that governments implemented during the pandemic was entirely justified due to the danger of the virus. Others took the position that the virus was basically safe for most of humanity and therefore governments should do little to nothing to combat it.
Personally, I have published scientifically arguing that it’s complex - some interventions were probably justified, some probably weren’t, and we have made little effort since 2020 to figure out which is which.
Now, headlines have called into question the idea that lockdowns could have done any good. According to a new study, it appears that lockdowns may have prematurely aged the brains of teenagers, which could impact everything from their mental health to future job prospects.
Fortunately, the data doesn’t really show this at all. This story is a case study in problematic science, and why it’s important to read the study even if it is published in a prestigious journal with a funny acronym.
The Science
The study in question is a neurological examination of teen brains. The researchers put a bunch of adolescents aged 9-17 into MRIs before the pandemic, and then looked at their brains again a few years later. They used this data to look at what had happened to the brains in the interim using a variety of statistical techniques.
As MRI studies go, this one isn’t totally woeful in design. It’s small, but the authors have done used some interesting methods to try and make useful inferences about the data they’ve collected. Unfortunately, there really is only so much you can do with data from about 100 children.
The usual thing to do with a sample like this would be to compare the changes before/after and make some inferences, but that has relatively limited usefulness. You can’t really say which changes might’ve happened just due to age, and which might be to external factors. So the researchers did something clever - they divided their sample randomly into three groups - a ‘normative’ model, a validation sample, and finally their test group.
The authors then used the ‘normative’ model group to give them an idea of what teen brain development looked like in this area of Washington State prior to the pandemic. They looked at the differences between 9, 11, 13, 15, and 17-year old brains in this group, which gave them this ‘normative’ model to compare to. It’s basically a way of estimating what normal brains should look like in this region of the world at different ages.
The authors validated this on a small sample of the other pre-pandemic brains, and then finally tested the post-pandemic group against the ‘normative’ development. They found that there were no meaningful differences for boys, but for girls there were a number of brain regions which had unexpected changes that are associated with greater age. They used a fairly complex statistical model to then estimate that these changes were similar to an additional 4.2 years of age for the teen girls.
There are obviously some weaknesses to this design. It’s much more useful than simply comparing a pre/post sample, because we have some idea about what we might expect the post sample to look like. However, the sample is still quite small, and we don’t really know that all of the differences pre pandemic were due simply to age. The authors created their ‘normative’ model based on about a dozen children of each age, which means that even one or two kids who diverged from the norm could’ve thrown their model off.
The 4.2 year number is also a bit meaningless. The authors compared 25 girls aged 12, 14, and 16 post-pandemic to their modelled estimates from prior to the pandemic, and had no longitudinal pre-pandemic samples. So we can say that the 25 girls who they tested after the pandemic were had differences that looked similar to a 4.2-year age difference in girl’s brains prior to the pandemic, but that doesn’t even tell us if these differences are particularly abnormal for this cohort.
Similarly, the authors couldn’t control for any factors aside from age and gender, because the sample is simply too small. Which brings us to an interesting point - what does any of this have to do with lockdowns?
Simple answer, really: nothing. The study does not, in any way, examine the effects of lockdown on teen brains.
Rather, the study shows that teen girls’ brains after the pandemic were different to the expected trends from brains before the pandemic. This could be caused by many things. Maybe the virus itself, which can cause some changes to brain chemistry, is to blame. Perhaps it was the global disruption brought about by a novel pandemic. Maybe the girls were more vulnerable than boys to things like relatives dying of COVID-19. We have no idea, because the authors didn’t do anything to investigate these myriad explanations. They don’t even report that the children in the study were present in Washington State for the lockdowns, nor whether they experienced similar lockdown impacts (i.e. school closures).
To make any inferences about lockdowns, the authors would’ve had to find some control group who’d had a different exposure to their intervention. Perhaps MRIs from kids in Florida, which had different COVID-19 restrictions, or a longitudinal sample from before the pandemic. These would all be inadequate samples for one reason or another, but they would’ve at least given some insight into whether lockdowns were associated with the cortical thinning seen in the research. As it stands, the study tells us nothing at all.
Problematic Publishing
You can’t just blame the media here - the authors put the word “lockdown” into their study. It’s the second word of the title of their paper. Despite the paper having nothing to do with lockdowns.
This is, in a word, bad. Bad science. Poorly thought-through. Inadequate in a very serious way. How did it get published in the prestigious Proceedings of the National Academy of Science (PNAS), you ask?
PNAS is certainly prestigious, but they also have a bizarre and outdated submission system. If you are not a member of the National Academy of Science in the US, you can submit papers to the journal and they are peer-reviewed as normal. But members can submit two papers per year under what is called the contributor track. In this track, PNAS gives the authors of the paper the ability to control aspects of the editorial process - in particular, they get to choose their own reviewers and manage the peer-review discussions themselves.
Of course, peer review is not the great bastion against scientific issues that some think it is, but it’s easy to see how this contributor track undermines even the more basic aspects of the process. If you can pick a few friends to glance over your paper, you can usually get whatever you want published. For example, you might be able to get a paper out talking about the negative impact of lockdowns even though you didn’t once assess lockdowns.
The new study mostly just shows that there may be some changes to the brains of teen girls in Washington State that happened sometime during the years of 2020-2022 which are unexpected. What caused these changes is impossible to pin down using this study design.
All of the headlines were wrong. It’s hard to know who to blame here - the overeager authors, the virality of social media, or the journalists who breathlessly reported the findings without once checking to see whether they were reasonable - but it’s clear that something broke down.
It’s certainly possible that lockdowns had some impact on teen brains. We know that restrictive COVID-19 interventions had a mixed effect on mental health for kids, with a range of negative effects but also some positive ones. The impact in this study was not necessarily negative - cortical thinning isn’t per se a problem - but it could be associated with other brain changes that were.
That being said, all the new study really proves is that there were some statistically significant changes to the brains of teens in Washington State between 2020 and 2022. What that means for lockdowns, public policy, or our knowledge of COVID-19 itself is anyone’s guess.
I had actually looked at this one myself and highlighted their small comment that the changes might have related to getting Covid - which they then all but dismissed as a cause because there was no indication that boys and girls caught Covid at different rates. If I recall correctly, more men died / were admitted to ICU and more women have ended up with long Covid so perhaps catching it wasn’t the most sensible indicator; the known differences in responses to it between sexes might have been worth mentioning, even in passing.
*screaming into the void*