Behind the Journal Article: Thinking Critically about Research Evidence and Eating Disorders

There’s been a fair bit of talk lately (ok, always) about evidence in eating disorders. In addition to the evidence for certain types of treatment, there’s talk about evidence for causes of eating disorders, evidence for whether recovery is possible, and more. The framing I generally see advanced is that we need to be using evidence-based practice only; presumably, this evidence comes from scientific research. I don’t disagree, but in this post I’ll be writing about how science is never wholly objective and is situated in social context.

Let the record show that I love science. I love all kinds of science: biological science, genetic science, neuroscience, social science, you name it, I think learning and research and scientific methods are interesting. I can’t do all kinds of science; as Tetyana says, this blog itself has moved away from “science” as she originally intended it as I continue to dominate posts with my social science-y takes on the eating disorders field. I write about social science because it’s what I know and what I’m trained to comment on – my perspectives on genetic studies will inevitably be coloured by my lack of knowledge about the intricate systems involved. So while I do consider what I write about “science,” it isn’t what everyone considers science, and it isn’t the same as biological or physical science.

However, there is a mistaken assumption, I find, that biological or physical sciences, or even quantitative social science, is somehow better or less biased. I am not here to contest the fact that some types of evidence is more compelling, or to argue that scientific methods and rigour are incredibly important. Instead, I’ll use this post to highlight how all science is generated in a socio-political-economic context, and there are multiple ways of interpreting “research evidence” based on where you are coming from.

I’m not going to use a biological or physical science study to make the point; as I mentioned, my read of this work would be rudimentary at best. Instead, I’ll use an older, highly-cited article by Stice & Spangler about binge eating onset in adolescent girls to explore how you can interpret and frame research evidence in different ways.

The Article

In 2002, Stice, Presnell & Spangler published their article, which highlights the results of a prospective research study on binge eating in adolescent girls. They sought to better understand what factors predict the onset of binge eating, and investigated this using a prospective study design – that is, having participants complete baseline measures and then following up with the same measures at 10 months and 20 months. For context into the influential character of the study, it has been cited 323 times (which is a fair bit for an academic study in a very specific field like eating disorders).

By all accounts, the study design seems fairly rigorous: the authors recruited 231 girls from grade 9 and 10 and administered validated scales to measure general thoughts and behaviours the girls had over the past 6 months and depressive symptoms, anxiety, anger, and eating pathology over the past month.

Despite the rigour, and not discounting the validity of the study as a whole, let’s take a deeper look at how even a well-designed, quantitative study has its own subjectivity.

Framing the Issue

The first section of any academic article is the literature review; here, the authors show their leanings, if you’re looking for it. Again, I don’t identify this to be boring or patronizing, or even to say that there’s anything wrong with having a perspective. I have a perspective, I am biased – and my research articles show clues about the way that I frame issues that can help readers to interpret my findings. It is harmful, I think, to pretend that these biases don’t exist, and to present things as objective truths when these “objective truths” are also based in particular versions of reality.

The first sentence of this particular article situates the authors within a very specific perspective on bodies and body size:

“Adolescent obesity is associated with serious health problems, including high blood pressure, adverse lipoprotein profiles, non- insulin-dependent diabetes mellitus, coronary heart disease, ath- erosclerotic cerebrovascular disease, colorectal cancer, and death from all causes (Must, Jacques, Dallal, Bajema, & Dietz, 1992)” (p. 131)

Spoiler alert for an article I’m in the process of writing: the perspective that the risks of eating disorders need to be “balanced” against the “risk of obesity” is an extraordinarily common framing in articles detailing eating disorder causes, correlates and outcomes. Here we might consider how research is not conducted in a social vector: public health priorities right now hinge on controlling what is presented as the epidemic state of fatness in Western cultures in particular.

In order to successfully publish about eating disorders, or to secure funding, researchers often have to make a nod to this “obesity epidemic,” to field the question “but what about obesity?” That research is framed by social priorities feels self-evident but perhaps isn’t; acknowledging this social contextual character also helps us to avoid blaming individual researchers for advancing perspectives with which we disagree.

Stick, Presnell & Spangler go on to highlight the context of their study, naming their perspective on weight and controllability: they endorse an energy balance theory of body mass – that is, calories in, calories out. Others have more thoroughly contested the energy balance theory (e.g., Aphramor, 2010, Bomback, 2014.)

Regardless of how you feel about it, there it is – an immediate acknowledgment of the frame of reference the authors will use to interpret their evidence. Their approach to knowledge seeking and the status of “objective truth” is also evident in the introductory section; they note which variables have been linked to risk for binge eating, like:

• Larger body size
• Body dissatisfaction
• Over-evaluation of appearance
• Endorsement of the thin ideal
• Eating disturbances
• Negative affect (mood state)

And, they cite a study by one of the authors to support these variables. Of course, self-citation is commonplace in academia – but still worth noting. Stice’s work is largely centered around the role of over-evaluation of thinness in promoting disordered eating; again, neither right nor wrong but impacting his interpretations.

So, so far we’ve gleaned that, for these authors:

• Weight is controllable
• Eating disorders are borne largely of an over-evaluation of weight and shape
• We can find out more about the risks for binge eating by asking people about their thoughts and behaviours, including about their body, and by measuring their body size

Investigating the Sample

Research evidence also only holds as well as the representativeness of the sample used to create it. In this case, all 231 participants were from a private school in Northern California. All were girls, though there is no mention of how this was assessed – did the researchers ask how people self-identified, in terms of gender? Most (65%) were identified as Caucasian, 20% identified as Asian, 4% African American, 2% Hispanic, 1% Native American, and 8% other. Presumably these questions were assessed using a self-report questionnaire.

This is a fairly common sample description in the eating disorders literature. The authors note that the all-girl makeup of the sample limits the generalizability of the findings, of course, but I do think it’s interesting that we keep doing this research in the same way and noting this limitation. I fall prey to this, too: again, let’s think about the context in which research is conducted. Generally, as researchers, we’re on a fairly tight time schedule, trying to publish (presumably, so as not to “perish”).

Often we have to justify our work, for instance noting why we’ve selected the sample we have. Though you could make an argument for the need to do research with those who haven’t been studied yet or studied well (that is, anyone other than the straight white thin cis girl, in eating disorders), but then you encounter the recruitment paradox: it is really, really hard to get people to participate in your research if they haven’t been included in narratives around eating disorders and/or have been de-legitimized in clinical settings.

Trust is a huge part of research – to get people to tell you their stories, they have to trust you and value what you’re doing. We have a horrible history, in research, of treating people as “subjects” to our research processes, and using stories for personal career gains. Sometimes research is harmful to people or perpetuates stereotypes. It’s not surprising, then, that some are taciturn about their experiences. If I’d been de-legitimized and studied as a passive subject, I wouldn’t want to tell my story either. Stories are powerful, personal things.

Surveying the Scales

The choice of scales also speaks to the way we interpret research evidence. In this case, the authors used a number of scales and reported on their psychometric properties – that is, how well they measure the phenomenon they are attempting to represent. The interesting thing about scales, though, is that they, too, are constructed in specific sociocultural contexts. Often, people from different social locations related differently to concepts the authors measured, including dieting, body mass, body dissatisfaction, depression, anxiety, emotional eating, and pressure to be thin.

They do not mention which groups of people were used in developing the psychometrics of the scales; we can’t be sure that the scales are actually measuring the same thing in the same way for all people. Of course, this is kind of a “working with what we have” scenario, again. And, admittedly, the scales the researchers chose do look strong – and, since the sample is a fairly normative one, we can be reasonably sure that the sample on which the scale was constructed is at least similar to the one in the study.

Reconsidering the Results 

In brief, the authors conclude that:

• Dieting, body mass, body dissatisfaction, appearance over-evaluation, perceived pressure to be thin, modeled eating disturbances, depressive symptoms, low self-esteem, emotional eating, and low social support from peers predict binge eating
• Binge eating leads to a “risk of obesity”

The idea is that dieting is related to body dissatisfaction, appearance over-evaluation, perceived pressure to be thin, and modeled eating disturbances. Then, binge eating is described as relating to the attempt to regulate the caloric deficit from dieting.

An interesting link the authors uncovered was between depressive symptoms and binge eating for those who did not overvalue appearance: those who did not report over-evaluation but had higher depressive symptoms were still at elevated risk for binge eating.

Amongst those who did show elevated appearance over-evaluation, this predicted binge eating onset only when their measured body size was “average-to-overweight.” This is where I’d like to pause and explore how framing shapes the interpretation of this finding. For me, as a systemically-oriented researcher, my first inclination would be to situate this: it is not at all surprising that those who are in larger bodies feel bodily discontent, because society tells them constantly that their bodies are wrong, and that they should attempt to change them. Instead of looking at the impact of weight stigma and multiple marginalizations, however, we get the suggestion that:

“A prevention program that helped adolescents achieve a healthy weight might mitigate the effects of appearance overevalution because their bodies would not deviate markedly from the cultural ideal” (p. 137)

Yes, you read that right. The suggestion is not that we change society to accommodate a more diverse array of physical forms, but rather that individual folks alter their bodies to be closer to the ideal. Again, of course, this is predicated on the idea that weight is controllable, based on that energy balance model of body mass.

Implications

I’m not here to argue that this study is in any way “wrong” or “bad.” It is just one example of how research is always and forever tied to social, political, and economic contexts. Sure, we can see this as “evidence,” particularly for the ways in which there are many routes to eating pathology – including those tied and those not tied to the way we think about our bodies. We can, and should, use research as evidence. But we can’t escape our subjectivity and need to acknowledge how our perspectives and paradigms inform what we put into the world as researchers.

We need to consider the forces that push us to draw certain conclusions and advance certain perspectives, whether these are borne of hard-wired tendencies to see the issue one way or another or of the opinions of a funder or reviewer three. No research article is perfect; we can’t take things at face value or objective truth, no matter how clean and rigorous the study. Every aspect of research, particularly social research, is situated in a political, social, and economic nexus. Science, and social science, are important and valuable – but evidence is a moving target.

Reference

Stice E, Presnell K, & Spangler D (2002). Risk factors for binge eating onset in adolescent girls: a 2-year prospective investigation. Health psychology : official journal of the Division of Health Psychology, American Psychological Association, 21 (2), 131-8 PMID: 11950103