Page
of 6
Pages
Assignment #9 – Identify the Threat to Internal Validity Using MR SMITH ID – Fall, 2022
5 Points Possible – Smartphone-Delivered Mindful Eating Intervention
Instructions: Below is a description of a study. Read over the description in Part One. In Part
Two, read variations of the study and determine which (if any) MR SMITH ID threat is present.
PART ONE: Study Description
Healthy eating is always a struggle in the United States, especially with cheap (but unhealthy)
food so readily available. As a result, numerous diets are available that claim that they can help
people better regulate their eating. One such diet involves “mindfulness”, which avoid being too
judgmental when it comes to eating but rather teaches the dieter to be curious about what, when,
and why they are craving food. Such mindfulness eating diets work by drawing the dieter’s
awareness to their eating habits through three steps: 1) Increasing the person’s awareness of
triggers that may lead them to eat (e.g. What emotions are they experiencing, Who they are with,
What foods are available, etc.); 2) Having the dieter think about the outcomes that follow from
their behavior (e.g. Will eating these cookies really make me feel better?), and 3) Having the
dieter learn to live with their cravings rather than trying to ignore them.
While mindfulness dieting does show promise, how to deliver the lessons of mindfulness diets to
participants can vary dramatically. Such lessons can be delivered in person (as in a dieting class),
though literature from a doctor or dietician, through websites, and through weight support groups,
to name just a few. But one method that might be very beneficial is delivering mindfulness
lessons through a smartphone app. After all, most Americans have access to a smartphone, so
why not use it to help them with dieting! There are a lot of benefits to the smartphone approach.
Users can access the app anywhere at any time without being tied to a computer screen. A mobile
phone avoids hassles involving time, travel, and scheduling commitments.
Set-up: Imagine you design a smartphone app that presents users with a mindfulness diet program
that encourages them to become aware of the “what, when, and why” questions surrounding their
current eating habits. You design a series of short modules (5 to 10 minutes) that provide dieters
with information about mindfulness and their diets. These modules include animations, short
lectures, or basic information about how to be mindful. You ask some participants to login to a
computer website to review the module information (control group). You ask other participants to
use a smartphone app that includes the same modules (experimental group).
Before the study officially begins, you gather a wide variety of pre-study information about each
participant. This includes information about how often they cave into cravings, self-reporting on
why they eat (to cope with emotions, when socializing, when merely hungry, etc.), whether they
binge eat, their starting weight, etc. These measures are also taken at the conclusion of the study.
The study design looks like the following:
Smartphone Intervention (Experimental Group)
Pre-study questionnaire Post-study questionnaire
Website Intervention (Control Group)
You predict that dieters in the Smartphone group will benefit more (e.g. lose more weight, be
more “mindful”, etc.) from the intervention than those in the Website group.
Identify which internal validity threat is most likely at work in each of the following Part Two
statements. You will receive .5 points for each correct response. Note that each threat should be
used only once, with one statement of the ten statements not having a threat. Also keep in mind
internal validity threats may overlap – in these instances, choose the BEST threat for the
scenario. Also note that the question order and answer order might differ in Canvas, so make
sure to focus on the CONTENT of the question and answer rather than their order.
PART TWO: Spot the Internal Validity Threat that is MOST likely at work for each of the
following scenarios: Use the key below to match the study validity description with the
specific validity threat.
a. Maturation
b. Regression to the mean
c. Selection
d. Mortality
e. Instrumentation
f. Testing
g. History
h. Interactions
i. Diffusion
j. No Threat
1). After completing the pre-study questionnaire, all dieters were randomly assigned to either
the Website intervention or the Smartphone intervention. Once the dieters were assigned to
their condition, they did not socialize with other dieters, which was easy to control as dieters
completed all study questions remotely. To adequately assess the effectiveness of the
intervention programs, the researchers made sure to use the exact same questions for the pre-
and post-study questionnaires, including information about how often they cave into their
cravings, self-reporting on why they eat (to cope with emotions, when socializing, when
merely hungry, etc.), whether they binge eat, their starting weight, and other items. Some of
the dieters even commented that they recalled seeing the same post-study questions before on
the pre-study questionnaire and that they based their post-study answers on their pre-study
answers. After comparing the post-exposure scores for the two conditions, you found no
differences between students in the Website intervention and the Smartphone intervention.
Which internal validity threat is most likely occurring in this scenario?
a. Maturation
b. Regression to the mean
c. Selection
d. Mortality
e. Instrumentation
f. Testing
g. History
h. Interactions
i. Diffusion
j. No Threat
2). After seeing the pre-study questionnaire scores, you were concerned about dieters who
were morbidly obese, and you had heightened medical concerns about these dieters. To make
sure such high-risk dieters would get the greatest benefit from your study, you made sure to
assign them to the Smartphone intervention while dieters with lower weights were randomly
assigned to either the Smartphone or Website interventions. After comparing the post-study
scores for the two conditions, you found that dieters in the Smartphone intervention lost
significantly more weight and had heightened mindfulness scores compared to those in the
Website intervention. Which internal validity threat is most likely occurring in this scenario?
a. Maturation
b. Regression to the mean
c. Selection
d. Mortality
e. Instrumentation
f. Testing
g. History
h. Interactions
i. Diffusion
j. No Threat
3). To compare the effectiveness of the Smartphone intervention compared to the Website
intervention, you kept careful records of how often dieters logged in for the modules on both
platforms throughout the month-long study. You noticed that while 90% of dieters in the
Smartphone intervention logged in at least once per day throughout the study, only 60% of
participants in the Website intervention logged in at least one per day throughout the study.
You found that dieters in the Smartphone intervention lost more weight and were more aware
of why, when, and how they experienced cravings than dieters in the Website intervention.
Which internal validity threat is most likely occurring in this scenario?
a. Maturation
b. Regression to the mean
c. Selection
d. Mortality
e. Instrumentation
f. Testing
g. History
h. Interactions
i. Diffusion
j. No Threat
4). As part of the pre-study questionnaire, you asked participants about their prior dieting
experiences. Reasoning that the dieters with previous diet failures might benefit more from
the Smartphone intervention, you make sure that those who failed at dieting at least five times
were placed in the Smartphone intervention. You found that dieters in the Smartphone
intervention benefited more from the mindfulness diet than those in the Website intervention.
Which internal validity threat is most likely occurring in this scenario?
a. Maturation
b. Regression to the mean
c. Selection
d. Mortality
e. Instrumentation
f. Testing
- History
h. Interactions
i. Diffusion
j. No Threat
5). Despite being assured that the Smartphone and Website modules were ready to run, you
found that there were a lot of glitches with the Website intervention program. It was often
sluggish, loading videos and images at a very slow rate. Dieters in this condition often needed
to logout and then log back in to get the Website program to work. Dieters in the Smartphone
intervention did not have the same glitches, with all dieters in that condition reporting that the
modules were very smooth. You found that dieters in the Smartphone intervention benefited
more from the mindfulness program than those in the Website intervention. Which internal
validity threat is most likely occurring in this scenario?
a. Maturation
b. Regression to the mean
c. Selection
d. Mortality
e. Instrumentation
f. Testing
g. History
h. Interactions
i. Diffusion
j. No Threat
6). About two weeks into the study, a new TV program started airing in town. The program
targeted potential dieters using entertaining videos to teach dieters concepts like being aware
of the source of their cravings or deciding whether the consequences of eating unhealthy food
made eating such food worthwhile. Many of the dieters in your study noted that they became
aware of the TV program and decided to check it out since it seemed to overlap with what
they were learning in your study. However, the show mostly appealed to women in your
study, who raved about the program. You found that there was no difference between female
dieters in the Smartphone intervention and female dieters in the Website intervention, though
male dieters seemed to benefit more from the Smartphone intervention than the Website
intervention. Which internal validity threat is most likely occurring in this scenario?
a. Maturation
b. Regression to the mean
c. Selection
d. Mortality
e. Instrumentation
f. Testing
g. History
h. Interactions
i. Diffusion
j. No Threat
7). After completing the pre-study questionnaire, all dieters were randomly assigned to either
the Website intervention or the Smartphone intervention. Once the dieters were assigned to
their condition, they did not socialize with other dieters, which was easy to control as dieters
completed all study questions remotely. Although pre-study measures and post-study
measures had a lot in common in terms of content area (including an emphasis on being
mindful), the questions on the post-study questionnaire differed from the pre-study
questionnaire. The researchers found that dieters in the Smartphone intervention benefited
more from the modules than dieters in the Website intervention. Which internal validity threat
is most likely occurring in this scenario?
a. Maturation
b. Regression to the mean
c. Selection
d. Mortality
e. Instrumentation
f. Testing
g. History
h. Interactions
i. Diffusion
j. No Threat
8). About two weeks into the study, a new TV program started airing in town. The program
targeted potential dieters using entertaining videos to teach dieters concepts like being aware
of the source of their cravings or deciding whether the consequences of eating unhealthy food
made eating such food worthwhile. Many of the dieters in your study noted that they became
aware of the TV program and decided to check it out since it seemed to overlap with what
they were learning in your study. You found that there was no difference between the
Smartphone intervention and the Website intervention. Which internal validity threat is most
likely occurring in this scenario?
a. Maturation
b. Regression to the mean
c. Selection
d. Mortality
e. Instrumentation
f. Testing
g. History
h. Interactions
i. Diffusion
j. No Threat
9). At the start of the study, you noticed that dieters in both the Smartphone condition and the
Website condition had a tough time understanding how to access the modules and how to
answer the questionnaires electronically. As the study progressed over the course of the
month, they became more familiar and comfortable with the electronic platforms. You found
no differences between dieters in the Smartphone intervention condition and the Website
intervention condition. Which internal validity threat is most likely occurring in this scenario?
a. Maturation
b. Regression to the mean
c. Selection
d. Mortality
e. Instrumentation
f. Testing
g. History
h. Interactions
i. Diffusion
j. No Threat
10). Before the study began, a lot of the participants knew each other from their attendance in
a weight loss support group. You did not know this when you randomly assigned them to the
Smartphone or Website interventions. Outside of the study, the participants continued get
together for support, and they started to compare notes about their experiences in your study.
Dieters in the Smartphone intervention learned about the Website intervention and vice-versa.
Since they often met in public and thus did not have access to a computer, they checked out
the Smartphone app instead. Thus, dieters in both the Website intervention and the
Smartphone intervention saw information in the Smartphone app. After comparing the post-
study results, you find no difference between the Smartphone intervention and the Website
intervention. Which internal validity threat is most likely occurring in this scenario?
a. Maturation
b. Regression to the mean
c. Selection
d. Mortality
e. Instrumentation
f. Testing
g. History
h. Interactions
i. Diffusion
j. No Threat
