Examining the Processes and Outcomes of Reading Comprehension (EXPO-RC)
Purpose
The purpose of this study is to better understand the test-taking behavior of 3rd, 5th, and 8th-grade students. Tests of reading comprehension (RC) are meant to measure the processes engaged in by readers when reading for understanding, yet research suggests that two individuals might engage in very different processes and earn the same score on a single test. Furthermore, the same student might earn very different scores on two different tests, with one test score suggesting the student has comprehension deficits and the other suggesting he does not. These differences occur because students’ test-taking strategies vary both within a testing session, and tests vary in their characteristics (e.g., text length, text type, question type) and the reading processes that they measure.
Project activities
The team will complete four studies using eye-tracking procedures to record students’ online reading and question reading and responding behavior. In the first study, the researchers will collect both online and offline measures of RC within the same study. This will allow for the assessment of the relationship between the processes and products of RC. Across the three remaining studies, the team will manipulate variables that might alter student’s reading behavior, response accuracy, and the extent to which their response accuracy reflects their ability to develop a coherent representation of the text.
Products
Researchers will produce preliminary evidence of malleable factors such as test-taking strategies and test formats that are associated with assessing elementary students’ comprehension skills, and information about how those factors differ as a function of students’ reading achievement, working memory, and motivation. The team will also produce peer-reviewed publications.
- Sponsor
Institute of Education Sciences
$1,392,920 - Principal investigator
Scott Patrick Ardoin
Professor and Department Head, Educational Psychology - Co-principal investigator
Katherine S. Binder
Mount Holyoke College - Active since
July 2017
Abstract
Setting
Three school districts in Georgia will participate in the study.
Sample
Participants include approximately 700 third, fifth, and eighth-grade students. Students without special education identification and whose first language is English will be recruited. Currently, 15% of students enrolled in the participating school districts are Hispanic or Latino. The racial composition of the sample is anticipated to reflect that of the school districts: 31% Black or African American, 48% White, 4% two or more races, 2 % Asian, less than 1 % American Indian/Alaska Native, and less than 1% Native Hawaiian or Other Pacific Islander.
Intervention
The research team will explore malleable factors such as test-taking strategies and test formats that are associated with assessing elementary students’ comprehension skills, and information about how those factors differ as a function of students’ reading achievement, working memory, and motivation. Results from these studies will have implications both for reading comprehension assessment and instruction.
Research Design and Methods
Researchers will carry out four studies to meet the project goals. In the first study, the team will collect both online and offline measures of RC. They will administer texts of different lengths and types as students’ eye movements are recorded. A similar text administration technique will be followed in each of the three following studies, where students will read and respond to test questions online while their eye movements are simultaneously being collected. In the second study, the researchers will manipulate the test-taking strategy that students must employ. In the third study, the team will vary the question response format (i.e., multiple-choice or constructive responses) and whether students have access to text when responding. In the fourth study, the researchers will manipulate the number of questions presented per screen as well as whether students are asked to provide evidence supporting their answers.
Control Condition
There is no control condition in this exploratory study.
Key Measures
Researchers will measure reading achievement (Woodcock Reading Mastery Tests, Third Edition), working memory, and motivation (Motivation Assessment Scale). Students’ eye movements will be recorded while they are reading texts and reading and responding to associated reading comprehension questions. The team will measure first fixation duration, gaze duration, fixation count, and total fixation duration.
Data Analytic Strategy
Researchers will carry out mixed analyses of variance and cross-classified hierarchical linear models to answer multiple research questions across the four studies.