About the 2007 Survey of State P-12 Data Collection Issues Related to Longitudinal Analysis

The Data Quality Campaign (DQC) and the National Center for Educational Achievement (NCEA), with support from The Bill & Melinda Gates Foundation, conducted a survey about state data systems to determine the number of states that have built the infrastructure to tap into the power of longitudinal data. Similar surveys were conducted in 2003, 2004, 2005 and 2006.

As of September 2007, responses to the survey were received from all 50 states plus the District of Columbia. The survey addressed which of the ten essential elements of a longitudinal student data system would be in place in the state as of the 2007-08 school year. All respondents were given the opportunity afterwards to review and correct their survey responses. Click here to see a copy of the survey.

Policy Questions and Associated Elements

The value of a robust longitudinal data system lies in the ability to answer the many questions that policymakers and educators face when deciding which policies and programs to endorse. The following six examples show how different subsets of the ten essential elements must be in place in order to answer key policy questions:

Which schools produce the strongest academic growth for their students? States must have Elements 1, 3, 4 to answer this question.

In order to be able to measure academic growth, a state must be able to track test scores over time. This requires both a student identifier (Element 1) that follows a student across years and the ability to evaluate test scores in multiple years (Element 3). Information on untested students (Element 4) provides a way to determine how many students are missing test scores in one or both years of data and why.
What achievement levels in middle school indicate that a student is on track to succeed in rigorous courses in high school? States must have Elements 1, 3, 6, 7 to answer this question.

The ability to track a student over time requires a unique student identifier (Element 1), and the achievement levels that are being evaluated over time come from the test database (Element 3). Information on courses taken in high school comes from the course transcript information (Element 6), and information on success on those courses comes from college readiness test scores (Element 7).
What is each school's graduation rate, according to the 2005 National Governor's Association graduation compact? States must have Elements 1, 2, 8, 10 in place to answer this question.

In order to be able to calculate the graduation rate defined in the compact, states must be able to track students over time with a student identifier (Element 1), verify their enrollment each year (Element 2), identify them on an individual basis as a graduate (Element 8), and have a system in place to audit and validate the graduation and dropout data collected each year (Element 10).
What high school performance indicators (e.g., enrollment in rigorous courses or performance on state tests) are the best predictors of students' success in college or the workplace? States must have Elements 1, 3, 6, 7, 8, 9 in place to answer this question.

The ability to track a student over time requires a unique student identifier (Element 1). The achievement levels that are being evaluated over time come from the test database (Element 3). Information on courses taken in high school come from the course transcript information (Element 6), college readiness test scores (Element 7), and/or graduation and dropout data (Element 8). Finally, states must have the ability to connect to the postsecondary system to follow students into college (Element 9).
What percentage of high school graduates who go on to college take remedial courses? States must have Elements 1, 8, 9 in place to answer this question.

States must be able to track students over time (Element 1) and identify them as graduates (Element 8). In addition, states must be able to connect to student data in the postsecondary system to determine which high school graduates have been enrolled in remedial courses in that system (Element 9).
Which teacher preparation programs produce the graduates whose students have the strongest academic growth? States must have Elements 1, 3, 4, and 5 in place to answer this question.

States that maintain information on teachers' preparation, certification, and experience, can match students to teachers (Element 5) and can analyze the academic growth of those teachers' students over time (Elements 1, 3, and 4 - see Policy Question 1) will be able to relate student academic growth to teacher preparation programs.

Questions and Comments

Questions and comments about the survey and/or the results should be directed to Nancy Smith at NCEA (nancy@dataqualitycampaign.org, 512.320.1888).