Statistical Learning to Identify Student Performance upon Kindergarten Entry

Project Rationale

In traditional education, students take standardized tests to determine academic proficiency according to subject-specific learning objectives assessed at grade level. However, those students at risk of poor performance cannot be flagged as needing additional support or intervention until they receive an unsatisfactory test result; this makes early corrective action nearly impossible. Alternatively, the present study introduces a prescriptive approach that leverages predictive analytics to forecast academic outcomes before the school year begins and remains consistent with classroom teacher observations.

Project Context

The FAU Lab School District (Florida #72) A.D. Henderson University School administers the DIAL-4 readiness screener to all incoming kindergarten students prior to admission and the Progress Monitoring assessments three times throughout the academic year. The Developing Indicators for the Assessment of Learning: Fourth Edition (DIAL-4) has three performance areas of 1) Motor (throwing, stand-hop-skip, building, etc.), 2) Concepts (colors, object naming, counting, etc.), and 3) Language (letters sounds, rhyming, etc.) with behavioral observations and intelligibility of speech (Mardell and Goldenberg, 2011). The Florida Assessment of Student Thinking (FAST) Progress Monitoring (PM) is the current standard for measuring grade-level proficiency across subject areas, namely 1) Early Literacy and 2) Math, with the initial assessment given within the first 30 days of the school year. Scale score results on both are readily available and complete for 117 kindergarten students from two cohorts (2022-2023 and 2023-2024). Additionally, student characteristics are captured by the admissions survey responses from a parent/guardian.

Supportive Literature

Classroom teachers help to define readiness for kindergarten by the skills they prioritize for children entering their grade. A survey by Hustedt et al. (2018) revealed skills such as students’ attention span, social ability, focus on tasks, and knowledge such as memorizing the alphabet or colors. One challenge is the variability of individuals: for example, the cognitive age of the child regardless of being in the same cohort (Elder and Lubotsky, 2009). Ohle and Harvey (2019) address the influences on kindergarten readiness besides academic proficiency, including student factors of previous daycare or preschool experience (Fram et al., 2012), race/ethnicity, socio-economic status, quality of instruction, and age (Elder and Lubotsky, 2009). Firsthand knowledge of these factors that characterize students helps to target efforts for early intervention (Duncan et al., 2018). Predictive analytics has contributed to data-driven decision making. In 2015, van Hartingsveldt et al. were able to predict handwriting proficiency based on kindergarten students’ entrance test submissions. Additionally, McWeeny et al. (2022) explored the relationship of a task known as rapid automatized naming (RAN) to English reading abilities of students and found a strong correlation. The researchers then used RAN as a strong explanatory variable to predict student performance in the English subject. The literature shows that a combination of standardized test results, student indicators, and classroom teacher observations are effective for better understanding academic performance of students.

Research Methods

The DIAL-4 “kindergarten readiness screener” provides a test result in the summer before the school year, which may indicate student performance. There are student characteristics such as age that are also known before the school year begins. The following independent variables were selected.

• Kindergarten Readiness (DIAL-4 Total Score)
• Age
• Demographics (Gender, Race/Ethnicity)
• Economics (Title 1 Status)
• Exceptionalities (ESE)
• Language (English Spoken in Home)

Kindergarten teachers will not have much information on their students in the first 30 days of the school year, after which their classroom observations will inform decisions on how best to instruct students throughout the school year. Hence, the dependent variable which is predicted is as follows:

• PM 1 STAR Early Literacy scale scores encoded as: 1) proficient 70^th to 99^th percentile, 2) borderline 50^th to 69^th percentile, and 3) unsatisfactory 49^th percentile or below.

A training dataset and a test dataset were split as 80% (n=94) used to train the model, while 20% (n=23) were held out for validation to evaluate the model predictions on previously unseen data.

A decision tree classifier was selected to balance the need for a simple model that performs well on small datasets and allows for interpretability. A decision tree will learn a series of recursive splits on the input features into smaller data subsets until a prediction can be made. Figure 1 shows an example diagram of tree-based methods. Once trained, this model was evaluated by overall accuracy and feature importances.

Figure 1. Example Diagram of a Decision Tree

Results

A moderate-to-strong linear relationship between the DIAL-4 Total Score and PM 1 Early Literacy Score was found as a Pearson correlation coefficient, r = 0.53, plotted in Figure 2. Therefore, the DIAL-4 Total Score is useful as an explanatory predictor of the PM 1 Early Literacy Score.

Figure 2. Linear Relationship between DIAL-4 and Progress Monitoring Assessment

Features of greater importance were revealed while training the decision tree classifier. The model learns how excluding a variable would impact accuracy, which indicates the contribution of variables to model performance. Age was found to be a strong explanatory predictor contributing 18.3% importance as shown in Figure 3.

Figure 3. Ranked importance of features in decision tree classification model

The Progress Monitoring assessments were introduced in the 2022-2023 academic year; hence, data were limited. There were more features available than could be used due to the general rule of thumb that there should be 10 times as many features (model input variables) as there are samples. The resulting classification model performed at 75% overall accuracy of predictions on the unseen test set (n=23).

The result that age upon kindergarten entry is a contributing factor to student performance was interpreted through discussions with the kindergarten teachers. Their observations noted great developmental variability at young ages under six years old. For example, teachers have observed how a student may exhibit great differences over just a few months in their own conceptual understanding, physical motor skills, emotional regulation, and social skills such as sharing or taking turns. In Florida, students are eligible for kindergarten at the age of five years on or before September 1. Those students born in August, for example, are at the youngest in their grade and teachers have observed noticeable differences.

Implications

Teachers provide ‘ground truth’ observations on students and have noted the importance of age in kindergarten student development. The findings of the present study augment this by quantifying age as an important independent, predictive variable.

Additionally, the DIAL-4 kindergarten readiness assessment is typically administered to at-risk students only. At A.D. Henderson University School, all incoming kindergartners complete the DIAL-4 assessment in the summer prior to admission. The DIAL-4 Total Score was found to have a moderate-to-strong correlation with Progress Monitoring 1 Early Literacy. Therefore, the decision to use the DIAL-4 as a screening tool to indicate student performance is supported.

Kindergarten teachers also indicated how previous daycare and preschool learning experiences vary widely and add value to their understanding of students upon kindergarten entry. However, this is a current data gap as the School does not collect this information. Further research on its potential impact is needed.

The present study leveraged “teacher in the loop” statistical learning to predict student performance. Teachers as the domain experts and computational analysis tools were found to be complementary. The teachers helped to interpret and validate results while predictive analytics served to support and augment their classroom observations.

References