Comparing Results for the Summative ELPAC
Aggregated results can be downloaded from the public Test Results for California’s Assessments website as well as from CERS (for properly credentialed users).
When making comparisons of the aggregated results across years within a given grade level and content area, it is important to understand that such comparisons are actually comparisons of different groups of students with different traits taking different tests. Generally, there will be more variance in scores from year to year when small numbers of students are tested. Even when the number of students is the same, the group’s composition from year to year may be quite different if student mobility (transiency) is high.
An individual can make comparisons within the same grade level, grade span, and overall score across years. Because the ELPAC results are scaled vertically, scale scores for a test at one grade level may be compared to scale scores at another grade level or grade span; this allows for the comparison of the same student’s performance over time, as well as comparison of student groups at different grade levels or grade spans. Thus, it can be said that proficiency for a given grade level or grade span was higher or lower one year as compared with another. However, caution should be taken when comparing scale scores from different grade levels or grade spans, especially nonadjacent grade levels, because the curricula are different across grades. Comparing scores obtained in different composite scores (i.e., oral language scores and written language scores) should be avoided because the results are not on the same scale.
Two types of comparisons are possible:
- comparing the average scale score within a grade level; or
- comparing the percent of students scoring at each performance level.
An example of how group-level scale scores for 2021–22 may be compared to the 2022–23 scale scores for the same grade level is shown in table 1 using ELPAC overall scores. In this table, hypothetical average scale scores for the ELPAC are compared between 2021–22 and 2022–23 for the students in a particular school. In addition to comparisons for all students, similar grade level–by–grade level comparisons of scale scores may be made for different student groups of interest. It is important to note that the difference in the average scale scores from two years may be a result of different student groups. When comparing the scale scores, it is not meaningful to average the scale scores from different grades for an average scale score for all students because each grade level or grade span has its own scale score ranges.
| Grade | 2021–22 No. of Students | 2021–22 Mean Scale Score | 2022–23 No. of Students | 2022–23 Mean Scale Score | Difference |
|---|---|---|---|---|---|
| Grade 5 | 120 | 1520 | 111 | 1518 | -2 |
| Grade 6 | 100 | 1525 | 124 | 1524 | -1 |
| Grade 7 | 90 | 1535 | 102 | 1537 | 2 |
Table 2 uses ELPAC scores to provide a second hypothetical example of how group-level ELPAC results may be compared. In this example, the percentage of students scoring at Level 4 (Well Developed) is compared between 2021–22 and 2022–23 across grade levels for the same school.
| Grade | 2021–22 No. of Students | 2021–22 % Reaching Level 4 | 2022–23 No. of Students | 2022–23 % Reaching Level 4 | Difference |
|---|---|---|---|---|---|
| Grade 5 | 120 | 20% | 111 | 19% | -1% |
| Grade 6 | 100 | 18% | 124 | 16% | -2% |
| Grade 7 | 90 | 20% | 102 | 22% | 2% |
| All Grades | 310 | 19% | 337 | 19% | 0% |
Note that table 2 also provides a comparison of overall results for the entire school. Because performance levels are standards-based classifications, 2021–22 and 2022–23 results for the entire school may be calculated by averaging across grade levels. The resulting school-level averages may be compared from year to year. However, for each year, these school-level averages should be weighted to reflect the number of students in each grade level. For example, the results for grade five carry more weight in the calculations for 2021–22 than the other two grade levels, but grade six carries more weight in the calculations for 2022–23 than the other two grade levels.
While these examples have made comparisons across only one year, it is important for program evaluation that results be compared across a number of years to verify that the trend is stable. The same sort of table could be used to compare year-to-year results for any test group.