flowchart TD r1(response1) r2(response2) r3(response3...) r1 --> A r2 --> A r3 --> A A[Rasch analysis] --> B([measure scores])
Illinois’ Essentially Thriving Schools
Ahh, the great state of Illinois, the Prairie State. Home to Abe Lincoln, Deep Dish pizza, and Da Bears. Illinois is also home to the Illinois 5Essentials Survey, a research-backed survey sent to schools across the state. The survey aims to identify five indicators, or essentials, that lead to school improvements such as increased test score proficiency and increased attendance. The 5Essentials Survey measures the student and teacher responses in the following five components found to be critical for school success:
- Effective Leaders: The principal works with teachers to implement a clear and strategic vision for school success.
- Collaborative Teachers: Teachers collaborate to promote professional growth.
- Involved Families: The entire school staff builds strong relationships with families and communities to support learning.
- Supportive Environment: The school is safe and orderly. Teachers have high expectations for students and support students to realize their goals. Classmates also support one another.
- Ambitious Instruction: Classes are academically demanding and engage students by emphasizing the application of knowledge.
GreatSchools, a national nonprofit school information site, gives out the Thrive Award to schools in Illinois that earn superior marks across these five essential indicators. The award methodology is designed to recognize public schools in Illinois that demonstrate positive, supportive learning environments for students and staff.
Let’s dive into the 5Essentials data to explore how schools earn the Thrive Award.
To start, each school that takes the survey is given three types of scores: a measure score, essential score, and essential level.
The Measure score is a 1-99 value that is not included in this data. It is a score that combines the responses across multiple survey questions using Rasch analysis.
The Essential score is also a 1-99 value that is the average of all the measure scores for each of the 5E indicators listed above.
The Essential level is a categorical indicator value ranging from one to five. Each level corresponds to 1-very_weak, 2-weak, 3-neutral, 4-strong, 5-very_strong where each level score is derived from the essential score value.
Let’s start with the essential scores and see how they morph into an essential level.
Here are the distributions of essential scores across the five essentials.
Note: the
instruction_scoresdistribution has a higher peak than the others with more values near the mean (higher kurtosis) and higher mean of 56.3.
From these distributions of essential scores, we can find the essential level for each of the five essentials for each school.
The 5Essentials calculations are set up so that every 20 points marks a standard deviation. Here we label essential score values accordingly to get our essential levels.
Essentially (pun intended), essential score values 1-19 are labeled 1-very_weak, 20-39 values are labeled 2-weak, etc.
Now we have created the essential levels from the essential scores. Here are the distributions of the level scores.
Note: The instructions_levels for this year have more ‘neutral’ and ‘strong’ levels and relatively few ‘very_weak’ or ‘weak’ levels compared to other essentials, probably due to the different distribution noted above.
Now that we know where the data comes from, let’s take a closer look at the GreatSchools Thrive Award methodology.
Our Thrive Award is very similar to the Illinois 5Essentials Survey overall score. The one main difference is we don’t provide varying levels of performance for our award. We simply offer a binary, yes/no, value. The Thrive Award is simple, either you earn an award or you don’t. To earn a Thrive Award, schools must earn a value of 3 or greater for their net score.
A net score is calculated as follows: +1 for ‘strong’ or ‘very_strong’ essential levels, 0 for ‘neutral’ essential levels, and -1 for ‘weak’ or ‘very_weak’ essential levels. Then, for each school, we add up the indicator scores (+1, 0, or -1) from the 5Essentials levels to get the school’s overall net score.
It’s probably easier to look at a few examples, let’s start with a non-winner first.
As we can see with this high school in Park Ridge, they miss out on a Thrive Award because their level values of 1, 3, 4, 4, and 3 correspond with level indicators of -1, 0, 1, 1, and 0.
When we sum their indicators, we get (-1 + 0 + 1 + 1 + 0) = 1. This is below the threshold of 3, so this school does not receive an award.
The good news for this school is that they know what areas they are prepared in and what areas they need to improve. This school can work to improve their low level scores while maintaining the processes in their areas of strength.
Here are a few examples of schools that do earn a Thrive Award.
Notice how Chappell Elementary School barely reaches the threshold net_score of 3 as they have 3 positive and two neutral indicators.
Similarly, Central Junior High School barely reaches the threshold as they have four positive indicators and one negative indicator for a net_score of 3 as well.
So any school with a net score that sums to 3 or greater receives a Thrive Award, everyone else does not.
That’s enough examples for now.
Feel free to investigate other Illinois schools via this interactive map. If viewing on a mobile phone, you may need to rotate your screen into landscape mode to interact with the map.
flowchart TD r1(response1):::prev_step r2(response2):::prev_step r3(response3...):::prev_step classDef prev_step fill:gray r1 --> A r2 --> A r3 --> A A[Rasch analysis]:::prev_step --> B([measure scores]):::prev_step B -->|average measure scores across each essential| C([essential scores])
flowchart TD r1(response1):::prev_step r2(response2):::prev_step r3(response3...):::prev_step classDef prev_step fill:gray r1 --> A r2 --> A r3 --> A A[Rasch analysis]:::prev_step --> B([measure scores]):::prev_step B -->|average measure scores across each essential| C([essential scores]):::prev_step C -->|every 20 points is one standard deviation| D([essential level]) linkStyle 4 fill:gray
flowchart LR C([net score]) C --> D[Thrive] C --> E[5E overall] D --> F[yes] D --> G[no] E --> H[Well Organized] E --> I[Organized] E --> J[Moderately Organized] E --> K[Partially Organized] E --> L[Not Yet Organized]
Non-winning Thrive example |
||
|---|---|---|
net_score: -1 + 0 + 1 + 1 + 0 = 1 |
||
| essential_name | level | indicator |
| leaders | 1 | -1 |
| teachers | 3 | 0 |
| families | 4 | 1 |
| environment | 4 | 1 |
| instruction | 3 | 0 |
Chappell Elementary School |
||
|---|---|---|
net_score: 0 + 1 + 1 + 0 + 1 = 3 |
||
| essential_name | level | indicator |
| leaders | 3 | 0 |
| teachers | 4 | 1 |
| families | 5 | 1 |
| environment | 3 | 0 |
| instruction | 4 | 1 |
Central Junior High School |
||
|---|---|---|
net_score: 1 + 1 + -1 + 1 + 1 = 3 |
||
| essential_name | level | indicator |
| leaders | 4 | 1 |
| teachers | 4 | 1 |
| families | 2 | -1 |
| environment | 4 | 1 |
| instruction | 4 | 1 |
Now that we’ve looked at the Thrive Award and the 5Essentials net score, there are a couple of other interesting ideas to look into:
- How are test scores related to the socioeconomic makeup of student populations?
- Does the 5Essentials net score correlate with positive school outcomes, as it claims?
We will start by looking into how the socioeconomic status of a school’s student population is correlated with test scores. We will use a school’s percent of students who are on the Free and Reduced-Price Lunch (FRL) program as our economic indicator to compare with various GreatSchools ratings.
Secondly, we will look at the 5Essentials net score to see how it correlates with positive school outcomes.
Let’s look at these two topics.
Test Scores and Socioeconomic status
The GreatSchools 1-10 Summary Rating is based on up to four themed ratings, each of which is designed to show different facets of school success. The Test Score Rating calculates how well a school performs on state standardized tests compared with other schools in the state. However, there is a problem with using test scores as the only measure of school success. This is because test scores are strongly negatively correlated with the socioeconomic status of the school’s student body. Test scores tell us something about the proficiency of the students in the school but they don’t tell the whole story.
Using school percent of FRL and comparing it to GreatSchools’ test score percentile rating, we can see that there is a fairly strong negative correlation (Pearson R correlation coefficient r = -0.76) between the two variables. Unfortunately, we can see as FRL percent increases test scores tend to decrease.
Using only test scores includes the bias of socioeconomic status of the student body. That is one of the reasons why GreatSchools calculates and weighs other ratings more heavily in our methodology.
Here are some other similar plots for our Summary, Student Progress, College Readiness, and Equity Ratings. You can learn more about our ratings on our Ratings Methodology page. Most of the ratings use test scores in some way while attempting to limit the effects of bias in state proficiency tests.
Notice that the Test Score Rating has a vastly larger negative correlation. We can take from this that using test scores alone as a metric includes the inherent bias associated with test scores and socioeconomics of the student body.
For these reasons it is good to take into account all information and multiple ratings such as Summary, Equity, and Student Progress Ratings when looking at schools.
Does the 5Essentials Survey align with measures of school outcomes and improvement?
The good news about the Thrive Award is that it does not take into account test scores at all when calculating the net score. Decoupling a school metric from test scores can offer other insights into the quality of a school that test scores may not.
We can see that there is really no pattern to the scatterplot and the correlation is weak between FRL percent and net score.
Not only does the net score avoid the socioeconomic bias of test scores, but it also correlates with GreatSchools’ measures of successful schools.
From the plots, we can see the net score has a positive correlation with GreatSchools ratings. Schools with high net scores tend to have higher GreatSchools ratings, too.
An alternative plot that tells the same story is offered via these boxplots. We can see that as the net_score increases, so does the median rating value.
Meaning that, according to both the 5 Essentials Survey and GreatSchools ratings, we can see that schools that are well organized on the five essentials are in fact set up to improve their outcomes.
That was a lot! If you’re still here, thanks for reading all the way through. We’ve seen how the Illinois 5 Essentials Survey and the corresponding net score is calculated. We also saw how the Thrive Award is created from the net score.
Then, we looked into how state proficiency tests are correlated with economic indicators such as FRL percentage. Our exploration used Illinois data only, but this trend is similar nationwide. Finally, we were able to see how the 5 Essentials data is less correlated with low-income indicators and aligns with GreatSchools’ measures of success.
Thanks for scrolling along and hopefully we were able to learn something together. That’s enough for now, I’m going to take a break to order a Deep Dish pizza and rewatch The Last Dance.