Below are answers to some frequently asked questions. For more on the most recent results, visit the Honesty Gap Fact Sheet.
What is the Honesty Gap?
In May of 2015, Achieve Inc. produced a report, “Proficient vs. Prepared,” which demonstrated that most states have historically misled parents on whether students were proficient in basic math and reading skills. Achieve’s analysis determined that more than half of all states demonstrated a 30-point differential between their reported proficiency rates and the National Assessment of Educational Progress (NAEP), widely considered the “gold standard” in student assessments.
The Collaborative for Student Success labeled this the “Honesty Gap” and its Executive Director, Karen Nussle, explained that it “is the result of a lack of political courage from some policymakers that do not want to be truthful with parents that our students are not prepared for college or the workforce.”
Thankfully, many states had already initiated, and have continued to persevere, with higher academic standards in math and English and also instituted new, modernized tests to measure student progress.
Why does the Honesty Gap exist and what are the consequences?
In most cases, the Honesty Gap results from a lack of political courage: some policymakers have not been willing to admit to parents that student are not prepared for the workforce or college, so they dilute the definition of proficiency to produce better results. This practice is not limited to just a few states; in last May’s analysis, over half the country had discrepancies of more than 30 percentage points.
Ultimately parents and students discover this inferior preparation when they accumulate debt through remedial college courses that they should have mastered in high school or they fail to successfully compete for jobs.
What can be done to eliminate the Honesty Gap?
The good news is that many states have taken steps to eliminate the Honesty Gap, such as: (a) raising standards and expectations in math and English, (b) employing new, higher quality tests that truly measure student competency and that can be compared across school districts and states; and (c) setting new levels of proficiency that are more demanding and aligned with what students need to know to be successful after high school.
What did the Honesty Gap analysis find this year?
Because states have now furnished new tests scores and the biennial NAEP reported results this past October, we are in a position to evaluate each state’s progress towards providing accurate, transparent information to parents. Towards that end, there are three major findings: (1) the “Honesty Gap” has significantly narrowed in nearly half of the states, (2) the number of states with small Honesty Gaps this year has tripled in comparison with last year, and (3) today there are just a handful of states that continue to grossly mislead parents about student proficiency.
Overall, the news is positive. State policymakers are taking the “Honesty Gap” very seriously in their states, which is why 26 states have “Significantly Improved” by narrowing their “Honesty Gap” by 10 percentage points or more.
How did my state do in closing the Honesty Gap?
The “Honesty Gap” has significantly narrowed in more than half of the states. In the past year, 26 states demonstrated an improvement of at least 10 percentage points by closing the “Honesty Gap” in either 4th grade reading and 8th grade math since 2013. The states have “Significantly Improved;” they are: Alaska, Arizona, Arkansas, Colorado, Delaware, District of Columbia, Georgia, Hawaii, Illinois, Indiana, Louisiana, Maine, Maryland, Michigan, Mississippi, Missouri, New Hampshire, New Jersey, New Mexico, Ohio, Oregon, Pennsylvania, Rhode Island, South Carolina, Washington, and West Virginia. These states join three others whose Honesty Gaps have held so close to NAEP (within 5 percentage points) over a two-year period that there was little room for improvement : Massachusetts, New York, and Utah. These three are the “Most Honest” states.
The number of states with small “Honesty Gaps” this year has more than quadrupled in comparison with last year. Achieve’s analysis from last May identified just six states with an Honesty Gap of 15 percentage points or fewer in both 4th grade reading and 8th grade math: Alabama, Massachusetts, Minnesota, New York, Utah and Wisconsin.
Today, there are twenty-five states with that same narrow difference. These states are the “Top Truth Tellers” in both subjects: Alabama, Alaska, Arizona, Arkansas, California, Connecticut, D.C., Georgia, Idaho, Illinois, Kentucky, Louisiana, Maine, Maryland, Massachusetts, Mississippi, New Hampshire, New Mexico, New York, Oregon, , South Dakota Utah, Vermont, Washington, West Virginia, and Wisconsin.
There remain just four states that continue to dramatically mislead parents. Achieve’s May 2015 analysis noted that 18 states exhibited an “Honesty Gap” of 35 percentage points or higher in either 4th grade reading or 8th grade math. Today, that list features just four “Honesty Challenged” states: Iowa, Oklahoma, Texas, and Virginia.
Why do some states, that are administering the same test, have different Honesty Gaps?
This report does not compare the quality of states’ tests, nor does it compare student performance on those tests across states. Rather, it examines the gap between the percentage of students deemed proficient on a state’s assessment and those deemed proficient on NAEP. As a result, states whose students perform better relative to other states that are taking the same test may have larger gaps between NAEP and their state assessments.
There are a different reasons why Honesty Gaps might be dissimilar in states that are administering the same test: Those states might be performing better overall on either the state assessment or NAEP; states might be administering the same assessment but be using different cut scores; or large numbers of students may not have taken the assessment in a particular grade or subject.
Why do the definitions of some categories look different from last year’s report?
- Top Truth Tellers: Last year, Top Truth Teller was defined as being within 15 percentage points of NAEP. This year, because so many states made significant improvement and closed their Honesty Gaps, we raised the bar and now define Top Truth Tellers as those with an Honesty Gap within 5 percentage points of NAEP in either subject. By pointing out states’ Honesty Gaps, we asked them to raise the bar and they did so – so we raised the bar for Top Truth Tellers to match it.
- Honesty Challenged: Last year, Achieve referred to states with the “biggest gaps” as those with Honesty Gaps of 35 percentage points or more. In that report, this included 18 states. This year, only 4 states remain in that category, so we have designated those states as Honesty Challenged.
Are the states designated as “Significantly Improved” the only ones that deserve recognition for narrowing their gaps?
No, they are not the only states that deserve recognition. Last year, many states already had small Honesty Gaps and therefore had less room to move to close their gaps, but deserve recognition for improvement nonetheless. For example, Washington cut its 8th grade Honesty Gap in half, but is not labeled as having significantly improved because it only reduced the gap by seven percentage points. Four out of the five states for which we do not have a previous year assessment score, and therefore cannot show gap improvement, are Top Truth Tellers in 8th grade math for reporting scores within five percentage points of NAEP. We’re proud of the states that have improved and all the states that provide parents with honest information. We believe that continued commitment to high-standards and aligned assessments that more accurately report student proficiency will help all states move into the Most Honest and Top Truth Teller categories – and close those Honesty Gaps for real.
Why do some states show huge changes in their Honesty Gap data, but small numbers of changes in percentage points? For example, NJ’s 8th grade math Honesty Gap shifts from reporting over 70% proficiency under the NJASK assessment to reporting just over 20% proficiency with the PARCC assessment, yet the corresponding text only counts this as a one percentage point change.
Achieve’s Proficient vs. Prepared report calculates how far away the state reported proficiency rates are from the NAEP reported proficiency rates, which we have termed the Honesty Gap. The analysis does not factor in whether the difference is positive or negative. However, we note that those states who under-report NAEP proficiency, like NJ, are doing so because they have adopted an even more rigorous proficiency standard. In other words, in some cases a difference in proficiency rates are a result of the more rigorous expectations, and an effort to ensure more students deemed as proficient are truly prepared for rigorous college-level work and entry-level careers.