Assessing the Medicare Advantage Star Ratings

Articles

Since Star Ratings were introduced a decade ago, average MA plan performance has improved, quality standards have risen, and more people are enrolled in higher-quality plans.

Every October since 2009, the Centers for Medicare and Medicaid Services (CMS) has released comprehensive data on Medicare Advantage (MA) health plan performance through its Star Ratings program. The program’s goals are to incentivize health insurers to improve their MA plans and encourage consumers to enroll in high-quality plans. To investigate whether these goals are being met, we analyzed CMS’s data for 2018 and previous years.

Our key findings:

  • On average, the plans’ scores have risen substantially, both overall and on the 22 individual quality measures that have been used consistently since the program’s inception.1
  • The percentage of MA members enrolled in high-performing plans (those with 4 or more Stars) has also increased significantly.
  • Average Star Ratings have improved even though the standards used to assess plan performance have been tightened in recent years, which suggests that plan quality is rising faster than the numerical scores indicate.

In this paper, we present our analyses of MA plan performance on Star Ratings. In addition, we briefly compare the financial performance of MA plans against fee-for-service (FFS) coverage, and discuss two factors that may be confounding Star scores. We then offer recommendations for how insurers can achieve additional improvements in Star Ratings and refine their strategies before the next ratings release in Fall 2019.

Overall ratings have risen

In 2011, CMS introduced overall scores for MA Prescription Drug (MA-PD) plans, which account for the majority of all MA contracts. Between 2011and 2018, the national average of the enrollment-weighted Star Ratingsforall MA-PD plans increased from 3.18 to 4.06, which reflects the plans’ overall improvement in performance even after membership shifts and changes to scoring criteria were accounted for (Exhibit 1).2

Since 2011, overall Medicare enrollment has grown by 17%,3 and MA penetration4 as a share of that enrollment has increased from 23% to 33%.5 In addition, the percentage of MA-PD members in plans with 4 or more Stars has risen from 24% to 73% (Exhibit 2). MA program enrollment reached 18.5 million in 2017, compared with 9.7 million in 2008,6 and the number of enrollees receiving the extra benefits associated with quality bonuses (e.g., reduced premiums, additional vision or dental coverage, and other additional benefits) swelled from 2.8 million people in 2010 to 13.2 million in 2017.

The overall scores are based on the scores assigned to specific measures, many of which have been modified since the program’s inception.7 However, 22 of the measures have been used consistently since 2009. (We call this set the consistent measures.) After calculating the enrollment-weighted average scores for each of the consistent measures, we found that all but two have risen over time (Exhibit 3).

Digging deeper into the ratings

Since the Star Ratings program’s inception, the financial incentives offered to MA carriers have varied, and CMS has periodically altered its ratings methodology (e.g., by raising the cut points for various scores). To investigate how these factors may have influenced Star scores, we conducted three sets of analyses, using data for the 22 consistent measures.

Impact of financial incentives

To investigate the possible impact of financial incentives on plan performance, we looked at the average scores for the consistent measures before, during, and after CMS introduced its quality bonus payment (QBP) demonstration. (For more details about the financial incentives, see the sidebar below.) We used the following time periods:

  • Years 2009–11, before financial incentives for quality were announced
  • Years 2012–13, after the financial incentives were announced, until the end of the QBP demonstration (i.e., during the second and third years of thedemonstration)
  • Years 2014–18, after the demonstration ended and the QBPs defined in the Affordable Care Act were reinstated

Our results show that average overall scores for the consistent measures have risen 16.7% since 2009. The rate of improvement differed among the three periods, however (Exhibit 4).

Impact on measure categories

We dug even deeper to analyze the rateof improvement among the four categories of measures (as defined by CMS). We found that, between 2009 and 2018, average aggregate raw scores for the consistent measures rose in three categories: outcomes, process, and access (Exhibit 5).8 However, there was little change in scoring for patient experience.

In the three categories with increased scores, the rate of improvement differed by time period. The greatest improvement in access metrics occurred in 2012 and 2013. The largest increases in outcome and process scores occurred between 2014 and 2018. These findings suggest that investments in performance may take time to pay off.

Impact of cut-point changes

Throughout the program’s history, CMS has periodically updated the performance levels required to achieve the rating for each measure.9,10 As a result, performance that might have achieved 4 Stars one year might be given only 3 Stars the next year.11 We therefore sought to understand the extent to which cut-point changes such as these were affecting Star Ratings.

We found that the cut points needed to achieve 4 Stars had increased for 8 of the 14 consistent measures that are scored using a single metric. Despite the increasing difficulty of getting a score of 4 Stars on those measures, the enrollment-weighted average Star Ratings increased for all but “improving or maintaining physical health” and “reducing the risk of falling” (Exhibit 6).

To further explore the impact of tightening standards, we applied the 2009 cut points for all of the 22 consistent measures to the contracts covered by the 2018 ratings to see how many MA enrollees would be in plans with at least 4 Stars if none of the cut points had changed. The result: about 1.2 million more beneficiaries (about 6.5% of total MA beneficiaries) would have been covered by plans with 4 or more Stars.12

These results indicate that many MA enrollees are receiving better care today. For example, approximately 76% of beneficiaries aged 50–75 received an appropriate screening for colorectal cancer in 2016, whereas only 54% of such enrollees were screened in 2007. It is possible, however, that some of the increases may simply reflect better reporting and documentation. (Again, more details can be found in the sidebar below.) Furthermore, whether increased screening rates are indicative of higher quality of care is a matter of some debate.

MA plans have also become more efficient

In a separate analysis, we found that MA plans have become more efficient than FFS coverage at providing Parts A and B benefits (Exhibit 7). Currently, the MA plans deliver Part A and B benefits at 90% of the costs of FFS coverage (or 92% if differences in risk coding are fully accounted for), compared with 102% in 2009.

Even though changes in bidding benchmarks were phased in from 2012 to 2016, MA plans continued to manage costs and bid below the declining benchmarks.13 This finding indicates that, in addition to the quality improvements previously discussed, MA plans have also improved their efficiency in recent years.

Confounding factors

The Medicare Payment Advisory Commission and others have identified several possible isues with the Star Ratings program, including the potential for plan consolidation to “erode the validity of the Star Ratings system as a measure of plan performance in a given area.”14 The quality of data collection has also been raised as a concern. We address both issues below.

Plan consolidation

Star Ratings are calculated at the contract level and apply to all plans that fall under the contract, regardless of quality differences between the individual plans. A plan that gets consolidated into an existing contract receives a Star Rating and quality bonus based on the score of the existing contract, even if its own score had been lower.

To understand the impact of plan consolidation, we looked at the 271 plans with less than 4-Star scores that were consolidated into other contracts between 2017 and 2018. A total of 1.8 million people had been enrolled in these lower-quality plans; 76% of them were covered by 4-Star plans in the 2018 ratings simply because of consolidation (Exhibit 8).

Thus, plan consolidation does have an impact on average enrollment-weighted Star Ratings, but it is unlikely to be the only factor contributing to the rising scores. (For more details on why this is the case, see the sidebar below.) We analyzed the seven contracts from the 2015; ratings year that had plans from the previous year folded into them. Of the seven, four were given 4 Stars in 2015; the remainder received 3 or 3.5 Stars. By 2018, three of the seven contracts had improved their ratings, three maintained their ratings, and one experienced a decline (Exhibit 9). This finding suggests that consolidation will not provide a long-term Star Ratings boost unless the underlying performance of the consolidated plans also improves—investments in quality must continue if the quality of the underlying plans is to continue to rise.

Caveat: Given the increased rate of consolidation in the past few years and the two-year gap between plan performance and ratings assignments, it remains to be seen if the same pattern will hold true for plans that have been consolidated since 2016. Furthermore, CMS has noted that the practice of “masking low quality plans under higher rated surviving contracts” confuses beneficiaries, and it recently proposed a new rule, under which Star Ratings for surviving contracts would be calculated based on the average enrollment-weighted scores of the consolidated contracts.15

Better data collection

For some measures, the raw scores may now be higher than they were historically because many MA plans have improved their ability to measure and track the factors that influence those scores (as well as their overall payments and Star Ratings). However, our research suggests that the recent score improvements are likely to be the result of both better documentation and better performance—e.g., actual increases in the rate of screenings or other interventions. (For more information about changes in data collection, see the sidebar below.)

Options for further improvement

Overall, these results show that, since the advent of the QBP system, MA plan quality (as measured by Star Ratings) has improved while controlling or reducing costs. Experience with Star Ratings indicates that health insurers can adjust and improve their performance over time based on incentives built into CMS-determined metrics.

Going forward, health insurers will need to continue investing in their Stars programs to maintain their current standard of quality and meet ongoing changes in cut points across measures. Insurers can improve their scores in two key areas:

Outcome and process measures. Insurers should continue to strive for excellence in these metrics—as cut points in these areas evolve, plans will have to keep up to stay competitive. Health Effectiveness Data and Information (HEDIS) scores affect both categories and thus should be an ongoing focal point for improvement. The improvements could require long-term investments and accountability across several areas of the enterprise, but insurers should see payoffs from increased performance within three to five years.16

Patient experience and access scores. Many insurers will also need to increase their focus on patient-experience and access measures, since scores for these measures have seen less improvement than other domains have. Scores from the Consumer Assessment of Healthcare Providers and Systems (CAHPS) surveys could be a good place to start—many are in the patient-experience category and have historically been less of a focus for insurers.  Concerted efforts to improve in this area could enable insurers to outperform competitors and provide a source of product differentiation in the form of enhanced member experience.

Some measures will remain difficult to achieve 4 Stars on, given cut-point evolutions, but continual improvements over time through dedicated investments, resources, and accountability could keep high-performing plans ahead of the competition. Health insurers can achieve such improvements in a few ways, as we discuss below.

Background: Evolution of the MA Star Ratings program

Any attempt to assess the impact of Star Ratings on the performance of MA plans overall must take into account three important variables: differences between the financial incentives offered in various years, the impact of plan consolidation, and requirements about data collection. We discuss each of these briefly below.

Financial incentives

MA Star Ratings have been published since 2009 (thus affecting plans for 2010) and were originally used as a quality indicator to help MA beneficiaries make informed enrollment decisions.1 However, the ACA added a financial incentive designed to encourage plans to increase their MA Star Ratings.2 It authorized bonus payments (in the form of a percentage increase in payment benchmarks and rebate amounts) to MA contracts with 4-, 4.5-, and 5-Star Ratings. The payments would be phased in over the 2012–14 plan years.

In addition, the CMS launched a quality bonus payment (QBP) demonstration for the same 2012–14 plan years.3 CMS extended the QBPs to include 3- and 3.5-Star contracts and complemented the phase-in of QBPs to 4-, 4.5-, and 5-Star contracts. Since the end of the demonstration in 2014, however, only 4-, 4.5-, and 5-Star contracts have been re­ceiving QBPs, as the ACA initially intended. Given that the financial incentives for plans differed in each of these phases, we analyzed data from the three phases separately.

Plan consolidation

In the past few years, the rate of plan consolidation has increased: 724 plans changed contracts between 2015 and 2018, compared with 361 between 2011 and 2014. In 2017, approximately 1.3 million additional enrollees were in plans receiving a bonus because their plans moved from a non-bonus contract to a bonus-eligible contract.

Star Ratings are calculated at the contract level and apply to all plans that fall under the contract, regardless of quality differences between the individual plans. A plan that gets consolidated (“cross-walked”) into an existing contract receives a Star Rating and quality bonus based on the score of the existing contract, even if its own score had been lower. The existing contract’s score is not affected by the quality of the cross-walked plan for two years (only then will the data from that plan begin to affect the score). Thus, a cross-walked plan may temporarily have a performance rating that does not reflect an actual increase in quality. However, if the performance of the cross-walked plan does not improve in the two years after consolidation, the score for the overall contract will suffer, and the health insurer may not be able to earn a quality bonus.

Data collection

Many vendors exist today to help MA plans improve quality measurement tracking and provide comprehensive care management and analytics. Since 2011, CMS has required MA plans and Part D sponsors to contract with external vendors to conduct independent data validation audits as “safeguards…to protect the Star Ratings from attempts to inflate performance or mask deficiencies.”4 CMS’s current policy is to reduce the rating of an individual measure within a contract to 1 if the measure’s data is incomplete, biased, or erroneous.5 CMS also includes an ongoing coding intensity adjustment to account for potential increases in the average risk score of MA beneficiaries, if it significantly outpaces FFS risk profiles.6 Although better data collection could have played a role in overall raw score improvement, it is likely not the only factor impacting quality improvements.


  1. MedPAC. Report to the Congress: Medicare Payment Policy. Chapter 3. March 2008.
  2. Health Care and Education Reconciliation Act of 2010. Pub. L. No. 111-152, (Section 1102, 124 Stat. 1029, 1040). March 30, 2010.
  3. CMS. Medicare program; Changes to the Medicare Advantage and the Medicare prescription drug benefit programs for contract year 2012 and other changes. Federal Register. April 15, 2011.
  4. CMS. Announcement of calendar year (CY) 2018 Medicare Advantage capitation rates and Medicare Advantage and Part D payment policies and final call letter and request for information. April 3, 2017.
  5. CMS. Announcement of calendar year (CY) 2018 Medicare Advantage capitation rates and Medicare Advantage and Part D payment policies and final call letter and request for information. April 3, 2017.
  6. Kronick R, Welch W. Measuring coding intensity in the Medicare Advantage program. Medicare & Medicaid Research Review. 2014.

Improved digitization and analytics

Improvements in these areas could help health insurers engage beneficiaries and physicians, reduce the administrative burden of managing care, and strengthen underlying program performance. For example, insurers could develop and promote use of these tools:

  • Better consumer digital interfaces could provide pricing transparency and scheduling information or allow consumers to more directly and easily access care.
  • Advanced analytics could identify and proactively close member care gaps, including better physician web portals and digital interfaces that provide strategies to address these issues.

Better provider engagement

By engaging more effectively with providers, health insurers could further improve patient experience and outcomes. Investments in patient-experience improvements have traditionally been low, given the difficulty of influencing physicians. Nevertheless, investments in this area represent an opportunity for further improvements in underlying Stars performance. Payers could achieve this in a few ways:

  • Invest in opportunities to change physician opinion and incentives to encourage and empower behaviors that could improve quality and Stars performance. For example, physicians could be included in programs that provide competitive incentives to close the care gaps.
  • Host educational events to educate physicians on the Stars program, including roundtables in which best-practice physicians share their experiences.
  • Develop physician- and office-based materials to encourage performance on Stars measures. For example, insurers could provide physicians with buttons that say, “Ask me about your flu shot” or provide training for office staff to encourage patients to inquire about critical prevention measures.

More effective data collection and coding

Improvements here could help insurers identify new sources of value within the existing population:

  • Natural language processing or other artificial intelligence mechanisms could help minimize coding errors.
  • Better tracking of Stars performance could help identify and address areas of underperformance, which could also provide incremental benefits by improving risk revenue.

APPENDIX

Changes in Star Ratings measures

Exhibit A contains a complete list of the measures used for the 2018 Star Ratings.

The following measures were affected by a change in rating reporting requirements and/or methodology since 2009

  • Colorectal cancer screening: Age group moved from 50–80 (2009) to 50–75 (2018)
  • Annual flu vaccine: Measurement period changed from September–December (2009) to July–December (2018)
  • Osteoporosis management: Age group changed from “67 and older” (2009) to 67–85 (2018)
  • Diabetes care, eye exam: Reporting requirements changed from diabetic MA enrollees who had a retinal or dilated eye exam (2009) to diabetic MA enrollees 18–75 who had an eye exam (retinal) (2018)
  • Diabetes care, kidney disease monitoring: Re­porting requirements changed from dia­betic MA enrollees who either had a urine micro­albumin test or received medical attention for nephropathy (2009) to diabetic MA enrollees 18–75 who had medical attention for nephro­pathy (2018)
  • Diabetes care, blood sugar controlled: Age group changed from “diabetic MA enrollees” (2009) to “diabetic MA enrollees 18–75” (2018)
  • Getting appointments and care quickly: Additional question added in 2018

As discussed in the main article, changes to the cut points for many measures have also been made since the Star Ratings program started. To understand the impact of these changes, we looked at how cut-point changes affected one measure, improving or maintaining physical health. We found that the cut points needed for 2-, 2.5-, 3-, and 3.5-Stars scores increased significantly more than did the cut points for 4, 4.5, and 5 Stars (Exhibit B). If the 2009 cut points for this measure were still in use today, all MA enrollees would be in plans with at least 4 Stars.

Glossary

On October 11, 2017, CMS released data on MA contracts and plans offered for 2017. Using this data and data from previous years, McKinsey calculated changes in enrollment-weighted Star Ratings averages over time and assessed performance over time on specific metrics to perform the analyses included in this paper. Data on enrollment and Star Ratings standards was used to conduct analysis on contract consolidation and cut-point changes. Specific definitions of enrollment-weighted average and enrollment are provided below.

Enrollment-weighted average. McKinsey calculated enrollment-weighted averages by taking the total number of enrollees in contracts and plans for 2017, assigning higher weights to plans with higher enrollment. The results were used to calculate the enrollment-weighted averages for 2018 Star Ratings. The enrollment-weighted average demonstrates Stars performance among carriers and products with the highest level of participation and thus allows us to understand overall trends.

Enrollment. The October 2017 summary Star Rating data from CMS was used as a filter for the April 2017 CMS MA enrollment by state, county, and contract.

Therefore, enrollment in contracts that did not exist in the October 2017 ratings file are not included in the enrollment data in this paper.

Related papers from McKinsey

  • Assessing the 2017 Medicare Advantage Star Ratings (October 2016)
  • Improving acquisition and retention in Medicare (March 2016)
  • Assessing the 2016 Medicare Advantage Star Ratings (October 2015)
  • 2016 Medicare Advantage rates: Perspectives for payors (April 2015)
  • Assessing the 2015 Medicare Advantage Star Ratings (November 2014)
  • 2015 Medicare Advantage rates: Perspectives for payors (April 2014)
  • Medicare Advantage: Dispelling market misperceptions (February 2014)

For questions about the methodology used for the calculations in this article, contact the authors.

The authors would like to thank Adam Rudin, Michael Sturm, and Jacqueline Wingfield for their assistance with the analyses used in this article.

  1. Overall scores were considered only for MA plans that include prescription drug benefits. However, these plans account for the vast majority of all MA contracts.
  2. Because of data reporting constraints, the ratings calculated in a given year reflect the performance of MA plans two years earlier (e.g., 2018 Star Ratings, published in the fall of 2017, reflect data from 2016).
  3. CMS. Medicare Enrollment Dashboard. Data as of January 2018.
  4. The calculation of MA penetration includes MA, MA-PD, special needs plans, and other MA plans, such as cost plans and demonstration plans.
  5. Jacobson G et al. Medicare Advantage 2017 spotlight: Enrollment market update. Kaiser Family Foundation. June 6, 2017.
  6. MedPAC. A Data Book: Health Care Spending and the Medicare Program. Chapter 9. June 2017.
  7. The appendix contains a complete list of the 2018 metrics by category.
  8. Based on changes in enrollment-weighted average raw performance scores.
  9. CMS. Medicare health & drug plan quality and plan performance ratings 2015: Part C & Part D technical notes. April 10, 2015.
  10. MedPAC. Report to the Congress: Medicare Payment Policy. Chapter 12. March 2016.
  11. For most measures, CMS now defines the Star cut points based on the natural gaps that arise among clusters of plans based on the distribution of scores. (CMS. Trends in Part C & D Star Rating measure cut points. November 8,2016.)
  12. This analysis calculated the total number of beneficiaries in 4+ Star plans on a given metric using 2018 raw data and 2009 cut points. For a view of how this analysis worked for one example measure, see the appendix.
  13. Congressional Research Service. Medicare Provisions in the Patient Protection and Affordable Care Act (PPafford): Summary and Timeline. June 30, 2010.
  14. MedPAC. Report to the Congress: Medicare Payment Policy. Chapter13. March 2017.
  15. CMS. Medicare program; Contract year 2019 policy and technical changes to the Medicare Advantage, Medicare Cost Plan, Medicare Fee-for-service, the Medicare Prescription Drug Benefit programs, and the PACE program. November 28, 2017.
  16. A rule finalized by CMS in April 2018 establishes that substantive changes to the Star Ratings program will go through rule-making rather than subregulatory updates in the annual call letter. This change underscores the importance of considering long-term investments, given that Stars program changes could take longer to implement under the new regulation.