Using the Quality roll-up indicator at the local level in your team

• Consider the impact of missing data.

If your team did not submit data for all 14 of the indicators included in the calculation, values for the missing data were estimated randomly to allow you to get a score for the quality roll-up indicator.  Using random values ensures that the membership-wide scores which are being used to demonstrate the value of teams at an aggregate level are solid estimates.  At the local team level, quality roll-up scores based on these random values are not as robust as scores based on complete data.  Teams with incomplete data for the quality roll-up score may therefore want to access more data prior to drawing definitive conclusions about their local score.

• Consider the role of the relationship with patients.

The quality roll-up indicator is intentionally weighted according to what matters most to patients in their relationship with primary care providers.  As the table below shows, some indicators are more important to this relationship than others.  You may wish to focus your improvement efforts on the indicators that are most important to patients. 

• Consider thresholds for performance.

AFHTO members have identified thresholds for performance on each of the 14 indicators included in the quality roll-up indicator. Indicators that are not yet meeting the lower threshold are areas to give priority consideration for quality improvement. Indicators scoring within the minimum and maximum range are performing within accepted norms but have room for improvement. Indicators scoring above the maximum threshold tell you that your team can look to other priorities for improvement efforts. 

[table id=87 /]

[Original Post: January 27, 2016] D2D might show you how your team stacks up.  And it might be hard for your team to take action on the data in D2D.  You might need more current, local, provider or patient-specific data to figure out what your team could do to make things better.  Here are some ideas to help you and your team drill down into data that can kick start some PDSAs or other efforts to improve quality.  Ideally, you would do the drill down in advance, preferably in collaboration with an influential clinician on your team.  This will give your clinicians something to talk about with their peers right away when you start looking at D2D. In the videos below, Carol Mulder provides an orientation to the D2D data review platform. The first provides general information about the core indicators, and the other provides a more detailed orientation geared to the needs of Board Chairs and EDs or Admin Leads. Read on to learn about actions you can take regarding the three categories of indicators: Patient Experience, Administrative (ICES/HQO), and EMR-Based. 

Patient experience indicators

Patient experience data is probably the most current of all the indicators in D2D.  However, it may still be useful to drill down into patients of a specific program or provider or who were targeted with a particular intervention.  This can help your team get a more local immediate sense of how things are going and increase interest in doing more to improve patient experience.  Ideas for drill down include the following:

• Track the next 10 (or other small number) patients that come in for a particular program or provider or do 2 weeks of patient surveys in the next month. This will give your team a sense of progress from D2D (ie are they holding at about the same level as in D2D or getting better/worse?) and also might help focus on specific groups or interventions.
• Ask a small group of patients one of the questions from D2D that was NOT in your survey. Teams may be more interested in indicators in D2D if they have local data – but they may want to see roughly how they are doing before going to the work of revamping their patient experience survey.
• Talk about sample size (see resources on web). Some teams are still working under the burdensome impression that they need to sample really large numbers of patients to get an idea of patient experience.  This is not true.  The more patients you sample, the more precise your results are.  However, you can get within 5 or 10% of the ‘true’ level of patient experience with relatively small sample sizes.  Often this is good enough for the purposes of tracking progress.
• Consider changing your patient experience survey process from once a year to an ongoing cycle using some of the new tools integrated with your EMR to enhance and track patient experience. Check out how other teams administer patient surveys.  You may also want to consider international and provincial patient survey initiatives
• Work with your patients to see what they think is important. AFHTO and HQO collaborated in Jan 2016 on a series of workshops with QIDSS and patients from health teams.  There may be some ideas coming out of those sessions that can help your team work with your patients to use your patient experience data to fuel improvements.  Check with your QIDSS or Carol Mulder for more information on these sessions.

Administrative (ICES/HQO) data indicators

Indicators based on administrative data tend to be the oldest of all indicators in D2D.  Improving the timeliness of administrative data is a priority for AFHTO and HQO and others.  And in the meantime, there are things teams can do to use these “old” data to fuel current, local efforts to improve.  These include the following:

• Get at your hospital data: Yes, there are provincial, information-technology-based solutions under way to improve access of primary care providers to hospital data. And your team can get hospital data now even while you are waiting to be connected to more automated, provincial solutions.  Check with your QIDS Specialists for ideas on how to get data from your local hospitals.
• Track the next 10 or 20 (or other small number) of hospitalized patients or patients who have been to the ER. A temporary manual process to check into a small number of patients may be more possible in the short term and will serve to give you some current, local information about what is REALLY happening with your team’s readmission or follow-up rates.  This data will not necessarily be comparable to what is in D2D but might be enough to start conversations in your team about what (if anything?) you can do to improve coordination of care for your patients as they go to and come from the hospital.
• Get current cancer screening data from your EMR: The QIDS Specialists have developed standardized EMR queries for cancer screening.  Try them.  Now that they are developed, they should take very little time to run on an ongoing basis (rather than just once a year for reporting purposes).  The data might not be directly comparable to what is in D2D (because it is from a different time-period and may have more information about patient eligibility for screening).  However, it will give you a sense of how your team is doing over time.  More importantly, have a list of specific patients that might be overdue for screening gives your team something concrete to do now about something they care about (ie patients).
• Sign your physicians up for monthly screening reports via CCO SAR. Once they get through the sign-up process, most physicians agree that these reports are very helpful, especially if you or they have trouble getting or trusting your EMR data for cancer screening.

EMR-based indicators (e.g. childhood immunization, diabetes, smoking status)

D2D indicators based on EMR data are relatively current.  And because EMRs are usually current up to the minute, your team can get even more timely, ongoing data for these indicators to guide efforts to improve on these indicators.  Ideas to increase the value of EMR data beyond the values reported in D2D include the following:

• Drill down to the patient or provider level: The queries to get EMR data for D2D are usually run at the patient level so you may just need to run the same D2D query again and look at the results BEFORE rolling them up the team level (as you do for D2D reporting). This will tell you and your team exactly which patients of which providers could benefit from interventions like a phone call or reminder about an appointment.  As with the cancer screening example above, this gives your team something concrete they can do to make a difference in the lives of their patients now.  This is invariably a compelling reason for teams to participate in the measurement and QI process.
• Work to improve the quality of your EMR data. Check out how your team is doing on the D2D data quality indicator relative to others. Consider picking a point of clinical interest with your team and working to improve the quality of EMR data in this very limited are.  Check out the experience of your peers in cleaning up the data and getting people interested in doing that –consider hiring a student to help you clean up your data (see suggestions in this handbook for cleaning up your roster and smoking/alcohol status).