For technical notes, please see page 22 of the Data Dictionary.

Interpretive Notes

Tips to help you understand the data and put it in context.

• Virtual rostering assigns patients to the primary care physician that provided the highest dollar amount of services within a defined set of primary care services. Your team may not be aware of which patients have been virtually rostered to them. Therefore, you may erroneously think that these patients are not “your” patients. As a result, your team’s sense of how many of “your” patients were screened may be different than shown in D2D.
• Screening tests performed in hospital laboratories or paid through alternate payment plans are not currently incorporated into this measure. Actual performance on this measure for teams that use hospital laboratories is therefore likely higher than the level presented in D2D.
• Please note: for colorectal cancer screening, a small proportion of FOBTs performed as diagnostic tests for other conditions besides cancer could not be excluded from the analysis.
• Inaccurate recording of exclusion criteria may result in an under-estimation of screening rates as patients who are not eligible for screening would be erroneously included in the denominator, artificially driving the observed rate down.
• The current measure does not consider patient choice in screening and therefore might reflect an under-estimate of the screening “interventions” (i.e., consultations/advice to undertake screening) by the team.

Steps to Improvement

Concrete steps you can take to improve care, based on your data. Assuming you have established that the data are good enough to direct action AND that improving performance in this area is a priority for your team, you may wish to discuss the following options with your clinical leaders, Quality Improvement committees, team staff and/or patients:

• Interventions to improve cancer screening rates using the CCO Cancer Screening Quality Improvement Toolkit designed specifically for Family Health Teams .
• Identify and set an improvement target and work towards it using the change ideas presented in HQO’s Primary Care Practice Group Report.
• Contact your peers to determine their performance and work with them to either spread any processes they find have helped them or collaboratively test some new changes that might work for you AND your peers. HQO’s QIP Navigator allows teams to query submitted QIPs, this tool is extremely usefully to identify peers who have focused on similar areas for improvement.
• Learn about what other family physician leaders are doing working as part of the Provincial Primary Care and Cancer Network.
• Sign your physicians up for monthly screening reports via CCO SAR. Once they get through the sign-up process, most physicians agree that these reports are very helpful, especially if you or they have trouble getting or trusting your EMR data for cancer screening.
• Check out what your peers are doing to improve colorectal cancer screening rates.
  • Presentations and posters from past AFHTO conferences:
    • My Cancer ‘IQ’ in Primary Care (AFHTO 2016).
    • Cancer Screening Day – A Wellness Day for Women (AFHTO 2016).
    • Using Screening Activity Report (SAR) Data to Increase Cancer Screening Rates (AFHTO 2017).

Data Quality Actions

Tips to help you understand the quality of your data and, if necessary, take steps to improve it. Estimate impact of data quality:

• Access the Imperfect Data Impact Calculator to find out whether the data quality issue(s) you think you have would change your initial decision regarding the need to improve.
• Additional information for estimating the impact of data quality for this measure.

Increase quality of the data If the “imperfect data impact calculator” shows that the issues in your data may point you to a different action than suggested in the report, you might consider:

• Get current cancer screening data from your EMR: The QIDS Specialist have developed standardized EMR queries for cancer screening. Try them. Now that they are developed, they should take very little time to run on an ongoing basis (rather than just once a year for reporting purposes). The data might not be directly comparable to what is in D2D because it is from a different time-period and may have more information about patient eligibility for screening. However, it will give you a sense of how your team is doing over time. More importantly, having a list of specific patients that might be overdue for screening gives your team something concrete to do now about something they care about (i.e., patients).
• Record exclusion criteria more completely and more consistently on both cancer screening test documentation and in the EMR.
  • e.g., Exclude patients who have had a total colectomy. Ask your patients about their participation in screening in hospital labs and update the EMR manually to provide a more accurate estimate of screening performance locally, even if this gap remains in the central database.
• Devise a mechanism to formally, consistently record invitations for screening for these and other cancers, to make it easier to extract screening information that incorporates patient choice.
• Perform a manageable sized chart audit (e.g., a maximum of 30 eligible patients has been suggested as a manageable number) and double check that appropriate screening was conducted and recorded in the chart or that patient choice to forego screening is noted. This is not a statistical exercise. It is intended only as a quick check to inform decisions about next steps to improve data quality.
  • Consider hiring a student to help generate a list of eligible patients and check against information in their chart.
• Other ideas: please share!

Additional information for estimating the impact of data quality for this measure:

The data are almost certainly not a definitive estimate of your team’s actual performance. However, they might be “good enough” to help you decide if your team needs to improve or not. To determine if the data are “good enough” for that, estimate how likely it is that one or more of the issues outlined in the interpretive notes are a problem with your team. Then, run the “imperfect data impact calculator” to see if the issue(s) could lead to a different decision related to the need for improvement. To do this, work with your clinical leaders and staff to establish an approximate impact of data quality – i.e., is the data quality issue causing your performance to look like TWICE or HALF or 10% (or other number) less or more than it actually is. Plug that number into the “imperfect data impact calculator.” It will show you whether the data quality issue(s) you think you have would change your initial decision regarding the need to improve. You may find it hard to generate consensus about the possible impact of data quality issues on the level of performance shown in the D2D report. In that case, try the following options:

• Track the next 10 (or 20 or other small number) encounters with patients who meet the inclusion criteria for cancer screening to get a better estimate of the extent of the data quality issues (i.e., check their chart for any record of screening, if absent ask patient whether they have been screened and educate patient on the purpose of screening, record patient choice, especially where the choice to not screen is made). Estimate how many of your patients get their tests done in hospital laboratories and estimate what the screening rate is among those patients.
• Estimate how many of your patients decline screening when offered.
• Extract data from the EMR to determine how many eligible patients were screened. Consult with your QIDS Specialist to ensure the definitions used for the data extraction are consistent with those being developed and deployed across the QIDS Specialist network of approximately 150 AFHTO member organizations. Perhaps the rate among these patients will shift your team’s overall rate to be TWICE or HALF or 10% (or some other number) of the rate in the report. Plug that number into the “imperfect data impact calculator” and proceed accordingly.
• If none of the above is helpful, consider instead experimenting with possible “error” rates to see how much error (i.e., TWICE or HALF or 10% of some other number) would be needed to change the decision made on the basis of the performance of the indicator in D2D. If, in the opinion of the team, such an amount of error is reasonable, then it may be worth considering efforts to improve data quality. Alternatively, if that amount of error is considered to be unlikely, then the data are likely good enough to support the initial decision regarding the need to improve, based on the performance shown in D2D.

If the “imperfect data impact calculator” points to the same decision (i.e., a need to improve or NOT) even after data quality issues are considered, the data are likely “good enough” to base your decision on regarding the need to improve. The next step is to consider strategies to improve, assuming the area of care measured by the indicator is a priority for your team. If your data are not “good enough,” you may then consider taking action to improve your data quality, before or at the same time as you try to improve processes of care.