Updated as of January 22, 2016

Estimate impact of data quality:

• Access the Imperfect Data Impact Calculator to find out whether the data quality issue(s) you think you have would change your initial decision regarding the need to improve

Increase quality of the data

…if the “imperfect data impact calculator” shows that the issues in your data may point you to a different action than suggested in the report. You might consider:

• Get current cancer screening data from your EMR: The QIDSS have developed standardized EMR queries for cancer screening. Try them. Now that they are developed, they should take very little time to run on an ongoing basis (rather than just once a year for reporting purposes). The data might not be directly comparable to what is in D2D (because it is from a different time-period and may have more information about patient eligibility for screening). However, it will give you a sense of how your team is doing over time. More importantly, having a list of specific patients that might be overdue for screening gives your team something concrete to do now about something they care about (i.e. patients).
• Record exclusion criteria more completely and more consistently on both cancer screening test documentation and in the EMR.
  • e.g., Exclude patients who have had cervical, endometrial or ovarian cancer and patients who have had a hysterectomy Ask your patients about their participation in screening in hospital labs and update the EMR manually to provide a more accurate estimate of screening performance locally, even if this gap remains in the central database.
• Devise a mechanism to formally, consistently record invitations for screening for these and other cancers, to make it easier to extract screening information that incorporates patient choice.
• Perform a manageable sized chart audit (e.g. 10% of eligible patients) and double check that appropriate screening was conducted and recorded in the chart or that patient choice to forego screening is noted.
  • Consider hiring a student to help generate a list of eligible patients and check against information in their chart.
• Other ideas: please share!

Additional information for estimating the impact of data quality for this measure:

The data are almost certainly not a definitive estimate of your team’s actual performance in the area of access.  However, they might be “good enough” to help you decide if your team needs to improve access or not.  To determine if the data are “good enough” for that, estimate how likely it is that one or more of the issues outlined in the interpretive notes are a problem with your team.  Then, run the “imperfect data impact calculator” to see if the issue(s) could lead to a different decision related to the need for improvement.  To do this, work with your clinical leaders and staff to establish an approximate impact of data quality – i.e. is the data quality issue causing your performance to look like TWICE or HALF or 10% (or other number) less or more than it actually is. Plug that number into the “imperfect data impact calculator”.  It will show you whether the data quality issue(s) you think you have would change your initial decision regarding the need to improve. You may find it hard to generate consensus about the possible impact of data quality issues on the level of performance shown in the D2D report.  In that case, try the following options:

• Track the next 10 (or 20 or other small number) encounters with patients who meet the inclusion criteria for cancer screening to get a better estimate of the extent of the data quality issues (i.e. check their chart for any record of screening, if absent ask patient whether they have been screened and educate patient on the purpose of screening, record patient choice, especially where the choice to not screen is made).Estimate how many of your patients get their tests done in hospital laboratories and estimate what the screening rate is among those patients.
• Estimate how many of your patients decline screening when offered.
• Extract data from the EMR to determine how many eligible patients were screened. Consult with your QIDSS to ensure the definitions used for the data extraction are consistent with those being developed and deployed across the QIDSS network of approximately 150 AFHTO member organizations. Perhaps the rate among these patients will shift your team’s overall rate to be TWICE or HALF or 10% (or some other number) of the rate in the report. Plug that number into the “imperfect data impact calculator” and proceed accordingly.

• If none of the above is helpful, consider instead experimenting with possible “error” rates to see how much error (i.e. TWICE or HALF or 10% of some other number) would be needed to change the decision made on the basis of the performance of the indicator in D2D. If, in the opinion of the team, such an amount of error is reasonable, then it may be worth considering efforts to improve data quality. Alternatively, if that amount of error is considered to be unlikely, then the data are likely good enough to support the initial decision regarding the need to improve, based on the performance shown in D2D.

If the “imperfect data impact calculator” points to the same decision (e.g. a need to improve or NOT) even after data quality issues are considered, the data are likely “good enough” to base your decision on regarding the need to improve. The next step is to consider strategies to improve, assuming the area of care measured by the indicator is a priority for your team.  If your data are not “good enough”, you may then consider taking action to improve your data quality, before or at the same time as you try to improve processes of care.