Estimate the impact of data quality

• Access the Imperfect Data Impact Calculator to find out whether the data quality issue(s) you think you have would change your initial decision regarding the need to improve.
• Please see below for more information about this tool.

Increase the quality of the data

…if the “imperfect data impact calculator” shows that the issues in your data may point you to a different decision than suggested in D2D 3.0. You might consider the following:

• Work with engaged clinicians to increase the consistency of data entry related to immunization. Refer to the Standardized EMR queries for guidance on ways to increase consistency in data entry such that extraction of the data can be more efficient and accurate over time and between teams.
• Track your immunization rates on an ongoing basis vs just at year-end. Work with QIDS Specialists in the application of consistent queries across teams and EMRs to increase access to these data in an efficient way.
• Ask patients about their immunizations, partly to update the EMR and partly to engage them in an important discussion about preventive strategies. Tilbury District FHT uses a simple paper form for patients to share information regarding flu shots received elsewhere. A similar idea could be applied to childhood immunizations. Better yet, contact them directly to find out exactly how this works!
• Support AFHTO’s efforts to work for inclusion of a standard process for recording and reporting immunization data aligned with PHAC guidelines as part of the technical specification for EMRs. This is already in progress through the EMR Data Management committee and individual EMR Communities of Practice.
• Develop data sharing processes with public health units and other partners engaged in immunization to ensure accurate records regarding immunization status, not only for reporting purposes but also for better risk management for patients.
• Other ideas: please share!

Additional  information for estimating the impact of data quality for this measure:

The data are almost certainly not a definitive estimate of your team’s actual performance. However, they might be “good enough” to help you decide if your team needs to improve or not. To determine if the data are “good enough” for that, estimate how likely it is that one or more of the issues outlined in the interpretive notes above are a problem with your team.  Then, run the “imperfect data impact calculator” to see if the issue(s) could lead to a different decision related to the need for improvement. To do this, work with your clinical leaders and staff to establish an approximate impact of data quality – i.e. is the data quality issue causing your performance to look like TWICE or HALF or 10% (or other number) less or more than it actually is. Plug that number into the “imperfect data impact calculator”.  It will show you whether the data quality issue(s) you think you have would change your initial decision regarding the need to improve. Click here to access the Imperfect Data Impact Calculator. You may find it hard to generate consensus about the impact of data quality issues on the level of performance shown in the D2D 3.0 report. In that case, consider the following options:

• Track the next 10 (or 20 or other small number) encounters to get a better estimate of the extent of the data quality issue. Perhaps the rate among these patients will shift your team’s overall rate to be TWICE or HALF or 10% (or some other number) of the rate in the report. Plug that number into the “imperfect data impact calculator” and proceed accordingly.

• Estimate how many of your patients get their immunizations elsewhere. Perhaps the rate among these patients will shift your team’s overall rate to be TWICE or HALF or 10% (or some other number) of the rate in the report. Plug that number into the “imperfect data impact calculator” and proceed accordingly. [If you are having trouble estimating how many patients are getting immunizations/flu vaccines elsewhere you might want to use 43% as a ball park figure. This was the rate observed by Tilbury District FHT who use a simple paper form for patients to share information regarding flu shots received elsewhere.]

• Estimate how many of your patient’s decline immunization when offered. Perhaps the rate among these patients will shift your team’s overall rate to be TWICE or HALF or 10% (or some other number) of the rate in the report. Plug that number into the “imperfect data impact calculator” and proceed accordingly.

• If none of the above is helpful, consider instead experimenting with possible “error” rates to see how much error (i.e. TWICE or HALF or 10% of some other number) would be needed to change the decision made on the basis of the performance of the indicator in D2D 3.0. If, in the opinion of the team, such an amount of error is reasonable, then it may be worth considering efforts to improve data quality. Alternatively, if that amount of error is considered to be unlikely, then the data are likely good enough to support the initial decision regarding the need to improve, based on the performance shown in D2D 3.0. If the “imperfect data impact calculator” points to the same decision (e.g. a need to improve or NOT) even after data quality issues are considered, the data are likely “good enough” to base your decision on regarding the need to improve. The next step is to consider strategies to improve, assuming the area of care measured by the indicator is a priority for your team. If your data are not “good enough”, you may then consider taking action to improve your data quality, before or at the same time as you try to improve processes of care.