Ultimate Blogging Championship

Category: Uncategorized

  • Childhood Immunization – all children

    Interpretive Notes Steps to Improvement Data Quality Actions

    For technical notes, please see page 25 of the data dictionary.

    Interpretive Notes

    Tips to help you understand the data and put it in context.

    • The D2D definition fully aligns with Public Health criteria. This means Rotavirus is now included in the definition and EMR queries. It is not part of the preventive care bonus for physicians. Therefore, performance may appear lower in D2D than in your preventive care bonus reports which exclude Rotavirus.
    • The D2D definition does not reflect patient choice. Patients who decline immunization and therefore are not immunized may be the reason why your rates are lower than you expect. See “data quality actions” for ideas to examine the extent to which this is affecting your rates.
    • Data for patients immunized outside of the primary care team (e.g., at a health unit) might not be recorded consistently in all EMRs and teams. The performance seen in D2D might therefore under-estimate actual immunization rates. See “data quality actions” for ideas to examine the extent to which this is affecting your rates.
    • Rates for teams with very few children in their panel may be more variable than rates based on larger eligible patient populations. For example, if 2 less children are immunized this year out of a population of 10 children, your immunization rate will drop by 20%. However, if 2 less children out of 100 are not immunized, your rate will only drop by 2%. Consider how many children are eligible for immunization when interpreting differences for your team year to year or relative to another team.
    • D2D includes data for all children as opposed to only rostered children. This might generate different rates than those the team might be used to seeing in reports based on rostered children only (g., the MOHLTC Preventive Care Target Population/Service Report (TPSR)). This report is provided by MOHLTC to eligible physicians in Patient Enrolment Models (PEMs) in April and September by MOHLTC to assist physicians in determining their Target Population and the delivery of preventive care services.
    • The timing of D2D reporting may not coincide with the reporting time period for the MOHLTC Preventive Care Target Population/Service Report. There may be differences in rates related to these differences in time periods.
    • Although a consistent definition was developed to create queries that were shared among members for purposes of extracting these data for D2D, it is possible that the extent to which these data were consistently recorded and therefore extracted in your team might vary. To explore this issue, look at the processes used in your team to record and extract immunization data and work to align it as much as possible with the standard process being developed by QIDS Specialists.

    Steps to Improvement

    Concrete steps you can take to improve care, based on your data. Assuming you have established that the data are good enough to direct action AND that improving performance in this area is a priority for your team, you may wish to discuss the following options with your clinical leaders, Quality Improvement committees, team staff and/or patients:

    Data Quality Actions

    Tips to help you understand the quality of your data and, if necessary, take steps to improve it. Estimate the impact of data quality

    Increase the quality of the data If the “imperfect data impact calculator” shows that the issues in your data may point you to a different action than suggested in the report, you might consider:

    • Work with engaged clinicians to increase the consistency of data entry related to immunization. Refer to the Standardized EMR queries for guidance on ways to increase consistency in data entry such that extraction of the data can be more efficient and accurate over time and between teams.
    • Track your immunization rates on an ongoing basis vs. just at year-end. Work with QIDS Specialists in the application of consistent queries across teams and EMRs to increase access to these data in an efficient way.
    • Ask patients about their immunizations, partly to update the EMR and partly to engage them in an important discussion about preventive strategies. Tilbury District FHT uses a simple paper form for patients to share information regarding flu shots received elsewhere. A similar idea could be applied to childhood immunizations. Better yet, contact Tilbury directly to find out exactly how this works!
    • Support AFHTO’s efforts to work for inclusion of a standard process for recording and reporting immunization data aligned with PHAC guidelines as part of the technical specification for EMRs. This is already in progress through the EMR Data Management committee and individual EMR Communities of Practice.
    • Develop data-sharing processes with public health units and other partners engaged in immunization to ensure accurate records regarding immunization status, not only for reporting purposes but also for better risk-management for patients.
    • Other ideas: please share!

    Additional information for estimating the impact of data quality for this measure:

    The data are almost certainly not a definitive estimate of your team’s actual performance. However, they might be “good enough” to help you decide if your team needs to improve or not. To determine if the data are “good enough” for that, estimate how likely it is that one or more of the issues outlined in the interpretive notes above are a problem with your team. Then, run the “imperfect data impact calculator” to see if the issue(s) could lead to a different decision related to the need for improvement. To do this, work with your clinical leaders and staff to establish an approximate impact of data quality – i.e., is the data quality issue causing your performance to look like TWICE or HALF or 10% (or other number) less or more than it actually is? Plug that number into the “imperfect data impact calculator.” It will show you whether the data quality issue(s) you think you have would change your initial decision regarding the need to improve. You may find it hard to generate consensus about the impact of data quality issues on the level of performance shown in the D2D report. In that case, consider the following options:

    • Track the next 10 (or 20 or other small number) encounters to get a better estimate of the extent of the data quality issue. Perhaps the rate among these patients will shift your team’s overall rate to be TWICE or HALF or 10% (or some other number) of the rate in the report. Plug that number into the “imperfect data impact calculator” and proceed accordingly.
    • Estimate how many of your patients get their immunizations elsewhere. Perhaps the rate among these patients will shift your team’s overall rate to be TWICE or HALF or 10% (or some other number) of the rate in the report. Plug that number into the “imperfect data impact calculator” and proceed accordingly. [If you are having trouble estimating how many patients are getting immunizations/flu vaccines elsewhere you might want to use 43% as a ball park figure. This was the rate observed by Tilbury District FHT who use a simple paper form for patients to share information regarding flu shots received elsewhere.]
    • Estimate how many of your patients decline immunization when offered. Perhaps the rate among these patients will shift your team’s overall rate to be TWICE or HALF or 10% (or some other number) of the rate in the report. Plug that number into the “imperfect data impact calculator” and proceed accordingly.
    • If none of the above is helpful, consider instead experimenting with possible “error” rates to see how much error (i.e., TWICE or HALF or 10% of some other number) would be needed to change the decision made on the basis of the performance of the indicator in D2D 3.0. If, in the opinion of the team, such an amount of error is reasonable, then it may be worth considering efforts to improve data quality. Alternatively, if that amount of error is considered to be unlikely, then the data are likely good enough to support the initial decision regarding the need to improve, based on the performance shown in D2D.

    If the “imperfect data impact calculator” points to the same decision (i.e., a need to improve or NOT) even after data quality issues are considered, the data are likely “good enough” to base your decision on regarding the need to improve. The next step is to consider strategies to improve, assuming the area of care measured by the indicator is a priority for your team. If your data are not “good enough,” you may then consider taking action to improve your data quality, before or at the same time as you try to improve processes of care.

  • Cervical Cancer Screening

    Interpretive Notes Steps to Improvement Data Quality Actions

      For technical notes, please see page 23 of the Data Dictionary.

    Interpretive Notes

    Tips to help you understand the data and put it in context.

    • Virtual rostering assigns patients to the primary care physician that provided the highest dollar amount of services within a defined set of primary care services. Your team may not be aware of which patients have been virtually rostered to them. Therefore, you may erroneously think that these patients are not “your” patients. As a result, your team’s sense of how many of “your” patients were screened may be different than shown in D2D.
    • Screening tests performed in hospital laboratories or paid through alternate payment plans are not currently incorporated into this measure. Actual performance on this measure for teams that use hospital laboratories is therefore likely higher than the level presented in D2D.
    • Inaccurate recording of exclusion criteria may result in an under-estimation of screening rates, as patients who are not eligible for screening would be erroneously included in the denominator, artificially driving the observed rate down.
    • The current measure does not consider patient choice in screening and therefore might reflect an under-estimate of the screening “interventions” (i.e., consultations/advice to undertake screening) by the team.

    Steps to Improvement

    Concrete steps you can take to improve care, based on your data. Assuming you have established that the data are good enough to direct action AND that improving performance in this area is a priority for your team, you may wish to discuss the following options with your clinical leaders, Quality Improvement committees, team staff and/or patients:

    Data Quality Actions

    Tips to help you understand the quality of your data and, if necessary, take steps to improve it. Estimate impact of data quality:

    Increase quality of the data If the “imperfect data impact calculator” shows that the issues in your data may point you to a different action than suggested in the report, you might consider:

    • Get current cancer screening data from your EMR: The QIDSS have developed standardized EMR queries for cancer screening. Try them. Now that they are developed, they should take very little time to run on an ongoing basis (rather than just once a year for reporting purposes). The data might not be directly comparable to what is in D2D because it is from a different time-period and may have more information about patient eligibility for screening. However, it will give you a sense of how your team is doing over time. More importantly, having a list of specific patients that might be overdue for screening gives your team something concrete to do now about something they care about (i.e., patients).
    • Record exclusion criteria more completely and more consistently on both cancer screening test documentation and in the EMR.
      • e.g., Exclude patients who have had cervical, endometrial or ovarian cancer and patients who have had a hysterectomy. Ask your patients about their participation in screening in hospital labs and update the EMR manually to provide a more accurate estimate of screening performance locally, even if this gap remains in the central database.
    • Devise a mechanism to formally, consistently record invitations for screening for these and other cancers, to make it easier to extract screening information that incorporates patient choice.
    • Perform a small chart audit (e.g., a maximum of 30 eligible patients has been suggested as a manageable number) and double-check that appropriate screening was conducted and recorded in the chart or that patient choice to forego screening is noted. This is not a statistical exercise. It is intended only as a quick check to inform decisions about next steps to improve data quality.
      • Consider hiring a student to help generate a list of eligible patients and check against information in their chart.
    • Other ideas: please share!

    Additional information for estimating the impact of data quality for this measure:

    The data are almost certainly not a definitive estimate of your team’s actual performance. However, they might be “good enough” to help you decide if your team needs to improve or not. To determine if the data are “good enough” for that, estimate how likely it is that one or more of the issues outlined in the interpretive notes are a problem with your team. Then, run the “imperfect data impact calculator” to see if the issue(s) could lead to a different decision related to the need for improvement. To do this, work with your clinical leaders and staff to establish an approximate impact of data quality – i.e., is the data quality issue causing your performance to look like TWICE or HALF or 10% (or other number) less or more than it actually is. Plug that number into the “imperfect data impact calculator.” It will show you whether the data quality issue(s) you think you have would change your initial decision regarding the need to improve. You may find it hard to generate consensus about the possible impact of data quality issues on the level of performance shown in the D2D report. In that case, try the following options:

    • Track the next 10 (or 20 or other small number) encounters with patients who meet the inclusion criteria for cancer screening to get a better estimate of the extent of the data quality issues (i.e., check their chart for any record of screening and, if absent, ask patient whether they have been screened and educate patient on the purpose of screening, record patient choice, especially where the choice to not screen is made). Estimate how many of your patients get their tests done in hospital laboratories and estimate what the screening rate is among those patients.
    • Estimate how many of your patients decline screening when offered.
    • Extract data from the EMR to determine how many eligible patients were screened. Consult with your QIDSS to ensure the definitions used for the data extraction are consistent with those being developed and deployed across the QIDSS network of approximately 150 AFHTO member organizations. Perhaps the rate among these patients will shift your team’s overall rate to be TWICE or HALF or 10% (or some other number) of the rate in the report. Plug that number into the “imperfect data impact calculator” and proceed accordingly.
    • If none of the above is helpful, consider instead experimenting with possible “error” rates to see how much error (i.e., TWICE or HALF or 10% of some other number) would be needed to change the decision made on the basis of the performance of the indicator in D2D. If, in the opinion of the team, such an amount of error is reasonable, then it may be worth considering efforts to improve data quality. Alternatively, if that amount of error is considered to be unlikely, then the data are likely good enough to support the initial decision regarding the need to improve, based on the performance shown in D2D.

    If the “imperfect data impact calculator” points to the same decision (i.e., a need to improve or NOT) even after data quality issues are considered, the data are likely “good enough” to base your decision on regarding the need to improve. The next step is to consider strategies to improve, assuming the area of care measured by the indicator is a priority for your team. If your data are not “good enough,” you may then consider taking action to improve your data quality, before or at the same time as you try to improve processes of care.

  • Regular primary care provider – individual and team

    Individual: Interpretive Notes Steps to Improvement Data Quality Actions
    Team: Interpretive Notes Steps to Improvement Data Quality Actions

      For technical notes (individual), please see page 20 of the Data Dictionary. For technical notes (team), please see page 21 of the Data Dictionary.

    Interpretive Notes (Individual)

    Tips to help you understand the data and put it in context.

    • Virtual rostering assigns patients to the primary care physician that provided the highest dollar amount of services within a defined set of primary care services. Physicians in your team may not be aware of which patients have been virtually rostered to them. Therefore, they may erroneously think that these patients are not “their” patients. As a result, an individual physician’s sense of how many of patients they see that are “their” patients may be different than the rate shown in D2D.
    • Visits to health care providers other than physicians are not included in this measure. However, this does not necessarily skew the measure. For example, if a patient visits a primary care team 10 times and sees a physician 8 times, and each time it is their “own” physician, they will score 100% (8 out of 8) for “regular care provider – individual.” If, however, they visit 10 times, receiving care from multiple providers but only saw a physician once and it was their own physician, they could still score 100% on this measure.
    • Efforts to improve access to same or next day appointments may result in patients seeing whichever physician is available for appointments at the time. While this may be valuable from the perspective of access, this process may be reflected in poor performance on the “regular care provider – individual” measure.
    • Teams with part-time physicians and teaching teams may have developed strong relationships between physicians to jointly care for patients, such that patients may feel equally comfortable and familiar with more than one physician. However, while this might embody team-based care, it may be reflected in poor performance on the “regular care provider – individual” measure.

    Steps to Improvement (Individual)

    Concrete steps you can take to improve care, based on your data. Assuming you have established that the data are good enough to direct action AND that improving performance in this area is a priority for your team, you may wish to discuss the following options with your clinical leaders, Quality Improvement committees, team staff and/or patients:

    • Explore interventions to increase proportion of patients with regular care provider and/or interventions to improve continuity of care (i.e., increase the chances that they see the same provider each time).
      • Ask patients what is most important to them: Train front reception staff to discuss options with patients as part of the appointment-booking process (i.e., difference in wait times if patient wants to see their regular primary care provider vs. any primary care provider in the team).
      • Improve same day/next day access for all physicians (i.e., ensure all physicians have same day/next day spots available exclusively for their patients).
    • Contact your peers to determine their performance and work with them to either spread any processes they find have helped them or collaboratively test some new changes that might work for you AND your peers. HQO’s QIP Navigator allows teams to query submitted QIPs, this tool is extremely usefully to identify peers who have focused on similar areas for improvement.

    Data Quality Actions (Individual)

    Tips to help you understand the quality of your data and, if necessary, take steps to improve it. Estimate impact of data quality:

    Increase quality of the data If the “imperfect data impact calculator” shows that the issues in your data may point you to a different action than suggested in the report, you might consider:

    • Most of the work to improve data quality for this indicator lies in refining the definitions as the data are captured via administrative information systems across all health care sectors and thus beyond the influence of primary care providers.
    • Consider generating a more local estimate of continuity based on who patients see in your team, as indicated by your EMR data. Consider collaborating with the QIDS Specialist group to improve definitions of encounter types and provider types so a more local, team-based measure can be extracted from the EMR.

    Additional information for estimating the impact of data quality for this measure:

    The data are almost certainly not a definitive estimate of your team’s actual performance in the area of access. However, they might be “good enough” to help you decide if your team needs to improve access or not. To determine if the data are “good enough” for that, estimate how likely it is that one or more of the issues outlined in the interpretive notes are a problem with your team. Then, run the “imperfect data impact calculator” to see if the issue(s) could lead to a different decision related to the need for improvement. To do this, work with your clinical leaders and staff to establish an approximate impact of data quality – i.e., is the data quality issue causing your performance to look like TWICE or HALF or 10% (or other number) less or more than it actually is. Plug that number into the “imperfect data impact calculator.” It will show you whether the data quality issue(s) you think you have would change your initial decision regarding the need to improve. You may find it hard to generate consensus about the possible impact of data quality issues on the level of performance shown in the D2D report. In that case, try the following options:

    • If you have part-time physicians in your team, try the following:
      • Extract data from the EMR to determine the number of visits that patients of part-time physicians made to other physicians in the team (i.e., NOT seen by their “own” doctor). Perhaps the rate among these patients will shift your team’s overall rate to be TWICE or HALF or 10% (or some other number) of the rate in the report. Plug that number into the “imperfect data impact calculator” and proceed accordingly.
      • If not possible to extract all these data from the EMR, extract as much as possible and project mathematically based on the number of patients, the average number of visits for all patients and how many hours the part-time physician normally works. Perhaps the rate among these patients will shift your team’s overall rate to be TWICE or HALF or 10% (or some other number) of the rate in the report. Plug that number into the “imperfect data impact calculator” and proceed accordingly.
    • Determine how many of the patients visiting your team are not formally rostered but might be virtually rostered and estimate how many of them usually see the same doctor. Perhaps the rate among these patients will shift your team’s overall rate to be TWICE or HALF or 10% (or some other number) of the rate in the report. Plug that number into the “imperfect data impact calculator” and proceed accordingly.
    • Attempt to estimate how many patients would have seen their own physician in the absence of efforts to increase same/next day access. This will almost certainly be a judgement call, rather than an “estimate” in the truest sense of the word. Perhaps the rate among these patients will shift your team’s overall rate to be TWICE or HALF or 10% (or some other number) of the rate in the report. Plug that number into the “imperfect data impact calculator” and proceed accordingly.
    • If none of the above is helpful, consider instead experimenting with possible “error” rates to see how much error (i.e., TWICE or HALF or 10% of some other number) would be needed to change the decision made on the basis of the performance of the indicator in D2D. If, in the opinion of the team, such an amount of error is reasonable, then it may be worth considering efforts to improve data quality. Alternatively, if that amount of error is considered to be unlikely, then the data are likely good enough to support the initial decision regarding the need to improve, based on the performance shown in D2D.

    If the “imperfect data impact calculator” points to the same decision (e.g., a need to improve or NOT) even after data quality issues are considered, the data are likely “good enough” to base your decision on regarding the need to improve. The next step is to consider strategies to improve, assuming the area of care measured by the indicator is a priority for your team. If your data are not “good enough,” you may then consider taking action to improve your data quality, before or at the same time as you try to improve processes of care.

    Interpretive Notes (Team)

    Tips to help you understand the data and put it in context.

    • “Team,” in this indicator, refers to the physician group i.e., FHO or FHN, not FHT.
    • HQO has removed this indicator from the PCPR for FHTs with multiple physician groups. 
    • Virtual rostering assigns patients to the primary care physician that provided the highest dollar amount of services within a defined set of primary care services. Physicians in your team may not be aware of which patients have been virtually rostered to them. Therefore, they may erroneously think that these patients are not “their” patients. As a result, an individual physician’s sense of how many of patients they see that are “their” patients may be different than the rate shown in D2D.
    • Visits to health care providers other than physicians are not included in this measure. However, this does not necessarily skew the measure. For example, if a patient visits a primary care team 10 times and sees a physician on this team 8 times, regardless of whether or not it was their “own” physician, they will score 100% (8 out of 8) for “regular care provider – team.” If, however, they visit 10 times, receiving care from multiple providers but only saw a physician once and it was physician on the team, they could still score 100% on this measure.
    • Teams with part-time physicians and teaching teams may have developed strong relationships between physicians to jointly care for patients, such that patients may feel equally comfortable and familiar with more than one physician. This principle is at the core of team-based care and is reflected in higher performance on the “regular care provider – team” vs. “regular care provider – individual” measure across most teams in D2D.

    Steps to Improvement (Team)

    Concrete steps you can take to improve care, based on your data. Assuming you have established that the data are good enough to direct action AND that improving performance in this area is a priority for your team, you may wish to discuss the following options with your clinical leaders, Quality Improvement committees, team staff and/or patients:

    • Contact your peers to determine their performance and work with them to either spread any processes they find have helped them or collaboratively test some new changes that might work for you AND your peers. HQO’s QIP Navigator allows teams to query submitted QIPs, this tool is extremely usefully to identify peers who have focused on similar areas for improvement.

     

    Data Quality Actions (Team)

    Tips to help you understand the quality of your data and, if necessary, take steps to improve it.

    Estimate impact of data quality:

    • Access the Imperfect Data Impact Calculator to find out whether the data quality issue(s) you think you have would change your initial decision regarding the need to improve.

    Increase quality of the data If the “imperfect data impact calculator” shows that the issues in your data may point you to a different action than suggested in the report, you might consider:

    • Most of the work to improve data quality for this indicator lies in refining the definitions as the data are captured via administrative information systems across all health care sectors and thus beyond the influence of primary care providers.
    • Consider generating a more local estimate of continuity based on who patients see in your team, as indicated by your EMR data. Consider collaborating with the QIDS Specialist group to improve definitions of encounter types and provider types so a more local, team-based measure can be extracted from the EMR.

    Review list of “rostered” patients with each physician and identify patients who are likely “virtually rostered” (i.e., see the doctor frequently but are not formally rostered) for consideration for formal inclusion in the roster.

    Additional information for estimating the impact of data quality for this measure:

    The data are almost certainly not a definitive estimate of your team’s actual performance in the area of access. However, they might be “good enough” to help you decide if your team needs to improve access or not. To determine if the data are “good enough” for that, estimate how likely it is that one or more of the issues outlined in the interpretive notes are a problem with your team. Then, run the “imperfect data impact calculator” to see if the issue(s) could lead to a different decision related to the need for improvement. To do this, work with your clinical leaders and staff to establish an approximate impact of data quality – i.e., is the data quality issue causing your performance to look like TWICE or HALF or 10% (or other number) less or more than it actually is? Plug that number into the “imperfect data impact calculator.” It will show you whether the data quality issue(s) you think you have would change your initial decision regarding the need to improve. You may find it hard to generate consensus about the possible impact of data quality issues on the level of performance shown in the D2D report. In that case, try the following options:

    • Determine how many of the patients visiting your team are not formally rostered but might be virtually rostered and estimate how many of them usually see the same doctor. Perhaps the rate among these patients will shift your team’s overall rate to be TWICE or HALF or 10% (or some other number) of the rate in the report. Plug that number into the “imperfect data impact calculator” and proceed accordingly.
    • Attempt to estimate how many patients would have seen their own physician, in the absence of efforts to increase same/next day access. This will almost certainly be a judgement call, rather than an “estimate” in the truest sense of the word. Perhaps the rate among these patients will shift your team’s overall rate to be TWICE or HALF or 10% (or some other number) of the rate in the report. Plug that number into the “imperfect data impact calculator” and proceed accordingly.
    • If none of the above is helpful, consider instead experimenting with possible “error” rates to see how much error (i.e., TWICE or HALF or 10% of some other number) would be needed to change the decision made on the basis of the performance of the indicator in D2D. If, in the opinion of the team, such an amount of error is reasonable, then it may be worth considering efforts to improve data quality. Alternatively, if that amount of error is considered to be unlikely, then the data are likely good enough to support the initial decision regarding the need to improve, based on the performance shown in D2D.

    If the “imperfect data impact calculator” points to the same decision (i.e., a need to improve or NOT) even after data quality issues are considered, the data are likely “good enough” to base your decision on regarding the need to improve. The next step is to consider strategies to improve, assuming the area of care measured by the indicator is a priority for your team. If your data are not “good enough,” you may then consider taking action to improve your data quality, before or at the same time as you try to improve processes of care.

  • Member news: PSHSA webinar, quality standards related to opioids, mental health and more

    Below are relevant updates and items for AFHTO members, some with fast-approaching deadlines:

    AFHTO News
    • Reasons to register for the AFHTO 2017 Conference today:
    1. It’s now less than two weeks left to register at early bird price!
    2. Hotel rooms more than 70% booked.
    3. André Picard and Deputy Minister Bob Bell
    4. So many Community of Practice meetings e.g. Physical Activity, EMR & IHP
    • Have you seen our new Strategic Plan: 2017-2020 plan now released. Thank you to all our members whose support has been invaluable in furthering AFHTO’s commitment to be an advocate, champion, network and resource to support FHTs, NPLCs and other interprofessional models of care.

     

     

     

     

     

    News Relevant to Primary Care
    • OHIP+ update: The ministry is working with clinicians and other stakeholders regarding feedback and potential concerns including ensuring appropriateness of the formulary for children and youth, collaboration with private insurance to ensure seamless coverage and exceptional access program pre-approvals. To learn more, visit the  website or join the mailing list by emailing OHIPplus@ontario.ca.

    Requests for Input and Resources for Patients and Teams

     

     

    Conferences and Events
    • HQO Quality Improvement Plans – Sneak Peek for 2018-19, Sep. 27, 2017: register now

     
    • Office Gynecological Procedures in Family Medicine, Oct. 3, 2017: register now
    • Learning Essential Approaches to Palliative and End-of-Life Care, Oct. 11, 2017: register now.
    • Workplace Harassment Webinar Series, Oct. 12, 2017 – Jan. 19, 2018: register now.
    • Health Quality Transformation 2017, Oct. 24, 2017: register now.
    • 6th Annual Central LHIN Oncology Day for Primary Care Providers: New Frontiers, October 20, 2017: register now.

  • EMR Data quality

    Interpretive Notes Data Quality Actions

    For UPDATED technical notes, please see page 28 in the Data Dictionary.

    Interpretive Notes

    Tips to help you understand the data and put it in context.

    • The EMR Data Quality (EMR-DQ) score is a snapshot of how well data are being recorded in your EMR based on measures of completeness (smoking status complete) and concordance (comparison of EMR-based colorectal and cervical cancer screening to CCO rates reported in the Screening Activity Report (SAR), coded diagnosis of diabetes). Because the indicator currently only includes 4 data elements, it may not reflect your team’s general approach to data quality in your EMR. Work continues to further refine the measures of EMR data quality even as efforts are underway to improve it. The EMR DQ measure is an attempt to operationalize a data quality framework described by Bowen and Lau.
    • Your EMR DQ score may be low if the cancer screening data in your EMR does NOT match the CCO cancer screening data. This might be due to lab results not coming in a consistent or standard way, or due to variations in documenting cancers, hysterectomies etc., so these patients can be excluded.
    • Your EMR DQ score might not represent what is happening across your whole team, if you were only able to get the CCO SAR and EMR cancer screening rates for a few doctors.
    • Your EMR DQ score might actually not represent the quality of data in your EMR, if you do better or worse on other aspects of data quality/standardization in your EMR (e.g. using SNOMED coding for your patients with diabetes or COPD).
    • Teams might think their EMR data quality is WAY better than their score shows. If this is the case, have a look at the data quality actions for other ways of improving data quality.

    Want to compare your data across iterations? The EMR-DQ Score has changed across multiple iterations of D2D, with the addition of new components. Moreover, teams may choose to enter data on more or different components from one iteration to another. For this reason, a simple comparison of EMR-DQ scores across two or more iterations of D2D is not necessarily a reliable indicator of whether your team’s data quality is getting better, getting worse, or staying the same. You could be comparing “apples to oranges” – i.e., a score based on one particular set of measures to a score based on a larger (or simply different) set of measures. The good news is you still have the necessary data to make an accurate “apples to apples” comparison. Here’s how:

    1.  Access your D2D data (from the Data-Input Toolkit) for each iteration you want to compare.
    2. Identify the measures for which you submitted data in all of these iterations.
    3. For each iteration you wish to compare, compile a straight average of the scores for each measure to calculate your modified EMR-DQ score.

    Example:  In D2D 3.0, you submitted EMR-DQ data for cervical and colorectal cancer screening and smoking status.  In D2D 4.0, you submitted data on cervical cancer screening, colorectal cancer screening, smoking status, and coded diagnosis of diabetes. In D2D 5.0, you submitted data on cervical cancer screening, smoking status, coded diagnosis of diabetes, and coded diagnosis of COPD (but not colorectal cancer screening, coded diagnosis of CHF, or coded diagnosis of depression)

    • To compare D2D 3.0 and D2D 4.0, you will add your scores for cervical cancer screening, colorectal cancer screening, and smoking status, then divide the total for each iteration by 3.
    • To compare D2D 3.0 and D2D 4.0, you will add your scores for cervical cancer screening, smoking status, and coded diagnosis of diabetes, then divide the total for each iteration by 3.
    • To compare D2D 3.0 through D2D 6.0, you will add your scores for cervical cancer screening and smoking status, then divide the total for each iteration by 2.

    Of course, every team is unique, and there may be other factors making it challenging to track your EMR-DQ performance across multiple years. For assistance, please contact improve@afhto.ca.

    Data Quality Actions

    Tips to help you understand the quality of your data and, if necessary, take steps to improve it. Estimate impact of data quality

    • Access the Imperfect Data Impact Calculator to find out whether the data quality issue(s) you think you have would change your initial decision regarding the need to improve.

    The data are almost certainly not a definitive estimate of your team’s actual performance. However, they might be “good enough” to help you decide if your team needs to improve or not. To determine if the data are “good enough” for that, estimate how likely it is that one or more of the issues outlined in the interpretive notes are a problem with your team. Then, run the “imperfect data impact calculator” to see if the issue(s) could lead to a different decision related to the need for improvement. To do this, work with your clinical leaders and staff to establish an approximate impact of data quality – i.e., is the data quality issue causing your performance to look like TWICE or HALF or 10% (or other number) less or more than it actually is? Plug that number into the “imperfect data impact calculator.” It will show you whether the data quality issue(s) you think you have would change your initial decision regarding the need to improve. You may find it hard to generate consensus about the impact of data quality issues on the level of performance shown in the D2D report. In that case, consider the following options:

    • Track the next 10 (or 20 or other small number) encounters to get a better estimate of the extent of the data quality issue. Perhaps the rate among these patients will shift your team’s overall rate to be TWICE or HALF or 10% (or some other number) of the rate in the report. Plug that number into the “imperfect data impact calculator” and proceed accordingly.
    • Estimate how many of your patients eligible for cervical or colorectal cancer screening have tests or labs recorded properly in the EMR. Perhaps the rate among these patients will shift your team’s overall rate to be TWICE or HALF or 10% (or some other number) of the rate in the report. Plug that number into the “imperfect data impact calculator” and proceed accordingly.
    • Estimate how many of your patients have a smoking status coded in a consistent manner in your EMR. Perhaps the rate among these patients will shift your team’s overall rate to be TWICE or HALF or 10% (or some other number) of the rate in the report. Plug that number into the “imperfect data impact calculator” and proceed accordingly.
    • Estimate how many of your patients with diabetes have a diagnosis code in the appropriate place in the EMR. Perhaps the rate among these patients will shift your team’s overall rate to be TWICE or HALF or 10% (or some other number) of the rate in the report. Plug that number into the “imperfect data impact calculator” and proceed accordingly.
    • Estimate how many of your patients with COPD have a diagnosis code in the appropriate place in the EMR. Perhaps the rate among these patients will shift your team’s overall rate to be TWICE or HALF or 10% (or some other number) of the rate in the report. Plug that number into the “imperfect data impact calculator” and proceed accordingly.
    • Estimate how many of your patients with depression have a diagnosis code in the appropriate place in the EMR. Perhaps the rate among these patients will shift your team’s overall rate to be TWICE or HALF or 10% (or some other number) of the rate in the report. Plug that number into the “imperfect data impact calculator” and proceed accordingly.
    • Estimate how many of your patients with CHF have a diagnosis code in the appropriate place in the EMR. Perhaps the rate among these patients will shift your team’s overall rate to be TWICE or HALF or 10% (or some other number) of the rate in the report. Plug that number into the “imperfect data impact calculator” and proceed accordingly.
    • If none of the above is helpful, consider instead experimenting with possible “error” rates to see how much error (i.e., TWICE or HALF or 10% or some other number) would be needed to change the decision made on the basis of the performance of the indicator in D2D 3.0. If, in the opinion of the team, such an amount of error is reasonable, then it may be worth considering efforts to improve data quality. Alternatively, if that amount of error is considered to be unlikely, then the data are likely good enough to support the initial decision regarding the need to improve, based on the performance shown in D2D 3.0.

    If the “imperfect data impact calculator” points to the same decision (e.g., a need to improve or NOT) even after data quality issues are considered, the data are likely “good enough” to base your decision on regarding the need to improve. The next step is to consider strategies to improve, assuming the area of care measured by the indicator is a priority for your team. If your data are not “good enough”, you may then consider taking action to improve your data quality, before or at the same time as you try to improve processes of care. Increase quality of the data The goal of this indicator is to inform and motivate action to improve data quality in the EMR and also serve as a measure to monitor progress for actions to improve data quality such as those described for all the other D2D indicators. If the “imperfect data impact calculator” shows that the issues in your data may point you to a different action than suggested in the report, you might consider:

    •  Exploring other aspects of EMR data quality: The actions to improve data quality do not need to be limited to cancer screening smoking status or diabetes diagnosis data, although these might be of immediate interest. Accurately identifying deceased patients could be a focus, or accurately recording lab results in appropriate fields and in appropriate language, or coding diagnoses consistently are other areas.
    • Confirming patient volume data: Confirm which patients are alive and active (by whatever definition you use in your EMR/team) as rates of indicators based on incorrect denominators (i.e., all patients ever seen, vs. just the patients who are alive and who are active) will be incorrect.
    • Measuring your EMR Maturity: Have your QIDS Specialist and/or physicians complete  OntarioMD’s EMR Progress Assessment tool (EPA) to measure the current and desired level of EMR maturity. By completing the EPA, teams will be connected with the resources of OntarioMD’s EMR Practice Enhancement Program to help enhance their EMR skills, efficiency and data quality.
    • Providing feedback to clinicians:
      • Create lists of patients with particular characteristics identified via standardized EMR queries for their review/confirmation. These have been developed by QIDS Specialists and the EMR CoPs (e.g., childhood immunization, COPD, diabetes, and CHF; a query for depression is coming soon).
      • Participate with external partners like CPCSSN, EMRALD, and eCHN to use your EMR data in more meaningful ways
    • Hiring a student to help you clean up your data (see suggestions in this handbook for cleaning up your roster and smoking/alcohol status).
    • Participating in the EMR communities of practice and join your peers in developing new tools and processes for standardizing access to EMR data. Contact us to get connected.
    • Tap into external resources to support clinical process changes using PDSAs from HQO (also check with your QIDS Specialist).

  • Three Draft Quality Standards Related to Opioids Available for Feedback

    To help tackle the growing opioid problem, Health Quality Ontario is developing three sets of quality standards. The first one outlines for patients and clinicians what quality care looks like for adults and adolescents with opioid use disorder and the other two provide guidance on prescribing opioids for the management of chronic and acute pain. Health Quality Ontario wants to hear from you. Click below to read the draft patient and clinician guides and let them know what you think. The deadline is September 28, 2017.

  • EDAC/PLC/NPLC News: Learning together and moving forward through change

    Below are highlights from summer 2017 meetings of AFHTO’s Executive Director Advisory Council (EDAC), Physician Leadership Council (PLC), and the NPLC Leadership Council. 

    Use the links below to navigate through the sections.

    FHT-MOHLTC Contract Renewal Building Relationships with the LHINs
    Ministry of Labour Safe at Work Ontario Initiative AFHTO Conference 2017 – Improving Primary Care Together
    Governance & Leadership Program Updates QIDS Program Updates
    NPs…This Focus Group is For YOU

    FHT-MOHLTC Contract Renewal

    Consultations between the Ministry and OMA on proposed changes to the FHT contract took place over the summer (see May 19 email update for the clauses under consideration for change in the new FHT contract). Given our common membership of family physicians, AFHTO also had the opportunity to meet with the OMA to discuss and compare member input in relation to the proposed changes. Joint meetings will continue as we support our members in contract implementation.

    We anticipate the Ministry will share the draft FHT contract with AFHTO over the coming weeks. A joint webinar will be held to review the changes and answer any questions you may have (most likely late Fall 2017).

    Building Relationships with the LHINs

    The Ministry has been making efforts to identify and address gaps in LHINs’ understanding of issues pertinent to interprofessional primary care, including an orientation to FHTs and NPLCs. As part of that effort, they are holding two “Primary Care 101” education sessions for LHIN CEOs this fall, and have shared teams’ AOPs with the LHINs.

    Under the Patients First Act, Family Health Teams (FHTs) and Nurse Practitioner-Led Clinics (NPLCs) can become Health Services Providers (HSPs), enabling LHINs to fund them (core funding for the FHTs and NPLCs remains with the Ministry). AFHTO will be offering a joint webcast with the LHINs and Ministry to provide education about what it means to be an ‘HSP’, the process to receive LHIN funding for certain programs/projects/initiatives, Service Accountability Agreements between the LHIN and HSPs they fund, and an overview of the LHIN Authority Guideline documents (anticipating October 2017).

    With Patients First implementation now underway, it’s important for primary care teams to build strong relationships with their LHINs and with their LHIN sub-region peers, to ensure our “place at the table” and to further our role as primary care leaders. We’re hearing that some LHINs have not been making room at the table for NPLCs, and we continue to advocate for them as part of our mandate.

    Ministry of Labour (MOL) Safe at Work Ontario Initiative

    Be prepared! The MOL has initiated their random site inspections of FHTs and CHCs to determine compliance with the OHSA and associated regulations. AFHTO has partnered with Public Services Health & Safety Association (PSHSA) to help our members get ready – our online Health & Safety Resources Webpage is now live! We have added sample policies and a Health and Safety Resource Manual, in addition to other resources…be sure to check it out!

    Missed the August health & safety webinar? By popular demand, we’re offering a second session on October 4th from 1-2 pm. Register now. Password- AFHTO2

    Regional training workshops are also available and are being planned in several LHIN regions. Ask your EDAC rep for more information.

    AFHTO 2017 Conference – Improving Primary Care Together

    REGISTER NOW! Early-bird registration closes October 2, 2017. Don’t miss out on the opportunity to meet and discuss topics relevant to your roles, teams and patients in sessions like:

    The Way Forward: Care Coordination Led by Primary Care

    This year’s focus for the Leadership Triad Session is on care coordination and building primary care as the foundation of the health care system, so more Ontarians can access comprehensive care, coordinated through primary care teams.

    Be a part of the dialogue and provide meaningful guidance to the LHINs by helping them to develop a plan for transitioning care-coordination resources to primary care. The May 2017 Minister’s mandate letter asked LHINs “to develop and implement a plan […] that embeds care coordinators and system navigators in primary care to ensure smooth transitions of care,” so we know that care coordination is a relevant and timely topic for both LHIN and AFHTO leaders.

    Note: This Leadership workshop is open only to board members, board chairs, EDs and Lead MDs or NPs of AFHTO member organizations and will include senior leadership members from the LHINs.

    Other sessions for FHT and NPLC leaders at the AFHTO 2017 Conference

    Governance & Leadership Program Updates

    Want to know what AFHTOs Governance and Leadership program is up to? Read here. Highlights include:

    • Online governance training, in partnership with OHA’s Governance Centre of Excellence – topics include board development, financial literacy, and risk management.
    • New knowledge products: Case study on FHT-Physician teamwork, and a new Physician Orientation Toolkit
    • Learning events on privacy and workplace safety
    • A sneak peek at what’s ahead for Fall 2017!

    QIDS Program Updates

    Big news about support for opioid management. AFHTO has been named as co-lead for a recently funded CAMH project to support primary care providers in better management of pain, addiction, and opioid prescribing. Interested in learning more about how to treat opiod use disorder in a primary-care setting? Consider registering for the 2017 Opioid Use Disorder in Primary Care Conference, November 24th in Toronto.

    Registration is open for our fall learning event: Managing Medication as a Team. This full-day workshop is an in-depth look at common aspects of interprofessional medication management, so you can pick up ideas to use in your own setting.

    Read here to find out what else AFHTO’s Quality Improvement & Decision Support program is doing. Highlights include:

    • QI Enablers Study: Let’s work together to find out how teams get better at what they do.
    • Patient Focus Groups: Finding out more about what matters to patients.
    • Information to Action: Learn more about our new basket of tools and resources for quality improvement
    • Optimum Project: Help for your older patients with treatment-resistant depression

    …and don’t forget to check our bi-weekly D2D eBulletin for regular program updates!

    NPs…This Focus Group is For YOU

    The Exceptional Access Program at the MOHLTC is developing a new online service for prescribers which will allow them to research, create, and manage exceptional access program requests for their patients. The ministry has been reaching out to various prescriber groups and drug manufacturers to make them aware of this new online service (The Special Authorization Digital Information Exchange [SADIE]).

    Provide your input and have a sneak peek into how SADIE will look and its functionality!

    When: Friday, September 29th, 2017 from 1:00 to 2:30
    ONLINE (with audio if your computer has mic/speakers) https://ali.health.gov.on.ca/AFHTO
    DIAL-IN for AUDIO: 416-212-8012/1-866-633-0848 / Conf Id 5749341#
    Please RSVP to Rob Oddi (rob.oddi@ontario.ca) if you can attend.

  • Inner City FHT Mix Cooking and Community in Life Skills Program

    Toronto Star article published September 14, 2017. Article in full pasted below. Emily Mathieu, Housing Reporter It is mid-morning in a packed upstairs kitchen at The 519 community centre and Kristen Ireland is brandishing an eggplant and making upbeat and practiced demands. “Somebody has got to cut the eggplant,” she declares, before flipping the tubular purple fruit into a set of ready hands, followed by an extremely large knife. “Knife skills, remember,” she said, offering it out handle first. “Pass it so you don’t kill somebody.” Ireland is a health promoter with the Inner City Family Health Team, a group of health professionals dedicated to caring for former or current residents of Seaton House and people who are experiencing homelessness. Brandon, who took and is diligently cutting the eggplant into cubes, is a health team client and has stayed for two years at Seaton House, Toronto’s largest shelter. At The 519 he is part of a team of chefs, working to learn how to prepare healthy and affordable meals, as part of a life skills program called Street Eats. “The most important thing is we learn from each other and master some skills,” said Brandon. People who don’t have housing, he felt it was important to say, are often isolated and lonely and benefit from group programs. The cooking program “warms my heart,” he said. Arnoldo Alcayaga, who at one time lived in an emergency shelter, was part of a group who came up with the workshop, with health team registered nurse Roxie Danielson and his own doctor. He was a health team client and wanted to find a way to give back because of the support he received. “Emotionally and spiritually you have to nourish your body and the best way to do that is to be aware of what you put in your body,” said Alcayaga, who is a chef. To do that properly, he said, you need to be instructed on how to pick, cook and find foods within your budget. Each session one of the men, all health team clients, picks out a recipe to make. Brandon chose pesto and pasta salad, topped with shredded cheese and tiny tomatoes from the health team’s garden. With Ireland’s help he coaches eight men through the hour and a bit it takes to turn out a pasta dish in a tiny and somewhat chaotic kitchen. The room is packed with jostling and laughing men, trying to communicate over the general din and a sputtering and roaring blender. The program runs every three weeks and is a partnership between the health team and The 519, a multi-service, city agency serving the LGBTQ community and other marginalized groups. The 519 donates the kitchen and the money for the food. “We have done everything from tamales to Chinese food,” as well as indigenous recipes, said Curran Stikuts, community organizer, who also does a fair bit of the grocery shopping. Stikuts said programs like Street Eats help provide a bit of additional food security for people struggling to get by in Toronto. He said in the last year The 519 provided more than 12,000 meals, just through their drop-in program and demand continues to rise. The men of Seaton House are facing an uncertain future. The shelter is scheduled for demolition in 2019, according to the city’s website. Construction is expected to take place from 2020 to 2023, provided council approves funding in 2018. There is a relocation plan in the works, but regardless of where they live many will continue to face the challenge of trying to live in Toronto on Ontario Works or the Ontario Disability Support Program. Most single men on ODSP who are living in social housing would be left with about $649 each month after paying rent. Those paying market rents would have far less. “ODSP just doesn’t cover the cost of living and eating well. It just doesn’t match what people need anymore,” said Danielson. Danielson, who helps run the class, said healthy food is a vital part of preserving health and preventing cardiovascular issues or conditions like diabetes from getting worse. Michael, a class participant and former Seaton House resident, now lives in supportive housing. In addition to ODSP, he receives what is known as a special food allowance, of $250 each month. “Seaton House was good to me in many ways,” particularly because of programs he was connected to, he said. He still relies on food banks for staple items, like pasta and rice and canned goods. The bulk of the allowance is spent on fresh produce and protein. One of the biggest appeals of the program, he said, is they are taught how to transform donated food items into decent meals, without a huge extra cost. “You get to meet people with a common purpose,” he said. Click here to access the Toronto Star article.

  • D2D Interactive report for greg

    [review_data]

  • Data to Decisions eBulletin #67: Team Work

    You did it! Over 120 teams contributed data to D2D 5.0! Watch for the interactive display platform coming October 4th. We’re putting together a webinar to showcase ways you can use the data you worked so hard on. Watch this space for more information and registration link! Help to convert your information to action: Do your own assessment to see how ready your team is to commit to (and get support for) reaching a few specific quality goals. For more info, register for the Information to Action session at the AFHTO 2017 Conference. Who IS that unmasked team?  In D2D 5.0, you will be able to see how individual teams are performing on each indicator– and then call them up to find out how they got there. (Note: this is ONLY for teams who explicitly agreed to be unmasked when they submitted their data.) Did you know your whole team can help patients with medication management? Join us in Toronto for Managing Medication as a Team, Friday, November 17th from 10:30am-3:30pm, to learn more. Come as a group, and put the inter– into interprofessional! In Case You Missed It: Check out eBulletin #66 or peruse other eBulletin back issues here!

    D2D 5.0 Timeline

    Help spread the word about D2D – invite others to sign up for the eBulletin online.