Ultimate Blogging Championship

Tag: Members Only

  • SAMI score

    Interpretive Notes Steps to Improvement Data Quality Actions

    For technical notes, please see page 15 of the Data Dictionary.

    Interpretive Notes

    Tips to help you understand the data and put it in context.

    • The ‘average’ patient or population has a SAMI score of 1.0.
      • A SAMI score of 1.40 can be interpreted as an expected need for primary health care that is 40% higher than in the average patient.
      • A SAMI score of 0.88 can be interpreted as a 12% lower-than-average expected need.
    • Patients who have very complex needs for specialized care (i.e., oncologist for cancer, endocrinologists for diabetes) might not have higher-than-average needs for PRIMARY care and therefore may not contribute to a higher SAMI score.
    • Among CHCs, where SAMI scores have been reported for several years, some patient populations have scores of nearly 3.0 (very high), with scores of 1.4 considered to be low.
    • The range of SAMI scores among FHTs contributing data to D2D 3.0 was 0.73 to 1.2 with a provincial range of 0.95 to 1.84 (depending on the type of primary care model).
    • Primary care documentation can theoretically affect SAMI score. If providers routinely use the same, non-specific code for visits by patients for different issues (e.g., “visit for medication renewal” instead of a more specific, diagnosis-related code) the SAMI score could theoretically under-estimate the needs for primary care. However, the scoring system has been validated in both Ontario and Manitoba and shown to be very stable, even with the current state of primary care documentation.

    Steps to Improvement

    Since SAMI score is essentially a description of patient primary care needs, it is not a reflection of quality of care and there is no “target” measure teams could or should strive for. Therefore, any improvement efforts are probably most usefully targeted at data quality, not at changing the SAMI score for a team.

    Data Quality Actions

    Tips to help you understand the quality of your data and, if necessary, take steps to improve it. If addressing SAMI score accuracy is an immediate priority for your team:

    • Work with engaged clinicians to increase the specificity of diagnosis data in your billing processes.
    • Other ideas: please share!

  • Cost

    Cost Sub-Categories Interpretive Notes Steps to Improvement Data Quality Actions

    For technical notes, please see page 11 of the Data Dictionary.

    Breakdown of Cost Sub-Categories

    1. Primary Care
    • General Practitioner – Fee-For-Service visits
    • FHO/FHN capitation costs
    • Non—Fee-For-Service General Practitioner/Family Physician visits
    1. Physician, Lab, drug, ED and outpatient costs
    • OHIP specialty physician Fee-For-Service costs
    • Ontario Drug Benefit drug cost (e.g., seniors, people on disability)
    • Home Care Services cost
    • National Ambulatory Care Reporting Service ED (e.g., ED visits)
    • OHIP lab cost
    • OHIP non-physician cost
    • Other non—Fee-For-Service visits
    • Emergency Department Alternate Funding Arrangement non-Fee-For-Service visits
    • Non—Fee-For-Service medical oncologists
    • Non—Fee-For-Service radiation oncologists
    • National Ambulatory Care Reporting Service cancer (e.g., day surgery or treatment)
    • National Ambulatory Care Reporting Service dialysis (e.g., day hospitalization for treatment)
    1. Inpatient and same day surgery costs
    • Inpatient (CIHI/DAD)
    • Same Day Surgery (SDS)
    • Inpatient Mental Health
    1. Long Term Care, Complex Continuing Care and Rehab costs
    • Long Term Care cost
    • Complex Continuing Care cost
    • Rehab (NRS)

    Interpretive Notes

    Tips to help you understand the data and put it in context. Cost has been identified as one of the priority measures for system-level performance by HQO and will therefore be eventually included in system-level performance reports. In the meantime, D2D remains the only primary care reporting process to include per capita cost data. D2D 1.0 was the first time cost data were shared with primary care providers at a team level, although these data have been used in research and policy decision-making for several years. The inclusion of cost data fully embodies the intent of D2D to be a “START-egy,” a tool to get started at meaningful measurement in primary care. As such, the main value of these data was to initiate conversations to refine the measure based on the wisdom of frontline primary care providers to make this measure meaningful and actionable over time. Another value of these data was to make it possible to demonstrate the relationship between lower costs and higher quality, based on data from D2D 2.0 and 3.0. While analyses and refinements continue with this indicator, it is possible that it will function more as a system-level indicator than a metric for particular attention at the team-level.

    • There has been a change in how cost is calculated between D2D 3.0 and D2D 4.0. Prior to D2D 4.0 costs were calculated over a 2-year period. For D2D 4.0, ICES changed that timeframe to 1 year. Teams looking to compare cost over iterations of D2D should refer to the team-level report, as comparing between D2D iterations will not be meaningful. Efforts are underway to investigate the feasibility to update previous iterations of D2D data with 1-year costs to make comparison over time easier in D2D.
    • Unadjusted total costs do not take into account how sick patients are. Consider focusing on ADJUSTED total costs to allow comparisons between teams to be more meaningful.
    • Because costs for long term care are considerably higher than costs in most other categories, costs are broken down into 4 categories: primary care; physician, lab, drug, ED and outpatient; inpatient and same day surgery costs; long term care, complex continuing care and rehab (see technical notes). Further exploration with AFHTO members may help clarify the extent to which any of these categories are sensitive to primary care interventions.

    Readers are referred to emerging research (Wodchis and Laberge and others, personal communication) on health care system costs which seems to indicate that differences in costs for patient care by different models, pre-dated the implementation of the models and thus may be related to factors beyond the model of care itself.

    Steps to Improvement

    Concrete steps you can take to improve care, based on your data. Assuming you have established that the data are good enough to direct action AND that improving performance in this area is a priority for your team, you may wish to discuss the following options with your clinical leaders, Quality Improvement committees, team staff and/or patients:

    • Check out how your peers are doing by looking at the D2D report to determine their performance with access and connect with them to either spread any processes they find helpful or collaboratively test some new changes that might work for you AND your peers. HQO’s QIP Navigator allows teams to query submitted QIPs; this tool is extremely useful to identify peers who have focused on similar areas for improvement.
    • Consider exploring the Choosing Wisely campaign for change ideas and share your ideas about inclusion of some of the Choosing Wisely metrics in subsequent iterations of D2D.

     

    Data Quality Actions

    Tips to help you understand the quality of your data and, if necessary, take steps to improve it. Estimate impact of data quality:

    Increase quality of the data If the “imperfect data impact calculator” shows that the issues in your data may point you to a different action than suggested in the report, you might consider:

    • Most of the work to improve data quality for this indicator lies in refining the definitions as the data are captured via administrative information systems across all health care sectors and thus beyond the influence of primary care providers. Primary care contributions to improving data quality would therefore be thoughtful reflections on refinements to the definition, to be considered for presentation in subsequent iterations of D2D.
    • Other ideas: please share!

     

    Additional information for estimating the impact of data quality for this measure:

    The data are almost certainly not a definitive estimate of your team’s actual performance. However, they might be “good enough” to help you decide if your team needs to improve or not. To determine if the data are “good enough” for that, estimate how likely it is that one or more of the issues outlined in the interpretive notes above are a problem with your team. Then, run the “imperfect data impact calculator” to see if the issue(s) could lead to a different decision related to the need for improvement. To do this, work with your clinical leaders and staff to establish an approximate impact of data quality – i.e., is the data quality issue is causing your performance to look like TWICE or HALF or 10% (or other number) less or more than it actually is. Plug that number into the “imperfect data impact calculator.” It will show you whether the data quality issue(s) you think you have would change your initial decision regarding the need to improve. You may find it hard to generate consensus about the possible impact of data quality issues on the level of performance shown in the D2D report. In that case, try the following options:

    • Explore the proportion of your patients who are long-term care residents to estimate how much impact their costs are having on your overall team cost. Perhaps the rate among these patients will shift your team’s overall rate to be TWICE or HALF or 10% (or some other number) of the rate in the report. Plug that number into the “imperfect data impact calculator” and proceed accordingly.
    • Alternatively, consider instead experimenting with possible “error” rates to see how much error (i.e., TWICE or HALF or 10% of some other number) would be needed to change the decision made on the basis of the performance of the indicator in D2D. If, in the opinion of the team, such an amount of error is reasonable, then it may be worth considering efforts to improve data quality. Alternatively, if that amount of error is considered to be unlikely, then the data are likely good enough to support the initial decision regarding the need to improve, based on the performance shown in D2D.

    If the “imperfect data impact calculator” points to the same decision (e.g., a need to improve or NOT) even after data quality issues are considered, the data are likely “good enough” to base your decision on regarding the need to improve. The next step is to consider strategies to improve, assuming the area of care measured by the indicator is a priority for your team. If your data are not “good enough”, you may then consider taking action to better understand the issues affecting data quality, before or at the same time as you try to improve processes of care.

  • Readmissions to hospital

    Interpretive Notes Steps to Improvement Data Quality Actions

    For technical notes, please see page 24 of the Data Dictionary.

    Interpretive Notes

    Tips to help you understand the data and put it in context.

    • This indicator has been risk adjusted for age, sex and co-morbidities. Adjustment takes into account the differences among patient populations to allow for more meaningful comparisons between your patients and other populations. Adjusted data are easier and more meaningful to compare between teams. However, unadjusted data may provide an estimate that better reflects what is actually happening in your team and thus might help guide local improvement efforts.
    • The readmission rate for a primary care organization is based on the experience of patients on the roster of that organization AND patients who are considered to be “virtually” rostered according to MOHLTC methodology. Virtual rostering assigns patients to the primary care physician that provided the highest dollar amount of services within a defined set of primary care services. Primary care organizations may not be aware that patients have been “virtually” rostered to them and thus might think the data related to these patients are erroneously attributed to their team (i.e., “they are not ‘our’ patients”). Hence, your team’s sense of how many readmissions should be attributed to the team may be different than the rate shown in D2D.
    • The data refer to hospitalization and readmissions that may have happened as much as 1.5 years ago (on average) because they are based on hospital data submitted to CIHI 2-6 months after discharge (on average), after which they must be compiled and validated prior to release for reporting purposes.
    • The current definition may under-estimate actual readmission rates for patients who have preventable readmissions because the denominator includes ALL patients who were hospitalized for any reason.
      • Readmissions may appear to be lower for teams with a higher proportion of child-bearing women because childbirth is one of the most common reasons for hospitalization and thus will increase the denominator, artificially decreasing the overall rate of readmissions.
      • The same is true for teams with high proportions of young, healthy patients needing elective surgeries, which are not nearly as common as birth as a reason for hospitalization, but still would reduce the overall readmissions rate because readmissions in such situations are rare.
    • Many primary care providers do not get timely information about recent hospitalizations of their patients. Teams who do not know if their patients have recently been in hospital may therefore have higher readmission rates than teams with timely access to data, who are better able to engage with patients and other providers to prevent readmissions.
    • There are many challenges in preventing readmissions, not all of which are solely under the control of primary care providers such as premature discharge from hospital and the natural progression of chronic conditions. Consider the possible impact of these factors on your team’s readmission rates.

    Steps to Improvement

    Concrete steps you can take to improve care, based on your data. Indicators based on administrative data tend to be the oldest of all indicators in D2D. Improving the timeliness of administrative data is a priority for AFHTO and HQO and others. And in the meantime, there are things teams can do to use these “old” data to fuel current, local efforts to improve. Assuming you have established that the data are good enough to direct action AND that improving performance in this area is a priority for your team, you may wish to discuss the following options with your clinical leaders, Quality Improvement committees, team staff and/or patients:

    Data Quality Actions

    Tips to help you understand the quality of your data and, if necessary, take steps to improve it. Estimate impact of data quality:

    Increase quality of the data If the “imperfect data impact calculator” shows that the issues in your data may point you to a different action than suggested in the report, you might consider:

    • Sign up with OntarioMD to implement direct EMR access to hospital discharge data via Hospital Report Manager (HRM). Some teams already have this access via HRM or similar regional systems such as SPIRE, POI or TDIS. This process takes some time to implement, so it is worthwhile to act as soon as possible to get the process started. Check out the Ontario MD site for progress reports on HRM implementation and stories about how teams are using it to improve work flow and patient outcomes.
    • Make arrangements with the hospitals most commonly visited by your team’s patients to receive extracts of data from hospital discharges and ER visits. You can then manually update the EMR with this information. Many teams are already doing this with the help of their QIDS Specialists. Contact your QIDS Specialist or AFHTO QIDS program staff for help.
    • Establish a communication process for direct notification of your team about hospitalizations on an individual-patient basis via fax, phone calls or other method. This also requires manual updating of the EMR. It also may involve some work on the part of the hospital and therefore may involve some negotiation.
    • Review hospital information systems to find hospitalizations of your patients and then manually update the EMR. This assumes appropriate permission to access these systems is in place, as is often the case when your team’s physicians also attend at the hospital’s Emergency Department or inpatient units.
    • Other ideas: please share!

    Additional information for estimating the impact of data quality for this measure:

    The data are almost certainly not a definitive estimate of your team’s actual performance in the area of access. However, they might be “good enough” to help you decide if your team needs to improve access or not. To determine if the data are “good enough” for that, estimate how likely it is that one or more of the issues outlined in the interpretive notes are a problem with your team. Then, run the “imperfect data impact calculator” to see if the issue(s) could lead to a different decision related to the need for improvement. To do this, work with your clinical leaders and staff to establish an approximate impact of data quality – i.e., is the data quality issue causing your performance to look like TWICE or HALF or 10% (or other number) less or more than it actually is? Plug that number into the “imperfect data impact calculator.”  It will show you whether the data quality issue(s) you think you have would change your initial decision regarding the need to improve. You may find it hard to generate consensus about the possible impact of data quality issues on the level of performance shown in the D2D report. In that case, try the following options:

    • Track the next 10 (or 20 or other small number) encounters to get a better estimate of the extent of the data quality issue. Perhaps the rate among these patients will shift your team’s overall rate to be TWICE or HALF or 10% (or some other number) of the rate in the report. Plug that number into the “imperfect data impact calculator” and proceed accordingly.
    • Estimate how many of your patients have chronic disease conditions that could be expected to result in hospitalizations as part of the normal course of the disease. Also, estimate how many of your patients are likely to be hospitalized for issues not usually associated with readmissions (e.g., birth, minor, elective surgeries). Compare to the proportion in other teams to get a sense of whether the difference is likely to lead to readmissions at TWICE or HALF or 10% (or some other number) of the rate in report. Plug that number into the “imperfect data impact calculator” and proceed accordingly.
    • Estimate how many hospitalizations your team is notified of and, by extension, how many you are NOT made aware of in a timely way. Consider how many readmissions might be related to your team’s lack of awareness of the initial hospitalization and estimate whether that would lead to readmissions at TWICE or HALF or 10% (or some other number) of the rate in report. Plug that number into the “imperfect data impact calculator” and proceed accordingly.
    • If none of the above is helpful, consider instead experimenting with possible “error” rates to see how much error (i.e., TWICE or HALF or 10% of some other number) would be needed to change the decision made on the basis of the performance of the indicator in D2D. If, in the opinion of the team, such an amount of error is reasonable, then it may be worth considering efforts to improve data quality. Alternatively, if that amount of error is considered to be unlikely, then the data are likely good enough to support the initial decision regarding the need to improve, based on the performance shown in D2D.

    If the “imperfect data impact calculator” points to the same decision (i.e., a need to improve or NOT) even after data quality issues are considered, the data are likely “good enough” to base your decision on regarding the need to improve. The next step is to consider strategies to improve, assuming the area of care measured by the indicator is a priority for your team. If your data are not “good enough,” you may then consider taking action to improve your data quality, before or at the same time as you try to improve processes of care.

  • Same/next day appointments

    Interpretive Notes Steps to Improvement Data Quality Actions

      For technical notes, please see page 18 of the Data Dictionary.

    Interpretive Notes

    Tips to help you understand the data and put it in context.

    • There is a chance that patients may not recall how long they waited for an appointment the last time they visited their provider.
      • There could be bias in either direction, with the patient wanting to be positive (i.e., under-estimating the time) or the patient being generally unsatisfied and therefore over-estimating the time.
    • There is likely inconsistency in the recall of patients regarding whether the person they saw was their “own” provider or someone else on the team.
    • The level of access to care in your team might be lower than indicated by D2D because it only considers the access of patients who eventually had an appointment. Patients who didn’t get an appointment soon enough (or when they wanted it) might never have shown up and instead may have either gone nowhere or to a walk-in clinic, an Emergency Department or another team. Therefore, these patients would never have been included in the survey.
    • For reasons listed below, this  indicator might not be the best measure of access, and you may wish to consider it in context of Reasonable wait for appointment, which measures how many patients report  they were able to get an appointment within a reasonable amount of time.

    Patient Choice:

    • Performance on this measure does not necessarily reflect patient choice because some patients would PREFER to have an appointment set several days in advance, instead of on the same or next day, as it makes it easier for them to plan.
    • The measure might also be counterintuitive to the known desire of patients to see their “own” regular provider. Efforts to increase the proportion of patients getting appointments within the same/next day might be counter-productive to efforts aimed at helping patients see their “own” regular provider as much as possible.

    Measurement Error:

    • In spite of best efforts, there may be inconsistency in the wording of the question between teams.

    Other Issues with Current Definition:

    • The measure is counterintuitive to the purpose of interprofessional team-based care when the wording specifies access to an appointment with the patient’s “own” regular provider. Team-based care suggests that anyone in the team could and should be able to help the patient, whenever they need it.

    Steps to Improvement

    Concrete steps you can take to improve care, based on your data. Assuming you have established that the data are good enough to direct action AND that improving performance in this area is a priority for your team, you may wish to discuss the following options with your clinical leaders, Quality Improvement committees, team staff and/or patients:

    • Get more current data to move beyond the idea that “the data are too old to make improvements” OR test out a new question that wasn’t previously included in your survey.
      • Step 1: Work with a physician who is interested in improving patient experience or work through your ED to get the conversation started with your team’s Medical Lead.
      • Step 2: Suggest tracking patient experience for a small subset of patients (i.e., the next 10 patients that come in) or for a narrowly defined timeframe (i.e., whatever you get in the next 2 weeks).
      • Step 3: Look at the data to see how this current “snapshot” compares to the “old” data
      • Step 4: Take this opportunity to discuss sample size. See resources on web. You will see that you do NOT need hundreds of surveys to know how your patients are feeling and to start making improvements.
      • Step 5: Consider setting up a frequent, ongoing automated patient experience survey process (See Data Quality Actions – how to increase quality of the data using new tools to capture and track patient experience data).

    Data Quality Actions

    Tips to help you understand the quality of your data and, if necessary, take steps to improve it. Estimate impact of data quality:

    Increase quality of the data If the “imperfect data impact calculator” shows that the issues in your data may point you to a different action than suggested in the report, you might consider:

    Additional information for estimating the impact of data quality for this measure:

    The data are almost certainly not a definitive estimate of your team’s actual performance in the area of access. However, they might be “good enough” to help you decide if your team needs to improve access or not. To determine if the data are “good enough” for that, estimate how likely it is that one or more of the issues outlined in the interpretive notes are a problem with your team. Then, run the “imperfect data impact calculator” to see if the issue(s) could lead to a different decision related to the need for improvement. To do this, work with your clinical leaders and staff to establish an approximate impact of data quality (i.e. is it the data quality issue that is causing your performance to look like TWICE or HALF or 10% (or other number) less or more than it actually is?) Plug that number into the “imperfect data impact calculator”. It will show you whether the data quality issue(s) you think you have would change your initial decision regarding the need to improve. You may find it hard to generate consensus about the possible impact of data quality issues on the level of performance shown in the D2D report. In that case, try the following options:

    • Track the next 10 (or 20 or other small number) encounters to get a better estimate of the extent of the data quality issue. Perhaps the rate among these patients will shift your team’s overall rate to be TWICE or HALF or 10% (or some other number) of the rate in the report. Plug that number into the “imperfect data impact calculator” and proceed accordingly.
    • Compare to other sources of data to see if the rate with other/better data is TWICE or HALF or 10% (or some other number) of the rate in the report. Plug that number into the “imperfect data impact calculator” and proceed accordingly. Other sources of data may include your scheduling system (e.g. actual appointment data), your EMR (queries or chart audits on a small number of patients), other pre-existing reports (CPCSSN, EMRALD, SAR, Physician Profile, MOHLTC, QIP, commercially provided or others) or personal interviews with a few patients, to name a few.
    • If none of the above is helpful, consider instead experimenting with possible “error” rates to see how much error (i.e., TWICE or HALF or 10% of some other number) would be needed to change the decision made on the basis of the performance of the indicator in D2D. If, in the opinion of the team, such an amount of error is reasonable, then it may be worth considering efforts to improve data quality. Alternatively, if that amount of error is considered to be unlikely, then the data are likely good enough to support the initial decision regarding the need to improve, based on the performance shown in D2D.

    If the “imperfect data impact calculator” points to the same decision (i.e., a need to improve or NOT) even after data quality issues are considered, the data are likely “good enough” to base your decision on regarding the need to improve. The next step is to consider strategies to improve, assuming the area of care measured by the indicator is a priority for your team. If your data are not “good enough”, you may then consider taking action to improve your data quality, before or at the same time as you try to improve processes of care.

  • Organizational / Operational Practices: Patient Relationships and Privacy

    Please share your resources to help build our library. 

    Patient Privacy & Personal Health Information Protection

    General Privacy Policies

    Breach of Privacy

    Consent to Disclose

    Confidentiality Agreements

    Email Consent Forms

    EMR & Computer Security

    Patient Access to Health Records

    Corrections to Personal Health Information

    Return to the Library of Organizational Policies and Procedures.

    Data-Sharing Agreements

    Patient Rights & Responsibilities

    Return to the Library of Organizational Policies and Procedures.

    Accessibility for Patients with Disabilities

    Return to the Library of Organizational Policies and Procedures.

    Anti-Discrimination, Anti-Harassment, and Anti-Abuse Policies

    Return to the Library of Organizational Policies and Procedures.

    Patient Complaint and Dispute Resolution Policies

    Return to the Library of Organizational Policies and Procedures.

     

     

  • Colorectal Cancer Screening

    Interpretive Notes Steps to Improvement Data Quality Actions

      For technical notes, please see page 22 of the Data Dictionary.

    Interpretive Notes

    Tips to help you understand the data and put it in context.

    • Virtual rostering assigns patients to the primary care physician that provided the highest dollar amount of services within a defined set of primary care services. Your team may not be aware of which patients have been virtually rostered to them. Therefore, you may erroneously think that these patients are not “your” patients. As a result, your team’s sense of how many of “your” patients were screened may be different than shown in D2D.
    • Screening tests performed in hospital laboratories or paid through alternate payment plans are not currently incorporated into this measure. Actual performance on this measure for teams that use hospital laboratories is therefore likely higher than the level presented in D2D.
    • Please note: for colorectal cancer screening, a small proportion of FOBTs performed as diagnostic tests for other conditions besides cancer could not be excluded from the analysis.
    • Inaccurate recording of exclusion criteria may result in an under-estimation of screening rates as patients who are not eligible for screening would be erroneously included in the denominator, artificially driving the observed rate down.
    • The current measure does not consider patient choice in screening and therefore might reflect an under-estimate of the screening “interventions” (i.e., consultations/advice to undertake screening) by the team.

    Steps to Improvement

    Concrete steps you can take to improve care, based on your data. Assuming you have established that the data are good enough to direct action AND that improving performance in this area is a priority for your team, you may wish to discuss the following options with your clinical leaders, Quality Improvement committees, team staff and/or patients:

    Data Quality Actions

    Tips to help you understand the quality of your data and, if necessary, take steps to improve it. Estimate impact of data quality:

    Increase quality of the data If the “imperfect data impact calculator” shows that the issues in your data may point you to a different action than suggested in the report, you might consider:

    • Get current cancer screening data from your EMR: The QIDS Specialist have developed standardized EMR queries for cancer screening. Try them. Now that they are developed, they should take very little time to run on an ongoing basis (rather than just once a year for reporting purposes). The data might not be directly comparable to what is in D2D because it is from a different time-period and may have more information about patient eligibility for screening. However, it will give you a sense of how your team is doing over time. More importantly, having a list of specific patients that might be overdue for screening gives your team something concrete to do now about something they care about (i.e., patients).
    • Record exclusion criteria more completely and more consistently on both cancer screening test documentation and in the EMR.
      • e.g., Exclude patients who have had a total colectomy. Ask your patients about their participation in screening in hospital labs and update the EMR manually to provide a more accurate estimate of screening performance locally, even if this gap remains in the central database.
    • Devise a mechanism to formally, consistently record invitations for screening for these and other cancers, to make it easier to extract screening information that incorporates patient choice.
    • Perform a manageable sized chart audit (e.g., a maximum of 30 eligible patients has been suggested as a manageable number) and double check that appropriate screening was conducted and recorded in the chart or that patient choice to forego screening is noted. This is not a statistical exercise. It is intended only as a quick check to inform decisions about next steps to improve data quality.
      • Consider hiring a student to help generate a list of eligible patients and check against information in their chart.
    • Other ideas: please share!

    Additional information for estimating the impact of data quality for this measure:

    The data are almost certainly not a definitive estimate of your team’s actual performance. However, they might be “good enough” to help you decide if your team needs to improve or not. To determine if the data are “good enough” for that, estimate how likely it is that one or more of the issues outlined in the interpretive notes are a problem with your team. Then, run the “imperfect data impact calculator” to see if the issue(s) could lead to a different decision related to the need for improvement. To do this, work with your clinical leaders and staff to establish an approximate impact of data quality – i.e., is the data quality issue causing your performance to look like TWICE or HALF or 10% (or other number) less or more than it actually is. Plug that number into the “imperfect data impact calculator.” It will show you whether the data quality issue(s) you think you have would change your initial decision regarding the need to improve. You may find it hard to generate consensus about the possible impact of data quality issues on the level of performance shown in the D2D report. In that case, try the following options:

    • Track the next 10 (or 20 or other small number) encounters with patients who meet the inclusion criteria for cancer screening to get a better estimate of the extent of the data quality issues (i.e., check their chart for any record of screening, if absent ask patient whether they have been screened and educate patient on the purpose of screening, record patient choice, especially where the choice to not screen is made). Estimate how many of your patients get their tests done in hospital laboratories and estimate what the screening rate is among those patients.
    • Estimate how many of your patients decline screening when offered.
    • Extract data from the EMR to determine how many eligible patients were screened. Consult with your QIDS Specialist to ensure the definitions used for the data extraction are consistent with those being developed and deployed across the QIDS Specialist network of approximately 150 AFHTO member organizations. Perhaps the rate among these patients will shift your team’s overall rate to be TWICE or HALF or 10% (or some other number) of the rate in the report. Plug that number into the “imperfect data impact calculator” and proceed accordingly.
    • If none of the above is helpful, consider instead experimenting with possible “error” rates to see how much error (i.e., TWICE or HALF or 10% of some other number) would be needed to change the decision made on the basis of the performance of the indicator in D2D. If, in the opinion of the team, such an amount of error is reasonable, then it may be worth considering efforts to improve data quality. Alternatively, if that amount of error is considered to be unlikely, then the data are likely good enough to support the initial decision regarding the need to improve, based on the performance shown in D2D.

    If the “imperfect data impact calculator” points to the same decision (i.e., a need to improve or NOT) even after data quality issues are considered, the data are likely “good enough” to base your decision on regarding the need to improve. The next step is to consider strategies to improve, assuming the area of care measured by the indicator is a priority for your team. If your data are not “good enough,” you may then consider taking action to improve your data quality, before or at the same time as you try to improve processes of care.

  • Patient experience: involved

    Interpretive Notes Steps to Improvement Data Quality Actions

      For technical notes, please see page 16 of the Data Dictionary.

    Interpretive Notes

    Tips to help you understand the data and put it in context.

    • The performance shown on these indicators in D2D is usually based only on patients either currently in the office or having just had a recent visit, since most teams survey patients in this way. It is possible that this may not be representative of patients who did not have appointments. For example, patients who feel they were not involved as much as they wanted or felt that the office staff were not courteous may be choosing to get care elsewhere (e.g., walk-in clinic, Emergency Department, other provider) or nowhere at all.
    • Teams that do not feel able to do anything to improve patient experiences in these areas may decline to ask these questions. They may be concerned that asking the questions may set false expectations among patients that their input will prompt changes. As a result, the performance level may represent only teams that would consider interventions to improve patient experience in these areas. Consequently, the rates may not be representative of all teams.
      • It could be an over-estimate of actual patient experience if one assumes that teams with good patient experience are more likely to consider interventions.
      • Alternatively, it could be an under-estimate if one assumes that teams for whom patient experience is not good are more likely to be considering interventions to improve their performance in this regard.
    • There appears to be little difference in performance in responses to patient experience survey (PES) questions over time. Most questions on most surveys show a high percent of patients with positive experiences. This may be real and it may also be what is known as “ceiling effect” (i.e., bunching of responses at the top end of the scale), possibly because patients want to be positive about their experience with their provider.

    Steps to Improvement

    Concrete steps you can take to improve care, based on your data. Assuming you have established that the data are good enough to direct action AND that improving performance in this area is a priority for your team, you may wish to discuss the following options with your clinical leaders, Quality Improvement committees, team staff and/or patients:

    • Contact your peers (in addition to those involved in the initiatives described above) to determine their performance with patient experience and work with them to either spread any processes they find have helped them or collaboratively test some new changes that might work for you AND your peers. HQO’s QIP Navigator allows teams to query submitted QIPs, this tool is extremely usefully to identify peers who have focused on similar areas for improvement.
    • Get your patients involved! It doesn’t end with the Patient Experience survey. Consider sharing your D2D data with them (check out an example of a waiting-room poster here), and getting them involved with your QI initiatives.

    Data Quality Actions

    Tips to help you understand the quality of your data and, if necessary, take steps to improve it. Estimate impact of data quality:

    Increase quality of the data If the “imperfect data impact calculator” shows that the issues in your data may point you to a different action than suggested in the report, you might consider:

    Additional information for estimating the impact of data quality for this measure:

    The data are almost certainly not a definitive estimate of your team’s actual performance in the area of access. However, they might be “good enough” to help you decide if your team needs to improve patient-centredness or not. To determine if the data are “good enough” for that, estimate how likely it is that one or more of the issues outlined in the interpretive notes are a problem with your team. Then, run the “imperfect data impact calculator” to see if the issue(s) could lead to a different decision related to the need for improvement. To do this, work with your clinical leaders and staff to establish an approximate impact of data quality (i.e., is it the data quality issue that is causing your performance to look like TWICE or HALF or 10% (or other number) less or more than it actually is?). Plug that number into the “imperfect data impact calculator.” It will show you whether the data quality issue(s) you think you have would change your initial decision regarding the need to improve. You may find it hard to generate consensus about the possible impact of data quality issues on the level of performance shown in the D2D report. In that case, try the following options:

    • Track the next 10 (or 20 or other small number) encounters to get a better estimate of the extent of the data quality issue. Perhaps the rate among these patients will shift your team’s overall rate to be TWICE or HALF or 10% (or some other number) of the rate in the report. Plug that number into the “imperfect data impact calculator” and proceed accordingly.
    • Compare to other sources of data to see if the rate with other/better data is TWICE or HALF or 10% (or some other number) of the rate in the report. Plug that number into the “imperfect data impact calculator” and proceed accordingly. Other sources of data may include your scheduling system (e.g., actual appointment data), your EMR (queries or chart audits on a small number of patients), other pre-existing reports (CPCSSN, EMRALD, SAR, Physician Profile, MOHLTC, QIP, commercially provided or others) or personal interviews with a few patients, to name a few.
    • If neither of the above is helpful, consider instead experimenting with possible “error” rates to see how much error (i.e., TWICE or HALF or 10% of some other number) would be needed to change the decision made on the basis of the performance of the indicator in D2D. If, in the opinion of the team, such an amount of error is reasonable, then it may be worth considering efforts to improve data quality. Alternatively, if that amount of error is considered to be unlikely, then the data are likely good enough to support the initial decision regarding the need to improve, based on the performance shown in D2D.

    If the “imperfect data impact calculator” points to the same decision (i.e., a need to improve or NOT) even after data quality issues are considered, the data are likely “good enough” to base your decision on regarding the need to improve. The next step is to consider strategies to improve, assuming the area of care measured by the indicator is a priority for your team. If your data are not “good enough,” you may then consider taking action to improve your data quality, before or at the same time as you try to improve processes of care.

  • Childhood Immunization – all children

    Interpretive Notes Steps to Improvement Data Quality Actions

    For technical notes, please see page 25 of the data dictionary.

    Interpretive Notes

    Tips to help you understand the data and put it in context.

    • The D2D definition fully aligns with Public Health criteria. This means Rotavirus is now included in the definition and EMR queries. It is not part of the preventive care bonus for physicians. Therefore, performance may appear lower in D2D than in your preventive care bonus reports which exclude Rotavirus.
    • The D2D definition does not reflect patient choice. Patients who decline immunization and therefore are not immunized may be the reason why your rates are lower than you expect. See “data quality actions” for ideas to examine the extent to which this is affecting your rates.
    • Data for patients immunized outside of the primary care team (e.g., at a health unit) might not be recorded consistently in all EMRs and teams. The performance seen in D2D might therefore under-estimate actual immunization rates. See “data quality actions” for ideas to examine the extent to which this is affecting your rates.
    • Rates for teams with very few children in their panel may be more variable than rates based on larger eligible patient populations. For example, if 2 less children are immunized this year out of a population of 10 children, your immunization rate will drop by 20%. However, if 2 less children out of 100 are not immunized, your rate will only drop by 2%. Consider how many children are eligible for immunization when interpreting differences for your team year to year or relative to another team.
    • D2D includes data for all children as opposed to only rostered children. This might generate different rates than those the team might be used to seeing in reports based on rostered children only (g., the MOHLTC Preventive Care Target Population/Service Report (TPSR)). This report is provided by MOHLTC to eligible physicians in Patient Enrolment Models (PEMs) in April and September by MOHLTC to assist physicians in determining their Target Population and the delivery of preventive care services.
    • The timing of D2D reporting may not coincide with the reporting time period for the MOHLTC Preventive Care Target Population/Service Report. There may be differences in rates related to these differences in time periods.
    • Although a consistent definition was developed to create queries that were shared among members for purposes of extracting these data for D2D, it is possible that the extent to which these data were consistently recorded and therefore extracted in your team might vary. To explore this issue, look at the processes used in your team to record and extract immunization data and work to align it as much as possible with the standard process being developed by QIDS Specialists.

    Steps to Improvement

    Concrete steps you can take to improve care, based on your data. Assuming you have established that the data are good enough to direct action AND that improving performance in this area is a priority for your team, you may wish to discuss the following options with your clinical leaders, Quality Improvement committees, team staff and/or patients:

    Data Quality Actions

    Tips to help you understand the quality of your data and, if necessary, take steps to improve it. Estimate the impact of data quality

    Increase the quality of the data If the “imperfect data impact calculator” shows that the issues in your data may point you to a different action than suggested in the report, you might consider:

    • Work with engaged clinicians to increase the consistency of data entry related to immunization. Refer to the Standardized EMR queries for guidance on ways to increase consistency in data entry such that extraction of the data can be more efficient and accurate over time and between teams.
    • Track your immunization rates on an ongoing basis vs. just at year-end. Work with QIDS Specialists in the application of consistent queries across teams and EMRs to increase access to these data in an efficient way.
    • Ask patients about their immunizations, partly to update the EMR and partly to engage them in an important discussion about preventive strategies. Tilbury District FHT uses a simple paper form for patients to share information regarding flu shots received elsewhere. A similar idea could be applied to childhood immunizations. Better yet, contact Tilbury directly to find out exactly how this works!
    • Support AFHTO’s efforts to work for inclusion of a standard process for recording and reporting immunization data aligned with PHAC guidelines as part of the technical specification for EMRs. This is already in progress through the EMR Data Management committee and individual EMR Communities of Practice.
    • Develop data-sharing processes with public health units and other partners engaged in immunization to ensure accurate records regarding immunization status, not only for reporting purposes but also for better risk-management for patients.
    • Other ideas: please share!

    Additional information for estimating the impact of data quality for this measure:

    The data are almost certainly not a definitive estimate of your team’s actual performance. However, they might be “good enough” to help you decide if your team needs to improve or not. To determine if the data are “good enough” for that, estimate how likely it is that one or more of the issues outlined in the interpretive notes above are a problem with your team. Then, run the “imperfect data impact calculator” to see if the issue(s) could lead to a different decision related to the need for improvement. To do this, work with your clinical leaders and staff to establish an approximate impact of data quality – i.e., is the data quality issue causing your performance to look like TWICE or HALF or 10% (or other number) less or more than it actually is? Plug that number into the “imperfect data impact calculator.” It will show you whether the data quality issue(s) you think you have would change your initial decision regarding the need to improve. You may find it hard to generate consensus about the impact of data quality issues on the level of performance shown in the D2D report. In that case, consider the following options:

    • Track the next 10 (or 20 or other small number) encounters to get a better estimate of the extent of the data quality issue. Perhaps the rate among these patients will shift your team’s overall rate to be TWICE or HALF or 10% (or some other number) of the rate in the report. Plug that number into the “imperfect data impact calculator” and proceed accordingly.
    • Estimate how many of your patients get their immunizations elsewhere. Perhaps the rate among these patients will shift your team’s overall rate to be TWICE or HALF or 10% (or some other number) of the rate in the report. Plug that number into the “imperfect data impact calculator” and proceed accordingly. [If you are having trouble estimating how many patients are getting immunizations/flu vaccines elsewhere you might want to use 43% as a ball park figure. This was the rate observed by Tilbury District FHT who use a simple paper form for patients to share information regarding flu shots received elsewhere.]
    • Estimate how many of your patients decline immunization when offered. Perhaps the rate among these patients will shift your team’s overall rate to be TWICE or HALF or 10% (or some other number) of the rate in the report. Plug that number into the “imperfect data impact calculator” and proceed accordingly.
    • If none of the above is helpful, consider instead experimenting with possible “error” rates to see how much error (i.e., TWICE or HALF or 10% of some other number) would be needed to change the decision made on the basis of the performance of the indicator in D2D 3.0. If, in the opinion of the team, such an amount of error is reasonable, then it may be worth considering efforts to improve data quality. Alternatively, if that amount of error is considered to be unlikely, then the data are likely good enough to support the initial decision regarding the need to improve, based on the performance shown in D2D.

    If the “imperfect data impact calculator” points to the same decision (i.e., a need to improve or NOT) even after data quality issues are considered, the data are likely “good enough” to base your decision on regarding the need to improve. The next step is to consider strategies to improve, assuming the area of care measured by the indicator is a priority for your team. If your data are not “good enough,” you may then consider taking action to improve your data quality, before or at the same time as you try to improve processes of care.

  • Cervical Cancer Screening

    Interpretive Notes Steps to Improvement Data Quality Actions

      For technical notes, please see page 23 of the Data Dictionary.

    Interpretive Notes

    Tips to help you understand the data and put it in context.

    • Virtual rostering assigns patients to the primary care physician that provided the highest dollar amount of services within a defined set of primary care services. Your team may not be aware of which patients have been virtually rostered to them. Therefore, you may erroneously think that these patients are not “your” patients. As a result, your team’s sense of how many of “your” patients were screened may be different than shown in D2D.
    • Screening tests performed in hospital laboratories or paid through alternate payment plans are not currently incorporated into this measure. Actual performance on this measure for teams that use hospital laboratories is therefore likely higher than the level presented in D2D.
    • Inaccurate recording of exclusion criteria may result in an under-estimation of screening rates, as patients who are not eligible for screening would be erroneously included in the denominator, artificially driving the observed rate down.
    • The current measure does not consider patient choice in screening and therefore might reflect an under-estimate of the screening “interventions” (i.e., consultations/advice to undertake screening) by the team.

    Steps to Improvement

    Concrete steps you can take to improve care, based on your data. Assuming you have established that the data are good enough to direct action AND that improving performance in this area is a priority for your team, you may wish to discuss the following options with your clinical leaders, Quality Improvement committees, team staff and/or patients:

    Data Quality Actions

    Tips to help you understand the quality of your data and, if necessary, take steps to improve it. Estimate impact of data quality:

    Increase quality of the data If the “imperfect data impact calculator” shows that the issues in your data may point you to a different action than suggested in the report, you might consider:

    • Get current cancer screening data from your EMR: The QIDSS have developed standardized EMR queries for cancer screening. Try them. Now that they are developed, they should take very little time to run on an ongoing basis (rather than just once a year for reporting purposes). The data might not be directly comparable to what is in D2D because it is from a different time-period and may have more information about patient eligibility for screening. However, it will give you a sense of how your team is doing over time. More importantly, having a list of specific patients that might be overdue for screening gives your team something concrete to do now about something they care about (i.e., patients).
    • Record exclusion criteria more completely and more consistently on both cancer screening test documentation and in the EMR.
      • e.g., Exclude patients who have had cervical, endometrial or ovarian cancer and patients who have had a hysterectomy. Ask your patients about their participation in screening in hospital labs and update the EMR manually to provide a more accurate estimate of screening performance locally, even if this gap remains in the central database.
    • Devise a mechanism to formally, consistently record invitations for screening for these and other cancers, to make it easier to extract screening information that incorporates patient choice.
    • Perform a small chart audit (e.g., a maximum of 30 eligible patients has been suggested as a manageable number) and double-check that appropriate screening was conducted and recorded in the chart or that patient choice to forego screening is noted. This is not a statistical exercise. It is intended only as a quick check to inform decisions about next steps to improve data quality.
      • Consider hiring a student to help generate a list of eligible patients and check against information in their chart.
    • Other ideas: please share!

    Additional information for estimating the impact of data quality for this measure:

    The data are almost certainly not a definitive estimate of your team’s actual performance. However, they might be “good enough” to help you decide if your team needs to improve or not. To determine if the data are “good enough” for that, estimate how likely it is that one or more of the issues outlined in the interpretive notes are a problem with your team. Then, run the “imperfect data impact calculator” to see if the issue(s) could lead to a different decision related to the need for improvement. To do this, work with your clinical leaders and staff to establish an approximate impact of data quality – i.e., is the data quality issue causing your performance to look like TWICE or HALF or 10% (or other number) less or more than it actually is. Plug that number into the “imperfect data impact calculator.” It will show you whether the data quality issue(s) you think you have would change your initial decision regarding the need to improve. You may find it hard to generate consensus about the possible impact of data quality issues on the level of performance shown in the D2D report. In that case, try the following options:

    • Track the next 10 (or 20 or other small number) encounters with patients who meet the inclusion criteria for cancer screening to get a better estimate of the extent of the data quality issues (i.e., check their chart for any record of screening and, if absent, ask patient whether they have been screened and educate patient on the purpose of screening, record patient choice, especially where the choice to not screen is made). Estimate how many of your patients get their tests done in hospital laboratories and estimate what the screening rate is among those patients.
    • Estimate how many of your patients decline screening when offered.
    • Extract data from the EMR to determine how many eligible patients were screened. Consult with your QIDSS to ensure the definitions used for the data extraction are consistent with those being developed and deployed across the QIDSS network of approximately 150 AFHTO member organizations. Perhaps the rate among these patients will shift your team’s overall rate to be TWICE or HALF or 10% (or some other number) of the rate in the report. Plug that number into the “imperfect data impact calculator” and proceed accordingly.
    • If none of the above is helpful, consider instead experimenting with possible “error” rates to see how much error (i.e., TWICE or HALF or 10% of some other number) would be needed to change the decision made on the basis of the performance of the indicator in D2D. If, in the opinion of the team, such an amount of error is reasonable, then it may be worth considering efforts to improve data quality. Alternatively, if that amount of error is considered to be unlikely, then the data are likely good enough to support the initial decision regarding the need to improve, based on the performance shown in D2D.

    If the “imperfect data impact calculator” points to the same decision (i.e., a need to improve or NOT) even after data quality issues are considered, the data are likely “good enough” to base your decision on regarding the need to improve. The next step is to consider strategies to improve, assuming the area of care measured by the indicator is a priority for your team. If your data are not “good enough,” you may then consider taking action to improve your data quality, before or at the same time as you try to improve processes of care.

  • Regular primary care provider – individual and team

    Individual: Interpretive Notes Steps to Improvement Data Quality Actions
    Team: Interpretive Notes Steps to Improvement Data Quality Actions

      For technical notes (individual), please see page 20 of the Data Dictionary. For technical notes (team), please see page 21 of the Data Dictionary.

    Interpretive Notes (Individual)

    Tips to help you understand the data and put it in context.

    • Virtual rostering assigns patients to the primary care physician that provided the highest dollar amount of services within a defined set of primary care services. Physicians in your team may not be aware of which patients have been virtually rostered to them. Therefore, they may erroneously think that these patients are not “their” patients. As a result, an individual physician’s sense of how many of patients they see that are “their” patients may be different than the rate shown in D2D.
    • Visits to health care providers other than physicians are not included in this measure. However, this does not necessarily skew the measure. For example, if a patient visits a primary care team 10 times and sees a physician 8 times, and each time it is their “own” physician, they will score 100% (8 out of 8) for “regular care provider – individual.” If, however, they visit 10 times, receiving care from multiple providers but only saw a physician once and it was their own physician, they could still score 100% on this measure.
    • Efforts to improve access to same or next day appointments may result in patients seeing whichever physician is available for appointments at the time. While this may be valuable from the perspective of access, this process may be reflected in poor performance on the “regular care provider – individual” measure.
    • Teams with part-time physicians and teaching teams may have developed strong relationships between physicians to jointly care for patients, such that patients may feel equally comfortable and familiar with more than one physician. However, while this might embody team-based care, it may be reflected in poor performance on the “regular care provider – individual” measure.

    Steps to Improvement (Individual)

    Concrete steps you can take to improve care, based on your data. Assuming you have established that the data are good enough to direct action AND that improving performance in this area is a priority for your team, you may wish to discuss the following options with your clinical leaders, Quality Improvement committees, team staff and/or patients:

    • Explore interventions to increase proportion of patients with regular care provider and/or interventions to improve continuity of care (i.e., increase the chances that they see the same provider each time).
      • Ask patients what is most important to them: Train front reception staff to discuss options with patients as part of the appointment-booking process (i.e., difference in wait times if patient wants to see their regular primary care provider vs. any primary care provider in the team).
      • Improve same day/next day access for all physicians (i.e., ensure all physicians have same day/next day spots available exclusively for their patients).
    • Contact your peers to determine their performance and work with them to either spread any processes they find have helped them or collaboratively test some new changes that might work for you AND your peers. HQO’s QIP Navigator allows teams to query submitted QIPs, this tool is extremely usefully to identify peers who have focused on similar areas for improvement.

    Data Quality Actions (Individual)

    Tips to help you understand the quality of your data and, if necessary, take steps to improve it. Estimate impact of data quality:

    Increase quality of the data If the “imperfect data impact calculator” shows that the issues in your data may point you to a different action than suggested in the report, you might consider:

    • Most of the work to improve data quality for this indicator lies in refining the definitions as the data are captured via administrative information systems across all health care sectors and thus beyond the influence of primary care providers.
    • Consider generating a more local estimate of continuity based on who patients see in your team, as indicated by your EMR data. Consider collaborating with the QIDS Specialist group to improve definitions of encounter types and provider types so a more local, team-based measure can be extracted from the EMR.

    Additional information for estimating the impact of data quality for this measure:

    The data are almost certainly not a definitive estimate of your team’s actual performance in the area of access. However, they might be “good enough” to help you decide if your team needs to improve access or not. To determine if the data are “good enough” for that, estimate how likely it is that one or more of the issues outlined in the interpretive notes are a problem with your team. Then, run the “imperfect data impact calculator” to see if the issue(s) could lead to a different decision related to the need for improvement. To do this, work with your clinical leaders and staff to establish an approximate impact of data quality – i.e., is the data quality issue causing your performance to look like TWICE or HALF or 10% (or other number) less or more than it actually is. Plug that number into the “imperfect data impact calculator.” It will show you whether the data quality issue(s) you think you have would change your initial decision regarding the need to improve. You may find it hard to generate consensus about the possible impact of data quality issues on the level of performance shown in the D2D report. In that case, try the following options:

    • If you have part-time physicians in your team, try the following:
      • Extract data from the EMR to determine the number of visits that patients of part-time physicians made to other physicians in the team (i.e., NOT seen by their “own” doctor). Perhaps the rate among these patients will shift your team’s overall rate to be TWICE or HALF or 10% (or some other number) of the rate in the report. Plug that number into the “imperfect data impact calculator” and proceed accordingly.
      • If not possible to extract all these data from the EMR, extract as much as possible and project mathematically based on the number of patients, the average number of visits for all patients and how many hours the part-time physician normally works. Perhaps the rate among these patients will shift your team’s overall rate to be TWICE or HALF or 10% (or some other number) of the rate in the report. Plug that number into the “imperfect data impact calculator” and proceed accordingly.
    • Determine how many of the patients visiting your team are not formally rostered but might be virtually rostered and estimate how many of them usually see the same doctor. Perhaps the rate among these patients will shift your team’s overall rate to be TWICE or HALF or 10% (or some other number) of the rate in the report. Plug that number into the “imperfect data impact calculator” and proceed accordingly.
    • Attempt to estimate how many patients would have seen their own physician in the absence of efforts to increase same/next day access. This will almost certainly be a judgement call, rather than an “estimate” in the truest sense of the word. Perhaps the rate among these patients will shift your team’s overall rate to be TWICE or HALF or 10% (or some other number) of the rate in the report. Plug that number into the “imperfect data impact calculator” and proceed accordingly.
    • If none of the above is helpful, consider instead experimenting with possible “error” rates to see how much error (i.e., TWICE or HALF or 10% of some other number) would be needed to change the decision made on the basis of the performance of the indicator in D2D. If, in the opinion of the team, such an amount of error is reasonable, then it may be worth considering efforts to improve data quality. Alternatively, if that amount of error is considered to be unlikely, then the data are likely good enough to support the initial decision regarding the need to improve, based on the performance shown in D2D.

    If the “imperfect data impact calculator” points to the same decision (e.g., a need to improve or NOT) even after data quality issues are considered, the data are likely “good enough” to base your decision on regarding the need to improve. The next step is to consider strategies to improve, assuming the area of care measured by the indicator is a priority for your team. If your data are not “good enough,” you may then consider taking action to improve your data quality, before or at the same time as you try to improve processes of care.

    Interpretive Notes (Team)

    Tips to help you understand the data and put it in context.

    • “Team,” in this indicator, refers to the physician group i.e., FHO or FHN, not FHT.
    • HQO has removed this indicator from the PCPR for FHTs with multiple physician groups. 
    • Virtual rostering assigns patients to the primary care physician that provided the highest dollar amount of services within a defined set of primary care services. Physicians in your team may not be aware of which patients have been virtually rostered to them. Therefore, they may erroneously think that these patients are not “their” patients. As a result, an individual physician’s sense of how many of patients they see that are “their” patients may be different than the rate shown in D2D.
    • Visits to health care providers other than physicians are not included in this measure. However, this does not necessarily skew the measure. For example, if a patient visits a primary care team 10 times and sees a physician on this team 8 times, regardless of whether or not it was their “own” physician, they will score 100% (8 out of 8) for “regular care provider – team.” If, however, they visit 10 times, receiving care from multiple providers but only saw a physician once and it was physician on the team, they could still score 100% on this measure.
    • Teams with part-time physicians and teaching teams may have developed strong relationships between physicians to jointly care for patients, such that patients may feel equally comfortable and familiar with more than one physician. This principle is at the core of team-based care and is reflected in higher performance on the “regular care provider – team” vs. “regular care provider – individual” measure across most teams in D2D.

    Steps to Improvement (Team)

    Concrete steps you can take to improve care, based on your data. Assuming you have established that the data are good enough to direct action AND that improving performance in this area is a priority for your team, you may wish to discuss the following options with your clinical leaders, Quality Improvement committees, team staff and/or patients:

    • Contact your peers to determine their performance and work with them to either spread any processes they find have helped them or collaboratively test some new changes that might work for you AND your peers. HQO’s QIP Navigator allows teams to query submitted QIPs, this tool is extremely usefully to identify peers who have focused on similar areas for improvement.

     

    Data Quality Actions (Team)

    Tips to help you understand the quality of your data and, if necessary, take steps to improve it.

    Estimate impact of data quality:

    • Access the Imperfect Data Impact Calculator to find out whether the data quality issue(s) you think you have would change your initial decision regarding the need to improve.

    Increase quality of the data If the “imperfect data impact calculator” shows that the issues in your data may point you to a different action than suggested in the report, you might consider:

    • Most of the work to improve data quality for this indicator lies in refining the definitions as the data are captured via administrative information systems across all health care sectors and thus beyond the influence of primary care providers.
    • Consider generating a more local estimate of continuity based on who patients see in your team, as indicated by your EMR data. Consider collaborating with the QIDS Specialist group to improve definitions of encounter types and provider types so a more local, team-based measure can be extracted from the EMR.

    Review list of “rostered” patients with each physician and identify patients who are likely “virtually rostered” (i.e., see the doctor frequently but are not formally rostered) for consideration for formal inclusion in the roster.

    Additional information for estimating the impact of data quality for this measure:

    The data are almost certainly not a definitive estimate of your team’s actual performance in the area of access. However, they might be “good enough” to help you decide if your team needs to improve access or not. To determine if the data are “good enough” for that, estimate how likely it is that one or more of the issues outlined in the interpretive notes are a problem with your team. Then, run the “imperfect data impact calculator” to see if the issue(s) could lead to a different decision related to the need for improvement. To do this, work with your clinical leaders and staff to establish an approximate impact of data quality – i.e., is the data quality issue causing your performance to look like TWICE or HALF or 10% (or other number) less or more than it actually is? Plug that number into the “imperfect data impact calculator.” It will show you whether the data quality issue(s) you think you have would change your initial decision regarding the need to improve. You may find it hard to generate consensus about the possible impact of data quality issues on the level of performance shown in the D2D report. In that case, try the following options:

    • Determine how many of the patients visiting your team are not formally rostered but might be virtually rostered and estimate how many of them usually see the same doctor. Perhaps the rate among these patients will shift your team’s overall rate to be TWICE or HALF or 10% (or some other number) of the rate in the report. Plug that number into the “imperfect data impact calculator” and proceed accordingly.
    • Attempt to estimate how many patients would have seen their own physician, in the absence of efforts to increase same/next day access. This will almost certainly be a judgement call, rather than an “estimate” in the truest sense of the word. Perhaps the rate among these patients will shift your team’s overall rate to be TWICE or HALF or 10% (or some other number) of the rate in the report. Plug that number into the “imperfect data impact calculator” and proceed accordingly.
    • If none of the above is helpful, consider instead experimenting with possible “error” rates to see how much error (i.e., TWICE or HALF or 10% of some other number) would be needed to change the decision made on the basis of the performance of the indicator in D2D. If, in the opinion of the team, such an amount of error is reasonable, then it may be worth considering efforts to improve data quality. Alternatively, if that amount of error is considered to be unlikely, then the data are likely good enough to support the initial decision regarding the need to improve, based on the performance shown in D2D.

    If the “imperfect data impact calculator” points to the same decision (i.e., a need to improve or NOT) even after data quality issues are considered, the data are likely “good enough” to base your decision on regarding the need to improve. The next step is to consider strategies to improve, assuming the area of care measured by the indicator is a priority for your team. If your data are not “good enough,” you may then consider taking action to improve your data quality, before or at the same time as you try to improve processes of care.