For technical notes, please see page 16 of the Data Dictionary.

Interpretive Notes

Tips to help you understand the data and put it in context.

• The performance shown on these indicators in D2D is usually based only on patients either currently in the office or having just had a recent visit, since most teams survey patients in this way. It is possible that this may not be representative of patients who did not have appointments. For example, patients who feel they were not involved as much as they wanted or felt that the office staff were not courteous may be choosing to get care elsewhere (e.g., walk-in clinic, Emergency Department, other provider) or nowhere at all.
• Teams that do not feel able to do anything to improve patient experiences in these areas may decline to ask these questions. They may be concerned that asking the questions may set false expectations among patients that their input will prompt changes. As a result, the performance level may represent only teams that would consider interventions to improve patient experience in these areas. Consequently, the rates may not be representative of all teams.
  • It could be an over-estimate of actual patient experience if one assumes that teams with good patient experience are more likely to consider interventions.
  • Alternatively, it could be an under-estimate if one assumes that teams for whom patient experience is not good are more likely to be considering interventions to improve their performance in this regard.
• There appears to be little difference in performance in responses to patient experience survey (PES) questions over time. Most questions on most surveys show a high percent of patients with positive experiences. This may be real and it may also be what is known as “ceiling effect” (i.e., bunching of responses at the top end of the scale), possibly because patients want to be positive about their experience with their provider.

Steps to Improvement

Concrete steps you can take to improve care, based on your data. Assuming you have established that the data are good enough to direct action AND that improving performance in this area is a priority for your team, you may wish to discuss the following options with your clinical leaders, Quality Improvement committees, team staff and/or patients:

• Get more current data to move beyond the idea that “the data are too old to make improvements” OR test out a new question that wasn’t previously included in your survey.
  • Step 1: Work with a physician who is interested in improving patient experience or work through your ED to get the conversation started with your team’s Medical Lead.
  • Step 2: Suggest tracking patient experience for a small subset of patients (i.e., the next 10 patients that come in) or for a narrowly defined timeframe (i.e., whatever you get in the next 2 weeks).
  • Step 3: Look at the data to see how this current “snapshot” compares to the “old” data.
  • Step 4: Take this opportunity to discuss sample size. See resources on web. You will see that you do NOT need hundreds of surveys to know how your patients are feeling and to start making improvements.
  • Step 5: Consider setting up a frequent, ongoing automated patient experience survey process (See Data Quality Actions – how to increase quality of the data using new tools to capture and track patient experience data).
• Collaborate with Patients Canada PFCC KPI initiative to discover what matters to patients and how to measure it. Learn about collaborating with patients by doing it, as described at the Trillium Research Day.
• Check out these resources for patient engagement from your AFHTO peers and partner organizations.
• Convene a patient advisory council, with the help of the Patient and Family Advisory Council Toolkit from CCO.
• Connect with teams who report progress with improving patient engagement and experience:
  • Nothing About Me Without Me: Applying Citizen Engagement Methods in a Family Health Team (AFHTO 2016)
  • Thunder Bay Regional Health Center.
  • Institute for patient and family centered care.
    • Toronto Western Family Health Team (TW-FHT) Patient Experience Survey.
    • TAPESTRY – Teams Advancing Patient Experience: Strengthening Quality; McMaster Family Health Team.
• Consider how your team’s mode of survey administration may or may not impact the results and explore alternative means to improve data capture and representativeness.
  • Measuring the Patient Experience: How to Select a Delivery Method for Best Results and Minimal Effort.
  • Feasibility and impact of using EMR to trigger automated patient experience surveying.
  • See Data Quality Actions to learn how to increase quality of the data using new tools to capture and track patient experience data.
• Read About Patient Partnerships to gather ideas for how your team can engage patients and caregivers as partners in healthcare.
  • Involving patients in setting priorities for healthcare improvement.
  • Meeting with Patients: their experiences and perspectives.

• Contact your peers (in addition to those involved in the initiatives described above) to determine their performance with patient experience and work with them to either spread any processes they find have helped them or collaboratively test some new changes that might work for you AND your peers. HQO’s QIP Navigator allows teams to query submitted QIPs, this tool is extremely usefully to identify peers who have focused on similar areas for improvement.
• Get your patients involved! It doesn’t end with the Patient Experience survey. Consider sharing your D2D data with them (check out an example of a waiting-room poster here), and getting them involved with your QI initiatives.

Data Quality Actions

Tips to help you understand the quality of your data and, if necessary, take steps to improve it. Estimate impact of data quality:

• Click here to access the Imperfect Data Impact Calculator to find out whether the data quality issue(s) you think you have would change your initial decision regarding the need to improve.
• Additional information for estimating the impact of data quality for this measure.

Increase quality of the data If the “imperfect data impact calculator” shows that the issues in your data may point you to a different action than suggested in the report, you might consider:

• Administer your patient survey in a way that includes patients not recently in the office.
• Revise your team’s patient survey to align with standard questions and/or add additional (standard) questions, even if it is one question at a time.
  • See the HQO Primary Care Patient Experience Survey
  • Tips and Tools to Capture the Patient’s Experience in Primary Care using the NHS’s Experience-Based Design (EBD) Methodology
  • What are they thinking? Approaches, Ideas and Tools to Measure Patient Experience in Your Practice
• Consider changing your patient experience survey process from once a year to an ongoing cycle using some of the new tools integrated with your EMR to enhance and track patient experience. Check out how other teams administer patient surveys (see below). You may also want to consider international and provincial patient survey initiatives.
  • Using new tools to capture and track patient experience survey data to improve the efficiency of administrating surveys, and especially improve the timing and quality of patient responses (e.g., administer surveys by email, phone, tablets – all integrated with your EMR).
  • Extending the EMR with Patient Tablets: Using Interactive, Point-of-Care Patient Surveys in the Waiting Room to Generate Clinical Content and Save Time.
  • Using an automated patient reminder service and survey to collect information on patient experiences.
  • Add some SaaS to your patient experience survey.
• Other ideas: please share!

Additional information for estimating the impact of data quality for this measure:

The data are almost certainly not a definitive estimate of your team’s actual performance in the area of access. However, they might be “good enough” to help you decide if your team needs to improve patient-centredness or not. To determine if the data are “good enough” for that, estimate how likely it is that one or more of the issues outlined in the interpretive notes are a problem with your team. Then, run the “imperfect data impact calculator” to see if the issue(s) could lead to a different decision related to the need for improvement. To do this, work with your clinical leaders and staff to establish an approximate impact of data quality (i.e., is it the data quality issue that is causing your performance to look like TWICE or HALF or 10% (or other number) less or more than it actually is?). Plug that number into the “imperfect data impact calculator.” It will show you whether the data quality issue(s) you think you have would change your initial decision regarding the need to improve. You may find it hard to generate consensus about the possible impact of data quality issues on the level of performance shown in the D2D report. In that case, try the following options:

• Track the next 10 (or 20 or other small number) encounters to get a better estimate of the extent of the data quality issue. Perhaps the rate among these patients will shift your team’s overall rate to be TWICE or HALF or 10% (or some other number) of the rate in the report. Plug that number into the “imperfect data impact calculator” and proceed accordingly.
• Compare to other sources of data to see if the rate with other/better data is TWICE or HALF or 10% (or some other number) of the rate in the report. Plug that number into the “imperfect data impact calculator” and proceed accordingly. Other sources of data may include your scheduling system (e.g., actual appointment data), your EMR (queries or chart audits on a small number of patients), other pre-existing reports (CPCSSN, EMRALD, SAR, Physician Profile, MOHLTC, QIP, commercially provided or others) or personal interviews with a few patients, to name a few.
• If neither of the above is helpful, consider instead experimenting with possible “error” rates to see how much error (i.e., TWICE or HALF or 10% of some other number) would be needed to change the decision made on the basis of the performance of the indicator in D2D. If, in the opinion of the team, such an amount of error is reasonable, then it may be worth considering efforts to improve data quality. Alternatively, if that amount of error is considered to be unlikely, then the data are likely good enough to support the initial decision regarding the need to improve, based on the performance shown in D2D.

If the “imperfect data impact calculator” points to the same decision (i.e., a need to improve or NOT) even after data quality issues are considered, the data are likely “good enough” to base your decision on regarding the need to improve. The next step is to consider strategies to improve, assuming the area of care measured by the indicator is a priority for your team. If your data are not “good enough,” you may then consider taking action to improve your data quality, before or at the same time as you try to improve processes of care.