Measuring the Patient/Client Experience with Virtual Care in Primary Care 

Purpose: 
In the last year, Ontarians attended nearly 1,000,000 virtual care visits (Ontario Telemedicine Network, 2019). Virtual care is provided through a vast number of modalities such as voice, video, teleconference, online platforms, and mobile texting (Pearl et al, 2014). While many primary care practices in Ontario have provided virtual care in varying degrees prior to the COVID-19 pandemic, virtually overnight, primary care providers shifted most of their practice to virtual care to reduce COVID-19 spread. With this major transition in the delivery of care, it is important to understand how patients/clients are experiencing virtual care so that care can be optimized to best meet their needs. 

This survey was created to help fill informational gaps to inform future planning and further support the implementation of virtual care in primary care. This survey can also be used for quality improvement purposes to allow providers to look at ways to improve their care delivery. 

Survey Development: 
The rapid adoption of virtual care during the pandemic not only addresses the current needs, but it will most likely shape the future of primary care delivery beyond the pandemic. As a result, the Association of Family Health Teams of Ontario (AFHTO) formed a working group with a number of primary care, health system and health research organizations to measure the patient/client experience with virtual care during the COVID-19 pandemic. These organizations included: Quality Improvement in Primary Care Council (QI in PC Council), Women’s College Hospital Institute for Health System Solutions and Virtual Care (WIHV), eHealth Centre of Excellence (eCE), Ontario Health (Quality) (OH(Q)), and Partnering for Quality. 

AFHTO created a first draft of the survey based on a literature review of virtual care surveys used in primary care. Due to the need to start surveying during the pandemic, and because many of the questions were adopted from existing and/or validated surveys, this survey did not undergo testing. The draft questions were reviewed and refined by the working group members. 

Additionally, AFHTO engaged with four Ontario Health Primary Care Quality Advisory Committee patient advisors and one of AFHTO’s Quality Steering Committee patient advisors to review and refine questions. The focus of the patient engagement is to identify questions that are meaningful to patients/clients and can help inform quality improvement of patient/client experience with virtual care in the primary care setting. 

How this Survey can be Used:
Data from this survey can be used to inform quality improvement and planning efforts. The survey was not designed to be used for benchmarking or comparative reporting; however, results can help guide practices as they plan service changes to improve their patients’ experience with virtual care. This survey complements practices’ existing patient/client experience surveys (ex: Primary Care Patient Experience Survey developed by OH(Q)).

Practices can deploy surveys based on the type of visit the patient had:
Thinking of your most recent visit, was it provided to you in person or virtually (i.e., phone, video, or messaging)? 
a)    In person ⇒ please complete practice’s existing patient/client experience survey
b)    Virtually ⇒ please complete the Patient/Client Virtual Care Experience Survey

The results from this survey will help provide a high-level overview of the patient/client experience with virtual care; however practices are  encouraged to capture few additional data elements to provide a more comprehensive picture of virtual care provided and support the interpretation of the patient/client experience. In particular, the collection of the following two elements will be helpful: 

a) Virtual Appointment – Reason for Visit/Appointment 

To accurately capture this information, it is recommended that this data is extracted from the EMR via searches/queries and provide the context for the results of this survey within similar timelines. Queries can be structured to search for which appointment modality (i.e. in-person, telephone, video etc.) was used with which reason for the visit/appointment.

b) Additional Demographic and Health Equity Data 

The pandemic has highlighted health care inequities already existing in the population. It is important for practices to understand the variances in their patients’/clients’ experiences which can help inform quality improvement and service planning. It is important to capture social determinants of health (SDOH) such as health-related behaviours, socioeconomic factors, and environmental factors, as they can account for 80-90% of a person’s health (Magnan, 2017). Towards the end of the survey, a few questions capture demographic and health equity data, however practices are strongly encouraged to examine the following resources for additional questions:

• Measuring Health Equity: Demographic Data Collection and Use in Toronto Central LHIN Hospitals and Community Health Centres
• The Upstream Lab
• We Ask Because We Care Questionnaire 

Data Collection: 
Practices may choose to implement the survey via various modalities such as website, online survey platforms (ex., Microsoft Forms, SurveyMonkey, RedCap, Qualtrics etc.), email, paper, and telephone. For online surveys, a template survey for Microsoft Forms has been created that can be duplicated. If you will be using SurveyMonkey, please email improve@afhto.ca and a copy of the survey will be shared with you. 

Ocean (CognisantMD) is offering a free platform to patient experience data; however, please refer to this guide to ensure that you have completed all the steps to qualify. For your convenience, eCE has created an Ocean eForm version of the survey, now available to be imported to your Ocean (CognisantMD) site, titled “Primary Care Patient/Client Virtual Care Experience Survey”. Instructions on how to import and configure the survey and data extraction can be found here. 

For telephone appointments there may be some options available such as adding telephone surveys as an additional duty to screeners at your clinic, creating a telephone survey student project, looking for volunteers and using a telephone survey software. 

Once you have identified the method of dissemination, it is recommended that the survey be fielded for a specific period (ex. 3 weeks) and that practices review results prior to relaunching the next round of the survey. This will allow you to conduct Plan-Do-Study-Act (PDSA) cycles that are manageable and will allow for time to implement quality improvement changes to improve patient/client experience. 

Survey PDFs:

• Primary Care Patient/Client Virtual Care Experience Survey Complete Implementation Guide
• Primary Care Patient/Client Virtual Care Experience Survey (PDF)
• Primary Care Patient/Client Virtual Care Experience Survey – Appendix A (Core 5 Questions)

AFHTO Provincial Initiative: Help Provide a Provincial Overview of the Patient/Client Virtual Care Experience! 
Some survey questions may not be applicable to your practice. In collaboration with patient advisors we have identified 5 core questions that we highly recommend remain consistent when being asked. The 5 core questions include: 2, 3, 5, 7, 8 (Appendix A).

We also highly recommend you report back to AFHTO to support AFHTO’s Provincial Initiative. To further support practices in expanding and solidifying virtual care in primary health at a provincial level, AFHTO will be collecting the practice-level aggregate responses to the survey. 

A minimum of 5 responses per question will be required for the submission of aggregate results. At the end of June 2021, an aggregate report will be generated with the findings from this survey without any identification of practices. This data will not be used for practice-level funding or public reporting and will only be used to support planning and quality improvement at a regional and provincial level. Participation is highly encouraged, however voluntary. If you choose to participate, please fill in this survey within the first week of the given months: 

• The week of November 1st, 2020
• The week of February 1st, 2021
• The week of May 1st, 2021

For more information, please contact: 

improve@afhto.ca

References: 
Magnan, S. 2017. Social Determinants of Health 101 for Health Care: Five Plus Five. NAM Perspectives. Discussion Paper, National Academy of Medicine, Washington, DC. doi: 10.31478/201710c
Pearl R. Kaiser Permanente Northern California: current experiences with internet, mobile, and video technologies. Health Aff (Millwood). 2014;33(2):251-257. doi:10.1377/hlthaff.2013.1005
OTN. “Virtual Care in Action.” OTN, 28 Aug. 2019, www.otn.ca/virtual-care-in-action-archives/