Is your practice meeting the needs of its patients? Assessing patient experience is critical to the success of every practice, no matter its size, specialty or location. One valuable tool for gauging provider performance and clinic operations is a patient satisfaction survey that helps you measure patient experience. When thoughtfully prepared, these surveys provide insight that practices can use to immediately improve quality and care.
To ensure this survey provides the data your practice needs, you need to identify the appropriate sample size (per provider) for your organization. This step requires that you combine mathematical principles along with industry standards. The following recommendations are based on the average national patient panel size of 2,300.1
Margin of error
When conducting survey research there will always be sampling errors, a term that refers to the differences between survey results from a sample and potential results achieved if every patient in the panel responded to the survey.2 Obtaining responses from every patient in the panel is generally not feasible, which is one reason why results can be thought of as estimates of potential results or true population values. To determine sample size:
- Select a maximum margin of error with which your organization is comfortable. See example below for details.
- Determine how many survey responses (per provider) are necessary to stay within a desired margin of error. For mathematical reasons, this number should be 30 or more.2
- Consider the minimum number of survey responses necessary to stay within your margin of error and the average response rate for your survey distribution method to determine an appropriate per provider sample size.
"Is your practice meeting the needs of its patients? Assessing patient experience is
critical to the success of every practice, no matter its size, specialty or location."
The industry standard for maximum acceptable margin of error is 14%, which means that the true population values are likely within 14% of the survey estimates.
If 85% of surveyed patients rated a physician’s performance as “excellent” on a survey question, the percentage of all patients in the entire panel who would rate the physician’s performance as excellent (i.e., the true population value) would likely be within 14% of
the survey estimate, or between 71% and 99%.
Survey responses
The maximum margin of error depends on the number of survey responses. According to the central limit theorem, a mathematical principle, a minimum of 30 responses should be used in data analysis to accurately estimate true population values regardless of patient panel size. A minimum of 30 responses applies to every unit, which means that every provider should have at least 30 survey responses before data analysis.2
The table below provides the minimum number of survey responses required to achieve specific margins of error. Selecting the industry standard margin of error of 14% means that the true population values are likely within 14% of the survey estimates. Note that a minimum of 50 survey responses are necessary (per provider) to stay within a 14% margin of error.
Identify sample size
When defining an appropriate sample size, the minimum number of survey responses necessary to stay within the predetermined margin of error and the average response rate for a given survey distribution method must both be considered. Mail survey distribution has an average response rate of 18% and email survey distribution has an average response rate of 15%. An appropriate provider sample size (i.e., the number of surveys to distribute per provider) can be determined using these average response rates and the minimum number of survey responses needed to stay within a maximum margin of error.
The table below provides sample size recommendations based on margins of error and survey distribution methods.
Using a 14% margin of error with a minimum of 50 responses per provider, 278 surveys per provider should be sent for mail distribution or 334 surveys per provider for email distribution.
If your response rate is different from the average response rate of the distribution method, it is possible that the maximum margin of error will change from what was anticipated. The table below illustrates this point.
If your response rate is lower than average for mailed surveys, the 50 responses necessary to stay within a 14% margin of error might not be received. If only 30 responses are received, use the cells selected in blue to see that the maximum margin of error has increased to 18%. The true population values would likely be within 18% of the survey estimates.
On the other hand, you might have a better than average response rate and receive more than 50 responses. If 75 responses are received, use the cells selected in green to see that your maximum margin of error has decreased to 11%. Now the true population values will likely be within 11% of the survey estimates.
The smaller the margin of error, the more precisely survey results estimate true population values.2 The recommendation is to select a moderate margin of error between 10% and 12%. If you achieve an average response rate for your survey distribution method, you will maintain this moderate margin of error that is less than the industry standard of 14%. Additionally, the larger sample sizes associated with 10% and 12% margins of error provide some insurance against dropping below the industry standard of 50 responses per provider or the minimum requirement of 30 responses per provider in case of lower than average response rates. If you have an above average response rate, it is possible to achieve a smaller margin of error than anticipated. These ideas are summarized in the table below.
More precise survey results provide a better understanding of the realities of patient satisfaction. Therefore, seek to achieve the smallest, yet realistic margin of error possible.
Notes
1. Alexander GC, Kurlander J, Wynia MK. “Physicians in retainer (‘concierge’) practice: A national survey of physician, patient and practice characteristics.” J Gen Intern Med. August 2005; 20(12): 1079-1083.
2. Norwood SL. Research Strategies for Advanced Practice Nurses. Upper Saddle River, N.J.: Prentice Hall Inc.; 2000.
3. Kish L. Survey Sampling. New York: John Wiley & Sons Inc.; 1995.
Resource
Sullivan LM. Essentials of Biostatistics in Public Health. Sudbury, Mass.: Jones and Bartlett; 2008.
