Surveys

A survey can provide quantitative information (numeric or rating) in a standardized way. Program-level survey data is collected annually through the Memorial University Student Experience Survey, and academic units can request survey results for their own students to help inform the self-study. The Student Experience Survey includes questions that were developed specifically for AUP purposes and units are strongly encouraged to avail of these results rather than develop a new survey. If there are circumstances where an academic unit feels they need to conduct their own survey, the survey administration will be the responsibility of the unit as the Student Experience Survey is the only centrally supported survey for academic planning purposes.

Considerations:

• Response rates: When a survey is sent to the entire population of majors/minors for a unit, only a portion of those students can be expected to respond. A low response rate or small sample size can cause challenges in terms of anonymity of respondents and usefulness of responses. Typically, a student survey will receive a maximum 30% response rate and AUP surveys will typically receive a 10-15% response rate.[1] If a unit expects a small sample size from a survey, they should consider an alternate engagement method.
• Policies: When surveying students, units must adhere to the Institutional Surveys Policy and the survey(s) must be reviewed and approved by the Institutional Survey Oversight Committee (ISOC) prior to administration. Additional consultations may be required in parallel with or on the condition of ISOC review, including the Committee on Ethical Research Involving Indigenous Groups (CERIIG). Surveys of alumni can only be administered by the Office of Alumni Engagement, and that Office should be notified of the survey as early as possible.
• Administrative tasks and timeframes: Units preparing to conduct their own student survey should consider the following items:
• Survey development - This will normally involve consultation, question development, formatting the instrument into the university’s approved survey tool (Qualtrics), and testing the instrument.;
• Submission to the ISOC - -The ISOC will normally meet monthly during the third week of the month, and the deadline for submission of draft surveys is one week prior to the meeting. Time should be allotted for receipt of a response and the integration of feedback. If the ISOC requires significant changes to the survey, a resubmission may be required.;
• Administration of survey: Units will need to coordinate the administration of their own unit AUP surveys. Qualtrics is the current approved survey tool for Memorial and should be used for the purposes of surveying students, so a delegated unit employee will require a Qualtrics account and will need to be familiar with using the tool.[2] As previously noted, surveys of past graduates must be distributed by the Office of Alumni Engagement and it will take a minimum of two weeks to schedule a survey invitation.  Student surveys should be live for a 2-4 week administration period.
• Integration of survey results into self-study report: Survey results will need to be analyzed and summarized, and the time required for this will depend on the survey complexity and the type of questions asked. Qualitative data (open-ended/written responses) must be summarized by themes to be referenced in a self-study report and this will take much longer to compile than numeric data.

Given the above-noted events, units should allow at least three months from development of the survey instrument  to compilation of survey results.

[1] Appendix A provides a table of response rates required for 95% and 90% confidence intervals based on population size.

[2] More information on Qualtrics can be found at https://www.mun.ca/surveysolution/

 

Appendix A

Population Size	Accuracy (+ or minus %) of 5% sampling error; and 95% Confidence Interval	Response Rate Needed	Accuracy (+ or minus %) of 5% sampling error; and 90% Confidence Interval	Response Rate Needed
5	5	100	5	100
9	9	100	9	100
10	10	100	10	100
14	14	100	13	92.86
15	14	93.33	14	93.33
16	15	93.75	15	93.75
20	19	95	19	95
25	24	96	23	92
30	28	93.33	27	90
35	32	91.43	31	88.57
40	36	90	35	87.5
50	44	88	42	84
60	52	86.67	49	81.67
70	59	84.29	56	80
80	66	82.5	62	77.5
90	73	81.11	68	75.56
100	80	80	73	73
125	94	75.2	86	68.8
150	108	72	97	64.67
175	120	68.57	107	61.14
200	132	66	115	57.5
250	152	60.8	130	52
500	217	43.4	176	35.2
1000	278	27.8	213	21.3

 

Calculations based on the formula Ns = (Np)(p)(1-p) / [(Np - 1)(B/C)^2 + (p)(1-p)]                                                                                            

 

Focus Groups

Units with small student populations may wish to engage students through focus groups. A focus group is a qualitative approach that allows units to engage in in-depth inquiry with students to obtain feedback on specific issues or questions. The ideal size for a focus group is 8-10 participants with a facilitator and separate note-taker if possible, and there should be a specific set of questions that the unit would like considered by participants. It is important to engage strong note-takers and independent facilitators. Units may wish to hold multiple focus groups if the participant population is larger than 10. While focus groups allow for in-depth inquiry and are effective for smaller units, it should be noted that it is not possible to attain feedback anonymously.

Considerations:

• Timelines: Since session scheduling will largely depend on the unit and will not involve other units or committees, this may allow units to conduct focus groups in cases where there is a short timeframe to collect information. When considering timing of focus groups, units should allow sufficient time to plan the session(s), including developing questions and identifying a facilitator; invite potential participants; host the session(s); and analyze and integrate insights into the self-study report.
• Resources: Focus groups require the commitment of one or two individuals to facilitate the session and take notes. The resulting qualitative data will involve more time for thematic analysis than would be the case for quantitative approaches but the information will be richer.

Town Halls

Town halls are another qualitative approach that offer the opportunity for students to provide feedback in an open setting and in a less structured format than surveys or focus groups. This method may be appropriate for units wishing to acquire open feedback from students in a larger sample than would be appropriate for focus groups. Town halls may also be set up as drop-in events, allowing more flexibility in student attendance. Units should have at least one facilitator and a note-taker and should prepare a semi-structured set of questions to guide the overall conversation while allowing time for students to freely express themselves.

Considerations:

• Timelines: Similar to focus groups, scheduling will depend on the unit and this approach would be suited for situations where feedback is to be collected in a short period of time. The unit will need time to identify a facilitator and note taker, develop a discussion outline, secure a venue/location for the town hall, advertise the event, and compile the information collected in a form that can be integrated into the self-study report.
• Resources: A facilitator and note taker will be needed to lead the town hall event, and to summarize the discussion afterwards. This approach has the advantage of providing detailed information and insights in a productive way as the session can be hosted once with a large number of participants in attendance.