Albeit irreplaceable, the frailties of traditional in-person data collection methods have been laid bare ever since the pandemic began in March 2020. With COVID-19 posing an ominous health risk to interviewers and respondents, ethnographic fieldwork, in-person surveys, and focus group discussions became near impossible. Organisations have had to transition towards remote systems of data collection for the research, monitoring, and evaluation of projects. In fact, conducting phone surveys has been the most widely adopted transition, but it is one that requires interviewers to relearn the basics of survey implementation.
Over the past sixteen months, the 21-member survey team at Society for Nutrition, Education and Health Action (SNEHA), a Mumbai-based NGO, conducted interviews with women, children, and adolescents in urban informal settlements on a range of health-related issues. Averaging 6-8 interviews per day, they recall how restrictions on movement impacted surveyors’ data collection methods, and the ways in which their adoption of digital platforms has helped transition towards effectively conducting telephonic health surveys.
The Challenges of Conducting Phone Surveys
The transition to remote data collection was not one without its initial hiccups, most of which were faced by our surveyors who were looking at a huge change in their working patterns. “I had no idea what was going to happen. [Surveyors] do not do any in-office work; we go out and collect data face to face,” said a field surveyor. “Sometimes, I feared that if we couldn’t go to the field, we would lose our jobs as well”.
SNEHA’s surveys would sometimes ask respondents questions about behaviour concerning their sexual and reproductive health. Initially, some surveyors reported being uncomfortable asking such sensitive questions from their homes, in the presence of family members. “We also have children at home… How could we ask such questions through mobile phones?” asks one surveyor. “But with time, I gained confidence. Now, when we have to ask [sensitive] questions, people [my family] go outside while I sit in my room”.
Dealing with Non-Response
The biggest challenge with conducting telephonic surveys is non-responses, mostly due to network issues, incorrect numbers, and switched-off phones. “We do not know whom we could not reach for our survey, perhaps owing to migration or insufficient phone credits. These people could be more vulnerable… we don’t know,” explains a Monitoring, Evaluation, Accountability and Learning (MEAL) specialist. Excluding individuals without functional mobile phones creates inherent biases in the data collected.
Vulnerable and low-resourced groups—which are the ones whose experiences are often most valuable in recording—risk remaining unheard. As one survey supervisor points out, “many women do not own a phone. When their husbands attend our call when they are at work, we ask for a good time to call back. Few assure us that they will go home and call us back, but they never really do”.
Building Rapport, Collecting Consent, and Completing Interviews
Collecting data without visual cues to guide them on the situation of their respondents wasn’t effortless for SNEHA’s surveyors. They had to consider the ‘affective environment’ of the research participants—whether people were feeling uncertain and worried—while making decisions about interview completion and had to be more empathetic than ever while asking questions. “That time [during lockdowns], people were very scared”, says a surveyor.
“At many times, I used to question whether to continue with the interview or not, because when we physically go to a place, we understand the exact situation in a particular house. [But] on the phone, we can’t see the person, we can’t judge whether this person is in an ideal [physical and mental] state for giving an interview or not”. Even obtaining consent over the telephone took greater time and effort on the part of the surveyors’ because of a lack of non-verbal cues.
During the lockdown, calls had to be rescheduled based on the respondent’s availability, which would often go beyond the surveyors’ working hours. “They call us back after their work shift at night, or early morning before leaving for work, or even on Sundays. Despite that, we try to complete the interview because we never know if we will be able to reach them again,” said a surveyor.
Interviewing Adolescents
Interviews with adolescents mandates parental consent, which requires interfacing with parents, which meant more phone calls being made and time invested by the surveyor. Surveyors found it challenging to retain adolescents’ attention on phone calls, especially with the younger ones. “As we continue asking questions [on the phone], their energy decreases. They start concentrating on their friends, going out, etc.,” recalls one surveyor.
Privacy was another issue: parents would often keep the call on speaker mode to listen in on the conversation. “We would request them to go to a corner or more private area in the house and then talk to us,” said one surveyor. “But in Mumbai’s informal settlements, children live in very cramped spaces, making it very difficult for them to find a comfortable private corner to answer sensitive questions,” said a supervisor. Adolescents’ interviews also took more time because surveyors had to ensure that questions and responses were being understood and recorded correctly.
Rising to the Challenge of Phone Surveys
Designing the Survey Instrument
To start with, the survey tool was shortened to 20 to 25 minutes, keeping in mind the limited attention spans of respondents over the phone. Only questions that were strictly in line with the survey’s objectives were retained to reduce respondent fatigue and ensure that interviews reached completion. “In the socio-demographic section, for example, the listing of ‘household assets’ was removed,” says a MEAL specialist. Sensitive questions, like those related to sexual and reproductive health, were placed at the end of the survey, once rapport was established, to lower the chances of non-responses.
Training the Surveyor
The switch to phone surveys necessitated the use of digital platforms for building capacity within the techniques of conducting phone interviews itself, besides for building capacity in surveyors’ routine functions like tool training, supervision, and debrief sessions.
Although SNEHA’s team was well-trained in Computer-Assisted Personal Interviewing (CAPI), it was not until the pandemic that they were introduced to data collection via phone interviews. The training was conducted to build surveyor capacity for effectively engaging with respondents over the phone. These sessions were interactive and conducted over Google Meet. “With the screen being shared by the trainer, it was similar to face-to-face training. We could clear doubts at any time, so, I don’t think any of us faced problems in understanding the content,” said a surveyor who participated.
Surveyors used CommCare during the interviews, which is an open-source survey application that allows for data to be submitted to a central cloud server. Standard operating procedures for contacting and obtaining consent from respondents were practised several times during the training in anticipation of the high refusal rates that were being recorded in virtual surveys. Surveyors were also trained in referring help-seeking respondents in distress or facing food insecurity. The training was followed by several mock interviews and practice sessions.
Building Rapport with the Respondent
Any data collection exercise hinges on trust. But instilling trust becomes challenging with phone interviews because the respondent cannot see the surveyor in order to establish their credibility and build rapport. In an effort to build credibility, surveyors introduced themselves, the organization, the purpose of the survey, how this telephone number was obtained, and the approximate duration of the interview.
“The first impression is the last impression, right?” a supervisor says. “We would start the conversation with a pleasant tone, ask them about their personal lives, let them get a little comfortable with us before getting into the interview.” Rapport was built by explaining the objectives of the survey and activities that go into data collection. If respondents wanted to speak privately at a later time, the interview was rescheduled accordingly. In this manner, trust was built by providing a detailed overview of the survey, clearly explaining confidentiality and taking into consideration the time and circumstances of the respondent.
Call-back Protocol and Timing Calls Right
Response rates depend on call-back mechanisms and the time at which the call is made. After testing the best times in the day for making them, the survey team ended up calling the respondents three times a day, with an interval of two hours between each call. This went on for three consecutive days after which the respondent was marked as ‘not available’.
Some times of the day had a higher probability of engaging with respondents; this is where understanding the local context and situation is vital. As for adolescents, one supervisor says, “Afternoons were when their parents would be resting and the adolescent would have access to the phone in privacy”.
Ensuring Quality Data
Since SNEHA used a sample from beneficiaries of their own interventions, they had some prior-collected data on the respondents’ characteristics such as their age and number of children. This data was useful in screening for a surveys’ specific target respondents.
The limitations of controlling the privacy of a telephonic survey was well-acknowledged by SNEHA’s surveyors. Yet, over time, they found that they were able to identify and adapt to non-verbal cues — such as noises in the background, or echoes when the phone is kept on speaker mode — which were meant to influence the respondent. About adolescents, one supervisor says, “If there is a lot of background noise, [it means they] are surrounded by people, so this may not be a good time to ask them questions.” Survey supervisors also adapted to virtual forms of supervision, including monitoring data quality through three-way calling and frequent online debriefing meetings held with surveyors.
As a supervisor, we pay attention to the tone of the surveyor, and whether all points of consent were explained. I would download the form and fill the responses myself, and later match it with the surveyor to see how they had recorded the responses.
From the Survey Team: How to Maximize Phone Surveys
When designed and implemented properly, phone surveys can be an effective data collection method. This is especially true in cases where there is a verified database of phone numbers and an established rapport within the sample community as it makes it easier to build confidence and trust. To that end, phone surveys would not be the best bet for a baseline evaluation. As one MEAL specialist says, “if you are going for a baseline survey where [say, for example] no one knows about SNEHA, then it is a little difficult to use a telephonic interview. There is also limited scope for using phone surveys in evaluations wherein anthropometric measurements like weight, height, waist and hip circumference need to be taken. Instead, a hybrid model can be adopted in these cases wherein teams collect anthropometric data in the field, and later conduct the remainder of the survey over the phone.
With all its challenges and breakthroughs over the past two years, telephonic data collection has proven to be a useful alternative for in-person data collection methods, albeit entirely new. They can be particularly useful for interviewing health workers, community volunteers, and other stakeholders wherein organisational rapport exists because the most common challenges of incorrect numbers or a lack of trust can be mitigated.
Editor’s Note: The views expressed in this article are based on inputs received through a qualitative inquiry of in-depth interviews and focus group discussions with the survey team at SNEHA. The authors would like to express their gratitude to the surveyors for their diligence and patience.
Featured image of SNEHA’s surveyors conducting telephonic surveys during the pandemic taken with permission from SNEHA.
