Data collection and surveys form the backbone of India’s social and development sectors. At a time when primary research on development issues is a global priority, the controversy surrounding the leak and scraping of the National Statistical Office (NSO’s) consumer expenditure survey 2017-18 by the Indian government raises serious credibility concerns over the quality, unavailability, and transparency of data.
Data collection projects typically struggle with survey methodologies and logistical inefficiencies. To make matters worse, data collectors — who are central to any surveys’ success — are usually treated poorly. Not only are they expected to be sharp and vigilant on the field, but the nature of some surveys requires collectors to adopt empathetic and sensitive approaches towards their research participants.
Any successful field experiment in the social sciences relies on careful design and even trickier implementation. This is a collective process that ranges from recruiting and training enumerators, to launching the experiment and managing the fieldwork. These elements form the kernel of SurveySights, which is a data collection and management organisation that aims to improve the quality of data collected in surveys by adopting a unique approach of 3Ts — Train, Track, and Treat. The SurveySights team reached out to 16 field investigators and 35 development sector professionals across India to hear their concerns and recommendations regarding data collection.
Sensitisation Training
“Training sessions were tightly scheduled… moreover, it was overwhelming to train people who had never worked on a detailed survey using tablets from scratch”
— Mahesh, Field Surveyor
Training data collectors is a pivotal step of fieldwork that is often carried out based on some ‘rulebook protocols’ instead of focussing on the elements of their interaction. Our respondents collectively agreed that training sessions are extremely critical to strengthening the quality of data being collected. These training sessions familiarise teams with the best practices in the field and help them understand the gaps in the survey tool. This way, they can actively detect and rectify common errors while conducting fieldwork. Investing time and effort in recruiting a well-trained field team adds value to a study and decides the future quality assurance of research projects.
However, despite its clear importance, our respondents reported training sessions to be insufficient. Training a team without incorporating their inputs on the survey tool could lead to the tool being revised multiple times even at the last moment. This significantly hampers the efficiency of the planning process. The field staff respondents also found training sessions to be stressful and felt undue pressure to complete what they believed to be an infeasible number of surveys daily. All these factors negatively impact a surveyor’s performance in the field and thereby affects the quality of data being collected.
A majority of our field staff respondents also admitted to having worked with teams that were unfamiliar with the local context and lacked relevant experience and digital training. Sumana, a field surveyor, tells us that she was not sensitized on the nuances of her research topic. “Despite being a woman, I still wasn’t sure how to subtly ask another female respondent about her sexual and reproductive health. I wish the training would have emphasized building trust and sensitivity during an interview”, she says.
Besides training the enumerators on standard field protocols to enhance data quality, we received recommendations on how to conduct project-specific survey tool training without compromising on time, as well as interview techniques on seeking informed consent. The need for training surveyors to be empathetic listeners, and to ensure the confidentiality of their respondents for maintaining data quality standards were also emphasized. These skills can be honed by making more room for face-to-face team-building exercises and mock interviews.
Most importantly though, given that research studies can often focus on stigmatising topics, it is important to sensitise field teams on tactfully striking trust and establishing a rapport with the respondents, while also acknowledging their dignity and freedom while administering personal questions.
Zeroing down on these pain points while training data collectors can significantly help improve the quality of data collected, and in turn, the quality of the research interventions produced.
Tracking Inconsistencies in the Field
“We had to interview women residents of a slum about their HIV awareness and contraceptive usage. A day before our fieldwork, we were acquainted with the frequent practice of within-slum prostitution, which is obviously an extremely sensitive topic. Training sessions were compact and restricted to the questionnaire, without sensitising the field team and conducting pre-visits in the community. Our data collection strategy fell through the very first day after a couple of interviews, and the entire plan had to be revised factoring in the sensitive topic”
— Rekha, a Field Supervisor for a research study on informal settlements
Launching a survey on the ground is an elaborate process beyond just recruiting and training field teams. Teams are confronted with several unanticipated challenges that are rarely given any consideration during the planning stage — the safety standards of the study area, possible entry barriers, respondent’s discomfort, and unwillingness to answer. It is crucial to acknowledge the wider socio-economic and cultural heterogeneity in respondent type and geography.
“To know the exact nature of the area, it is important to create a local contact and bond with the community”
— Susheela, Field Supervisor
Insufficient pre-visits to develop trust with the said community’s representatives could lead to potential risks in a study area. Our respondents reported difficulties in locating participants and verifying their credentials because valid authorisation documents lacked. Incorrectly translated survey tools could also lead participants to misconstrue questions and negatively impact the overall communication chain. It could increase the survey duration and induce fatigued answers.
“There were times when we worked overtime in the field without a powered phone and means of transportation to go back home.”
— Hitesh, Field Surveyor
To address all of these niggles, the field staff in our sample highlighted the need for careful logistical pre-planning where safe transportation, accommodation, and medical provisions are arranged.
Once the fieldwork is live, having well-established quality assurance procedures in place can help identify and overcome erroneous data, double-check completed forms and conduct regular debrief sessions to track field worker’s progress. Feedback loops are essential for efficient communication within teams. The need for conducting data quality checks has become imperative given the rising quality concerns of primary surveys in India, even more so for remote surveys in the COVID-19 era.
Role of Incentives
“How is this study going to benefit me as a data collector?”
— Anita, Field Team Lead
Field teams perform one of the most demanding jobs within research, but they are paid paltry salaries, often disregarded at work, and exposed to unfair treatment by impatient participants and researchers alike. Despite the indispensable role that field teams play in a survey’s success, they are rarely in the spotlight.
So, what kind of incentives drive data collectors to go the extra mile and meet data quality targets? The respondents underscored the importance of complimenting fieldwork with upskilling opportunities to improve their learning curve and skill-set. Tardy payment structures act as a colossal dis-incentive. The majority of respondents highlighted the need for monetary incentives such as promotions and salary hikes to boost their morale and confidence.
Due to the short-term nature of the projects, the respondents felt under-utilised and unmotivated to conduct quality surveys. This links to their increased willingness to seek stable jobs elsewhere, leading to staff attrition at different stages of data collection. A dearth of organisational incentives like the timely release of salaries due to delayed paperwork and non-financial rewards may also affect the quality of the data that is collected. Therefore, providing field teams with secure and reliable jobs may encourage staff continuity.
In addition to many nuanced incentive structures discussed above, prioritising the field team’s inclusion at every phase of implementation for improved data quality is key. Recognising their role as more than just data collectors by investing in their growth through a hybrid of performance assessments and rewards will catalyse better performances and encourage staff retention.
Emphasizing on the psychological well-being of field teams who could be sensitive to traumatic experiences in the field that may leave them unsettled is also as important as ensuring sensitive engagement with respondents. To build such an ecosystem, these experiences should be freely discussed and acknowledged.
Data quality targets are met better as a team
The data collectors demonstrate a strong inclination to work in the social sector – which now brings the onus upon researchers to engage with them, prioritise their inclusion at every phase of the study, and incentivise them to achieve high-quality results.
These insightful interactions brought to light distinct field nuances and helped better understand the challenges that are often overlooked before undertaking a survey. Being aware of these recommendations and placing human values at the forefront can ensure the data collected to be robust and reliable. It is crucial to acknowledge that these recommendations form the standard protocols for many organisations ‘in theory’, and rightly so. Yet, in reality, they easily slip through the cracks due to inadequate resources and stringent deadlines.
All respondents have been anonymised. Featured image of an all-women capacity building workshop for raising awareness about climate change in Savda Ghevra courtesy SurveySights.
