Israel Oladejo, Laterite’s Data Quality Intern, shares what he learned about how to collect high quality data via a phone survey during his internship in summer 2021.
I am a Master of Science in Information Technology student at Carnegie Mellon University Africa, with a concentration in Applied Machine Learning. My areas of interest include data science and analysis, research, and entrepreneurship. I was interested in working at Laterite because I wanted to play a part in Africa’s growing research capacity.
According to the World Economic Forum, Africa accounts for less than 2 percent of global research output. But I observed that Laterite is building its research capacity to tackle some of the biggest challenges facing the continent – using research and various information analytic tools to drive decision making. I wanted to contribute to that and learn how to use data analytics to improve research in Rwanda.
During my internship, I designed monitoring systems to screen incoming data for data collection projects, and cleaned data collected for analysis. One of the projects I worked on was a phone survey of teachers and school leaders in Rwandan secondary schools in 14 districts of the country. Here’s what I learned.
1. Data quality processes must be set up before data collection begins.
This ensures quality control is consistent throughout the data collection phase, leading to accurate, reliable, and valid data.
Before data collection, we stress-test the surveys internally to identify coding errors, test built-in consistency checks and survey logic, and provide initial survey length estimates to ensure all aspects of the survey are working as intended before launch. We set up high-frequency checks which allow the data team to observe and flag suspicious patterns in the data so they can be resolved quickly. This might include duplicate observations or outliers, discrepancies between the number of calls started and the number of calls ended, the length of each call, the outcome of call attempts, etc.
When data collection commences, we flag submitted records for review on areas in the survey that could be susceptible to inaccurate data in a Google sheets monitoring dashboard. This allows the data and research teams to maintain oversight of the survey’s progress and compliance to protocols in real-time. It also creates a continuous feedback process for enumerators, improving their accountability.
2. Preparing your sample for the survey can help build trust and confidence.
For the phone survey I worked on, we scheduled appointments with school leaders and notified teachers of the data collection schedule before the start of data collection.
Some teachers in the original sample for this survey had left their schools or profession, while at the same time new teachers had joined. To address this, we built questions in the form to ascertain whether the respondents were still working at the same schools and subjects.
During the pilot, we observed that teachers felt generally more confident taking part in the survey when they knew that their school leaders had been interviewed. So we changed the interview sequence by interviewing school leaders before teachers so they would feel comfortable talking to our enumerators.
3. Phone surveys bring specific challenges.
A phone survey of 20-30 minutes can be difficult to complete in one call, because it is harder for enumerators to build up rapport over the phone. Connectivity issues add to this challenge, as calls can drop halfway through an interview. Respondents, in this case teachers, also have other priorities during the course of their day. So we coded questions in the survey form to ascertain whether the respondents were still on the call during phone interviews, and enabled form progress to be saved and continued at a rescheduled date and time if a call dropped. Also, we simplified the form by reviewing answers to open-ended questions and converted them into closed questions with choices.
Working on the phone surveys of teachers and school leaders showed me the importance of tracking interventions in schools to measure their effectiveness. This will help to better understand the experience of respondents and highlight any gaps and areas for improvement. I believe that with high-quality data, policymakers and program implementers will be able to make informed decisions and data driven investments to boost development in African states.