Phone Survey Methods

Overview

The Living Standards Measurement Study (LSMS) launched the High Frequency Phone Surveys (HFPS) initiative in April 2020, in response to the urgent data needs that the COVID-19 crisis brought about in low- and middle-income countries (LMICs), to monitor and understand the socioeconomic impacts of the pandemic.

Until then, phone surveys were not widely used in LMICs, but lockdowns, mobility restrictions, and social distancing measures meant that face-to-face surveys had to be halted. The LSMS team has been collaborating closely with National Statistical Offices (NSOs) in Burkina Faso, Ethiopia, Malawi, Nigeria, Tanzania, and Uganda to conduct phone surveys and support capacity building. The LSMS-HFPS were made possible thanks to the existing infrastructure of longitudinal face-to-face surveys the LSMS team had supported as part of another LSMS initiative, the Integrated Surveys on Agriculture (LSMS-ISA).

As part of this effort, the LSMS team and its partners invested in methodological research that informs how to successfully implement phone surveys in LMIC contexts and how to collect reliable and relevant data. This work builds on survey experimentation as part of the regular phone survey data collection and includes insights and guidance for phone survey sampling, questionnaire design, data reliability and imputation.

Work Areas

1. Sampling

A first challenge when conducting phone surveys in LMIC contexts is obtaining a list of mobile phone numbers that can be used as a sampling frame. The LSMS-HFPS were conducted as re-contact surveys using phone numbers collected as part of the face-to-face LSMS-ISA surveys.

Another challenge is related to sample selection: only households with access to a phone can participate in phone surveys, which means that phone survey samples can bias estimates and analysis that are based on phone survey data. Research by the LSMS team showed that certain techniques of adjusting survey weights ex-post are effective in reducing biases related to sample selection at the household level.

2. Respondent Selection and Individual Level Data

It is difficult to collect individual-level data in phone surveys because they are more limited in scope, but this data is important for many policy variables related to employment, education, health and other areas. Research by the LSMS team highlights different options for obtaining better individual-level data, including collecting information on several individuals, interviewing different respondents and adjusting survey weights ex-post.

3. Mixed Mode Surveys and Mode Effects

Survey mode effects are differences in responses and measured outcomes that arise due to the method or mode of survey administration - by phone or in person. They potentially complicate comparing face-to-face and phone survey data and the implementation of mixed-mode surveys.

Yet, little is known about their extent in phone survey in LMICs and what can be done to reduce them. The LSMS team fielded a survey experiment to assess mode effects in Nigeria, building on ongoing data collection as part of the Nigeria General Household Survey Panel and the Nigeria Longitudinal Phone Survey.

4. Imputation

Since phone surveys are more limited in scope than face-to-face surveys, as previously mentioned, some important welfare variables cannot be collected over the phone. For example, household consumption, which underlies poverty measurement, is too complex and time-consuming for mobile phone surveys. Survey-to-survey imputation allows imputing variables based on a detailed base survey (the face-to-face survey) to a less complex target survey (the phone survey) that collects predictors for variables like poverty and consumption estimates.

The LSMS team is working on operationalizing survey-to-survey imputation for high frequency welfare measurement based on phone surveys. The evidence comes from survey experiments in Tanzania and Malawi.

Resources

Data

Living Standards Measurement Study