Academic Funding

Gathering Data for Policy Makers, Business and the Public

May 5, 2013 1562

Federal surveys have been getting more expensive to administer, in part because the number of people who actually respond to surveys has been progressively declining.  As a result, researchers are developing and implementing new methods for collecting the information that our policy makers, business leaders and the public officials depend on. Many of these new approaches are highlighted in volume 645 (January 2013) of the Annals of the American Academy of Political and Social Science.  These approaches use new technologies and combine strategies to collect needed data. Below is an explanation of some of the technical terms and new approaches presented in the Annals.

Multi-modal Surveys

Combining several different methods of approaching people for surveys is a promising approach to maximize response rates. For example, the American Community Survey now is conducted through a multi-modal survey. First, a statistically valid random sample of people to be surveyed is selected from an address database. These people are mailed a letter asking them to fill out the survey on a secure website. Response rates for the website are fairly high, especially considering that the cost of this first step is very low. If people don’t respond, they are mailed a survey they can submit by mail. Then, a sub-sample of those who don’t respond are called, and some of those who don’t respond by phone are visited in-person by survey-takers. While the last steps are more costly, since they are only used for a subset of the survey participants, the multi-modal approach increases the response rate while keeping costs down. The response rate for this survey has been consistently near 98 percent of all household units.

This approach can be further refined: one possibility for the decennial Census, for example, is a promotional campaign asking people to pre-register electronically so they can receive the questionnaire by email.

Other Improvements in Survey Design Can Improve Response Rates

  • Response rates improve significantly when the survey targets are contacted before the survey, with a letter or email that explains why the survey is relevant to the person being asked to take the survey.
  • Multiple calls or other approaches to the person also increase response rates.
  • Reducing the burden of the survey, for example by limiting the number of questions, filling in information in advance where it can be obtained from administrative sources, using a “matrix” approach so that not every person surveyed has to answer every question and using proxy reports by others who are knowledgeable when the person doesn’t respond, can all help improve the response rate and reliability of survey data.
  • Financial incentives can improve response rates.

Surveys Can be Adjusted for Non-Response

Survey reliability can be improved by weighting the answers of the responses from subgroups to ensure that the sample reflects as much as possible the population being surveyed. Similar statistical measures can be used to adjust where people choose not to respond to individual questions.

Data Can be Obtained by Linking to Administrative Records

Federal administrative agencies such as the Internal Revenue Service, the Social Security Administration, the Center for Disease Control (which houses vital statistics) and the Department of Housing and Urban Development, as well as state and local agencies such as school boards and Departments of Public Health all maintain important data. If cross-linked appropriately, with great care to protect individual privacy, administrative data can provide important information that bolsters the accuracy and reliability of surveys. However, it does have limitations; for example, health records will not capture those uninsured people who do not seek care and education records won’t include the homeschooled child.

Digital Data

Digital data refers to information that is created electronically by government, business and individuals. These data might be gathered when someone makes a phone call, purchases an item, conducts a web search, or drives on an HOV lane, to give just a few examples. Commercial firms collect digital data on everything from phone records to purchasing patterns to travel records. The government also collects digital data, especially for its security and anti-terrorism programs. Some of these data, again with proper privacy protections, could be used to provide the kind of information that business, in particular, relies on.

The American Academy of Political and Social Science, one of the nation’s oldest learned societies, is dedicated to the use of social science to address important social problems. For over a century, our flagship journal, The Annals of the American Academy of Political and Social Science, has brought together public officials and scholars from across the disciplines to tackle issues ranging from racial inequality and intractable poverty to the threat of nuclear terrorism. Today, through conferences and symposia, podcast interviews with leading social scientists, and the annual induction of Academy Fellows and presentation of the Daniel Patrick Moynihan Prize, the Academy is dedicated to bridging the gap between academic research and the formation of public policy.

View all posts by American Academy of Political and Social Science

Related Articles

Megan Stevenson on Why Interventions in the Criminal Justice System Don’t Work
Social Science Bites
July 1, 2024

Megan Stevenson on Why Interventions in the Criminal Justice System Don’t Work

Read Now
Why We’ve Had to Dramatically Shift How We Talk About UK Politics
Insights
June 25, 2024

Why We’ve Had to Dramatically Shift How We Talk About UK Politics

Read Now
Pandemic Nemesis: Illich reconsidered
News
June 14, 2024

Pandemic Nemesis: Illich reconsidered

Read Now
Beyond Net-Zero Targets: When Do Companies Maximize Their Potential to Reduce Carbon Emissions?
Business and Management INK
June 4, 2024

Beyond Net-Zero Targets: When Do Companies Maximize Their Potential to Reduce Carbon Emissions?

Read Now
Rob Ford on Immigration

Rob Ford on Immigration

Opinions on immigration are not set in stone, suggests Rob Ford – but they may be set in generations. Zeroing in on the experience of the United Kingdom since the end of World War II, Ford – a political scientist at the University of Manchester – explains how this generation’s ‘other’ becomes the next generation’s ‘neighbor.’

Read Now
Biden Administration Releases ‘Blueprint’ For Using Social and Behavioral Science in Policy

Biden Administration Releases ‘Blueprint’ For Using Social and Behavioral Science in Policy

U.S. President Joseph Biden’s administration has laid down a marker buttressing the use of social and behavioral science in crafting policies for the federal government by releasing a 102-page Blueprint for the Use of Social and Behavioral Science to Advance Evidence-Based Policymaking.

Read Now
New Fellowship for Community-Led Development Research of Latin America and the Caribbean Now Open

New Fellowship for Community-Led Development Research of Latin America and the Caribbean Now Open

Thanks to a collaboration between the Inter-American Foundation (IAF) and the Social Science Research Council (SSRC), applications are now being accepted for […]

Read Now
0 0 votes
Article Rating
Subscribe
Notify of
guest

This site uses Akismet to reduce spam. Learn how your comment data is processed.

4 Comments
Newest
Oldest Most Voted
Inline Feedbacks
View all comments