Academic Funding

Gathering Data for Policy Makers, Business and the Public

May 5, 2013 1754

Federal surveys have been getting more expensive to administer, in part because the number of people who actually respond to surveys has been progressively declining.  As a result, researchers are developing and implementing new methods for collecting the information that our policy makers, business leaders and the public officials depend on. Many of these new approaches are highlighted in volume 645 (January 2013) of the Annals of the American Academy of Political and Social Science.  These approaches use new technologies and combine strategies to collect needed data. Below is an explanation of some of the technical terms and new approaches presented in the Annals.

Multi-modal Surveys

Combining several different methods of approaching people for surveys is a promising approach to maximize response rates. For example, the American Community Survey now is conducted through a multi-modal survey. First, a statistically valid random sample of people to be surveyed is selected from an address database. These people are mailed a letter asking them to fill out the survey on a secure website. Response rates for the website are fairly high, especially considering that the cost of this first step is very low. If people don’t respond, they are mailed a survey they can submit by mail. Then, a sub-sample of those who don’t respond are called, and some of those who don’t respond by phone are visited in-person by survey-takers. While the last steps are more costly, since they are only used for a subset of the survey participants, the multi-modal approach increases the response rate while keeping costs down. The response rate for this survey has been consistently near 98 percent of all household units.

This approach can be further refined: one possibility for the decennial Census, for example, is a promotional campaign asking people to pre-register electronically so they can receive the questionnaire by email.

Other Improvements in Survey Design Can Improve Response Rates

  • Response rates improve significantly when the survey targets are contacted before the survey, with a letter or email that explains why the survey is relevant to the person being asked to take the survey.
  • Multiple calls or other approaches to the person also increase response rates.
  • Reducing the burden of the survey, for example by limiting the number of questions, filling in information in advance where it can be obtained from administrative sources, using a “matrix” approach so that not every person surveyed has to answer every question and using proxy reports by others who are knowledgeable when the person doesn’t respond, can all help improve the response rate and reliability of survey data.
  • Financial incentives can improve response rates.

Surveys Can be Adjusted for Non-Response

Survey reliability can be improved by weighting the answers of the responses from subgroups to ensure that the sample reflects as much as possible the population being surveyed. Similar statistical measures can be used to adjust where people choose not to respond to individual questions.

Data Can be Obtained by Linking to Administrative Records

Federal administrative agencies such as the Internal Revenue Service, the Social Security Administration, the Center for Disease Control (which houses vital statistics) and the Department of Housing and Urban Development, as well as state and local agencies such as school boards and Departments of Public Health all maintain important data. If cross-linked appropriately, with great care to protect individual privacy, administrative data can provide important information that bolsters the accuracy and reliability of surveys. However, it does have limitations; for example, health records will not capture those uninsured people who do not seek care and education records won’t include the homeschooled child.

Digital Data

Digital data refers to information that is created electronically by government, business and individuals. These data might be gathered when someone makes a phone call, purchases an item, conducts a web search, or drives on an HOV lane, to give just a few examples. Commercial firms collect digital data on everything from phone records to purchasing patterns to travel records. The government also collects digital data, especially for its security and anti-terrorism programs. Some of these data, again with proper privacy protections, could be used to provide the kind of information that business, in particular, relies on.

The American Academy of Political and Social Science, one of the nation’s oldest learned societies, is dedicated to the use of social science to address important social problems. For over a century, our flagship journal, The Annals of the American Academy of Political and Social Science, has brought together public officials and scholars from across the disciplines to tackle issues ranging from racial inequality and intractable poverty to the threat of nuclear terrorism. Today, through conferences and symposia, podcast interviews with leading social scientists, and the annual induction of Academy Fellows and presentation of the Daniel Patrick Moynihan Prize, the Academy is dedicated to bridging the gap between academic research and the formation of public policy.

View all posts by American Academy of Political and Social Science

Related Articles

Why Might RFK Jr Be Good for US Health Care?
Public Policy
December 3, 2024

Why Might RFK Jr Be Good for US Health Care?

Read Now
Tenth Edition of The Evidence: Why We Need to Change the Narrative Around Part-Time Work
Bookshelf
December 2, 2024

Tenth Edition of The Evidence: Why We Need to Change the Narrative Around Part-Time Work

Read Now
Joshua Greene on Effective Charities
Social Science Bites
December 2, 2024

Joshua Greene on Effective Charities

Read Now
From the University to the Edu-Factory: Understanding the Crisis of Higher Education
Industry
November 25, 2024

From the University to the Edu-Factory: Understanding the Crisis of Higher Education

Read Now
The End of Meaningful CSR?

The End of Meaningful CSR?

In this article, co-authors W. Lance Bennet and Julie Uldam reflect on the inspiration behind their research article, “Corporate Social Responsibility in […]

Read Now
Deciphering the Mystery of the Working-Class Voter: A View From Britain

Deciphering the Mystery of the Working-Class Voter: A View From Britain

How is class defined these these days – asking specifically about Britain here but the question certainly resonates globally – and when […]

Read Now
Doing the Math on Equal Pay

Doing the Math on Equal Pay

In the UK, it’s November 20. In France, it’s today, November 8. For the EU, it’s November 15. It’s the day of […]

Read Now
0 0 votes
Article Rating
Subscribe
Notify of
guest

This site uses Akismet to reduce spam. Learn how your comment data is processed.

4 Comments
Newest
Oldest Most Voted
Inline Feedbacks
View all comments