FREE ELECTRONIC LIBRARY - Theses, dissertations, documentation

Pages:     | 1 |   ...   | 9 | 10 || 12 | 13 |   ...   | 56 |

«Moving to opportunity voluMe 14, nuMber 2 • 2012 U.S. Department of Housing and Urban Development | Office of Policy Development and Research ...»

-- [ Page 11 ] --

Abstract The Institute for Social Research (ISR) at the University of Michigan successfully led an intensive, long-term, in-person survey for the Moving to Opportunity (MTO) for Fair Housing demonstration final impacts evaluation (Sanbonmatsu et al., 2011), achieving final effective response rates (ERRs) of 89.6 percent among MTO adults and 88.7 percent among youth, well above what response rates of surveys with comparable low-income populations have accomplished. A variety of survey field strategies ISR employed— careful staff selection, strategic use of financial incentives, and close collaboration between ISR and the National Bureau of Economic Research—all contributed to these high ERRs.

The high costs associated with achieving high ERRs for in-person surveys like that employed in MTO raises questions about added value. Costs per survey interview nearly quadrupled during the last 4 fielding months. This extra investment increased the MTO adult survey ERR by only about 3.2 percentage points. A reanalysis of intention-to-treat estimates on selected outcomes suggests the merits of such an investment. If survey fielding had stopped at an 81-percent ERR for adults, we would have falsely concluded that MTO had no effect on two of four key health outcomes, that MTO had no effect on female youth mental health, and that MTO increased female youth idleness.

–  –  –

Introduction As early as the 17th century, scientists observed that individuals who live in economically disadvantaged neighborhoods fare worse on a range of outcomes—from physical and mental health to employment and earnings, schooling, crime, and consumer bankruptcy filings—than individuals who live in economically well-off neighborhoods (Macintyre and Ellaway, 2003). Untangling whether neighborhoods per se, or the variety of characteristics of the individuals residing in particular neighborhoods, drive this observed association has been difficult. The question of whether neighborhoods matter is further complicated by the fact that we cannot always observe or measure the reasons why individuals decide to live in particular types of neighborhoods, and these very same reasons might be highly related to their outcomes. The Moving to Opportunity (MTO) for Fair Housing demonstration is a study uniquely positioned to contribute to our understanding of whether neighborhoods have causal effects on individuals’ well-being. The MTO experiment produced changes in housing mobility and subsequent experiences in low-poverty neighborhoods that can help isolate the effects of neighborhood circumstances on outcomes from a host of other individual, household, or local community characteristics.

To maximize MTO’s contribution to science and policy, the long-term survey for the final impacts evaluation1 (Sanbonmatsu et al., 2011) had an ambitious data collection strategy that included a broad set of outcomes measured from administrative records sources and an intensive in-person survey, occurring up to 15 years after study entry, led by the Institute for Social Research (ISR) at the University of Michigan. The National Bureau of Economic Research (NBER) research team set very high response rate goals to ensure that survey data collection adequately represented the eligible MTO sample and captured a breadth of outcomes across the domains of housing, neighborhood safety, physical and mental health, employment, education, financial security, and youth risky behavior. ISR successfully reached a final effective response rate (ERR)2 of 89.6 percent among MTO adults and 88.7 percent among MTO youth (ages 10 to 20 as of December 2007) in the long-term survey for the final impacts evaluation. These ERRs are much greater than what some studies of low-income populations have accomplished (Weiss and Bailar, 2002) and on par with several longestablished and well-resourced survey initiatives such as the Panel Study of Income Dynamics3 (American Association of Public Opinion Research, 2012; Gouskova, 2008; Groves et al., 2004).

Reaching such high response rates not only required substantial time investment from the NBER research team (to fundraise and design the survey) and financial commitment from the U.S.

Department of Housing and Urban Development (HUD) and various other funders, but it also required creativity and flexibility among ISR staff to navigate and strategize in real time while Research on MTO originally launched separately for each site, with a series of academic research investigators leading each site. The followup survey for the MTO interim impacts evaluation (Orr et al., 2003) that Abt Associates Inc. conducted was the first effort to administer a comparable data collection for MTO families overall. HUD also funded Abt Associates to canvass MTO families through 2007 to maintain an updated contact list.

Ludwig (2012) describes the calculation of the effective response rate, which reflects the weighted proportion of interviews completed for the eligible adult and youth samples.

The Panel Study of Income Dynamics (PSID) obtains response rates of 93 to 94 percent, and recent studies of PSID youth have response rates of between 87 and 91 percent.

–  –  –

interviewers worked in the field. Following and finding thousands of economically disadvantaged families who lived or currently live near resource-poor, potentially unsafe neighborhoods is a complex task. This complexity was compounded by the amount of time that had passed since the last in-person or phone contact with MTO study members and changes in MTO households—many of the youngest cohort at MTO study entry have since split off to create their own households.

The previous in-person interview with an MTO study household member had been a minimum of 5 years before the start of the long-term survey data collection for the final impacts evaluation, and some of the sample (37 percent, or 3,830) had not been interviewed at the followup survey for the interim impacts evaluation (Orr et al., 2003), meaning the most recent contact may have occurred more than 10 years before the start of this data collection. In addition to facing the pure locational challenges of finding the eligible MTO survey sample in light of the high overall ERR aims, MTO researchers wanted to maintain balance in the temporal flow of completed interviews by site and by treatment status. Maintaining sample balance in this way required particular monitoring, nimbleness, and flexibility among ISR’s data collection staff to target eligible survey sample members strategically, on a week-by-week basis, for extra attention from interviewers.

Some challenges that the ISR data collection staff faced for the MTO long-term survey effort are common to survey data collection efforts in general, but many were relatively unique or new to the experiences of ISR. The first sections of this article describe the various data collection design strategies ISR employed to maximize the probability of achieving the high ERR and strategies ISR implemented to address unanticipated challenges that protected, as much as possible, the quality of data and research design after survey data collection was in the field. These sections address factors that contributed to (and worked against) achieving a high response rate for both adults and youth.

Overall, the MTO in-person long-term survey for the final impacts evaluation was a reliable, efficient, and essential resource for capturing multiple aspects of life circumstances and individual outcomes that otherwise would have been difficult to capture at scale compared with lower cost alternatives. As one very poignant example, researchers would not have discovered MTO’s surprising effects on mental health outcomes if not for a survey instrument with diagnostic questionnaires used to measure mental health disorders. The intensive efforts required to achieve the very high MTO response rates do raise questions, however, about the relative worth of extra resources necessary to complete interviews among those last, difficult-to-find respondents. If NBER researchers and the MTO study funders had spent fewer resources and stopped data collection at a lower response rate, what would the estimated effects on survey-based outcomes have looked like? In an attempt to evaluate the ex post scientific value of expending additional resources on increasing the survey response rate in a study such as MTO, the last section of this article describes the cost of MTO long-term survey data collection and a few back-of-the-envelope calculations of MTO’s effects under varying response rate assumptions.

Background The MTO demonstration began in the mid-1990s at five sites (Baltimore, Boston, Chicago, Los Angeles, and New York City). Low-income families with children living in public housing in highly disadvantaged areas who volunteered for the MTO program were randomly assigned to one of

Cityscape 59Gebler, Gennetian, Hudson, Ward, and Sciandra

three groups: an experimental group that was offered housing vouchers that had to be used in a low-poverty area along with mobility counseling from nonprofit agencies, a Section 8 group that was offered a traditional housing voucher with no locational restrictions, and a control group that was not offered a housing voucher but remained eligible for any public assistance to which they were otherwise entitled.

MTO long-term survey fielding launched in June 2008 and continued through April 2010, with a carefully staged release of sample by each of the initial five MTO sites across three waves,4 with second-stage subsampling of the hardest-to-locate cases triggered at a predetermined initial response rate threshold of 75 percent. The ISR data collection staff conducted interviews in person, using a laptop computer and averaging 108.3 minutes for adult interviews and 116.7 minutes for youth interviews. The staff interviewed one adult and up to three youth ages 10 to 20 in each MTO family. In families with more than three eligible youth, ISR randomly selected three for inclusion in the sample. In addition to employing computer-assisted interviews, the data collection protocol included taking physical measurements, collecting dried blood spot samples from adults, and facilitating achievement assessments for youth.

Conducting an extensive and complex data collection effort with a highly disadvantaged and mobile population posed many challenges. First locating the family; then convincing respondents to participate; and finally completing a survey, physical measurements, and achievement assessments on multiple individuals in a single household combined to make this data collection operation extremely difficult. Some MTO families included foster children or youth who left home several years before the interview and had not kept in contact with other family members. Address information was often outdated or incorrect, and many families were living under the radar, without credit cards, mortgages, driver’s licenses, or other identification that could help locate respondents. As a result, interviewers often conducted tracking with a labor-intensive, door-to-door search, checking address information and asking neighbors if they had information about where the respondent or family may have moved.

In addition to tracking challenges, the ISR data collection staff faced considerable challenges working in and around economically disadvantaged neighborhoods. The interviewing protocol required interviewers to carry a laptop computer and a large bag of supplies and equipment.5 Many areas had no public parking, requiring interviewers to carry the equipment long distances in inclement weather and up multiple flights of stairs in highrise buildings. Interviewers often had to work in unsafe neighborhoods and conducted many interviews in suboptimal locations, including small and crowded living rooms, sometimes with no heat or electricity. Family members and friends coming and going, loud televisions and radios, and other distractions often made it difficult to maintain respondents’ focus and confidentiality. Finally, ISR experienced staffing shortages in some areas because of the inability to recruit and retain qualified interviewers willing and able to The first release of sample was in June 2008, the second in September 2008, and the third in February 2009, upon securing enough funding to survey a random two-thirds of Section 8 group adults.

The combined weight of the laptop computer, equipment, and supplies needed for completing an interview was approximately 30 pounds.

–  –  –

work successfully under the challenging conditions MTO required. As a result, production progress across the five MTO cities was at times uneven and required adjustments in field protocols to take into account differences in completion rates by site.

MTO Survey Data Collection: Strategies for Optimizing Response Rates ISR used a multifaceted set of approaches to address the previously listed challenges. This section discusses the ISR field structure and tracking efforts and the tools and strategies it used to maximize respondent participation.

Field Team Structure In any study, building an effective field team is an essential element of successful data collection.

ISR developed its field staffing model for MTO to take advantage of the clustered sample and to address the challenges of working with a sample that was highly mobile and lived in disadvantaged areas. The ISR data collection staff was made up of seven teams: a team of approximately 8 to 20 field interviewers in each of the five MTO cities, a team of Internet trackers supporting the field interviewers, and a travel team of field interviewers. The tracking team was composed of individuals who had Internet access to public records and were skilled at conducting Internet searches, networking, and piecing together information from many different sources. The travel team comprised experienced field interviewers with a demonstrated ability to work effectively and efficiently in the field and who lived in other parts of the country and were available to travel to interview respondents who had moved away from the main MTO cities. The travel team also supplemented data collection in cities that were understaffed.

Pages:     | 1 |   ...   | 9 | 10 || 12 | 13 |   ...   | 56 |

Similar works:

«The use of Te Reo Māori for Assessment Policy Student Policy Group 1. Purpose: This policy provides for students to use Te Reo Māori in work being submitted for assessment as part of a course of study.2. Organisational Scope: This is a University-wide policy.3. Definitions: Assessment: The measurement of a student’s performance 4. Policy Content and Guidelines:4.1 Policy (a) Students may use Te Reo Māori in assessment except where: (i) a course is taught fully or partly in a language other...»

«Case Study Approach Examining Local Wellness Policy Development and the Perceived Impact to the School Community National Food Service Management Institute The University of Mississippi 1-800-321-3054 This publication has been produced by the National Food Service Management Institute – Applied Research Division, located at The University of Southern Mississippi with headquarters at The University of Mississippi. Funding for the Institute has been provided with federal funds from the U.S....»

«7. Making Policy and Winning Votes: Election promises and political strategies in the 2013 campaign Nicholas Reece This chapter examines the intersection of public policy and politics in the 2013 federal election campaign. More than any other point in the political cycle, election campaigns are a time in which candidates and political parties release a large amount of new policy in the hope that it will win them increased public support. The candidates and the parties also attack the policies...»

«ACCREDITATION POLICY AND PROCEDURE MANUAL Effective for Reviews During the 2014-2015 Accreditation Cycle Incorporates all changes approved by the ABET Board of Directors as of October 26, 2013 Please Note: The ABET Board of Directors adopted revisions to ABET Constitution and the ABET By Laws in October 2012. This ABET Accreditation Policy & Procedure Manual is undergoing review and potential revision to ensure alignment with the ABET Constitution and By Laws. As this work proceeds, and until...»

«UNDERSTANDING POLICY FORMULATION: A TOOLS PERSPECTIVE 1 John Turnpenny Senior Lecturer in the School of Politics, Philosophy, Language and Communication Studies, University of East Anglia, Norwich, UK Andrew Jordan Professor of Environmental Policy, Tyndall Centre School of Environmental Sciences, University of East Anglia.ABSTRACT Policy formulation is a key element of policy-making, but in spite of increasing interest it is still greatly under-explored. This paper develops a research agenda...»

«Booking Policy – VOSA examiners at ATFs Introduction This document describes the policy and procedures that apply for the booking of VOSA examiners to attend at an ATF Site to conduct testing. VOSA Booking Objective VOSA’s aim is to provide examiners for testing at ATFs at times that are convenient and best suit each individual ATF. VOSA’s ability to meet applications for all dates and times requested will be constrained however, for example, by the total number of examiners available on...»

«Bright Home Insurance Policy Summary This is a summary of the policy and does not contain the full terms and conditions of the cover, which can be found in the policy documentation. It is important that you read the policy documentation carefully when you receive it. One policy booklet covers all sections of this policy other than Family Professional Fees which is covered by a separate policy booklet. Who is the Insurer? The Insurer of all sections of this policy other than Family Professional...»

«Public Policy Research and Training in Vietnam Public Policy Research and Training in Vietnam Edited by Toru Hashimoto Stefan Hell Sang-Woo Nam Asian Development Bank Institute Hanoi December 2005 Asian Development Bank Institute Kasumigaseki Building, 8th Floor 3-2-5 Kasumigaseki, Chiyoda-ku Tokyo 100-6008, Japan Tel: (81-3) 3593-5500 Fax: (81-3) 3593-5571 Email: info@adbi.org Website: www.adbi.org © 2005 Asian Development Bank Institute Publishing Permit/GPXB164-54/XB-QLXB dated 30/11/2005...»

«TOWARDS A WATER AND FOOD SECURE FUTURE Critical Perspectives for Policy-makers WHITE PAPER TOWARDS A WATER AND FOOD SECURE FUTURE Critical Perspectives for Policy-makers The outlook for 2050 is encouraging, globally, but much work is needed to achieve sustainable water use and ensure food security for all. Revised reprint FOOD AND AGRICULTURE ORGANIZATION OF THE UNITED NATIONS Rome, 2015 WORLD WATER COUNCIL Marseille, 2015 The designations employed and the presentation of material in this...»

«Technical Report 2010 049 Soil biodiversity: functions, threats and tools for policy makers February 2010 Europe Direct is a service to help you find answers to your questions about the European Union New freephone number: A great deal of additional information on the European Union is available on the Internet. It can be accessed through the Europa server (http://ec.europa.eu). ISBN : 978-92-79-20668-9 doi : 10.2779/14571 © European Communities, 2010 Reproduction is authorised provided the...»

«82-01-06 DATA SECURITY MANAGEMENT THE SECURITY POLICY LIFE CYCLE: FUNCTIONS AND RESPONSIBILITIES Patrick D. Howard, CISSP INSIDE Policy Functions; Policy Responsibilities; Policy Function–Responsibility Matrix It is time to let out a great sigh of relief. After countless months of tedious effort, one has succeeded in writing one’s company’s Internet Usage Policy. Time to celebrate, right? Well, maybe. It is true that the greatest hurdle for many organizations is documenting its...»

«Chapter 1 L InTroducTIon Science Wars and Policy Wars W hen considering the importance of science in policymaking, common wisdom contends that keeping science as far as possible from social and political concerns would be the best way to ensure science’s reliability. This intuition is captured in the value-free ideal for science—that social, ethical, and political values should have no influence over the reasoning of scientists, and that scientists should proceed in their work with as...»

<<  HOME   |    CONTACTS
2016 www.theses.xlibx.info - Theses, dissertations, documentation

Materials of this site are available for review, all rights belong to their respective owners.
If you do not agree with the fact that your material is placed on this site, please, email us, we will within 1-2 business days delete him.