«Moving to opportunity voluMe 14, nuMber 2 • 2012 U.S. Department of Housing and Urban Development | Office of Policy Development and Research ...»
Indeed, the leading social science thinking of that time (Kain, 1968; Wilson, 1987) emphasized nonracial aspects of the isolation of high-poverty neighborhoods as critical to their apparent negative effects. Kain’s spatial mismatch theory pointed to the physical distance between the residents of such neighborhoods and the jobs for which they were qualified. Wilson’s theory stressed the departure of the African-American middle class, after the Fair Housing Act of 1968, from ghettos to which open discrimination had previously confined them. As beneficial as that departure might prove to those who had left, Wilson held that the loss of role models and community leaders had severely affected those who remained.
HUD2 proposed a rigorous experiment testing differences in outcomes between two groups of very low-income families with children, drawn from high-poverty neighborhoods in large metropolitan areas: one group that was offered counseling intended to result in a low-poverty subsidized rental placement (later known as the experimental group, or the low-poverty voucher group), and one group simply received a rent subsidy voucher with no such assistance (later known as the Section 8 group, or the traditional voucher group). Congress approved the study in the Appropriations Acts funding HUD operations in fiscal years 1992 and 1993 and authorized it in Section 152 of the Housing and Community Development Act of 1992, deviating from the HUD language only in narrowing the target population to residents of public and assisted housing in high-poverty census tracts and in mandating both short- and long-run reporting on the results.3 The change in targeting was significant: it required greater cooperation from public housing authorities (PHAs) and project owners; furthermore, unsubsidized tenants in high-poverty neighborhoods have somewhat different incentives and might have different characteristics from their assisted neighbors.4 The Appropriations Acts provided $70 million in additional voucher funding to support the demonstration.
HUD implementation of the demonstration required further decisionmaking. First, high- and low-poverty areas had to be defined. The high-poverty criterion—40 percent or more of the census tract population—was readily available from Jargowsky and Bane (1991). HUD chose as a criterion for target tracts that less than 10 percent of the population lived below the poverty line.
HUD derived that standard from U.S. Census maps, which showed that very large portions of the landmass in most metropolitan areas were located in tracts meeting that specification, a fact that, Shroder wrote most of the original legislative language.
HUD Assistant Secretary John Weicher and Senate Appropriations staff member Bruce Katz negotiated the narrower targeting in a meeting in 1992, but neither remembers who proposed it. Both HUD Secretary Jack Kemp and Senator Barbara Mikulski of Maryland, Katz’s principal, were committed to the orderly demolition of distressed public housing without reconcentration of poverty in other places.
Giving a housing voucher to someone living in private-market housing is a fundamentally different “treatment” from giving a voucher to someone who is already living in public housing. The latter is already receiving a substantial housing subsidy, although the person has no choice about where to use it; the former is unsubsidized. For some consequences, see Jacob and Ludwig (2012) and Mills et al. (2006).
Cityscape 33Shroder and Orr
presumably, would enhance the possibility for successful placements.5 In a competitive process requiring a joint application from a PHA and a nonprofit provider of counseling services, HUD selected five metropolitan areas as demonstration sites: Baltimore, Boston, Chicago, Los Angeles, and New York.6 HUD also had to determine whether the experimental group would face any special limitations in its use of vouchers. In general, geographic limitations on the use of vouchers would expressly violate the Housing Act of 1937. The authorization for the demonstration, however, could validly be interpreted as permitting deviation from that general rule. In 1993, HUD decided that, for purposes of testing the effect of neighborhood on families, it would limit the experimental group vouchers to low-poverty census tracts.7 Without that decision, MTO would have been a test of the effect of counseling on achieving mobility to low-poverty neighborhoods. Without the constraints on use, however, many fewer low-poverty placements would have been in the sample.
HUD used an existing contract with Abt Associates Inc. to design and implement the experiment.8 The first and most important issue for the contractor was how many random-assignment groups to create. Although $70 million for vouchers, in 1992 money, was a substantial support for a social science project, the number of vouchers available would be quite limited. The appropriations supported vouchers not for 1 year, which is current practice, but for 5; given inflation, that funding pattern effectively reduced the number of new families assisted by a factor of more than five. Moreover, families with children, in general, require larger units than the voucher program average unit size, and units in metropolitan areas, especially in better neighborhoods, require more subsidy because area rents are higher than the national average. Finally, the authorizing language required that some of the vouchers be used within the regular Section 8 program rules.
Given that HUD could not furnish large numbers of eligible families with vouchers in any case, Abt9 argued that it was both ethical and scientifically necessary to have a control group that would receive neither a low-poverty nor a traditional voucher. The creation of a control group in addition to the two voucher treatment groups would allow for strong comparisons of the effect of neighborhood on low-income families with children.
Random assignment began in Baltimore in 1994 and concluded in Los Angeles in 1998. We leave the results of the experiment to other articles in this symposium. The remainder of this article concerns the following issues: (1) Was a full-blown randomized social experiment necessary, or During implementation of the demonstration, families in the experimental group with large numbers of children were allowed to use their vouchers in somewhat higher poverty tracts.
Shroder and Bill Murphy were the authors of the Notice of Funding Availability for the site competition. The selection of the 40-percent poverty standard proved not to be innocuous, because some metropolitan areas did not have large numbers of such tracts or large amounts of public and assisted housing in them. This standard worked against the selection of otherwise strong proposals from Seattle and Fort Worth, among others.
The principal policymaker was Assistant Secretary Michael Stegman.
Shroder and John Goering were the authors of the initial Statement of Work.
Orr was Abt’s principal investigator on the project.
could “natural experimental” or “quasi-experimental” studies have produced equally valid results?
(2) What was unique about MTO? (3) Where do we stand today regarding the concentration of poverty? Are the issues the same or are they different?
Was a Randomized Social Experiment Necessary?
The use of administrative records about families placed in different neighborhoods on a quasirandom basis has led to claims, both in the Gautreaux program and elsewhere, that the experience of those families constitutes a “quasi-experiment” or a “natural experiment.”10 Unless these phrases are stretched well beyond their intended meaning, this claim is mistaken. Housing agencies in every case used by researchers to date have not kept data on the families who refused the placement, and the families who refuse placement in a “good” neighborhood are probably different from the families who refuse placement in a “bad” one. Consequently, even if the offer of the unit was made randomly, the placement was not.
Mendenhall, DeLuca, and Duncan (2006) reported known characteristics of the Gautreaux households that actually leased up at the time the offer of a city or suburban unit occurred. Exhibit 1 displays the results.
observables is itself of no great importance from an evaluation perspective, because one can control for observables in multivariate analysis. Necessarily, however, one cannot control for unobserved differences that might also bias the analysis.
Differences that would normally be unobserved (because information about them is generally not collected) turned out to be important in MTO. Shroder (2002) analyzed the probability of a family in the experimental group making use of the offer of a low-poverty voucher. Exhibit 2 shows that uncertainty about liking a new neighborhood, level of comfort with sending a child to a nearly all-White school, dissatisfaction with the current neighborhood, and preferred distance from that current unit that the family head would like to move are all strongly associated, either positively or negatively, with actual lease up. These factors associated with placement might also be associated with other outcomes, such as employment and education, and could bias studies based on observational data. These attitudinal indicators would not normally be collected and, therefore,
researchers could not control for them. Even with all the variables shown, however, the model in exhibit 2 correctly predicts only 65 percent of lease-up outcomes. We simply do not know all the factors that affect residential decisions for a given family, and asserting that what we don’t know won’t hurt us when we analyze other outcomes would be presumptuous. The value of a large-sample randomization is to ensure that these unmeasured or unmeasurable factors will balance out among the two treatment groups and the control group.
As long as tenants can refuse a unit that is offered to them, and housing authorities do not track the families that refuse units, it is difficult to say whether differences in the realized outcomes of families who accept random placements reflect the effect of neighborhood or merely the differing unmeasured characteristics of the families themselves.11 Readers can find additional evidence on the critical importance of randomization in Kling, Liebman, and Katz (2007) and Ludwig and Kling (2007). Their evidence is entirely different from what we present here but is entirely supportive of this point. For example, Ludwig and Kling wrote that applying nonexperimental techniques to MTO data yields evidence that crime is contagious—that is, bad examples from peers in bad neighborhoods tend to cause young men to commit crimes— but that after making use of the experimental design of MTO, they find “no evidence that contagion is as important as much of the previous research would suggest…” (Ludwig and Kling, 2007: 511).
In short, a randomized social experiment was necessary to estimate the effects of neighborhood on very low-income families with children. Nonexperimental techniques cannot solve the fundamental obstacles to valid inference.
What Makes MTO Unique?
MTO was remarkable in (1) the questions it asked, (2) the depth and scope of the effects it analyzed, (3) the range and quality of data sources it tapped, and (4) the long period over which it tracked participating families. Taken together, these attributes make MTO pathbreaking social policy research.
In this section, we explore each way that MTO distinguished itself from other social research.
The Research Question The question that MTO set out to address—What is the effect of neighborhood on low-income families?—is absolutely fundamental to our understanding of the nature of poverty. At least since the 1960s, various social scientists have hypothesized that concentrated poverty engenders a “culture of poverty” that encourages shortsighted, self-defeating behavior that traps low-income families in the underclass (see Harrington, 1962; Lewis, 1959). Although the culture of poverty is not synonymous with spatial concentration of low-income residents, such concentration would at least facilitate the social isolation that is advanced as a hallmark of that culture. Although the concept of a culture of poverty is controversial among social scientists, it is at least plausible that If housing authorities both randomized placement offers and maintained good records on families who rejected them, the baseline situation would be equivalent to that of a randomized experiment.
the lack of community resources and successful role models in areas of concentrated poverty could undermine the motivation and aspirations of low-income families and limit their access to highquality education, jobs, and other prerequisites of a healthy, prosperous life.
Nonetheless, the personal attributes of low-income people that may serve as barriers to success— for example, low education, lack of skills and job experience, poor health—would be present regardless of their residential location. The relative contributions of personal characteristics and residential environment to the poverty problem have been an unsolved riddle since the inception of the War on Poverty in the 1960s. Unlike most demonstrations, which seek only to test some specific approach to a problem, MTO set out to help us understand the problem in the only way we could hope to—with a rigorous experimental design capable of separating the effects of neighborhood from the effects of personal characteristics.
MTO did more than that, however. It also sought to measure the effectiveness of the two principal
approaches that HUD had used throughout most of its history to address the poverty problem:
public housing and rent vouchers. This ambitious combination of basic research and policy research objectives makes MTO unique among the major social experiments of the past 40 years.