Essential: 13 Ways To Distort Rapid Re-housing Outcomes

Family Options shows us that much of the reported Rapid Re-housing data, particularly around stability, was in error. Why? This list from April 2014 offers insight.

Appendix A: Recognizing Research Distortions

Due to a variety of pressures Rapid Re-housing Research is often misreported. Quality research avoids any of the following distortions:

1. By definition the Rapid Re-housing program exit rate to permanent (rental) housing is necessarily near 100%. Simultaneous to program exit a final rental payment is made by Rapid Re-housing program to the household’s landlord. This means that the household, by definition, the household must exit to permanent (rental) housing. Anyone directly comparing exits from shelters, or programs not subsidizing rental housing, to exits from Rapid Re-housing either do not understand the nature of these interventions or is intentionally disingenuous. Any citation of comparative program exit rates without acknowledgement of this distinction is illustration that one is reviewing Rapid Re-housing propaganda rather than research.

2. It is common for the “successes” of Rapid Re-housing households in avoiding homelessness to be reported during a period when the household is still rent subsidized. Improved Rapid Re-housing research will only begin six months after rental subsidies end so that the residual financial and social capital created by the subsidies has dissipated to a point more like those that did not receive the subsidy.

3. It is common for as much as 85% (HPRP) of the reported success of Rapid Re-housing to come in the form of reports regarding prevention households that were often above the sub-15% Area Median Income (AMI) typical of homelessness, were never homeless, and were therefore never re-housed. Further HPRP caseworkers were required to assist only those households who would be able to sustain rent post-subsidy. Suggesting that sheltered households will have the same outcomes as households who received rental assistance and were never removed from their homes creates false expectations. HPRP had an AMI maximum of 50% and thereby served innumerable households who would never have entered a shelter. Even interventions limited to 30% of AMI are often serving households not typical of those in shelters (note Hennepin discovery that those given prevention funding had two to four times the income of those in shelters - http://www.endhomelessness.org/page/-/files/3642_file_Promising_Strategy_Prevention_Targeting_in_Hennepin.pdf).

4. One category of “permanent housing” reported as a statistical success in homelessness (HMIS) research, often at levels above 20%, is assignment of the exiting households to the category of being “permanently housed with family or friends”. Service providers, and likely many researchers, all know that people who were formerly homeless, who are now housed with family and friends, are far more unstably housed than, for example, households in Interim/Transitional Housing. Despite this the household housed with family and friends is statistically placed in the category of “permanently housed” while the household in Transitional Housing is deemed still homeless. Savvy service providers, under pressure to show statistical success, know that their outcomes improve by the excessive use of the HMIS “permanently with family or friends” category. Research that does not carefully follow outcomes for individual households tend to be far less valuable in understanding true outcomes.

5. Much of the reporting on successes often comes in regions where additional subsidies are being made available for a particular type of household that is homeless. Veterans are one example. Another example is that there may be a report that family homelessness declined in Alameda County, California without reference to the unique “bonanza” of short-term investment in these households stemming from, for example, a research project. “Bonanza” is the description of Martha Burt discussing the estimated 604 additional housing vouchers released in the 12 communities of the ongoing $10 million Family Options study. Data regarding diminished family homelessness should control for the statistical improvements resulting from these unique vouchers. 

6. Rapid Re-housing programs are often mandated to “cream” clients by offering housing only to households that have very low barriers to stability and reporting results as if applicable to all household that are homeless. For example households with low-incomes are screened out of participation while the “research” suggests that these results are applicable to all shelter residents. Often unaccounted for in Rapid Re-housing research are the additional screening of clients by the sheer nature of navigating the benefits system, the support of a caseworker, and the approval of a landlord. These subtle forms of creaming can lead to a selection bias favorable to positive outcomes. In Philadelphia the Office of Supportive Housing described their process this way, “The social worker submitting the application makes a judgment call on the client’s ability to maintain housing then we screen the application and make a decision of whether or not to move the application forward based on the clients income or potential income, severity of behavioral health needs and the questions in the social summary. We approve about half of the applications submitted to us and about half of those lease up. We’ve found that in Philadelphia, based on rents, a households (2-3 bedroom size) needs to make a minimum of $900 a month to sustain housing.” Based on their research Philadelphia went on to request from HUD that Prevention funding only be given to those with incomes exceeding 22% AMI (http://www.regulations.gov/api/contentStreamer?objectId=0900006480fad025&disposition=attachment&contentType=pdf).

7. The Homelessness Prevention and Rapid Re-Housing Program (HPRP) remains the primary source of Rapid Re-housing research. As the Office of the Inspector General summarized “HUD does not have reliable data for outcome-based measurements of the HPRP (http://www.hudoig.gov/reports-publications/audit-reports/homelessness-prevention-and-rapid-re-housing-program). After a $1.5 billion dollar investment HUD is left seeking subjective outcome stories https://www.onecpd.info/news/hud-requests-hprp-program-participant-outcomes-information-and-promising-practicessuccess-stories/.

8. Permanent Supportive Housing (PSH) programs that serve only those with access to permanent disability income have been also been reported on as if their outcomes were applicable to the households without access to disability income. Often the PSH programs include an income payee so that clients cannot receive their remaining income until after their rent is paid. 

9. In cost comparisons services provided to Rapidly Re-housed households by community agencies, which are included in shelter cost estimates, are not included in cost calculations. This tremendously inflates the apparent shelter cost. Undoubtedly researchers could create another list noting cost estimate distortions.

10. Note that reported research outcomes that do not make note of immediate negative household outcomes, or of households evicted and/or lost contact with during the course of the research, are likely to be engaged in significant underestimates of negative outcomes. If the percentage of households experiencing these outcomes is not noted one is working with fairly low quality research.

11. A Research Note: The best outcomes research always involves direct contact with the household over a longitudinal period or access to independently verifiable variables such as earned income. 

12. A Research Note: Without a control group of households as part of research one cannot say with certainty what the outcomes would have been without an intervention or what benefit the intervention brought.

13. A Research Note: To some degree it is important to recognize the degree to which the funder of the research will, if the research has integrity, not bias the research outcomes but bias the way the research is reported. Researchers are not inclined to be explicit about a failed intervention (at least in terms of cost/benefits) if the funder is hopeful for a different outcome. There are other examples from Rapid Re-housing but the Washington State study is a great example: http://www.dshs.wa.gov/pdf/ms/rda/research/11/185.pd