Housing discrimination lawsuits are about to become much harder to prove.
Imagine if the auto industry successfully lobbied lawmakers to give them a complete defense from product defect liability if their cars were made by complicated, but increasingly commonly used, industrial robots.
Alternatively, imagine that the law didn’t go quite so far, but instead provided two escape valves from product defect lawsuits. The first escape valve exempts the carmaker from lawsuits for defects like random engine explosions. The second valve exempts one from any responsibility, if the robots were made by another company—no matter how defective the resulting cars are. Sound good? Those robots are pretty complicated, after all.
The Proposed Algorithmic Defense
The U.S. Department of Housing and Urban Development (“HUD”) received comments from lenders and housing agencies “expressing concern that complicated, yet increasingly commonly used, algorithmic models to assess factors such as risk or creditworthiness, should be provided a safe harbor” from disparate impact discrimination lawsuits.[1] The HUD responded with just what they were looking for: a set of complete algorithmic defenses to disparate impact liability.
The proposed HUD rule creates three complete defenses.[2] If any of the three defenses apply, the plaintiff who relies on the defendant’s use of an algorithm to show disparate impact will lose. First, the defendant can show the inputs of the algorithm are not “substitutes or close proxies for protected classes.”[3] Second, the HUD gives immunity to the company using the algorithm if it is in the control of a third party and is an industry standard .[4] Finally, the HUD sanctions models validated by a “neutral third party.”[5]
There is limited guidance on the definition of important terms. For example, it is unclear how close the “substitutes” or “proxies” must be to the protected class or who counts as a “neutral third party.”
Opposition to the Algorithmic Defense
Comments closed on the proposed rule on October 18. The rule was met with widespread and vigorous opposition. Among the 4057 comments, statements from the National Fair Housing Alliance,
A common criticism of the proposed rule is that the third-party defense will incentivize housing agencies, banks, and insurers to use third-party algorithms to shield themselves from liability. While the HUD argues that this allows plaintiffs to challenge discrimination at its source,[15] this result could just as easily be accomplished if lenders were able to bring third parties into court as third-party defendants. Further,
How the HUD Misunderstands Algorithms
Further, the HUD misunderstands the nature and threat of algorithmic discrimination, as argued by many commentators. At a fundamental level, predictive statistical models, like those used to assess whether or not someone is likely to repay a loan, make predictions by detecting correlations in historical data.[17] The HUD tests each correlation with what it calls “inputs.” Others call these predictor or independent variables, and in the context of machine learning they have been re-branded to be called “features.” Some statistical correlations can be meaningful, enabling a model to make accurate predictions into the future. Other correlations, however, are based on chance and do not provide future predictive accuracy; still others, like the tragic but measurable correlations between socioeconomic status and race, are based on historical factors like institutional discrimination which, if removed and alleviated, would not hold predictive accuracy in the future.[18] For all but the simplest tasks, tuning a model to make correct predictions is difficult and requires statistical expertise, good data, and a good understanding of that data.
Preventing discriminatory impact through the use of a statistical model is therefore much more complicated than ensuring that the individual inputs are not “substitutes or close proxies for protected characteristics.”[19] Many other factors could create a dramatic discriminatory effect: several of the inputs could be proxies for protected characteristics without being close proxies (think zip code, wealth, or home-ownership); there could be much more data available for some groups than others, leading to disparities in predictive accuracy; and even more challenging, the data itself could be tainted with historic discrimination such that it reflects neither the world as it should be nor the world as it may become through efforts to remove the impact of discrimination.
Conclusion
For these reasons and others, many commentators urge that the focus should be on the outputs of the model rather than the inputs .[20] Because I believe we should be as concerned about eliminating discrimination as we are about preventing defects in our cars , I join the many voices calling for a reconsideration of the proposed rule. There should be no get-out-of-jail free card for outsourcing your discrimination to an algorithm.
Footnotes
[1] HUD’s Implementation of the Fair Housing Act Disparate Impact Standard, 84 Fed. Reg. 42584, 42859 (August 19, 2019), https://www.federalregister.gov/d/2019-17542/p-96
[2] Proposed C.F.R. §100.500(c)(2). https://www.federalregister.gov/d/2019-17542/p-157. The full text of the affirmative defense is:
(2) Where a plaintiff alleges that the cause of a discriminatory effect is a model used by the defendant, such as a risk assessment algorithm, and the defendant:
(i) Provides the material factors that make up the inputs used in the challenged model and shows that these factors do not rely in any material part on factors that are substitutes or close proxies for protected classes under the Fair Housing Act and that the model is predictive of credit risk or other similar valid objective;
(ii) Shows that the challenged model is produced, maintained, or distributed by a recognized third party that determines industry standards, the inputs and methods within the model are not determined by the defendant, and the defendant is using the model as intended by the third party; or
(iii) Shows that the model has been subjected to critical review and has been validated by an objective and unbiased neutral third party that has analyzed the challenged model and found that the model was empirically derived and is a demonstrably and statistically sound algorithm that accurately predicts risk or other valid objectives, and that none of the factors used in the algorithm rely in
[3] Proposed C.F.R. §100.500(c)(2)(i), supra.
[4] Proposed C.F.R. §100.500(c)(2)(ii), supra.
[5] Proposed C.F.R. §100.500(c)(2)(iii), supra.
[6] https://nationalfairhousing.org/2019/10/10/huds-proposed-rule-is-a-direct-assault-on-the-supreme-courts-decision-in-inclusive-communities/
[7] http://www.nareb.com/site-files/uploads/2019/10/Real-Estate-Trade-Associations-Disparate-Impact-Comment-Letter.pdf
[8] https://www.aclu.org/sites/default/files/field_document/submitted_-_aclu_comment_re_hud_disparate_impact_nprm2.pdf
[9] https://ag.ny.gov/sites/default/files/hud_di_proposed_rule_ag_comment_final.pdf [“State AGs comment”]
[10] https://www.eff.org/deeplinks/2019/10/tell-hud-algorithms-are-no-excuse-discrimination [“EFF comment”]
[11] http://clinic.cyber.harvard.edu/files/2019/10/HUD-Rule-Comment-ONEIL-10-18-2019-FINAL.pdf
[12] https://ainowinstitute.org/ainow-cril-october-2019-hud-comments.pdf [“AI Now Institute comment”]
[13] https://www1.nyc.gov/assets/hpd/downloads/pdf/nyc-comment-hud-proposed-rule-disparate-impact-standard.pdf
[14] See, e.g., EFF’s comment, supra note 10 (“This defense gets rid of any incentive for defendants not to use models that result in discriminatory effect or to pressure model makers to ensure their algorithmic models avoid discriminatory outcomes.”).
[15] https://www.federalregister.gov/d/2019-17542/p-100
[16] See AI Now Institute comment, supra note 12, at 10 (“For instance, the Center and AI Now, in 2018 and 2019 Litigating Algorithms Reports, highlighted legal challenges to algorithmic tools used in education, public benefits, and criminal justice, where third party vendors used broad, and ultimately illegitimate, trade secrecy or confidentiality claims to obstruct efforts to examine the algorithm”).
[17] See Solon Barocas and Andrew Selbst, Big Data’s Disparate Impact, 104 Cal. L. Rev. 671, 677 (2016) (“In particular, [data mining / predictive modeling] automates the process of discovering useful patterns, revealing regularities upon which subsequent decision making can rely. The accumulated set of discovered relationships is commonly called a “model,” and these models can be employed to automate the process of classifying entities or activities of interest, estimating the value of unobserved variables, or predicting future outcomes.”)
[18] David R. Williams et al., Understanding Associations between race, Socioeconomic Status and Health: Patterns and Prospects, 35 Health Psychology 407 (2016), https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4817358/.
[19] See generally Joshua A. Kroll et al., Accountable Algorithms, 165 U. Pa. L. Rev. 633, 637 (2017), https://scholarship.law.upenn.edu/cgi/viewcontent.cgi?article=9570&context=penn_law_review
[20] See State AGs comment, supra note 9 (“[I]n the context of disparate impact claims the output of the algorithm, and the weight that output is given in a user’s decision-making is where liability should be primarily focused”).