class: bottom, left, title-slide .title[ # Fair algorithms, causality, and measurement ] .author[ ### Joshua Loftus (LSE Statistics) ] --- class: split-four <style type="text/css"> .remark-slide-content { font-size: 1.2rem; padding: 1em 4em 1em 4em; } </style> .column[  Matt Kusner (UCL)  Julia Stoyanovich ] .column[  Chris Russell (AWS/ELLIS)  Ke Yang ] .column[  Ricardo Silva (UCL)  Lucius Bynum ] .column[ ## Turing group ### NYU  Margarita Boyarskaya ] --- ### High level intro Motivating examples, guiding philosophies ### Causal fairness In prediction and ranking tasks, with intersectionality, interference, ... ### Concluding thoughts And totally unfounded speculations --- ### Serious business... .pull-left[ "New products and services, including those that incorporate or utilize **artificial intelligence and machine learning**, can raise new or exacerbate existing *ethical, technological, legal*, and other challenges, which **may negatively affect our brands** and demand for our products and services and **adversely affect our revenues** and operating results" ] .pull-right[  Source: [WIRED](https://www.wired.com/story/google-microsoft-warn-ai-may-do-dumb-things/), Feb. 2019. ] --- ### Not a new problem for psychometrics!  See [50 Years of Test (Un)fairness: Lessons for Machine Learning](https://dl.acm.org/doi/10.1145/3287560.3287600) (Hutchinson and Mitchell, 2019) --- ### Simpson's paradox <iframe src="https://en.wikipedia.org/wiki/Simpson%27s_paradox#UC_Berkeley_gender_bias" scrolling= "no" height="490" width="760" ></iframe> --- ### Berkeley graduate admissions example > The bias in the aggregated data stems **not from any pattern of discrimination on the part of admissions committees**, which seem quite fair on the whole, but apparently from **prior screening at earlier levels** of the educational system. Women are shunted by their **socialization and education** toward fields of graduate study that are generally more crowded, less productive of completed degrees, and less well funded, and that frequently offer poorer professional employment prospects. From the final paragraph of Bickel et al (1975) #### What information / covariates should we condition on? --- ### Directed Acyclic Graph (DAG) models  - Assumption: paths show conditional dependence - Assumption: intervention to change one variable also affects all variables on paths away from it --- ## Two interpretations ### Interventions **Can we change the DAG**, for example via socialization or education (emphasizing more women role models in STEM curricula, say?) ### Counterfactual fairness I was admitted to grad school. **If I had been different**, had a different gender for example, maybe I would have had different socialization, applied to a different department, and not been admitted. **Is that fair**? --- ## Recidivism prediction  Source: [ProPublica](https://www.propublica.org/article/how-we-analyzed-the-compas-recidivism-algorithm), 2016. --- ### (Infamous?) policing example "Algorithm does not *explicitly* use race" -- in an unfair world, **other things can correlate with race**, e.g. prior convictions .pull-left[ <div id="htmlwidget-f843a680a1fcd62a0dd1" style="width:504px;height:504px;" class="DiagrammeR html-widget"></div> <script type="application/json" data-for="htmlwidget-f843a680a1fcd62a0dd1">{"x":{"diagram":"\ngraph LR\n R-->M\n S-->M\n S-->P\n M-->P\n P-->C\n"},"evals":[],"jsHooks":[]}</script> ] .pull-right[ Race `\(R\)`, structural racism `\(S\)`, arrested for marijuana `\(M\)`, prior conviction `\(P\)`, risk score `\(C\)` If `\(C\)` is computed using only `\(P\)` (and not `\(R\)` *directly*), is that "fair"? ] --- ## How do we get a DAG? Simple examples are easy to understand Too simple? Don't correspond to the real world? ### How do we know if it's "true"? -- My opinion: we could use a healthy dose of (philosophical) > [Pragmatism](https://en.wikipedia.org/wiki/Pragmatism) is a philosophical tradition that considers words and thought as *tools and instruments for prediction, problem solving, and action*, and rejects the idea that the function of thought is to describe, represent, or mirror reality. --- #### Statistical wisdom: models as (thinking) tools .pull-left[  [George Box](https://en.wikipedia.org/wiki/George_E._P._Box) ] .pull-right[ > [All models are wrong](All_models_are_wrong) but some are useful therefore, > ... the scientist must be alert to what is **importantly wrong** > ... **cannot obtain a "correct" one** by excessive elaboration ] --- class: inverse, center, middle ## My research focus: **causal fairness** --- ## Defining fairness causally Kusner, Loftus, Russell, Silva. *Counterfactual fairness* ([NeurIPS 2017](https://papers.nips.cc/paper/2017/hash/a486cd07e4ac3d270571622f4f316ec5-Abstract.html)): Given a DAG, the predictor `\(\hat Y\)` is **counterfactually fair** if `$$P(\hat Y_a | x, a) = P(\hat Y_{a'} |x, a)$$` #### Proposition (sufficient, not necessary): Any predictor `\(\hat Y\)` which is a function of only non-descendents of `\(A\)` in the DAG is counterfactually fair --- ### Pathway analysis / decomposition .pull-left[  - [Kilbertus et al (2017)](https://papers.nips.cc/paper/2017/hash/f5f8590cd58a54e94377e6ae2eded4d9-Abstract.html): proxies and **resolving variables** ] .pull-right[ - [Kusner et al (2017)](https://papers.nips.cc/paper/2017/hash/a486cd07e4ac3d270571622f4f316ec5-Abstract.html): path-dependent counterfactual fairness (see supplement) - [Zhang et al (2017)](https://www.ijcai.org/proceedings/2017/549) - [Nabi and Shpitser (2018)](https://ojs.aaai.org/index.php/AAAI/article/view/11553) - [Zhang and Bareinboim (2018)](https://aaai.org/ocs/index.php/AAAI/AAAI18/paper/view/16949/0) - [Chiappa (2019)](https://ojs.aaai.org//index.php/AAAI/article/view/4777) - ... ] --- ### The Fundamental Contradiction of Fairness See, e.g.: Kleinberg et al (2016), Chouldechova (2016) #### (Simplified) impossibility theorem > Unless the world is already fair, the only solutions satisfying both equal treatment (or opportunity) and equal outcomes (demographic parity) are trivial ones (e.g. jail everyone) Many versions can be proven with different sets of assumptions but basically the same conclusion: some fairness definitions are contradictory --- class: split-four ### Every DAG is (importantly?) wrong .column[ <div id="htmlwidget-ba32e0541469b5ed4906" style="width:504px;height:504px;" class="DiagrammeR html-widget"></div> <script type="application/json" data-for="htmlwidget-ba32e0541469b5ed4906">{"x":{"diagram":"\ngraph TB\n Race-->Outcome\n"},"evals":[],"jsHooks":[]}</script> ] .column[ <div id="htmlwidget-484ed42468d86de4a421" style="width:504px;height:504px;" class="DiagrammeR html-widget"></div> <script type="application/json" data-for="htmlwidget-484ed42468d86de4a421">{"x":{"diagram":"\ngraph TB\n Racism-->Outcome\n"},"evals":[],"jsHooks":[]}</script> ] .column[ <div id="htmlwidget-697d58abce2053a24549" style="width:504px;height:504px;" class="DiagrammeR html-widget"></div> <script type="application/json" data-for="htmlwidget-697d58abce2053a24549">{"x":{"diagram":"\ngraph TB\n A-->X\n X-->Y\n A-->Y\n"},"evals":[],"jsHooks":[]}</script> ] .column[ <div id="htmlwidget-29a56e939b13f92b7856" style="width:504px;height:504px;" class="DiagrammeR html-widget"></div> <script type="application/json" data-for="htmlwidget-29a56e939b13f92b7856">{"x":{"diagram":"\ngraph TB\n A-->X\n A-->R\n X-->Y\n R-->Y\n A-->Y\n"},"evals":[],"jsHooks":[]}</script> ] Transparency: we can say specifically what we disagree on Interpretation: meanings of variables, edges (mechanisms) --- ### Impossibility: competing DAGs / pathways When Worlds Collide: Integrating Different Counterfactual Assumptions in Fairness (Russell et al, *NeurIPS 2017*) - Approximate counterfactual fairness (relax equality constraint) - Approximately satisfy fairness across both (all) models - Limitation: the more contradictory are the competing models, the more trivial the predictions (constant) - Causal framing of fundamental contradiction #### Resolving the contradiction I think this is a good *path*. It's now about understanding the causes of unfairness well enough to reach consensus. --- ## [Intersectional](https://en.wikipedia.org/wiki/Intersectionality) fairness Yang, Stoyanovich, Loftus. *Causal Intersectionality and Fair Ranking* ([FORC 2021](https://drops.dagstuhl.de/opus/portals/lipics/index.php?semnr=16187)) - Multiple sensitive attributes, e.g. race and gender - Variety of relationships with other mediating variables - Some of these mediators may be resolving/non-resolving for different sensitive attributes Lots of scholarship, not much using formal mathematical models. See [Bright et al (2016)](https://www.journals.uchicago.edu/doi/abs/10.1086/684173), [O'Connor et al (2019)](https://www.tandfonline.com/doi/abs/10.1080/02691728.2018.1555870), and a few other references in our paper --- ## "Moving company" example .pull-left[ Race, gender, weightlifting test, application score <div id="htmlwidget-457827ab1876f7c57926" style="width:504px;height:504px;" class="DiagrammeR html-widget"></div> <script type="application/json" data-for="htmlwidget-457827ab1876f7c57926">{"x":{"diagram":"\ngraph TB\n R-->X\n G-->X\n R-->Y\n X-->Y\n G-->Y\n"},"evals":[],"jsHooks":[]}</script> ] .pull-right[ Weightlifting considered a **resolving variable** (company argues it is a necessary qualification) <img src="../moving.png" width="100%" style="display: block; margin: auto;" /> (See paper for results) ] --- ### Untangling intersectional relationships Causal mediation with **multiple mediators** and **multiple causes** is a hard problem, limiting realistic application until better methods/data are available #### Opportunity for statistical research  --- ## Interventions under [interference](https://en.wikipedia.org/wiki/Spillover_(experiment%29) Kusner, Russell, Loftus, Silva. Making Decisions that Reduce Discriminatory Impacts ([ICML 2019](http://proceedings.mlr.press/v97/kusner19a.html)) - Designing an optimal policy / intervention / allocation - Relaxing common assumption that intervention on individual/unit `\(i\)` does not effect other individuals/units (in fairness/justice applications that will often be **importantly wrong**) - Constraint: bound **counterfactual privilege**, preventing "rich get richer" effect --- ### Optimal fair policies under interference .pull-left[ Intervention `\(\mathbf Z\)` trying to increase `\(\mathbf Y\)` Privilege constraint, for `\(\tau \geq 0\)` $$ \mathbb{E}[\mathbf{\hat Y}({\color{red}a},\mathbf{Z})] - \mathbb{E}[\mathbf{\hat Y}({\color{blue}a'},\mathbf{Z})] \leq \tau $$ ] .pull-right[  ] Optimization problem (with budget constraint `\(b\)`) `$$\mathbf{Z} = \arg \max \sum_i \mathbb{E}\left[\mathbf{\hat Y}^{(i)}(a^{(i)}, \mathbf Z) | \mathbf A^{(i)}, \mathbf X^{(i)} \right]$$` `$$\quad s.t. \quad \sum_i \mathbf{Z}^{(i)} \leq b$$` --- ### Allocating resources to (NYC) schools .pull-left[ Without constraint  ] .pull-right[ With constraint  ] (See paper for discussion of results) --- ### Exciting advances in causal modeling Observational data, machine learning, causal reinforcement learning, ... ### Applications in ethical data science Prioritizing normative values: fairness, privacy, interpretability, replicability, "alignment," ... --- class: inverse, center, middle ## Discussion and speculation --- ### DAGs can be useful tools [Imbens (2019)](https://siepr.stanford.edu/sites/default/files/publications/19-022.pdf) on DAGs, reviewing The Book of Why: > TBOW and the DAG approach fully deserve the attention of all researchers and users of causal inference as one of its leading methodologies. But they have limitations: > In economics the endogeneity often arises from agents actively making choices regarding the causal variable on the basis of anticipated effects of those choices --- ### Bet on causality Claim: **causality** points us in good directions for research - Choosing which covariates to condition on in fair prediction/decisions - Changing focus from prediction to action, interventions, policy design, etc - Making models/assumptions transparent --- [Why focus on causality?](https://twitter.com/whitesphd/status/1308094245669081089)  --- ### Actions, decisions, trade-offs > Various models have only predicted the world in various ways. The point, however, is to change it Why trade-offs? Constraints of time and other resources... -- #### Constructs and measures Validity, reliability, and **malleability** (e.g. "math ability" measured in 1088 (pre-calculus) vs 1988 (post-computer)) --- ### Key trade-off of the "information age" <img src="causal_files/figure-html/unnamed-chunk-8-1.png" width="504" /> --- class: inverse ### Parting wisdom (hard won, not by me) Remember pragmatism, Box's rule, and what's **important** and what's **useful**? (to who? for what?) How can we [change the objective function](https://en.wikipedia.org/wiki/Twelve_leverage_points) before it's [too late](https://en.wikipedia.org/wiki/Instrumental_convergence#Paperclip_maximizer)? -- ### Thank you for listening! Reading for a fairly general audience: The long road to fairer algorithms ([Nature, 2020](https://www.nature.com/articles/d41586-020-00274-3)) [joshualoftus.com/talks.html](joshualoftus.com/talks.html) slides [joshualoftus.com](joshualoftus.com) other research, teaching, etc