A few years back I was working with a client that was optimizing their marketing and product offerings by clustering their clients according to several attributes, including ethnicity. I was very uncomfortable with that. Ultimately I did not have to deal with that dilemma, as I left that project for other reasons. But I’m inclined to say that using ethnicity as a predictor in such situations is unethical, and I would have recommended against it, even at the cost of having a model that performed worse than the one that included ethnicity as an attribute.

​

Do any of you have real world examples of cases where you went with a less accurate/worse performing ML model for ethical reasons, or where regulations prevented you from using certain types of models even if those models might perform better?