The Role of Bias in Criminal Proceedings: Implications for the Design of Fair and Accurate AI-Based Tools for Recidivism Prediction

Conscious and unconscious biases impact workplaces, social interactions, public policy, and criminal justice. These biases have different sources. They can stem from historical and cultural stereotypes, skewed or insufficiently informed perceptions, or strongly held prejudices. Biases are part of interpersonal relations; however, they can lead to discrimination and unfair treatment. This is particularly problematic in the case of the criminal justice system, where essential rights are at stake, including the right to liberty, the right to a fair trial, and the presumption of innocence. Understanding how bias can impact the outcomes of criminal justice proceedings is key to strengthening the transparency of due process and ensuring fairness in justice delivery. Such an understanding is also critical as regards the development and deployment of AI-enabled systems in the field of justice, and the tools for predictive analytics and risk assessment are not an exception. Indeed, insights into how human biases can play a part in judgment and decision-making have important implications for the design of AI-based systems for recidivism prediction.

Data shows that characteristics such as gender, race, ethnicity, age, religion, and disability can and often do turn out to be crucial factors that influence the outcomes of criminal justice. Disparities in sentencing, incarceration, and recidivism as regards gender and other sensitive characteristics are observed across jurisdictions around the world.

For example, men still comprise the majority of convicted individuals across all categories of crime. In Europe, according to the 2023 Space I report, the average prison population across the Member States of the Council of Europe consisted of 93.4% males and 5.2% females, highlighting a significant gender disparity among incarcerated individuals. Racial biases also play a part. The stop-and-search data from the European Union Agency for Fundamental Rights (FRA) 2023 report on the “Experiences of People of African Descent” showed that more than half of respondents who were stopped by police considered these searches to be racially motivated.

Ethnicity is another divisive factor. The questionable nature of ethnic profiling is a case in point, whereby law enforcement practices are based mainly on assumptions about an individual’s ethnicity or national origin rather than their behavior or objective evidence. Concerns regarding disparities in pre-trial detention have been raised by the United Nations Committee on the Elimination of Racial Discrimination (CERD). The Committee warns that racial or ethnic groups, particularly non-citizens – including immigrants, refugees, asylum seekers, and stateless persons – as well as Roma, indigenous peoples, and other marginalized communities, are disproportionately subjected to exclusion, marginalization, and lack of integration into society; and that an excessively high number of non-nationals are held in pre-trial detention.

It is evident that protected characteristics can have a significant impact on individuals’ entry into the criminal justice system, their experiences within it (including sentence length, severity, and the prejudice and bias encountered in legislative decision-making processes), and their likelihood of reoffending and returning to recidivism. At the same time, newly developed AI tools designed to support judicial decision-making could amplify human biases by replicating gendered prejudices, contributing to further discrimination, and exacerbating existing vulnerabilities. Given these realities, decision-makers must remain aware of the potential for bias in their judgments, as well as in the AI-driven tools they rely on for support. Achieving a legal and technological system free from bias requires a rigorous critical examination of both innate and socially constructed prejudices to uphold fairness and impartiality while acknowledging their potential to contribute to a more equitable judicial process.

Article provided by CSD