Algorithms have biases too
Posted: Sun Apr 06, 2025 8:54 am
AI ethics isn’t just about copyright. Another problem is that AI is only as reliable as its input. That means it’s just as susceptible to bias as a human is .
Two years ago, Anna M. Górska and Dariusz Jemielniak from the Kozminski University conducted an interesting study on how AI “sees” people working in medicine, law, and universities. Almost 80% of the representation of doctors or university professors were… exclusively men.
This is just one study, but there have been plenty of situations in which AI advertising database made important decisions—such as evaluating job candidates—based on gender or racial biases, including Amazon , which had major problems with this .
Look What is DataRobot?
By the way, it is worth asking how AI actually evaluates data. The truth is that no one knows for sure. The largest models, especially those based on neural networks, are so-called black boxes. That is: we know the input data, we know the result, but we have no idea what processes are happening “inside”. So we do not know, for example, on what specific basis the algorithm evaluates candidates’ CVs or how it diagnoses cancer.
Two years ago, Anna M. Górska and Dariusz Jemielniak from the Kozminski University conducted an interesting study on how AI “sees” people working in medicine, law, and universities. Almost 80% of the representation of doctors or university professors were… exclusively men.
This is just one study, but there have been plenty of situations in which AI advertising database made important decisions—such as evaluating job candidates—based on gender or racial biases, including Amazon , which had major problems with this .
Look What is DataRobot?
By the way, it is worth asking how AI actually evaluates data. The truth is that no one knows for sure. The largest models, especially those based on neural networks, are so-called black boxes. That is: we know the input data, we know the result, but we have no idea what processes are happening “inside”. So we do not know, for example, on what specific basis the algorithm evaluates candidates’ CVs or how it diagnoses cancer.