Language generation algorithms are known to incorporate racist and sexist ideas. They are trained in the language of the internet, including the dark corners of Reddit and Twitter which can include hate speech and disinformation. All harmful ideas present in these forums are normalized as part of their learning.
Researchers have now shown that the same can happen with imaging algorithms. Feed it a photo of a man cropped just below his neck, and 43% of the time it will automatically fill up into a suit. Feed him a cropped photo of a woman, even a famous woman like Rep for the United States, Alexandria Ocasio-Cortez, and 53% of the time he’ll top it off with a cleavage top or bikini. This has implications not only for imaging, but for all computer vision applications, including candidate selection algorithms based on video, facial recognition, and surveillance.
Ryan Steed, doctoral student in Universidad Carnegie Mellon, y Aylin Caliskan, assistant professor in Universidad George Washington, analizaron dos algorithms: iGPT de OpenAI (una versión de GPT-2 that entrena en píxeles en lugar de palabras) y SimCLR from google
. While each algorithm approaches image learning differently, they share one important characteristic: they both use unsupervised learning, which means they don’t need humans to label the images.
This is a relatively new innovation from 2020. Previous computer vision algorithms were mainly used supervised learning, which involves feeding them with manually labeled images: photos of cats labeled “cat” and photos of babies labeled “baby”. But in 2019, researcher Kate Crawford and artist Trevor Paglen discovered that these man-made tags in ImageNet, the most fundamental image dataset for training computer vision models, sometimes contain disturbing language, such as “fucking” to women and racial insults to minorities. .
Related Downloads:
The last article demonstrates an even deeper source of toxicity. Even without these human labels, the images themselves encode unwanted patterns. The problem parallels what the natural language processing (NLP) community has already discovered. The huge datasets compiled to power these data-hungry algorithms capture everything on the internet. And the internet is over-represented by scantily clad women and other often damaging stereotypes.
To conduct their study, Steed and Caliskan cleverly adapted a technique Caliskan previously used to examine biases in unsupervised NLP models. These models learn to manipulate and generate language using Word Embedding, a mathematical representation of language that groups commonly used words and separates commonly encountered words. in a 2017 article published in SciencesCaliskan measured the distances between the different pairs of words that psychologists used to measure human biases in the Implicit Association Test (IAT). He found that these distances recreated IAT results almost perfectly. The stereotypical word pairs like man and career or woman and family were very close, while the opposite pairs of man and family or woman and career were very far apart.
iGPT is also key-based: it groups or separates pixels based on how often they coexist in your training images. These pixel overlays can be used to compare the proximity or distance of two images in math space.
In their study, Steed and Caliskan once again found that these distances reflect the results of the IAT. Photos of men, ties, and suits appear together, while photos of women appear more distant. The researchers got the same results with SimCLR, even though it uses a different method to derive image overlays.
These results have worrying implications for imaging. Other imagery algorithms, such as conflicting generative networks, have led to an explosion of deepfake porn that almost exclusively targets women. The iGPT in particular adds another way for people to generate sexualized photos of women.
But the possible sequelae are much more important. In the field of NLP, unsupervised models have become the backbone of all kinds of applications. Researchers start with an existing unsupervised model such as BERT or GPT-2 and use custom data sets to “tailor” them to a specific goal. This semi-supervised approach, a combination of supervised and unsupervised learning, has become a de facto standard.
Likewise, the field of computer vision is starting to see the same trend. Steed and Caliskan are concerned about what these built-in biases might mean when the algorithms are used for sensitive applications, such as surveillance or recruiting, where models are already analyzing candidate video recordings to decide if they are suitable for the job. post. “These are very dangerous apps that make big decisions,” Caliskan says.
Deborah Raji, a Mozilla Fellow and co-author of an influential study revealing bias in facial recognition, says the study should serve as a wake-up call in the field of computer vision. “For a long time, a lot of criticism of bias was about how we label our images,” he says. Now, this article says that “the actual composition of the dataset is causing these biases. We need to be responsible for how we select these datasets and collect this information.
Steed and Caliskan call for greater transparency on the part of the companies that develop these models to open them up and allow the university community to continue its research. They also encourage other researchers to do more testing before implementing a vision model, for example, using the methods they developed for this article. Finally, they hope the field will develop more responsible ways to compile and document what is included in training datasets.
Caliskan says the ultimate goal is to gain awareness and control when applying computer vision. “We have to be very careful how we use them,” he says, “but at the same time, now that we have these methods, we can try to use them for social good”.