A while back, one of the image generation AIs (midjourney?) caught flack because the majority of the images it generated only contained white people. Like…over 90% of all images. And worse, if you asked for a “pretty girl” it generated uniformly white girls, but if you asked for an “ugly girl” you got a more racially-diverse sample. Wince.
But then there reaction was to just literally tack “…but diverse!” on the end of prompts or something. They literally just inserted stuff into the text of the prompt. This solved the immediate problem, and the resulting images were definitely more diverse…but it led straight to the sort of problems that Google is running into now.
“Data, stop. Data. Stop. Data, SHUT UP!”