When training an AI to analyze pictures, you show it tons of examples of different things: cats, dogs, and apparently in Apple’s case, brassieres.
Twitter users on Monday discovered that searching for “brassiere” in Apple’s default Photos app returned images that weren’t entirely safe for work.
Tests by the Quartz newsroom and others on Twitter confirm that “brassiere” is searchable in Photos—however, results were mixed. One Quartz reporter’s search yielded only an image of her in a dress, skipping over photos of friends at the beach. Another wore a bra as a part of a costume, but the AI didn’t surface those pictures. The AI often included photos of dresses with skinny straps, or sports bras. Others confirmed that the folder had—disconcertingly—worked as intended.
For another Quartz reporter, the Photos app catalogued an image of a t-shirt (featured in a past story), which seems to confirm our working theory: It’s looking for shapes that resemble bra straps.
The Photos AI is trained to find objects or people that it knows and group them into folders when you search for them. Try searching, for example, “dogs,” “wedding,” or “cars.” Apple made this possible this by showing an algorithm tons of examples of different items, and then telling the algorithm the word associated with each. Somehow, brassieres made it into the mix.
Don’t worry—your phone’s AI is doing this only on the device, meaning nobody at Apple is seeing your photos, and they’re not stored anywhere besides your phone (and potentially iCloud, depending on your storage settings). Photos does not allow you to turn the AI off.
Apple isn’t alone in using brassiere as a category of images that AI understands. ImageNet, the library of images that created the AI boom we see today, uses it as well, also including the words “bra” and “bandeau.” Coworkers tested the same “brassiere” search on Google Photos and found similar results.
We’ve reached out to Apple and will update with any response.