Snapchat, the only social media platform left where millennials can escape their parents, has been notoriously secret about how it packed advanced augmented reality features into its mobile app.
Snapchat, the only social media platform left where millennials can escape their parents, has been notoriously secret about how it packed advanced augmented reality features into its mobile app.
In a research paper published June 13 on the open publishing platform Arxiv, the company seems to detail one of its tricks for compressing crucial image recognition AI while still maintaining acceptable performance. This image recognition software, if indeed used by Snap $SNAP, could be responsible for tasks like recognizing users’ faces and other objects in the app’s World Lenses.
Join 500,000+ readers who start their day with Quartz.
By subscribing, you agree to our Terms of Service and Privacy Policy.
Snap’s method hinges on two techniques: simplifying the way that its convolutional neural networks (a flavor of machine learning common in image recognition) recognize shapes, and proposing a slightly different configuration of the network to offset that simplification.
With these tweaks, Snap claims to fit its algorithm into just 5.2 MB—about the size of a standard song in MP3—with accuracy that just edges out Google $GOOGL’s latest research attempt to scale down its mobile AI. With both networks taking that same 5.2 MB space, Snap scored 65.8% accuracy while Google scored 64.7% on a standard image recognition task, according to the paper. (For AI nerds, this is top-1 accuracy, or when the network is only given one shot at guessing.)
Snap isn’t the first to attempt to downsize AI for mobile, but publication of the research reveals a few key points:
We’ve reached out to Snap for more information, and will update if we hear back.
Snapchat has raised its AI profile in recent months by hiring a new director of engineering, Hussein Mehanna, according to a CNBC report. Mehanna had previously worked as a director of engineering in Facebook’s Apple $AAPL Machine Learning division.
Facebook released code for Caffe2Go, an entire framework for running AI on mobile devices in late 2016, and Google released a mobile version of the hugely-popular TensorFlow last month at its I/O developer conference. Snap’s work was built using Caffe, the open-source library developed by University of California Berkeley.
