Snapchat quietly revealed how it can put AI on your phone

We can finally see a piece of Snap’s vision.
We can finally see a piece of Snap’s vision.
Image: Reuters/ Brendan McDermid
We may earn a commission from links on this page.

Snapchat, the only social media platform left where millennials can escape their parents, has been notoriously secret about how it packed advanced augmented reality features into its mobile app.

In a research paper published June 13 on the open publishing platform Arxiv, the company seems to detail one of its tricks for compressing crucial image recognition AI while still maintaining acceptable performance. This image recognition software, if indeed used by Snap, could be responsible for tasks like recognizing users’ faces and other objects in the app’s World Lenses.

Snap’s method hinges on two techniques: simplifying the way that its convolutional neural networks (a flavor of machine learning common in image recognition) recognize shapes, and proposing a slightly different configuration of the network to offset that simplification.

With these tweaks, Snap claims to fit its algorithm into just 5.2 MB—about the size of a standard song in MP3—with accuracy that just edges out Google’s latest research attempt to scale down its mobile AI. With both networks taking that same 5.2 MB space, Snap scored 65.8% accuracy while Google scored 64.7% on a standard image recognition task, according to the paper.  (For AI nerds, this is top-1 accuracy, or when the network is only given one shot at guessing.)

Snap isn’t the first to attempt to downsize AI for mobile, but publication of the research reveals a few key points:

  • Snapchat is now interested in proving its AI chops, directly comparing itself to Google’s research efforts.
  • Phones are quickly becoming the front line for AI, rather than the algorithms running on servers.
  • Despite its $2 billion cloud contract with Google, Snapchat isn’t relying on Google or Facebook’s AI tech for its research.

We’ve reached out to Snap for more information, and will update if we hear back.

Snapchat has raised its AI profile in recent months by hiring a new director of engineering, Hussein Mehanna, according to a CNBC report. Mehanna had previously worked as a director of engineering in Facebook’s Apple Machine Learning division.

Facebook released code for Caffe2Go, an entire framework for running AI on mobile devices in late 2016, and Google released a mobile version of the hugely-popular TensorFlow last month at its I/O developer conference. Snap’s work was built using Caffe, the open-source library developed by University of California Berkeley.