The crowdsourced map of internet traffic that keeps your favorite websites running

August 18, 2014
August 18, 2014

It isn’t easy keep the internet connected. A large country will bite off a chunk of a smaller one, and switch the network connection; or hackers will program a swarm of robots to overload a website, cutting it off from everyone else; or sharks will just eat undersea fiber optic cables:

So what’s a company that depends on its website to do in such an unpredictable world? Just like mobile apps that guide cars around accidents and traffic, so too are there are ways to make sure internet traffic takes the most reliable route to its destination.

When you type a URL like “www.qz.com” into your web browser, the Domain Name System (DNS) converts it into a unique internet address. Since “the internet” is really just ton of different private networks and computers all connected together, companies that host data for users have a lot of options for how to make it available, and often store information at different servers across the globe, accessible by many different routes:

Tap image to zoom

That’s where a company called Dyn (pronounced “dine”) comes in. It is hired by internet firms like Twitter, Etsy, and Zappos to manage their DNS traffic and prevent interruptions, accidents, and hackers from bringing their sites down. Dyn ensures that, when users try to access a site, data is delivered to them via the most efficient delivery service.

One way it does this uses technology developed by Renesys, a firm recently acquired by Dyn, that works with 400 internet service providers and deploys a global network of real-time probes across 1.5 million data points in order to determine the most effective routes to content, akin to the way Waze’s technology is used by Google to feed routing data into its map product.

But Dyn’s overall service matters a lot today because of a couple key trends driving internet usage:

Distributed everything

Dyn CEO Jeremy Hitchcock groaned when I asked him about “the cloud,” but the increasing reliance on distributed storage and software makes traffic management even more important than it once was. Many of the companies Dyn works with provide enterprise service to users and are literally out of business when their site goes down.

Peering at peering

The relationships between different internet service providers (ISPs), backhaul networks, and content providers are coming under increasing scrutiny thanks to fights between Netflix and ISPs over who should pay to upgrade the infrastructure where they exchange data. While regulators are examining solutions, companies need to avoid congestion points where fights between different networks result in degraded service.

“Splinter net”

Making internet traffic seamless across the entire globe is hard enough already (Dyn has 18 data centers around the world to facilitate this), but in the wake of Edward Snowden’s revelations, many countries are considering protecting their citizens’ data by requiring it to be hosted within their national borders. Internet companies aren’t thrilled about this balkanization because it threatens the idea of the internet as a unified, open network, but it does mean that traffic management will become even more important.

Hackers everywhere

There are a variety of attacks hackers can use to shut down a website by exploiting vulnerabilities in its DNS, the most common of which is probably the distributed denial of service (DDOS) attack. The best solution, according to Dyn, is to have more redundant networks than the hackers can overload, allowing users to continue accessing content.

Top News

Powered by WordPress.com VIP
Follow

Get every new post delivered to your Inbox.

Join 26,263 other followers