What really caused the blackout of Facebook, Instagram and WhatsApp?

Facebook World

Hundreds of millions of people were unable to access Facebook, Instagram and WhatsApp for more than six hours this Monday (10/05/2021), underscoring the global dependence on platforms owned by the Silicon Valley giant. But what actually caused the disruption?   

In an apology blog, Santosh Janardhan, Facebook’s vice president of infrastructure, said that “configuration changes to the backbone routers that coordinate network traffic between our data centers caused problems that disrupted this communication.” 

Inadvertent mistake or sabotage?

For their part, several Facebook employees, who declined to be named, had previously told Reuters that they believed the outage was due to an internal error in the way Internet traffic was routed to their systems. 

Failures in internal communication tools and other resources that depend on that same network to function compounded the error, employees said. Security experts have said that both an inadvertent mistake and sabotage by an insider were plausible.

The Border Gateway Protocol system

On the other hand, cyber experts believe that the problem comes down to something called BGP, or Border Gateway Protocol, the system that the Internet uses to choose the fastest route to move information packets.  

Sami Slim of data center company Telehouse compared BGP to “the Internet’s equivalent of air traffic control.” In the same way that air traffic controllers sometimes make changes to flight schedules, “Facebook made an update on these routes,” Slim said.  

Wrong message: Facebook servers no longer exist

But this update contained a crucial bug. It’s still unclear how or why, but Facebook routers essentially sent a message to the internet announcing that the company’s servers no longer existed.  

Experts say Facebook’s technical infrastructure is unusually reliant on its own systems, and that proved disastrous on Monday.  

After Facebook sent the fateful route update, its engineers were left out of the system that would allow them to report that the update was, in fact, a mistake. So they couldn’t fix the problem.  

All eggs in the same basket

“Normally it is good not to put all your eggs in the same basket,” said Pierre Bonis of AFNIC, the association that manages domain names in France. “For security reasons, Facebook has had to concentrate its infrastructure very heavily,” he said.  

“That speeds things up on a day-to-day basis, but since everything is in the same place, when that site has a problem, nothing works.” 

Facebook employees stay out of their offices

The aftermath of the shutdown included some Facebook employees not even being able to enter their buildings because their security cards no longer worked, further slowing response.         

Social media outages are not uncommon: Instagram alone has suffered more than 80 in the past year in the United States, according to website builder ToolTester. However, this week’s Facebook outage has been unusual in its length and magnitude.  

There is also a precedent for BGP meddling at the origin of a social media shutdown. 

In 2008, when a Pakistani Internet service provider tried to block YouTube for domestic users, it inadvertently shut down the global website for several hours.   

Among Facebook, Instagram, WhatsApp and Facebook Messenger, “billions of users have been affected by the total disconnection of services,” according to the tracking service Downdetector. 

Facebook shares drop due to outage 

Facebook, whose shares fell nearly five percent due to the outage, has stressed that “there is no evidence that user data has been compromised as a result of this downtime.” 

However, although it only lasted a few hours, the impact of the disruption was profound. As a result of these incidents, according to The Wall Street Journal, the fortune of Facebook CEO Mark Zuckerberg was reduced this Monday by about $5.9 billion.  

And this was due to the terrible performance of the company’s shares on Wall Street, in which the social network fell by about 4.9%.  

According to Forbes figures, this decrease means that Zuckerberg’s fortune has been at 117,000 million dollars, which relegates him to the sixth place of the richest people in the world.  

Bloomberg, for his part, estimates that the creator of Facebook now accumulates $121 billion, making him the fifth richest on the planet, behind Bill Gates.