Home » News » The simple typo that brought down Amazon Cloud

The simple typo that brought down Amazon Cloud

Tuesday, 7 March, 2017

Human error was at fault when a web storage system powered by Amazon Cloud wobbled and fell over at the end of February, the internet giant has admitted.

Around 150,000 websites and top apps use Amazon Web Services, including filesharing on business messaging service Slack, the reader-led question and answer service Quora, and the GIF image search engine Giphy.

Considered (rightly or wrongly) to be more secure than traditional hosting services, it now appears the cloud is just as fragile and open to error as any other operation.

Amazon Cloud ‘unkillable?’

But it wasn’t a hack attack or malicious code that brought down Amazon’s £10bn-a-year service.

No, it was an “input” error by an Amazon employee.

In a series of tweets when the huge service started struggling with outages on 28 February 2017, Amazon detailed how S3 – their ‘Simple Storage Service’ – “was experiencing high error rates”.

The simple typo that brought down Amazon Cloud

The simple typo that brought down Amazon Cloud 1

The simple typo that brought down Amazon Cloud 2

In a statement published on their own website Amazon later admitted that an “authorized S3 team member using an established playbook executed a command which was intended to remove a small number of servers for one of the S3 subsystems that is used by the S3 billing process.”

“Unfortunately, one of the inputs to the command was entered incorrectly and a larger set of servers was removed than intended.”

With healthcare apps and interconnected home devices relying on the Amazon cloud network, outages like this could cause havoc if mistakes go unchecked.

Advice: Back up, back up and back up again

Amazon apologised for the error and said it would monitor its systems to make sure that commands entered incorrectly could not bring down web services.

The statement continued: “While we are proud of our long track record of availability with Amazon S3, we know how critical this service is to our customers, their applications and end users, and their businesses.

“We will do everything we can to learn from this event and use it to improve our availability even further.”

It’s yet another reminder that no system is infallible.

If you’ve got information or personal files that you just can’t bear to lose, you should be backing them up and making copies to store in a number of locations; whether that’s online, in the likes of cloud storage service Google Drive, or in a more traditional offline external hard drive.

Tom Rodgers author picture

By:

Tom is a tech journalist and former Editor at BroadbandDeals.co.uk.

News What's the story?

Keep up with the latest developments in UK broadband.

The biggest malware threats of 2020…so far

It’s been a year few of us will forget in a hurry, and we're only halfway through.

The biggest malware threats of 2020…so farThe biggest malware threats of 2020…so far Read more

Instagram could become the main news source for young people.

Reuters finds changes in the way younger users consume the news.

Read more

BT launches second line service

BT launches second broadband home line service for the new crop of home workers.

Read more

Best broadband areas for online gaming in the UK.

Read more

Help Learn with us

Make the most of the internet with our broadband library.

How to check if your broadband is down

It might seem obvious that an outage has occurred, but there are easy ways to check if your broadband is down, or whether the problem is more localised

How to check if your broadband is downHow to check if your broadband is down Read more

A guide to Big Tech alternatives.

It seems like we’re reliant on a small group of companies, are there alternatives?

Read more

Quick tips for boosting home broadband speed

Boosting speed can transform activities like streaming, gaming and accessing cloud storage

Read more

What’s the difference between hardware, firmware and software?

Read more