ProtonBlog(new window)

Downtime for August 11, 2014

Share this page

UPDATE August 12, 2014We have managed to recover most of the missing emails, and will work to restore them over the next couple days.

On Monday, August 11, 2014 we took Proton Mail offline for upgrades, which we estimated would take just a few minutes. Part of the upgrades for Monday were to optimize our database table structure, which would speed up the performance of the site for all users.

Unfortunately, a wrong command typed by one of our developers resulted in corrupting the messages table which resulted in losing messages for most of Sunday and the earlier part of Monday (about 20 hours). Needless to say this was a terrible mistake and we’re very sorry that this happened.

This seemingly benign update took a terrible turn because our standard operating procedure (SOP) wasn’t followed. Because a backup was not made prior to beginning database operations (as mandated by our SOP), we lost a day’s worth of user data. The Proton Mail team is growing right now, and unfortunately, we did not properly communicate our standard operating procedures to some of the new team members.

In the next few weeks we will be rolling out a more robust server architecture and deployment process that will not only speed up the site, but also increase reliability, reduce the need for downtime and prevent these sort of errors from happening. We will also be increasing oversight of staff and centralizing operations to improve our internal communications and ensure our established procedures are always followed.

These procedures have helped Proton Mail avoid data loss and extended downtime up until today, and could have prevented the incident today.

The road to bringing privacy back to email is a challenging one, but its all been possible thanks to your support. Over the coming weeks, we will be building in multiple safeguards to prevent data loss in the future. Thanks for bearing with us through our beta and understanding that human errors like this do happen.

Protect your privacy with Proton
Create a free account

Share this page

Jason Stockman(new window)

Jason is a co-founder of Proton Mail. He works on building Proton Mail's user interface and websites. Jason has over 10 years experience building websites and applications.

Related articles

Can you password-protect a folder in Google Drive?
Protecting a folder with a password is a simple yet effective way of securing files. You may wonder whether you can password-protect a folder in Google Drive. We explain what access controls Google Drive offers and what you can do to improve your sec
Proton Pass now supports passkeys on all devices and plans
We’re excited to announce that Proton Pass supports passkeys for everyone, allowing you to manage and use passkeys across all devices seamlessly. Passkeys are an easy and secure alternative to traditional passwords that can help prevent phishing atta
what is a passkey?
Passkeys are a new way to secure your online accounts using cryptographic keys instead of passwords. They offer a high level of convenience and security, and are a real game-changer in the way we access and secure sites. What is a passkey, though, an
Apple’s marketing team has built a powerful association between the iPhone and privacy. The company’s ad campaigns claim that “what happens on your iPhone, stays on your iPhone.” And, “Privacy. That’s iPhone.” But Apple’s lawyers are telling a diffe
A cyberattack on national public employment service France Travail has exposed the personal data of as many as 43 million people.  The latest breach is the second major cybersecurity attack to happen in France in the past month, raising concerns abo
If I share a folder in Google Drive, can anybody see my other folders
Google Drive makes it easy to share files and folders, but you may have wondered at some point whether the people you’ve shared a folder with can see your other folders. We answer this question below and also share some tips for truly secure link sha
In 2014, Proton Mail was introduced as a web app, revolutionizing how we think about email privacy. Today, we’re excited to broaden the horizons of secure communication by launching the Proton Mail desktop app. Anyone can now use the new Proton Mail