Lack of investment in protection can lead to downtime and lost revenue.
Last week, the United States Federal Aviation Association (FAA) was forced to ground all flights in the country. “Our preliminary work has traced the outage to a damaged database file,” The FAA said. Apparently they had a backup of the database but “when a redundant backup kicked in…the data was corrupted and wasn’t considered reliable,” officials said.
Later we learned that Delta Air Lines has their own backup to the FAA system that failed, but the airline didn’t use it because their CEO described “Delta’s secondary system as ‘fairly old technology”. At the time of this writing, we now know that the system went down due to someone unintentionally deleting a critical file.
All of this news was of particular interest to us because, well, we are all about backups! We’ve been in the midst of getting clients and agency partners’ clients renewed on service plans for 2023. We’ve been reviewing and sharing reports with clients about their websites’ traffic, uptime, plugin fixes, and yes the backups that have been performed and stored.
An example of the backup section of the reports we provide to clients
When redundancy is a good thing.
An important aspect of a good service plan is redundancy, which is why we don’t just rely on one or even two backups, but instead ensure we have many copies stored up. There’s always a chance that one backup could fail, but highly unlikely that multiple copies would, though it sounds like that may be what happened with the FAA.
“Our focus now is making sure we understand not just the system itself but the redundancies. As you can imagine a system this important for safety has backups but in this case that was not sufficient to prevent a disruption.” – Secretary of Transporation Pete Buttigieg told Walter Isaacson of Amanpour and Company.
A couple of years ago we had a client website on a service and security plan experience a disruption as well. Unfortunately, this client’s site was one of a number of sites that were targeted by hackers initiating a DDoS (distributed denial of service) attack.
Screenshot of the now suspended account that contacted our client.
When we were alerted that their site had been hacked we took immediate steps to jump in and work to restore traffic to the site, while ensuring we had viable backups to migrate the site to a different service. We were simultaneously providing information to intelligence officers who called us as they investigated who was behind the hack. We were able to get our client’s website back up within a matter of hours.
Plans like the ones we offer our clients and those of our agency partners are like insurance policies. They don’t ensure that covered websites will never experience a problem, but they do provide greater assurance that they are better protected and supported if the unexpected occurs.
Going the extra mile.
A good example of our being able to help a client by providing a backup, happened a few weeks ago. An agency partner reached out to us and inquired if we still retained a backup of a site that we launched a few years ago. This particular client had hired a separate company to redesign the site, but they apparently did such a poor job that the client wanted to revert to the original site that we created. The rub was, though: the new company never took a backup of the site before they launched! We dug into our archives and were able to restore the site to its former glory from a backup snapshot that we took when we launched the site and got them up and running (again).
Contingency plans for keeping your custom website online.
When you get a custom site built, it’s somewhat similar to the effort that goes into building a custom house: they can be big purchases and take a lot of time and energy into getting just right. Once it’s built, you want to do what you can to protect that investment, and that usually comes in the form of some kind of insurance policy. We also encourage our clients to think of a service plan as like having a specialist on-call who has a vested interest in making sure their site stays performant and more importantly, has contingency plans in place…so their website stays up safely or at least isn’t as difficult for stakeholders as the FAA database disruption was last week.
