Using Amazon S3 to Store Database Backups

Using S3 to store your MySQL or PostgreSQL Backups

"I'd love to restore a Database Backup" said no one ever. When you're forced to do so, that means that the production system you're maintaining is going belly up and your company (or the company you're working for) is probably losing an outrageous amount of money for every second of downtime. It can happen for a variety of reasons:

  • Someone running a DELETE query with a WHERE clause larger than intended, or worse with no WHERE clause at all.
  • TRUNCATE and DROP queries being executed on the wrong tables or columns.
  • A faulty Hard Disk that suddenly stops working.
  • A hacker decided to ruin your company and deleted everything or even worse, you got a ransomware that is now asking you to pay some Bitcoins to get your data back (spoiler: you will not get it back).

So, if you worked in IT for more than a day, it should be clear that storing your Database Backups appropriately is of extreme importance. We live in the hope that we won't need them, but when urgency calls, you'll be glad of having gone the extra mile to protect your data and to have that database backup ready to be restored.

In this situations you, the DevOps engineer, or SysAdmin, or Cloud expert or even "the only developer in the company", can either be the hero that saved the day or the one who didn't care about taking backups.


If you are using Amazon RDS you are pretty much sorted out as RDS will create and safely store backups for you every night automatically.

But if you are not in the Cloud, or running a Database server on an EC2 instance to retain more control, or on a third-party VPS to save some cents, you may have to manage this yourself.

In this article, we will use Amazon S3 as a safe heaven for one of our most precious assets.

The Definitive Guide to Amazon S3

The definitive guide to Amazon S3

Amazon S3 or Simple Storage Service is probably the most widely used and well-known service provided by AWS. The console interface is very familiar as it looks just like your average file explorer. However S3 is anything but a simple way to store files, as it provides an outstanding number of functionalities, many of which came free of use, to help you comply with the most strict regulations and to implement complex and scalable applications very easily.

Since I'm now studying for be an AWS Certified Solution Architect, I will use this blog to collect notes and examples from books and websites around the web, trying to give you the most complete and dense informations about the AWS services that I came by.

Studying for the AWS Certified Solutions Architect - Associate Exam

I will study to get AWS certified

As a developer, I appreciated using AWS many times. Needless to say, Cloud Computing gives you superpowers as a developer or system administrator.

The ability to scale up and down following traffic patterns, the resilency offered by a properly designed Cloud infrastructure and the power of hundreds of tools like limitless storage or Machine Learning for mere mortals, have made Cloud technologies like Amazon Web Services or the Google Cloud Platform something that cannot be ignored anymore. At times it may be costly but the ease of use is incomparable to an on-premise instance.


Linux Server Hardening

Most of the time I enjoy System Administration tasks, but there is a number of steps that I take on every new instance that I create that are boring repetitive and easy to get wrong. It usually takes me a few minutes to recollect all commands, then a few more minutes to make sure that I'm doing it right. This guide is for the future me and anyone who may come here.

These are the very steps that I used on the server hosting this blog.


How a new project starts

Day 3 into my IndieWeb journey.

One of the principles of IndieWeb is selfdogfood (a name that according to me and to others can be improved). The principle declares that you should be

using your own creations on your own personal site that you depend on, as an aspect of your primary online identity.

Having been a web developer since I can remember this looks nice to me. I've never feel comfortable at using Wordpress, Joomla or any other CMS. I've always felt constrained by their developer choices and at the end of the day, I didn't feel like I knew what was happening under the hood.

This has not been the case with Rails (which is not a CMS of course), that I feel comfortable enough to say that I have a wide and deep knowledge of it.

So when it comes to starting a new project rails new has been my tool of choice for the last 5 years now.

Rails comes packed with lots of goodies like ActionCable or the newly added Webpack integration. What I really appreciate about it is the strong set of rules and conventions that guide you through your development journey.

Before even starting out I had to decide which features I want to implement first, which, in the IndieWeb vocabulary, are my itches. Thankfully the IndieMark page came to help.

So the first step is registering what will be your domain for life. It took me a few hours to choose, as I wanted to avoid the surname, but I settled on the plain and simple I used NameCheap to buy the domain.

Time to setup the project. After checking that I have the latest versions of Ruby and Rails installed on my MacBook, I gave birth to it with:

rails new IndieRails -T

The -T flag is to disable the default testing framework used by Rails, as I am using RSpec. After a few customisations to the Gemfile (Postgres driver, Slim, RSpec, DatabaseCleaner, Rubocop) and having converted the default .erb files to slim, I can finally deploy this thing. But where?


Trying this IndieWeb thing

Last Sunday I was reading Hacker News. Zapping through the links I ended up on this Ask post where, as usual on HN, a thread about Facebook, Google and other web giants (or silos) are literally taking control of our lives, collecting way more data about us than our governments and selling our personal informations to advertisers.

For the first time, thanks to Aaron Parecki, I found about the IndieWeb. It took me a while to understand what it is (the documentation not the best), and surely I just scratched the surface. What I found out almost shocked me.

The fundamental idea is very simple yet incredibly powerful: instead of keeping on publishing your updates to social networks, own your own website and publish there; then, federate with other websites to create the same network structure that you find on the Silos.

The benefits of such choice are multiple:

  • You do not link the destiny of your productions to the destiny of the Silo. Should Facebook bankrupt tomorrow, we would all lose hundreds of posts, thousands of photos, tens of thousands of messages and many other social breadcrumbs that we only published there.
  • By publishing on your own domain, you retain actual ownership of your contents. You are not forced to give to Facebook, to Instagram, to Google or anyone else, greatly reducing their power to analize and target you. Besides, you are the ultimate authority for what can and cannot be on your website. No form of censorship or walled garden can be applied to your own domain.
  • This is your small, yet unique, place on the Internet. You are free to express your creativity here as you wish. Be it in the content or in the design, everything here speaks about you.
  • By owning a piece of the Internet, you help shape the Internet. That's right. You are an internet citizen, with passport and batteries included. You're not just using it anymore, you are creating it.

Without mentioning any other benefit, there is one, extremely important reason for adopting this format: this is how the web was meant to be. When in 1992 Tim Berners-Lee invented the Web, he wasn't thinking at this large corporations owning huge pieces of the traffic. The idea was to allow people to write, share and talk about content, in the most free and diverse way you can think of. To be honest, I am so surprised that this took this long to exist and spread, even a little.

This is where the web of tomorrow should be aiming at. A Web, after all, is a Web only when there are links to be clicked, edges to be explored and spiders (or humans) navigating it.

In the next few weeks I'll try to write more about the IndieWeb and my own implementation of its principles. This website runs on a scratch new Ruby on Rails 5.1.1 project, that I am working on right now.

This is my place on the Web. Be welcome and feel at home.