ETCD – Backup and Restore

So you broke ETCD. It happens! The easiest way back is usually to restore a good backup from before things went wrong. Here’s how I do it.

First, grab the etcd-manager binary. Go to the tags page and download the latest stable version.

When I wrote this (Feb 2, 2024), that was 3.0.20210228.

ETCD keeps two separate stores: main and event. Let’s list the backups for each one.

Start with main:

etcd-manager-ctl-linux-amd64 --backup-store=s3://my-app-kops/my-cluster/backups/etcd/main list-backups

Pick the backup you want and note its name, like 2024-01-29T23:10:15Z-000165.

Now do the same for events:

etcd-manager-ctl-linux-amd64 --backup-store=s3://my-app-kops/my-cluster/backups/etcd/events list-backups

Again, note the name you want, like 2024-01-29T23:06:42Z-000164.

Always restore main first:

etcd-manager-ctl-linux-amd64 --backup-store=s3://my-app-kops/my-cluster/backups/etcd/main restore-backup 2024-01-29T23:10:15Z-000165

Then restore event:

etcd-manager-ctl-linux-amd64 --backup-store=s3://my-app-kops/my-cluster/backups/etcd/events restore-backup 2024-01-29T23:06:42Z-000164

That’s it. ETCD will come back up using the backup you picked.