In my recent recovery test, I found that not having a good checklist to follow really hurt my ability to recover my backup quickly. A lot of the things I got hung up on I had though, "Oh, I'll remember that when the time comes." It turned out that, in fact, no, I didn't remember, or I didn't remember everything. In the heat of the moment, I didn't always remember all the steps involved in, say, setting up my printer or which applications I had installed. That's why I think making a couple of lists and cheat sheets now can go a long way in helping you when the time comes to get back up and running.

So what lists should you make?

1. What equipment needs special set up on the new computer?

   For me, this is things like my printer and wireless internet. By having a list written out with all the steps involved and the drivers my operating system need in one place, I can quickly get these set up without having to remember every step needed along the way. If you have any special equipment that will need configured, have a step by step process of how to get it set up on a new computer.

2. What applications do I need to install?

   As I said in the application backup post, make a list of all your installed applications and locations (either web addresses or location in your backup) of where to download them. You'll also need a list of license keys written up in a text file in your remote backup. If you do have a fire, you won't be able to go back to the CD to find the key.

3. What special things do I need to do to get back to my current state?

   I have a network drive share on my laptop that I use to share video with a couple of other computers in the house. I had forgotten how involved this was to set up and so I wrote all the steps down this time and put it on Dropbox. Any complicated set up like this should be written down and stored in your remote backup so that you won't have to spend the time to figure it out again.

If this was a real recovery, and not just a planned test, I would have gotten myself in a slight panic over all the things I couldn't remember how to do. The number one reason this test took six hours was because of all the little things I just forgot about.

Now, all of these lists have gone straight into my Dropbox under a recovery folder, including any special drivers I need for my printer and scanner. Next time, I should be able to run through these lists a lot quicker to get my computer back up and running.

On Dec. 13th, I tested my backup system the best way I know how; I put an empty hard drive in my computer and got myself back to my original configuration with only my remote and local backups.

It worked, but it was a little bumpy at times and showed that I still have a couple holes in my system, as far as speed of recovery is concerned. I also found some things that I need on a remote backup that I don't currently have. Here are some things I learned:

1. Make a list

   There were a couple of applications I knew I didn't have to backup since I could just download them from the internet once I had my computer back up. The problem was, once I had my computer back up, I couldn't remember what those applications were! If I had made a list in a text file of what those applications were and where to get them, it would have made things a lot easier.

2. When you install new applications, make sure their settings are included in the remote backup

   There were a couple of application settings that I didn't have in the remote backup. I did have them on the Drobo, but I would have been in trouble if I only had the remote backups. I'll be setting up a periodic remote backup setting review so that these don't fall through the cracks again.

3. Properly backup SQL databases

   This doesn't apply for most, but it did for me. I was backing up my PostgreSQL database via the file system, but I should really be exporting the data to a file first. I was able to recover the data, but it wasn't quick or pretty. I'll need to figure out a way to dump that data somewhere on my file system so that it can get backed up remotely.

4. It's good to test

   There are some things I found out that I never would have known if I hadn't run this test. I'm not quite ready to say that everyone should do this, but I'm close. I think that the Merlin Mann quote I put up earlier is exactly correct: Backup is a little like Taekwondo; it's hard to know whether you're really good at until you're in the midst of getting your ass kicked.

   I'm now planning on running a recovery once every six months or so. My biggest realization: I don't have a solid process around my recovery. I'm backing everything up better than I thought, but it still wasn't easy to get up and running and took way too much time. I need certain lists and procedures written up before hand before I'd feel comfortable with the whole thing. And I need to shorten the time from the current record of six and a half hours.

All in all, it went well. I still have my recovered drive in the computer and plan on keeping it there. I don't see a big difference between this new hard drive and the old one except this one is faster (attributable to a fresh install of the operating system). As I figure out ways to streamline everything, I'll be sure to let you all know.

I've also been writing up quick little procedure/checklists and putting them on a folder in Dropbox. It's little things that I didn't know I didn't know until I ran a recovery. Now they're captured in text in a place I know I can get to when my computer decides to give up the ghost. Future posts will flesh out what you might need to write down to make your own recovery successful.

Bottom line? I did good, but I've got a long way to go to get great.

3:38 - And I'm back up and running! Everything's looking good, but I learned a lot about my set up and what's needed to make it easier to recover. I'll be writing about that, though, in a future post.

3:23 - That install took a while. I think I need to prune my list of applications. Things are looking good except for some permissions problems on the files.

11:53 - Recovering from the Drobo nicely. Looks like I've got a list of things to add to the JungleDisk backup.

11:17 - I don't think I'm storing enough of my settings on JungleDisk. Nothing's working quite the way I had it working before. I know the Drobo has it all, but that's really not good enough in a real recovery situation. Finishing up some final updates from Ubuntu.

11:06 - Pulling files fine from JungleDisk. It looks like everything worked as I hoped! Going to restart and get the rest from the Drobo since I don't really want a massive Amazon S3 bill.

10:52 - Getting near the two hour mark, most of it spent reading Anathem. Now on to recovering user settings and restarting the computer.

10:25 - Should have looked up before hand how to install a large number of packages in Ubuntu. Updated my repositories from JungleDisk and now install my old system. Says it'll take another 30 minutes or so.

9:57 - Just went on a wild goose chase with TrueCrypt when trying to install it. You need dmsetup installed first on Linux and I thought I had to install from source. No, just apt-get update first. Now to install JungleDisk. Note: I haven't touched my local backups yet.

9:35 - Actually, no. I need to get JungleDisk up first to restore my repository list, so first I need to get my encrypted passwords off of Dropbox and then install TrueCrypt.

9:32 - Base OS install is done. Now to install all my software.

9:16 - Partition tables! Good question. That's one thing I didn't write down before starting. I guess I'll just take the defaults.

9:11 - It has no idea what to make of the drive. Yippie! I've lost all my data! Now to install Ubuntu from the CD. Should take about 30 to 45 minutes.

9:08 - It's in. Let's see if it actually works.

9:00 - Computer shutting down. Getting ready to swap the hard drive.

8:37 - My last JungleDisk backup ran at 9 last night and I did a Drobo backup around 5 yesterday. That should cover everything I need for this morning.

8:32 - Getting ready to start. Got my tea and my Ubuntu CD, what more do I need?

The backup recovery test will start Dec. 13th at 9:00AM. Will it be exciting enough for a live blog? I have no idea.

I'm going to attempt to recover my computer on Dec. 13th from a full (simulated) hard drive meltdown. In preparation, I've developed a plan as to how I'll recover everything from my local and remote backups. I could do everything from the local backup, but I want to make sure the remote backup is working too. So here's my current recovery strategy.

Local Backup

As I've stated many times before, I use a Drobo for my local backups. I backup everything in my local user directory, where all my documents and user settings files are stored, and some applications that I've had to install myself. I plan on not using this right away when I recover, to prove to myself that I can get back on my feet with just my remote backups.

Secure Key Backup

I have a local memory card in a fire-proof lock box that's encrypted via TrueCrypt and has most of my passwords on it, including what I need to access my remote backup, JungleDisk.

First Remote Backup

The first thing I'll need to recover are files off of Dropbox. I have my TrueCrypt volume with all of my passwords on Dropbox as a redundant backup of my local encrypted memory card and I'll need those to get at my JungleDisk passwords. This should be simple using Dropbox's web interface, which will give me access to everything on my Dropbox without having to install the software right away.

Second Remote Backup

My main remote backup is JungleDisk. Once I get my passwords from the TrueCrypt volume, I'll be able to install and start JungleDisk to get my files back in order. After that, it should be smooth sailing (I hope).

Why?

The main reason I have it all set up this way is redundancy. I want local backups that are easy to access and with less hoops to jump through to get them working, but I also need remote backups because local backups won't cover me in case of a fire or other natural disaster (like a spilt can of pop!)

The key is, I need to be able to access everything from remote backups because I might not have anything local to recover with. I have my passwords locally in a lock box, but also strongly encrypted on Dropbox so that I can get them if the lock box is gone. (They're also on the Drobo and JungleDisk, because I really don't want to lose my passwords.)

Any plan you create should allow you to recover a new computer back to your current computer's working state even if you don't have access to your local work area. In fact, I would say that a remote backup is way more important than a local backup because with a remote backup, you should be able to recover from anywhere. A local backup is really just for convenience and speed, not to be depended upon for the really important stuff.

So that's my plan. What's yours?

I just had a conversion with my wife about my recovery experiment on the 13th.

She said, "If you're going to do it on purpose, you better do a full backup first."

Should I? If I'm really testing my backup solution, it better work whether I do it on purpose or not. Hard drives fail and I won't be able to do a full backup before mine fails, so why should my test be any different.

I feel fairly confident that I can recover nearly everything, without a full backup first. I may lose emails from my computer, but I keep everything on the server for seven days before deleting it, so even that should be fine.

I feel okay about trying this, and I'll still have my old hard drive if I need to go back. I feel that it's really important to try a real world scenario and not fudge the results by running a backup right before I try this.

"Well, you're doing it with your hard drive."

Yes, dear. I won't touch your computer.