On Dec. 13th, I tested my backup system the best way I know how; I put an empty hard drive in my computer and got myself back to my original configuration with only my remote and local backups.

It worked, but it was a little bumpy at times and showed that I still have a couple holes in my system, as far as speed of recovery is concerned. I also found some things that I need on a remote backup that I don't currently have. Here are some things I learned:

1. Make a list

   There were a couple of applications I knew I didn't have to backup since I could just download them from the internet once I had my computer back up. The problem was, once I had my computer back up, I couldn't remember what those applications were! If I had made a list in a text file of what those applications were and where to get them, it would have made things a lot easier.

2. When you install new applications, make sure their settings are included in the remote backup

   There were a couple of application settings that I didn't have in the remote backup. I did have them on the Drobo, but I would have been in trouble if I only had the remote backups. I'll be setting up a periodic remote backup setting review so that these don't fall through the cracks again.

3. Properly backup SQL databases

   This doesn't apply for most, but it did for me. I was backing up my PostgreSQL database via the file system, but I should really be exporting the data to a file first. I was able to recover the data, but it wasn't quick or pretty. I'll need to figure out a way to dump that data somewhere on my file system so that it can get backed up remotely.

4. It's good to test

   There are some things I found out that I never would have known if I hadn't run this test. I'm not quite ready to say that everyone should do this, but I'm close. I think that the Merlin Mann quote I put up earlier is exactly correct: Backup is a little like Taekwondo; it's hard to know whether you're really good at until you're in the midst of getting your ass kicked.

   I'm now planning on running a recovery once every six months or so. My biggest realization: I don't have a solid process around my recovery. I'm backing everything up better than I thought, but it still wasn't easy to get up and running and took way too much time. I need certain lists and procedures written up before hand before I'd feel comfortable with the whole thing. And I need to shorten the time from the current record of six and a half hours.

All in all, it went well. I still have my recovered drive in the computer and plan on keeping it there. I don't see a big difference between this new hard drive and the old one except this one is faster (attributable to a fresh install of the operating system). As I figure out ways to streamline everything, I'll be sure to let you all know.

I've also been writing up quick little procedure/checklists and putting them on a folder in Dropbox. It's little things that I didn't know I didn't know until I ran a recovery. Now they're captured in text in a place I know I can get to when my computer decides to give up the ghost. Future posts will flesh out what you might need to write down to make your own recovery successful.

Bottom line? I did good, but I've got a long way to go to get great.

I've gone back and forth on whether or not to backup applications. Backing up user preferences for applications is a must, but the value of backing up the actual application files is more up in air.

One of the things that makes this hard to mandate a backup rule about applications is the fact that applications act differently on each platform.

Mac

The Mac might be the one platform where I could see backing up the applications. Most Mac applications are self contained and can be recovered by simply dragging and dropping them back into the Applications directory. The only issue to watch for is an application's size, which could get pricey for remote backups. If that's not a concern I would say, for the Mac, backup your applications.

Windows

Windows applications are different since they install, not just the applications, but a lot of other auxiliary files and registry entries. Running applications without all these other files can have unexpected results. It makes more sense to backup the installer programs and not the installed application.

Remember to also backup the Application Data folder in your user directory. That's where all your preferences are stored for most of your applications.

Linux

Although Linux doesn't have a registry, applications do install a lot of data into various directories in the system. There's /usr, /usr/local, /var and so many others that it's hard to tell if you've gotten everything or if the permissions are right once you've recovered your data.

Luckily with most Linux distributions, all the applications you have installed are in the central repositories and can just be reinstalled with a simple command line call. There's no need to backup most of them when they can all be gotten from a central online location.

I will backup things that I've installed myself, usually under my home directory or in /opt. It's also important to backup application preferences in your home folder. But, you don't need to backup your applications under Linux.

Conclusion

In general, for Mac, go ahead and back them up. For Windows, don't bother, but backup the application installers if you can. For Linux, backup a list of the applications you have installed, then just run your package manager to get them all back out again.

Have any other ideas on how to backup installed applications?

3:38 - And I'm back up and running! Everything's looking good, but I learned a lot about my set up and what's needed to make it easier to recover. I'll be writing about that, though, in a future post.

3:23 - That install took a while. I think I need to prune my list of applications. Things are looking good except for some permissions problems on the files.

11:53 - Recovering from the Drobo nicely. Looks like I've got a list of things to add to the JungleDisk backup.

11:17 - I don't think I'm storing enough of my settings on JungleDisk. Nothing's working quite the way I had it working before. I know the Drobo has it all, but that's really not good enough in a real recovery situation. Finishing up some final updates from Ubuntu.

11:06 - Pulling files fine from JungleDisk. It looks like everything worked as I hoped! Going to restart and get the rest from the Drobo since I don't really want a massive Amazon S3 bill.

10:52 - Getting near the two hour mark, most of it spent reading Anathem. Now on to recovering user settings and restarting the computer.

10:25 - Should have looked up before hand how to install a large number of packages in Ubuntu. Updated my repositories from JungleDisk and now install my old system. Says it'll take another 30 minutes or so.

9:57 - Just went on a wild goose chase with TrueCrypt when trying to install it. You need dmsetup installed first on Linux and I thought I had to install from source. No, just apt-get update first. Now to install JungleDisk. Note: I haven't touched my local backups yet.

9:35 - Actually, no. I need to get JungleDisk up first to restore my repository list, so first I need to get my encrypted passwords off of Dropbox and then install TrueCrypt.

9:32 - Base OS install is done. Now to install all my software.

9:16 - Partition tables! Good question. That's one thing I didn't write down before starting. I guess I'll just take the defaults.

9:11 - It has no idea what to make of the drive. Yippie! I've lost all my data! Now to install Ubuntu from the CD. Should take about 30 to 45 minutes.

9:08 - It's in. Let's see if it actually works.

9:00 - Computer shutting down. Getting ready to swap the hard drive.

8:37 - My last JungleDisk backup ran at 9 last night and I did a Drobo backup around 5 yesterday. That should cover everything I need for this morning.

8:32 - Getting ready to start. Got my tea and my Ubuntu CD, what more do I need?

The backup recovery test will start Dec. 13th at 9:00AM. Will it be exciting enough for a live blog? I have no idea.

I'm going to attempt to recover my computer on Dec. 13th from a full (simulated) hard drive meltdown. In preparation, I've developed a plan as to how I'll recover everything from my local and remote backups. I could do everything from the local backup, but I want to make sure the remote backup is working too. So here's my current recovery strategy.

Local Backup

As I've stated many times before, I use a Drobo for my local backups. I backup everything in my local user directory, where all my documents and user settings files are stored, and some applications that I've had to install myself. I plan on not using this right away when I recover, to prove to myself that I can get back on my feet with just my remote backups.

Secure Key Backup

I have a local memory card in a fire-proof lock box that's encrypted via TrueCrypt and has most of my passwords on it, including what I need to access my remote backup, JungleDisk.

First Remote Backup

The first thing I'll need to recover are files off of Dropbox. I have my TrueCrypt volume with all of my passwords on Dropbox as a redundant backup of my local encrypted memory card and I'll need those to get at my JungleDisk passwords. This should be simple using Dropbox's web interface, which will give me access to everything on my Dropbox without having to install the software right away.

Second Remote Backup

My main remote backup is JungleDisk. Once I get my passwords from the TrueCrypt volume, I'll be able to install and start JungleDisk to get my files back in order. After that, it should be smooth sailing (I hope).

Why?

The main reason I have it all set up this way is redundancy. I want local backups that are easy to access and with less hoops to jump through to get them working, but I also need remote backups because local backups won't cover me in case of a fire or other natural disaster (like a spilt can of pop!)

The key is, I need to be able to access everything from remote backups because I might not have anything local to recover with. I have my passwords locally in a lock box, but also strongly encrypted on Dropbox so that I can get them if the lock box is gone. (They're also on the Drobo and JungleDisk, because I really don't want to lose my passwords.)

Any plan you create should allow you to recover a new computer back to your current computer's working state even if you don't have access to your local work area. In fact, I would say that a remote backup is way more important than a local backup because with a remote backup, you should be able to recover from anywhere. A local backup is really just for convenience and speed, not to be depended upon for the really important stuff.

So that's my plan. What's yours?

I just had a conversion with my wife about my recovery experiment on the 13th.

She said, "If you're going to do it on purpose, you better do a full backup first."

Should I? If I'm really testing my backup solution, it better work whether I do it on purpose or not. Hard drives fail and I won't be able to do a full backup before mine fails, so why should my test be any different.

I feel fairly confident that I can recover nearly everything, without a full backup first. I may lose emails from my computer, but I keep everything on the server for seven days before deleting it, so even that should be fine.

I feel okay about trying this, and I'll still have my old hard drive if I need to go back. I feel that it's really important to try a real world scenario and not fudge the results by running a backup right before I try this.

"Well, you're doing it with your hard drive."

Yes, dear. I won't touch your computer.