Skip Ribbon Commands
Skip to main content

Cory's Blog

:

Quick Launch

Stenoweb Home Page > Cory's Blog > Posts > Disk Backups
September 02
Disk Backups

A fairly common theme in my life is that I accidentally a big ol' server that has a whole bunch of disk capacity in it. As a side-effect of this, I spend what might be considered an inordinate amount of time thinking about how to preserve the data that could be on that system. My personal theory about backups has long been that if one of my systems has a failure and I need to restore from a backup, there's a reasonably good chance that I'm going to need to restore the whole system anyway. If I don't need to restore the whole system, most backup tools let you get at just a single file, and I have had good luck on several occasions pulling single files (even from hidden folders) out of my full system backups.

And so, when I set out to design a backup system for TECT, it was just presumed on my part that I would need and want to account for the entire 12 terabytes of disk capacity. (It's 16TB raw, and I could have 14TB if it were provisioned as a single volume, but I've set it up differently.) With that requirement in mind, I set out and found that unless I'm willing to give up some (or several) principles of a reasonably good backup system, it was going to be extremely expensive to set up.

The solution I'd eventually settled on was to buy a single LTO-6 tape mechanism, which is just shy of $2900 for a bundle that includes all of the necessary parts, and then buy tape cartridges as necessary, probably starting with enough to back up the whole system three times. In theory, the only negative aspect of this arrangement is that the tape system is not automatic. I need to remember to run the full backup of the data drive once per week, and then load the cartridges to do the incremental backups. But, for the amount of data I was looking at eventually being able to accommodate, it was the most convenient and one of the more economical solutions. Not counting the cost of the mechanism, LTO 6 cartridges are beaten by only one type of media for the least expensive way to store data. Incidentally, LTO 5 cartridges are a much better deal at the moment, and work with LTO 6 mechanisms, but they hold less data and are therefore slightly less convenient.

But, after some investigation which was prompted by a discussion on YOCF (sometimes they do have good ideas), I decided to see exactly what was going on on the server that I was actively consuming six terabytes of disk space. It happens that it wasn't so hard to find out at all. TECT is laid out in a fairly logical way…

Location on Disk

Data

Size

C:\

Server OS, Exchange, SharePoint, WSUS

200 GB

D:\Users

Personal Data

315 GB

D:\Video

Shared Video (TV/Movies)

630 GB

D:\Music

Shared Music

80 GB

D:\sw

Software share

320 GB

D:\LapBak

Laptop Backups

5400 GB

 

If I move all of my important personal data from laptops and external disks to TECT, I expect that my iTunes library would add another 240 gigabytes and my photos directory (which should really be in my home directory rather than on a public share) would be about 330 gigs.

It turns out that I really do have less than two terabytes of really important stuff. The laptop backups would be inconvenient to lose, but that whole area merits a cleaning anyway. I have, for example, backups from laptops I haven't owned for a year or more in there, and I've "archived" a copy of the backup from my work PC that's now also more than a year old there.

Backing up two terabytes is a lot easier than backing up six of them, let alone with the potential for the need to back up ten or twelve terabytes being imminent. Fortunately, I don't think that the core data set will grow that quickly. It will grow, as I do things like buy movies from iTunes and take additional photographs, but I certainly don't expect to be shopping to double the array's capacity in the next few years.

And so, I thought what about those inexpensive, pre-packaged 3TB external hard disk drives? A year or so back I had bought a lot of cheap Seagate Expansion and GoFlex Desk external hard disk drives (three 2TB Expansions and two 4TB GoFlex Desks, respectively) for some cheap DIY network storage experiments, and those are terrible products, but I have more recently bought a 3TB Western Digital MyBook that has done very well at its job of "sitting at my desk connected to a machine, running backups and holding my VMs, photos, and iTunes library."

And the whole thing could be more automated, too. Just connect a disk on Sunday, let the full backup run, and then let daily or twice-daily incremental backups run to catch small (or big) changes in data. Change out the disk on Sunday for the next oldest one and then move last week's backups off-site. It complies at least partially with all of the principles of good backups, and it allows for some flexibility in software choice. (Not much, but some.)

And so, I went out and bought three more of the 3TB Western Digital MyBook drives. I'm sure the people at the local Target think I'm crazy, but at $130 a pop, data preservation for TECT (and really, TECT is where almost all of my data lives anyway) can get started for $400, rather than $4000.

It's a good starting point and to be honest, I would be a lot less devastated if one of my three $130 backup disks failed than if my $2900 tape drive failed. I have yet to actually acquire and start using backup software, but I have been making copies of the data on the D:\ disk over to the backup drives and it has been working very well for now. (I'll be moving to a setup that catches all the data on the system and then moving my remaining personal data over to the server in due time.)

And, if in the future I have the budget to buy a tape mechanism and it's necessary because I have grown the amount of data I want or need to be backed up, it shouldn't be a big deal. And now I get to move onto different issues like testing different SharePoint configurations, and possibly rationalizing my collection of user-facing computers.

Comments

There are no comments for this post.