Updating Windows without a network connection

Problem: I have to get three Windows servers patched up to date tomorrow. I found this out about 3 this afternoon.

Second problem: No network connection to the outside world, under any circumstances.

Third problem: Any rewritable media used on said servers must be destroyed after use.

Impossible? Believe it or not, no.Normally we keep a copy of Hfnetchk Pro in this environment for pushing out patches (copied from an Hfnetchk Pro server that does have a connection to the outside world), but someone saw fit to blow that server away. Ahem. Someone can expect a thank-you letter from me. And perhaps a thank-you present from my dog.

As for why servers with no connection to the outside world need patches to protect them from the outside world, well, I don’t make the rules.

So the answer in this case is to get my grubby mitts on ctupdate, a tool written by the wonderful German IT magazine c’t (their few English-translated articles are so brilliant, I wonder sometimes if I should learn German just so I can read the magazine).

Ctupdate will go download your updates, make an ISO image for you to burn to CD or DVD, and the result includes a nice menu so brain-dead easy that even a CIO could use it. (Oh, did I say that out loud?)

The catch? At present, a full collection of Windows XP or 2003 updates is nearly 800 MB in size, so make sure you have a fast network connection and either a DVD burner or a big USB disk if you plan to use it.

With a ctupdate-created DVD in hand, I can walk up to those isolated servers, pop in the disc, click a couple of buttons, have a cup or two of coffee, and then move on to the next one. Or better yet, copy the DVD to a network share, run the executable, click those buttons, have some coffee, and get on with the day. Problem solved.

This works for some slightly less convoluted situations too. If you expect to be asked to fix Windows PCs for a relative or twelve while you’re on Christmas vacation, prepare by downloading ctupdate, downloading all the updates, and either burning them to DVD or copying them over to a USB device. It works with Windows 2000, XP, and 2003 updates.

Integrated components vs discrete

Integrated components vs discrete is an old argument. I distinctly remember setting up a server for a new big-shot in 2004. I opened the server up to put memory in, and found its PCI slots filled with cards that duplicated all of the on-board components.

I asked my boss about this, and he said the guy had insisted on doing this, because “discrete components are better.”

I’ve been making jokes at the guy’s expense ever since.

Read more

I guess crate-trained means the dog is trained to get out of the crate

We should have named the dog Houdini.

The first night with the dog was difficult, because she wanted to play all night. Since they told her she was crate trained, my wife went out and got a crate the next morning.

Read more

Thoughts on backups

Backups have weighed heavily on my mind lately. When you have 125 servers to tend to at work, chances are one of them is going to fail eventually. Really what seems to happen is they fail in bunches.

One of my clients has a problem. He’s out of capacity. And that’s gotten me thinking about backups in general.You see, my client’s golf buddies are telling him nobody backs up to tape anymore. Backing up to disk is the hot thing now. Here’s the theory. Your network is fast, right? Why make it wait on the tape drive? Back up all your servers to disk instead, and they can all back up at once, and hours-long backups take minutes instead, and restores take seconds. And no more paying $3,000 for tape drives and $6,000 for a rotation of tapes for it!

Now here’s the problem. A CIO hears "disk" and he thinks of that 400-gigabyte IDE drive he saw in the Sunday paper sales ad for $129 with a $60 mail-in rebate. (It wasn’t really quite that big, and it wasn’t really quite that cheap, but these things are always better on Monday morning than they were the day before.)

No enterprise bases something as important as backups on a single consumer-grade IDE disk. For one thing, it won’t be fast enough. For another, they’re not designed to be used that heavily, that frequently. An enterprise could get away with something like HP’s $1200 entry-level NAS boxes, which use cheap IDE drives but in a RAID configuration, so that when one of those cheap disks fails, it can limp along for the rest of the night until you swap out the failed drive. The chances of one drive failing are small but too large for comfort; the chances of two drives failing at once are only slightly better than Ronald Reagan winning the Republican primary this year. With Abraham Lincoln as his running mate.

One can set up some very nice backups on a Gigabit Ethernet setup. Since Gigabit’s theoretical bandwidth is about 3 time that of Ultra320 SCSI’s theoretical bandwidth, you can back up three servers at once at full speed. Drop in a second NIC, and you can back up six. In reality, the disks in the NAS box can’t come close to keeping up with that rate, but the disk can still back up everything much faster than tape will. Even a lightning-fast state of the art 200/400 GB LTO drive.

Frankly, with such a setup it becomes practical to back up your most important servers over the lunch hour, to avoid losing half a day’s work.

But you don’t get it for $129.

And in reality, no enterprise in its right mind is throwing out tapes either. If they back up to disk, they spool that backup to tapes the next day, so they can store the tapes offsite for archival and/or disaster recovery purposes.

How important is this? I remember about a year ago getting a request for a file that was changed in the middle of a week, and the person wanted that copy from the middle of the week, not from our Friday backups that are archived longer. Even with a tape rotation of 40 tapes, I couldn’t get the file. The tape had been overwritten in the rotation a day or two before.

While rare, these instances can happen. A 40-tape rotation might not be enough to avoid it. Let alone just a couple hundred gigs of disk space.

But what about home?

Consumer tape drives had a terrible reputation, and based on my experience it was largely deserved. The drives had a terrible tendency to break down, and the failure rate of the tapes themselves was high too. The lack of comfort with enterprise-grade tape that I see in my day-to-day work may stem from this.

The last time I was in a consumer electronics store, I don’t think I saw any tape drives.

I suspect most people back their stuff up onto optical disks of some sort, be it CD-R or RW, or some form of writable DVD. The disks are cheap, drives that can read them are plentiful, and if floppies are any indication, the formats ought to still be readable in 20 years. My main concern is that the discs themselves may not be. Cheap optical discs tend to deterriorate rapidly. Even name-brand discs sometimes do. We’ve had great luck with TDK discs ever since Kodak took theirs off the market, but all we can say is that over the course of three years, we haven’t had one fail.

The last time my church’s IT guy called asking about backups, we happened upon a solution: a rotation of USB hard drives. Plug it in, back it up, and take the drive home with you. It’s cheap and elegant. Worried about the reliability of the drives? That’s why you use several. Three’s the minimum; five drives would be better. Use a different drive every day.

It’ll work, and it’s pretty affordable. And since the drives can be opened up and replaced with internal drives, it has the potential for cheap future upgrades.

How about the reliability of hard drives? Well, I have a box full of perfectly readable 120-meg drives in my basement. They date from 1991-1993, for the most part. I bought them off eBay in the mid 1990s, intending to put them in computers I would donate to churches. The computers never materialized, so the drives sat. I fire one up every once in a while out of curiosity. The copies of DOS, Windows 3.1, and the DOS Netware client that were on them when I got them are still there.

Some technology writers have observed that modern IDE hard drives seem to have a use-by date; they just seem to have a tendency to drop dead if they sit unused for too long. I see this tendency in a lot of devices that use inexpensive electric motors. Starting them up every once in a while and giving them a workout to keep the lubricants flowing and keep them from turning glue-like seems to be the best way to keep them working.

At this stage, I’m less worried about the long-term viability of hard drives than I am about optical discs. Ask me again in 20 years which one was the better choice, and I’ll be able to answer the question a lot better.

If you’re stuck using optical discs, the best advice I can give is to use a brand of media with a good reputation, such as TDK, make multiple copies, and store them in a cool, dark, dry place. The multiple copies should preferably be stored in different cool, dark, dry places. Light seems to break down optical discs, and cooler temperatures as a general rule slow down chemical reactions. Dryness prevents chemical reactions with water and whatever the water might manage to pick up.