How to use compression to help life with an SSD

Since pretty much everyone thinks my love of SSDs is insane, I’ll throw another insane idea on top of it: using data compression. It makes sense. Doing it selectively, you help performance, while saving space. At a much higher cost per gig, that saved space is very nice to have.

Here’s why compression makes sense. Under many circumstances, an SSD can saturate your IDE bus. Then you run into the 56K modem problem. The bus is saturated, but you want more speed, so what do you do? Compress the data. Although data compression makes people nervous (shades of DoubleSpace I’m sure), modems have been doing this for two decades. Why? Because it works.

So while your drive is happily shoving 200 megs per second through your IDE bus, if you can compress that file by 20 percent, guess what? You’ll get 20% better throughput.

CPU usage is the main objection to this. But in my experience, NTFS compression uses 20-40% of a recent (P4-class or newer) CPU when compressing. That’s the hard part. When decompressing, overhead is a lot less. The objections to NTFS compression really date to the days when 200 MHz was a fast CPU.

I don’t recommend just compressing your whole disk. Selective compression is a lot better. There’s no use trying to compress data that’s already compressed, and a lot of our data is.

Use the command COMPACT to do the job for you. Here’s my sequence of commands:

CD \
COMPACT /S /C *.doc *.xls *.rtf *.txt *.1st *.log readme* *.bmp *.wav *.wmf *.bat *.cmd *.htm *.html *.xml *.css *.hlp *.chm *.inf *.pnf *.cat

If you have other compressible files, of course you can add those.

This is a one-time event, but you can schedule it to happen daily or weekly if you want. Just put the two lines in a batch file and create a scheduled task to run it. The command will skip any files that are already compressed. While the compression itself doesn’t take a lot of CPU time, scanning the drive does, so you might want to run it while you’re away if you’re going to schedule it.

Don’t bother trying to compress your My Music or My Pictures directories; that data is all highly compressed already, so all you do is tax your CPU for no reason when you compress that kind of data. Of course the main reason people buy 1 TB drives is because they have hundreds of gigabytes of music and movie files. It’ll be a while before storing that kind of data on SSD is practical. In that case, buy an SSD to hold the operating system and apps, and a conventional drive to hold all that data.

Some people compress their C:\Program Files directory. This can work, but some programs are already compressed. I would be more inclined to experiment with subdirectories on a case-by-case basis. Try compressing one program directory, see if it packs down any, and if it does, great. If not, uncompress it and move on.

UPX does an outstanding job of packing down program files but it’s not completely transparent. I found enough programs didn’t run afterward that I gave up on it. NTFS compression is a lot less effective, but a lot more transparent. As long as you don’t compress your swap file or hibernation file (and Windows will warn you incessantly if you even try to do that), you won’t break anything with it.

If you enjoy tinkering with things, by all means feel free to experiment with UPX. There was a time when I would have probably done it, but given a choice today between playing with data compression or playing with metalworking tools, I’d rather play with my metalworking tools.

But I do really like this SSD. For the first time in a very long time, I can sit down at a computer running modern software and it still feels fast.

Psst… Wanna compete with Best Buy?

Best Bait-n-Switch is offering a service where they’ll remove crapware from a PC for 30 bucks.

You can offer to do the same thing for 30 bucks, but do a better job. Here’s how.Of course, the first thing you do is go into Add/Remove Programs and remove everything in sight, unless it’s something the client actually wants. That’ll take about 20 minutes, tops, and it’s probably the extent of what Best Buy does. That’ll help, but it doesn’t bring back all of the new PC peppiness.

Next, you need to install and run a couple of utilities. Start out with CCleaner to remove any stray registry entries that may linger behind. Hopefully there won’t be too much. Then grab the unbeatable Donn Edwards bundle of JK-Defrag, NTREGOPT, and Pagedefrag.

Run NTREGOPT to remove the slack space from the registry, then run Pagedefrag and reboot. You’ll end up with a defragmented pagefile and a fresh-as-a-new-install registry.

Finally, run JK-Defrag to move all the useless data to the end of the drive, and all the stuff people actually use to the front. It’ll do a much better job than Microsoft’s built-in defragmenter, even on a new system.

The tuneup should take less than an hour, and most of it is time you can just walk away from the system and let it do its thing. You can advertise your service as better than Best Buy’s and compete solely on that, or beat them on price by a few bucks while providing a better and more worthwhile service.

If you’re feeling really industrious, you can even consult the appropriate Black Viper services list and disable unnecessary services to free up a little RAM and CPU time. If you don’t want to do a lot of reading, Computer Browser and Remote Registry are two services that always make sense to disable in home environments. My personal list used to be a lot longer, but Windows’ defaults are a lot more optimal than they were 5-8 years ago. The other stuff I always used to disable is disabled by default now.

And here’s one last piece of valuable advice you can give your clients. Rather than buy the Norton or McAfee antivirus product that’s probably installed on their computer as trialware, delete it and have your client buy NOD32 instead. The price is comparable to the other products, but it consumes a lot less CPU time and memory than the rest. So if you want antivirus protection but also want the computer to stay peppy, that’s the best choice in town.

Why Firefox will probably always have mixed acceptance in corporate environments

I saw an article in Information Week today about Firefox in the enterprise.

The fanboys on both sides took offense, of course.

I’m a longtime Firefox user and an IT professional, but yet I agree with the premise that Firefox will always have trouble in that environment.The biggest reason is inside the firewall, in the corporate intranet. Some commenters complained about lazy in-house design, but that’s not the whole story. Many web-based enterprise applications are designed for Internet Explorer and only Internet Explorer. One app that I support takes it a step further, and only works with IE 5.5 or IE 6. That’s going to be a problem when the order comes down to deploy IE 7. The product is discontinued, so at that point we’ll have to either migrate to something else, or have people connect to a terminal server so they can run IE 6.

I have another web-based application I support (but if I ever change jobs I’ll deny ever hearing about it) that works with IE 7, but if and only if an administrator logs on and manually registers some ActiveX controls. That product is called Microsoft Project Server 2003 Web Access.

Yes, you read that right. Even Microsoft can’t properly support its own web browsers.

Any corporate web-based app that uses ActiveX will never run on Firefox. Those that check for a specific IE version might run on a hacked version of Firefox, but if you ever have any problems, you’re on your own. Corporate suits don’t like that.

And since computers and applications tend to live almost forever once they’re deployed, IE’s stranglehold on those environments may not be measured in years. We may be talking a decade, or even more.

I’ll submit the refrigerator-sized VAX systems I walk past nearly every day in the server room as evidence of the longevity of some systems. The computers themselves may not be quite 20 years old, but the applications they’re running are at least that old.

Firefox also tends to go against corporate culture in other ways. One of the first questions a corporate suit will ask is who they can sue if it breaks. Never mind that if a Microsoft product breaks, they probably waived all legal rights as part of the EULA. The guys in corner offices who wear ties know more about that than anyone who works on computers. A wave of the hand makes that problem go away.

Yeah, right. But don’t bother trying to tell them that.

A second problem is that many IT decisions are made, or approved, by people who admire Bill Gates’ wealth. Since Bill Gates became the world’s richest man by selling computer software, his computer software must be the best, period, end of story.

Many of the books decision-makers read perpetuate this belief. One example is the highly popular and influential book Naked Economics by Charles Wheelan. In many circles, this book is a must-read. I have to admit I’m getting as much out of this $11 book than I got out of my college economics class, if not more. But Wheelan trots Gates out again and again as a master visionary, a master programmer, and lots of other things that he clearly isn’t. The examples serve to make Wheelan’s point, which is the most important thing, but they also perpetuate the myth that Bill Gates is the greatest computer scientist and visionary of all time, when the fact is he’s an astute and ruthless businessman who happened to find himself in the computer industry. His track record as a programmer and visionary isn’t all that great.

But because of this myth, spread largely outside of the computer industry proper, many influential people will insist on using the Microsoft product any time there’s a choice. They’re not interested in Wordperfect or Quicken or Dreamweaver or Firefox any other product not made by Microsoft, as long as Microsoft makes something that competes with it.

The Millionaire Mind by Thomas Stanley explains this mentality somewhat. When a person’s job is to make money, they don’t want to do product research and they don’t want to take chances. When they buy tires, a dishwasher, or a refrigerator, they walk into the store and buy the most expensive one, because the most expensive one must be the best. They don’t want to spend time doing market research because they could spend that time making money. And they want something they believe won’t break, because time spent dealing with broken stuff is time they can’t spend making money.

Basically, any time spent discussing or researching a purchase is time that can’t be spent making money. So in the mind of a bean-counter or an executive type, it’s much cheaper in the long run to just choose the Microsoft product and forget about it.

The logic is completely faulty–it’s an excellent example of a red herring logical fallacy, as Bill Gates’ wealth has nothing to do with the quality of his competitors’ products–but arguing that point isn’t likely to get you anywhere. Even if the decision maker is wrong, the time spent arguing about it is probably worth more than the potential savings by going with a different product.

At home, none of this matters. And at home, I’ll keep using Firefox. I’ve been using Firefox since 2002 when it was an obscure project called Phoenix, so I think you can call me a longtime fan.

Firefox made remarkable progress from 2002 to now, while IE has gone from IE 6 to IE 7 in the same timeframe.

But in the corporate world, very little of that matters. Incumbency has its advantages. Some companies will embrace it because of its many advantages. In other companies, users will sneak it in the door, the same way they snuck in PCs in the 1980s and 1990s while the mainframe-centric IT staff wasn’t looking. But in the majority of companies, it’s likely to stay shut out, perhaps because something important requires IE, but if not, the mere absence of Microsoft’s name on the product will be enough to keep it out of some doors.

I don’t expect to ever have Firefox on my PC at my current job. It’s my employer’s loss, but it’s not my decision.

Firefox 3 is out

I downloaded and installed Firefox 3 today. I didn’t try any of the release candidates, so today is the first I’ve seen it.

So far, I like it.Appearance-wise, it looks a little more slick and polished. I can’t exactly quantify it. Looking good isn’t as important as running well, but appearance is the first thing you notice.

Installation is lightning fast, especially off flash media. If you have multiple computers, put the installer on a USB flash drive and install from there.

It imported everything, and I mean everything, from my old 2.x configuration.

The "Awesome bar" is the most touted feature. Given that it imported my (extensive) browser history, its guesses where I want to go aren’t exactly awesome yet. But I like how I can start typing the title of a page I viewed and its URL will appear. As it learns more about my browsing habits and I learn more about it, I’ll like it.

Overall it does seem faster than its predecessor. It renders my blog considerably faster. We’ll see how it fares on Friday when I throw 20-30 addresses at Google Maps for my Saturday ritual. Firefox 2.x really bogged down when I did that.

Memory usage right now is a reasonable 72 megs, which is probably about where Firefox 2 would be after an hour. I’ve kept Firefox sessions open for weeks, so we’ll see how 3.0 looks in a month. The session I’m using right now will probably still be open, knowing me.

These are all just first impressions, but so far I like it. It’s familiar, but some of the improvements are obvious pretty early.

Ve hev vays to uninstall Symantec Antivirus

We use Symantec Antivirus where we work, and somehow I got put in charge of it. It’s not my favorite product, but I’m not sure what would be better. So we live with it.

Recently I had two systems that decided they didn’t want to be managed anymore, and my usual fix, copying the server’s certificate file and grc.dat back into place, didn’t work. The official solution? Uninstall and reinstall.

So what if it refuses to uninstall and reinstall?I didn’t like the answer I got (rebuild the server), so I did some digging. I noticed that one of the services hung in the stopping state, which gave me a clue. I found manual instructions for uninstalling, but one of the prerequisites is that you stop all the services.

Unable to stop the services, I set everything with “Symantec” in its name to Disabled and rebooted. When the server came back, SAV wasn’t running.

At that point, the manual uninstallation would have worked, but that process takes 30-60 minutes, depending on how much junk you have to wade through in the registry (the more applications you have installed, the longer it takes). While I was snowed in this weekend, I built a machine and installed SAV on it so I could step through the process. With nothing else installed, it took me about 30 minutes to complete all the steps.

I decided to be lazy and see if I could pull it out with Add/Remove Programs. It would take 5-10 minutes to find out, and if it worked, it would be a good investment of time.

In my case, it worked, so I got to trade 60-120 minutes of active work for 20 minutes of mostly passive work. That’s a good trade, especially when the active work involves registry editing.

If the official method, via Add/Remove Programs, had failed, I can think of one option besides the manual uninstall. If your local bureaucracy will allow you to install a tool without jumping through a zillion hoops, you could install Revo Uninstaller and see if it can clean up the mess. Odds are it would leave fewer traces of SAV strewn about on your computer, so you’d get a cleaner uninstall, and, perhaps, less chance of whatever caused your problem in the first place lingering and rearing its ugly head again.

That wasn’t an option for me, so I was glad that I was able to get Add/Remove Programs to work.

Why I like MS Office better than OpenOffice

I saw a story on Digg talking about why MS Office is so much better than OpenOffice. The argument was pretty shallow–pretty much everything it said was either untrue or could be simplified to "because it is" or "because it costs money."

I’ve used both. I have both installed on a couple of machines. I generally use MS Office. Here’s why.For virtually everything I do, OpenOffice is fine. There’s no feature in Office 2000 that I actually use that isn’t in recent builds of OpenOffice. None. I wrote a book in Office 97, and the only thing that would keep me from writing the same book again in OpenOffice might be the template I used. If OpenOffice could interpret my old publisher’s template and save it in a format my editor’s copy of Word could understand, I’d be OK.

And honestly, I think during the process of writing that book, I pushed my system a lot harder than most people do. Word 97 would crash hard on me once or twice a month, and I don’t think anyone else has ever done that.

I’ve never crashed Word 2000. I don’t know if it’s because Word 2000 is more stable or if it’s because Windows 2000 is a lot more stable than Windows 98 was. I never ran Office 97 on Windows 2000.

My complaint with OpenOffice is speed. Word launches in five seconds or less, even if I don’t have its quick-launch application in memory. Usually less. OpenOffice components load slowly, sometimes taking 30 seconds to load. If I wanted to wait 30 seconds for my word processor to load, I’d use my Commodore 128.

And while I can’t quantify it, once Word is loaded, it’s faster and more responsive. OpenOffice Writer seems to hesitate just a fraction of a second longer when I pull down a menu or hit a hotkey. There’s not a lot of difference, but it drives me nuts.

I’m spoiled, I know. I used to use a word processor called TransWrite on my Amiga. There were a lot of things TransWrite wouldn’t do, but it was lightning fast. Even on a 7 MHz Amiga, it did everything instantly.

I can’t speak for anyone else, but what I want is something that gives me all the features of, say, Word 95, and runs as fast as TransWrite did. Given that 1 GHz is considered a slow computer nowadays, I don’t think that’s too much to ask. Neither OpenOffice nor Microsoft totally deliver, but Microsoft’s product comes a lot closer.

I absolutely, positively do not buy the argument that MS Office is more capable. Microsoft’s eternal struggle has been figuring out how to get people to upgrade their old versions of Office, because frankly when I started working in desktop support in August 1995, the existing Windows 3.1 versions of Word and Excel did everything that the people I supported wanted, even then. When I became a full-time IT worker in March 1997, one of my first jobs was rolling out Office 97. Its draw was that it was 32-bit and crashed less. It had some new features but aside from the real-time spelling and grammar checking, nobody really talked about them. Some people loved the real-time checks, and other people fell all over themselves turning them off.

Two years later, Office 2000 came out. A hotshot in the accounting department told me how much better it was, but when we really talked about the new features, his opinion was mostly due to the excitement of being the first to have the new version. Outlook was considerably better in Office 2000 than it had been in previous versions, but outside of that the only new feature I ever heard anyone mention was that the font menu displayed font names in the actual font. Access was better, but not a lot of people used it.

I’ve used Office XP and 2003. Outlook was incrementally better in both versions. But aside from Word’s booklet printing capabilities, I’ve never found anything in the newer versions of Office that I miss when I come home and use Office 2000 on my now-ancient computers.

And whenever I shift gears from Office 2000 over into OpenOffice, a few obscure features might be in a different place in the menu structure but I’ve always found what I needed.

But if for some reason I had to ditch MS Office tomorrow, I wouldn’t switch to OpenOffice. I’d load the Windows versions of AbiWord and Gnumeric.

In some regards, AbiWord and Gnumeric are closer to the 1992 versions of Word and Excel when it comes to capabilities. But they’re fast. And I’ve always been willing to sacrifice a few capabilities for a program that can operate as quickly as I can think. My only complaint about those two programs is that I never figured out how to make .doc and .xls the default file format for them.

Don’t overlook thrift stores when looking for software

Need a cheap copy of Windows or Office? Don’t need the newest, buggiest, clunkiest version?

Visit your local Salvation Army Thrift Store.I was flipping through CDs at a Salvation Army store over the weekend. The software was mixed in with the music. I found several copies of Windows 95 and Windows NT 4.0, and numerous copies of Office 97, all marked at $3.

Windows 98 is probably more useful, which is probably why I didn’t find any copies of it. But NT4 is reasonably fast and stable (by Microsoft standards) as long as your hardware is supported.

Office 97, on the other hand, had all the major functionality of later versions but is a lot less CPU- and memory-intensive. Remember, when it came out, 133 MHz PCs were above average, and 32 MB of RAM was usually considered excessive.

Just make sure the disc is original, the right disc is in the case, and it includes the CD key. I found a number of odd things in Windows 95 CD cases–some more useful than Win95 and some a whole lot less. None of it would have mattered since they would have required a different CD key from the one on the jewel case.

And make sure that if you’re going to run this stuff and connect the computer to the Internet that you’re sitting behind a reasonably good firewall. A Linksys router or wireless access point is perfectly adequate. Microsoft no longer provides security fixes for this old software, so you could be more susceptible to attacks than someone running the latest and worst.

I was definitely glad to stumble across a source of legal and useful commercial software. I know it’s just a matter of time before I’ll need it, and I’d much rather pay $3 for Office 97 than $300 for a newer version that didn’t really add anything useful besides ribbon toolbars, new Clippy animations, and a soundtrack by Robert Fripp.

How to organize your thoughts and notes

I know a lot of people keep notebooks pertaining to their hobbies. Any time they find something good on a discussion forum or elsewhere, they made a copy, print it out, punch holes in it, and put it in the notebook. Some people even put their own notes, from experience or discussions, in there.

These notebooks are a good way to learn a lot and retain it–you may forget some things over the years, but reading through the notebook again will refresh your memory.

There’s just one problem with notebooks–finding the information buried within.Notebooks are, after all, reference works, not something intended to be read cover to cover. The key to a good reference work, though, is a good table of contents and index.

Who wants to spend the time putting that together? But that’s the perfect use of the piece of software I discussed yesterday.

Just install LyX, learn how to insert a table of contents and index, learn how to flag something for inclusion in the index, and then, as you find things, copy and paste them into your “virtual notebook.” As you add things to it, you can print new copies of your notebook, and your notebook will have page headings and page numbers and a table of contents and an index so you can find stuff quickly. Keep your working copy for adding new stuff in LyX, but output a PDF from Lyx that you can read on your computer(s), and print a copy that you can keep in the basement where you’re actually going to use a paper copy.

The better you organize it, the better it’ll work, but even that isn’t strictly necessary since you can flip to either the front or the back to find stuff.

I can’t think of a hobby that this wouldn’t help. I don’t believe that information overload is so much a problem of too much information as it is a problem of not being able to sort through the information available to find the bit that you really need.

Where’s this software been all my life?

Playing around with LyX

In what little free time I’ve had the past few days (we have a project that has us in the midst of a death march at work), I’ve been messing around with LyX, a typesetting program for Windows, Unix, and most other operating sytems. I remember messing with it about six years ago, when there wasn’t much else resembling a word processor available for Linux, but this time, I’m more impressed with what I see.LyX is a front-end for a typesetting system called TeX. TeX was developed by the legendary computer scientist Don Knuth when he was dissatisfied with the appearance of his galley proofs for the second edition of The Art of Computer Programming. Knuth had an eye for fine typography, and because hand-set type was increasingly being replaced by machines, he looked for a way to make a computer play by the same set of rules that experienced typesetters have used for the past 500 years.

I had my first exposure to TeX when I was working on a business analysis project with Charlie Sebold. There was a department Charlie and I both did a lot of work for, and supporting these 8 users had ballooned into a full-time job in itself. We had an expensive contractor billing an average of 45 hours a week to the department alone over the course of a year, and when I replaced him, I wasn’t able to knock that down much below 40. We believed there was something wrong with a department of 8 users spending $200,000 a year in computer support. Come to think of it, that may have something to do with why I don’t work there anymore, but I digress. Charlie and I embarked on a project to figure out what we could do to cut those costs. I don’t remember anymore how the writing duties got split up, but Charlie typeset the report in TeX. I remember him being surprised to hear that I didn’t know much about TeX, especially since I had written a book for O’Reilly at that point, and if you look at the early O’Reilly books, they look like they were produced by TeX on the default settings.

Well, intentionally or unintentionally, using TeX for the report was a stroke of brilliance, because the most influential people in the department were design snobs, and TeX produces better-looking output than anything PageMaker could ever do. The text is beautifully justified, with no rivers through it, and the kerning is always set just right, and it will even use ligatures when appropriate. Basically, it does all of the hallmarks of elegant design that they taught me in journalism school–stuff that takes hours to do by hand–and it does it in minutes.

So when Charlie handed that report out at the first meeting, he got us a whole bunch of instant credibility.

What I like about LyX is that it removes the markup stage from TeX. You apply an appropriate document style–book, letter, article, report, or whatever–and you mark lines as whatever they happen to be–standard paragraphs, headings, chapter titles, document titles, author, or whatever–and it handles all of the layout and everything else for you. It’ll even generate the table of contents for you. And if you want an index, just flag words as you write or edit, and it can generate an index.

It also handles the most frustrating aspect of writing that I faced when I was writing my book back in 1999. A good book shouldn’t spent a lot of time repeating itself, so there’ll be times when you’ll refer the reader to a specific chapter, or even a specific page. The problem is, these things change. I not only re-ordered the chapters about halfway through the writing process, I actually took a couple of chapters, combined the like topics, and turned them into two completely differently titled chapters. Finding my cross-references and keeping them straight was such a pain that I really didn’t do it all that much. With LyX, cross-references are easy. You just label a section, and insert a cross-reference to the label, and it inserts the page number and the name of the section for you. You can put a cross-reference on every page and not slow down a bit.

Now that I’ve spent a few hours with it, I heartily recommend LyX. In college I found I got better grades when I turned in papers using fonts other than Times and Arial, and the output from LyX adds a whole new degree of elegance to it. Succeeding in college is as much about playing the game as it is anything else, and LyX gives you that slight edge.

And, as you might suspect, I’ve been playing with LyX for a reason. I’m writing again. Over the course of the past year, I’ve prepared a 133-page manuscript (that’s single-spaced Times with no pretty pictures or formatting, so it’s more than it sounds). I’m in the process of editing and typesetting it now. It’s highly specialized, so I’ll be self-publishing it, rather than using a publisher. I’ll be happy if it sells 1,000 copies and thrilled if it sells 10,000, and no publisher is willing to touch a book anymore if they think a book will only sell 10,000 copies. If it sells 1,000 copies, it will have been worth my while to write. Modern print-on-demand technology makes that a much safer risk than it was in 1999, when I wrote and published my first book.

And while there are times when the help of a traditional publisher definitely makes a better book, I think this is a case where I can create a better product working on my own.

I’ll keep you posted.

Speeding up Openoffice

This week the usual sources were flooded with stories about how slow and bloated Openoffice is. I guess this came on the heels of the release of version 2.0; it’s never been much of a secret that Openoffice was big and slow. It’s descended from Staroffice, after all, and it was big and slow too.

Speedup tips ensued.

Read more