How to use compression to help life with an SSD

Dave Farquhar Hardware, Software, Windows , ,

Since pretty much everyone thinks my love of SSDs is insane, I’ll throw another insane idea on top of it: using data compression. It makes sense. Doing it selectively, you help performance, while saving space. At a much higher cost per gig, that saved space is very nice to have.

Here’s why compression makes sense. Under many circumstances, an SSD can saturate your IDE bus. Then you run into the 56K modem problem. The bus is saturated, but you want more speed, so what do you do? Compress the data. Although data compression makes people nervous (shades of DoubleSpace I’m sure), modems have been doing this for two decades. Why? Because it works.

So while your drive is happily shoving 200 megs per second through your IDE bus, if you can compress that file by 20 percent, guess what? You’ll get 20% better throughput.

CPU usage is the main objection to this. But in my experience, NTFS compression uses 20-40% of a recent (P4-class or newer) CPU when compressing. That’s the hard part. When decompressing, overhead is a lot less. The objections to NTFS compression really date to the days when 200 MHz was a fast CPU.

I don’t recommend just compressing your whole disk. Selective compression is a lot better. There’s no use trying to compress data that’s already compressed, and a lot of our data is.

Use the command COMPACT to do the job for you. Here’s my sequence of commands:

CD \
COMPACT /S /C *.doc *.xls *.rtf *.txt *.1st *.log readme* *.bmp *.wav *.wmf *.bat *.cmd *.htm *.html *.xml *.css *.hlp *.chm *.inf *.pnf *.cat

If you have other compressible files, of course you can add those.

This is a one-time event, but you can schedule it to happen daily or weekly if you want. Just put the two lines in a batch file and create a scheduled task to run it. The command will skip any files that are already compressed. While the compression itself doesn’t take a lot of CPU time, scanning the drive does, so you might want to run it while you’re away if you’re going to schedule it.

Don’t bother trying to compress your My Music or My Pictures directories; that data is all highly compressed already, so all you do is tax your CPU for no reason when you compress that kind of data. Of course the main reason people buy 1 TB drives is because they have hundreds of gigabytes of music and movie files. It’ll be a while before storing that kind of data on SSD is practical. In that case, buy an SSD to hold the operating system and apps, and a conventional drive to hold all that data.

Some people compress their C:\Program Files directory. This can work, but some programs are already compressed. I would be more inclined to experiment with subdirectories on a case-by-case basis. Try compressing one program directory, see if it packs down any, and if it does, great. If not, uncompress it and move on.

UPX does an outstanding job of packing down program files but it’s not completely transparent. I found enough programs didn’t run afterward that I gave up on it. NTFS compression is a lot less effective, but a lot more transparent. As long as you don’t compress your swap file or hibernation file (and Windows will warn you incessantly if you even try to do that), you won’t break anything with it.

If you enjoy tinkering with things, by all means feel free to experiment with UPX. There was a time when I would have probably done it, but given a choice today between playing with data compression or playing with metalworking tools, I’d rather play with my metalworking tools.

But I do really like this SSD. For the first time in a very long time, I can sit down at a computer running modern software and it still feels fast.

Very brief first SSD impressions

Dave Farquhar Hardware , , , ,

My OCZ Vertex SSD arrived yesterday. I don’t have it working yet–not completely. In retrospect, I should have just installed the drive and rebuilt the system from scratch. I’d be time ahead by now. But I can tell you a few things.It’s fast. I booted Windows XP in 20 seconds off the Vertex. Not a fresh install either–this was my existing installation I’ve been using for 18 months. Granted, off my factory Seagate Barracuda 7200.7, it booted in about a minute. (I’m still pretty good at optimizing a PC.)

Read more

The Maryville church killing suspect was denied insurance for mental health treatments

Dave Farquhar General

There’s no explanation for why some crimes happen, but in the case of the murder of Pastor Fred Winters last week during his Sunday sermon at First Baptist Church in Maryville, Ill., there is an explanation. And it’s troubling.

The suspect was receiving treatment for mental illness. It seemed to be working. But the insurance company didn’t want to pay for it.

Thanks to this decision, a wife is without a husband, two teenage daughters are without a father, and a church is without a pastor.This part of the story is buried in paragraph 6 of a story that ran in the St. Louis Post-Dispatch on March 10. The money quote:

Drug treatments that seemed to help weren’t covered by insurance. A doctor recommended treatments in a hyperbaric chamber, but they also weren’t covered by insurance. And the chamber Abernathy wanted to use was in Florida.

Where’s the outrage?

Most likely, Terry Sedlacek is going to plead not guilty by reason of insanity. Already, early reports say his family claims he’s insane due to lyme disease. By other accounts, there’s only ever been one case study demonstrating that lyme disease causes insanity, and it’s never been duplicated.

But the cause of his insanity doesn’t matter. By all accounts, something was certainly wrong with Terry Sedlacek. His neighbors were afraid of him. So were his coworkers. He was getting treatment, and the treatment seemed to be working before the insurance company cut it off. The treatments they were willing to pay for didn’t work all that well. And the evidence available today suggests Sedlacek marked last Sunday as “death day” in his planner, loaded up enough ammunition to kill 30 people, and drove to church.

To many people, this story is all about gun control. Left-wingers say if Sedlacek couldn’t have gotten guns, this wouldn’t have happened. Right-wingers say if there’d been a few people with guns at church that morning, someone would have gunned down Sedlacek before he killed Pastor Winters.

Well, mass shootings happen in countries with strict gun control also, and it would have taken a really good, really quick shot to save Pastor Winters’ life since four shots went off in those few seconds before he died.

I don’t see this as a gun control case. Had Terry Sedlacek’s doctor been permitted to practice medicine without interference from insurance company employees–who by all rights would be practicing medicine themselves if they actually knew anything about medicine–then he would have either been in a better mental state on March 10, or he might have been in an institution. In either case, nobody dies.

It’s time for the Winters family, First Baptist Church, or someone–anyone–to file a wrongful death lawsuit against Sedlacek’s insurer. This isn’t just about money. This is about calling attention to a broken system that should have been abolished years ago.

How many more people have to die before we fix this?

I didn’t cause the depression

Dave Farquhar Saving money , , , , , , , ,

Various analysts are blaming the current depression on people like me. The reasoning goes like this: I have money in the bank, therefore, I should be out spending it, for the greater good, to stir the economy.

Let’s correct that right now.People like me “hoarding” cash didn’t cause this depression. I played by the rules. I didn’t lie on my mortgage application. I bought less house than the bank said I could afford, because I didn’t see how I could make that payment and still buy groceries. I bought a Honda Civic because I didn’t see how I could afford a car that cost $25,000 or $30,000 and I really didn’t see how I could afford to put gas in it. I made this decision when gas cost $1.59 a gallon in Missouri.

Basically, I made a budget and then I made the decision to stick with it. It wasn’t rocket science. Any time I thought about buying something, I sat down with a spreadsheet, entered in all the money I paid out each month, entered what I made, and figured out if the money left over was enough to buy whatever it was I wanted.

We were due for a depression, or at least a recession, at the beginning of the century. The dot-com boom and Y2K was a bonanza, but then two things happened. Y2K came and went, the world didn’t end, and people quit buying survival supplies in large quantities. Meanwhile, these startups failed to come up with viable business plans, continued to spend money faster than the government, and ended up going out of business. This hurt those companies, but it also hurt companies like Cisco and IBM and Intel, because as these companies went bust, their inventory of technology equipment, some of it unused, went on the market at bargain prices. There was no reason to buy a new Cisco router from CDW when you could buy the same thing, still sealed in the package, from a liquidator for half the price.

Then 9/11 happened and it really looked like we’d get our recession. But the government slashed interest rates, changed bank regulations, and encouraged people to buy like there was no tomorrow. GM started offering 0% financing on its cars in order to move them. Soon you could get free financing on anything but a house, and interest rates on houses were ridiculously low. And anyone could get a loan. Republicans loved it because it made the economy go boom-boom again. Democrats loved it because people at any income level could get mortgages.

But the problem was that many of these loans had onerous terms and conditions, and just because you could afford the payments one day didn’t mean you’d be able to afford them in two, three, or five years after some of the back-loaded terms kicked in. Of course, nobody worried about that because they were living the high life.

And then it all fell apart. It wasn’t quite as rapid as it seems. I think people started having problems paying their bills in 2005 or so, but it didn’t quite hit critical mass yet. It hit the smaller banks first. I know because the banks who had my mortgage kept going under, and every year or so, a slightly bigger bank would end up with my mortgage. But those weren’t any match for this monster either. Countrywide got my loan in 2007, but Countrywide wasn’t a dinky little bank. It went under, and when I made my final house payment, that payment went to Bank of America. Now it looks like even the mighty Bank of America might make me look like the kiss of death.

But that wasn’t the only problem. These bad loans got packaged up and re-sold. And somehow, these bad loans got higher grades than they deserved. A guy working as a slicer at Arby’s making $9/hour living in a $150,000 house isn’t a good investment. When everything’s going right, he can afford to make his payments, but the minute something goes wrong, he’s going to start missing payments and might not ever recover. So unless the guy gets a decent job, he’s not going to be able to afford to stay in that house. Yet somehow, a bank could package a bunch of loans like this and spin it as a grade-A investment.

Imagine me going around to my neighbors’ houses on trash day, filling boxes with trash, and selling the boxes, legally able to tell the buyer that the box contains something valuable. That’s great, until someone opens the box and realizes it’s just a box of trash.

No, this depression wasn’t caused by people like me. It was caused by people living beyond their means for too long, and not being able to pay the piper when the time came.

There’s another word for what’s happening right now, besides recession or depression. That word is “correction.” When the economy has been going in one direction for too long, it corrects itself. Sometime in the future, there will be another correction, and the economy will start improving again.

But I read my ultimate proof yesterday. Supposedly, if people like me would just spend their money, things would get better. So why does someone walk into a Jeep dealership with $24,000 in cash, intent on driving home in a new Jeep, and end up driving himself and his still-heavy wallet home in his old car?

And let’s look at people like me one other way. When I nearly lost my job in January, I had almost six months’ worth of income in the bank and a plan in place to be able to live off it for a couple of years, potentially. It wouldn’t have been a comfortable living, but it would have been doable. There would have been no need for me to go collect unemployment. I would not have been a burden on society. And when I retire, I’ll retire with enough money to get me through the rest of my life, with or without Social Security. I won’t be a burden on society either.

People who save their money might not spend it at the most opportune time for everyone else, so they might fail to even the economy out like a capacitor evens out electrical power. But they are never, ever a drag on society.

The kind of guy who could save America

Dave Farquhar Society , , , , , , ,

I went to several estate sales today (it’s what I do on Saturdays, after all), but one was memorable. Some sales just jump out at you, and this one had evil genius/mad scientist written all over it.The estate belonged to a man named Carl. From what I could gather, Carl was Catholic, diabetic, and from my wife’s comments, must not have been married at the time he died. She mostly stayed upstairs while I rollicked around in the basement, which was tinkerer’s heaven.

“This guy was just like you!” my wife marveled when I resurfaced once. Well, she’s half right. I very much would have liked Carl. And yes, Carl liked computers and models and trains and didn’t see any point in buying anything he could make himself. But Carl’s knowledge of physics and other sciences went far, far beyond mine, as did his knowledge of electronics. I pulled out box after box after box of electronic components. Some of the stuff was pretty new, and some of it obviously dated to the early 1970s, if not earlier. It pains me to think most of that stuff is going to get thrown away, but there’s no sense in me buying it, even for pennies on the dollar, when I don’t know what it is, let alone what to do with it.

It’s entirely possible that Carl and I did cross paths, sort of. In the 1980s and early 1990s, BBSing was a common hobby among people who enjoyed electronics, amateur radio, and computers. People exactly like Carl. For that matter, it’s possible he might not have just dialed into BBSs, he fit the stereotype of a BBS operator like a hand in a glove. Who knows, maybe Carl ran a BBS I used to call.

Digging around Carl’s work area, I found lots of different things. I bought some moldmaking supplies and casting resin, Bondo body filler, and some tools. Carl took care of his tools. But on his workbench, I found a single file laying there that still had metal shavings on it. Perhaps Carl died before he was finished with it and cleaned it. I found a brush, cleaned off the file, and could picture Carl looking down, nodding approval. I bought the file and the brush. Both were better than the ones I owned previously.

Unfortunately, Carl is the type of person our society has been trained to fear, rather than respect, especially during this decade. I found plenty of literature that Homeland Security wouldn’t approve of. Instructions for making Tesla coils, and lots of instructions for making things that go boom in the back yard. I also found literature that dealt with alternative car fuels, converting cars to electric power, and generating your own electricity.

He was also obviously very interested in robotics and using computers to control things. In a spare bedroom, I found a pile of old Timex Sinclair 1000 computers and peripherals. He added I/O ports to most of them, and hacked another one to use a Texas Instruments keyboard instead of the cheap membrane keyboard that came with it. He must have used that Sinclair for programming. Another spare bedroom had a couple of barely started robotics projects.

Unfortunately, many people look at people like Carl, and are too quick to label him a deviant, or worse yet, a terrorist. The label is unfair. In fact, during natural disasters, amateur radio operators often are the people with the best information early, giving invaluable information to relief workers.

But the most important thing is the tendency not to think within the boundaries that “normal” people usually confine themselves to. Among his things, I found a book titled How to Patent Your Ideas.

Now I don’t know what kind of ideas he had floating in his head. As far as I can tell, he never published any of them (I have his last name, and I searched out of curiosity).

But with all this talk today about energy independence, I think it’s great that some guy in Crestwood, Missouri was thinking along those lines. I don’t know if any of those thoughts turned into anything tangible or not. But frankly, that kind of work is important–much more so than the tinkering I’m doing in my basement, which so far has resulted only in some wooden toys for my son to play with, and metal toys for me.

We need some new ideas, rather than just buying everything from abroad. I know there are still people like Carl out there, but I hope they aren’t a dying breed.

Now, if you’ll excuse me, I have a sudden desire to go see what I can do with some of the tools I bought from Carl’s workbench.

How Scotch tape got its name

Dave Farquhar Useless Trivia

I am Scottish. I try to stretch nickels into dollars. Sometimes it’s because I feel the need, but sometimes I do it just to see if I can.

Don’t try to insult me by calling me a stingy miser. It won’t work, because I’ll take it as a compliment.

And yes, this has something to do with Scotch tape.Recently I was researching to see when Scotch tape was invented. I was thrilled to find out the name came from an insult. Originally, the tape only had adhesive on the edges, to keep costs down. A car dealer said the stingy Scotch bosses need to put more adhesive on it.

Eventually they did put more adhesive on it, but the name stuck.

And for what it’s worth, I was researching the history of the stuff because I wanted to know if I could legitimately use it when repairing really old toys. I try to use materials that would have been available when the toys weren’t so old. It turned out the stuff came out in the 1930s, so yes, if I want to use it to hold window inserts in place, I can do that and stay true to period.

And I will use Scotch brand. It’s more expensive, but it lasts longer. A Scot will pay more money for something if it saves money in the long run. It’s why I buy Compaqs instead of Dells.

It’s what I do.

Dave Farquhar Personal ,

I awoke this morning at my usual time. It was Saturday. I really just wanted to roll back over, pull the covers over me, and sleep another 30 minutes, but it was Saturday. And that’s not what I do on Saturdays. It didn’t matter that I was tired, and it didn’t matter that it was 10 degrees out. Staying out of the cold isn’t what I do on Saturdays.I got dressed, grabbed my coat and a map, and headed to my car. I knew where I had to be and when, and I was running late. I don’t know how I can get up at my usual time and still find a way to run late, but I guess I’m just talented that way.

I drank my morning cup of coffee in the car as I made my way into the city. Google would say to take the interstate, but I avoid interstates. It was Saturday. I might miss something interesting. Nothing interesting happens on Saturday when it’s 10 degrees out and sensible people are still in bed, taking cover under flannel sheets and a half-dozen quilts. But it was Saturday, and that’s what I do, whether it’s January or June.

I got excited when I saw someone putting signs out. Aha! Something interesting I didn’t know about! Then I realized the signs were advertising my planned destination. I turned onto a lonely road. There weren’t a lot of cars parked on the street, and most of the cars that were parked weren’t running. I started getting hopeful. Maybe it wouldn’t matter that I was running 15 minutes late. Then I saw some faces I recognized, sitting in cars, trying to keep warm. I angled into a spot a few doors down from my destination. I took a last drink of coffee, pulled my hood over my head, tucked my hands into my pockets, and trotted down the sidewalk, up the steps, and onto the patio where a box of numbers was waiting.

Number 47!?

I took my number and headed back to my car. I didn’t get too dejected, because it’s Saturday, and that’s not what I do on Saturdays. Saturday is like Christmas when you’re a kid. Even the most disappointing Christmas is still the best day of the year when you’re a kid. That means the most disappointing Saturday is still better than the best day at the office. Even if I got number 47.

Besides, getting number 47 meant that 46 other people decided it was better to be out in the 10-degree cold than under flannel sheets and a half-dozen quilts. Maybe that meant I wasn’t crazy. Or maybe it meant they were crazier than me, since they probably got up earlier than I did.

I walked back to my car, motivated no longer by excitement but rather by the prospect of a warm place and a good book to pass the time. But of course I didn’t pick up the book right away. I checked the time. Eight twenty-five. I had 35 minutes. I checked my map. I weighed my options. Something else was going on about four miles away, but did I have time? I decided to stay put. About half the time I stay put in that situation, and about half the time I go, and about 99 percent of the time I wind up second-guessing the decision. It was Saturday, and that’s what I do.

So I sat in my semi-warm car, reading a 50-year-old book about metalworking, wondering where on earth one might find the tools described in the book now that we just buy things made half a world away instead of making them. And the only answer I could come up with was in the basements of people old enough to have read the same book, only way back when it was still possible to buy stuff like that.

I looked around. More cars were coming, more people were taking numbers and then taking shelter. But there was only one person who looks for the same things I do. The others must have decided to go someplace else. Or maybe they were less dedicated than me, still keeping warm under flannel sheets and a half-dozen quilts like sensible people.

Eight fifty-five came, and people abandoned the warmth of their cars for the 10-degree cold and the privilege of waiting in line. Someone standing next to me had number 42. Another had 45. Close enough. I watched a latecomer walk up the stairs and take number 94 out of the box. That meant at least 93 other people were about as crazy as me.

A man opened the front door and announced he’d only have room for the first 25 people. He started calling out numbers. A few didn’t show, so numbers 26 through 30 got in, including the guy who looks for the same things I do. But I can still find stuff in his wake. There’s another guy who’s a lot more likely to beat me to things I want, and he wasn’t there, so that didn’t bother me.

I looked around, trying to see who I recognized, and trying to remember what they look for. I wondered if they were as cold as me. I already knew they were as crazy as me. I bounced my knees up and down and wiggled my toes to try to keep warmer. It didn’t help much.

Three people left, and the man returned to the door and let five people in. The people who left came out empty-handed. That’s never a good sign. But the guy who looks for the same things I do was still inside, which meant he might be finding good stuff. Hopefully there would be something left for me too.

Another person left and five more people were allowed in. I didn’t complain. One more left, and then another, and finally I heard the man call number 47. I was in.

I surveyed the house. It was small, but nice. It had lots of nice woodwork and was solidly built–the kind of house that can stand for centuries. But there are fewer and fewer of those now, because tastes have changed and many houses like it get bulldozed to make way for what’s popular today–or for yet another Walgreen Drug Store. So I went out of my way to admire the woodwork, because in 20 years there might not be any of it left outside of the City Museum.

Based on a number of things in the house, I surmised the owners had been of Italian descent and Catholic. Given the area, neither was a surprise. Neither was what I found and what I didn’t find. Spend enough Saturdays doing what I do, and you start to notice patterns.

I lost track of time but I spent three dollars. I didn’t have to wait in line, so I guess most people weren’t buying much. I never saw the guy who looks for the same things as me inside, and I never saw him leave. Sometimes he’s sneaky that way. I put my change in my wallet, tucked my treasure under my arm, pulled my hood over my head, and walked out the door and to my car. After quickly double-checking my map, I headed to my next estate sale.

It was Saturday, and that’s what I do.