Author: Dave Farquhar

Hey hey! It works! The server was down all day yesterday, which was a shame. I wanted to try a new experiment. So I’ll try it today.

I saw criticism over at Storage Review on Wednesday morning for their critiques of other hardware sites’ reviews. I disagree with this criticism; many of the reviews out there are atrocities, with poor methodology, hearsay, reviewer ignorance, and other shortcomings. Sometimes these reviews are more misleading than the information in the products’ advertising or packaging! I believe Storage Review is well within professional bounds to point out these shortcomings when they find them.

The mainstream media does this all the time. Columnists and editors will criticize the reporting done in other publications. Most newspapers also employ one person, known as the ombudsman, whose job it is to criticize and/or defend, as appropriate, the publication’s own work.

Seeing as the hardware sites out there often do very sloppy work, even compared to the mainstream media, some policing of it is a very good thing.

Then, over lunch, the idea hit me. Why not do some critiquing myself? I’m trained in editorial writing and editing. I have some experience as a reviewer. And I’ve published a fair bit of my own work in the arena of technology journalism–newspaper columns, a book, individual magazine articles, a series… So I’m qualified to do it, even though I’m not the biggest name out there. And that kind of content is certainly more useful than the “this is how my day went” stuff I’ve been posting way too often.

I’m not so arrogant as to assume that the webmasters of these large sites are in my readership and would take my advice. I don’t expect to change them directly. What I do expect to do is to raise people’s expectations a little. By pointing out what’s good and what’s not so good, hopefully I can raise the public consciousness a little, and indirectly influence some of these sites. If not, then at least my readers are better informed than they otherwise would be, and that’s definitely a good thing.

KT-133A roundup (Tom’s Hardware Guide)

This is a roundup of six VIA KT133a boards. Good review overall. It doesn’t get bogged down in three pages of history that tend to look like a cut-and-paste job from the last similar review, unlike some sites. But it does give just enough history to give proper perspective, though it would have been nice to have mentioned it took EDO and SDRAM some time to show their advantages as well–DDR is no more a failure than the technologies that came before. Unusual for Tom’s, this review isn’t obsessed with overclocking either. Lots of useful information, such as the memory modules tested successfully with each board. Inclusion of the DFI AK74-AC, which will never be released, is questionable. I can see including a reference design, but a cancelled commercial board doesn’t seem to make much sense. You can get an idea from its scores why it got the axe; it was consistently one of the bottom two boards in the roundup.

Emphasis was on performance, not stability, but Pabst and Schmid noted they had no compatibility or stability problems with these boards. Stability in benchmarks doesn’t guarantee stability in the real world, but it’s usually a good indication. As tight as the race is between these boards, stability is more important than speed anyway, and since the majority of people don’t overclock, the attempt to at at least mention compatibility and stability is refreshing.

Socket 7 Upgrade Advice (AnandTech)

This is a collection of upgrade advice for Socket 7 owners. This review, too, doesn’t get too bogged down in history, but the mention of fake cache is noteworthy. This was a PC Chips dirty trick, dating back to 1995 or so, before the K6 series. It wasn’t a very common practice and didn’t last very long–certainly not as long as the article suggests.

Lots of good upgrade advice, including a short compatibility list and pitfalls you can expect. Also included are some benchmarks, but it would have been nice if they’d included more vintage chips. The oldest chip included was the K6-2/450, and AMD sold plenty of slower chips. You can’t extrapolate the performance of a K6-2/300 under the same conditions based on the 450’s score.

Also, the rest of the hardware used is hardly vintage–you’re not likely to find an IBM 75GXP drive and a GeForce 2 video card in an old Socket 7 system. Using vintage hardware would have given more useful results, plus it would have given the opportunity to show what difference upgrading the video card and/or CPU makes, which no doubt some Socket 7 owners are wondering about. Testing these chips with a GeForce does demonstrate that a more modern architecture will give better peformance–it exposes the weaknesses of the CPU–but indication of how much a new CPU would improve a three-year-old PC would be more useful to most people. Few people have the delusion that a K6-3+ is going to challenge an Athlon or P3. They just want to know the best way to spend their money.

No deceiving graphics or lack of knowledge here; what’s in this article is good stuff and well written. It’s just too bad the testing didn’t more closely resemble the real world, which would have made it infinitely more useful.

Memory Tweaking Guide (Sharky Extreme)

This is a nice introduction to the art of memory tweaking, and it explains all those weird acronyms we hear about all the time but rarely see explained. Good advice on how to tweak, and good advice on how to spend your memory money wisely. They disclosed their testbed and included the disclaimer that your results will vary from theirs–their benchmarks are for examples only. The only real gripe I have is that the benchmark graphs, like all too many on the Web, don’t start at zero. From looking at the graph, it would seem that Quake 3 runs six times as fast at 640x480x16 than at 1600x1200x16, when in reality it runs about twice as fast. Graphing this way, as any statistics professor will tell you, is a no-no because it exaggerates the differences way too much.

Asus CUSL2C Review (Trainwrecker)

This is a review of the Asus CUSL2C, an i815-based board intended for the average user. This review has lots of good sources for further information, but unfortunately it also has a little too much hearsay and speculation. Some examples:

“Of course, Asus won’t support this [cable] mod and we’re pretty sure that doing it will void your warranty.” Of course modifying the cable on an Asus product, or any other manufacturer’s product, will void your warranty. So will overclocking, which they didn’t mention. Overclockers are either unaware or apathetic of this. In matters like this, assertiveness is your friend–it gives a review credibility. One who is assertive and wrong than is more believable than one who is wishy-washy and right.

“Arguably, Asus provides the best BIOS support in the business. We believe Asus develops their BIOS’s at their facility in Germany.” Indeed, Asus claims to have re-written over half the code in their BIOSes, which is one reason why Asus boards perform well historically. Most motherboard manufacturers make at least minor modifications to the Award, AMI, or Phoenix BIOS when they license it, but Asus generally makes more changes than most. This claim is fairly well known.

I was also disappointed to see a section heading labeled “Windows 2000,” which simply consisted of a statement that they didn’t have time to test under Windows 2000, followed by lots of hearsay, but at least they included workarounds for the alleged problems. Including hearsay is fine, and some would say even beneficial, as long as you test the claims yourself. This review would have been much more useful if they had delayed the review another day and tested some of the claims they’ve heard.

There’s some good information here, particularly the links to additional resources for this board, but this review is definitely not up to par with the typical reviews on the better-known sites.

DDR Analysis (RealWorldTech)

Good perspective here, in that DDR is an incremental upgrade, just like PC133, PC100, PC66 SDRAM, and EDO DRAM were before it. But I don’t like the assertion that faster clock speeds would make DDR stand out. Why not actually test it with higher-speed processors to show how each of the technologies scale? Testing each chipset at least at 1 GHz in addition to 800 MHz would have been nice; you can’t get a P3 faster than 1 GHz but testing the Athlon chipsets at 1.2 would add to the enlightenment. Why settle for assertions alone when you can have hard numbers?

Also, the assertion “And don’t forget, even though things like DDR, AGP, ATA/100 and other advancements don’t amount to a significant gain all on their own, using all of latest technology may add up to a significant gain,” is interesting, but it’s better if backed up with an example. It’s possible to build two otherwise similar systems, one utilizing AGP, ATA-100 and DDR and another utilizing a PCI version of the same video card, a UDMA-33 controller, and PC133 SDRAM, and see the difference. Unfortuantely you can’t totally isolate the chipsets, so minor differences in the two motherboards will keep this from being totally scientific, but they’ll suffice for demonstrating the trend. Ideally, you’d use two boards from the same manufacturer, using chipsets of like vintage from the same manufacturer. That pretty much limits us to the VIA Apollo Pro series and a Pentium III CPU.

And if you’re ambitious, you can test each possible combination of parts. It’s a nice theory that the whole may be greater than the sum of the parts, and chances are a lot of people will buy it at face value. Why not test it?

This reminds me of a quote from Don Tapscott, in a Communication World interview from Dec. 1999, where he spelled out a sort of communication pecking order. He said, “If you provide structure to data, you get information. And if you provide context to information, you get knowledge. And if you provide human judgment and trans-historical insights, perhaps we can get wisdom.”

This analysis has good human judgment and trans-historical insights. It has context. It has structure. The problem is it doesn’t have enough data, and that’s what keeps this from being a landmark piece. Built on a stronger foundation, this had the potential to be quoted for years to come.