CMD.EXE and its shellshock-like qualities

“So did you know there’s a Windows version of Shellshock?” a coworker asked the other day.

“What, Cygwin’s bash?” I asked.

“No, in CMD.EXE.”

I thought for a second, back to some really nasty batch files I’ve seen that do goofy stuff with variables and parenthesis and other reserved characters. Suddenly it made sense. Those cryptic batch files are exploiting the command interpreter to do things that shouldn’t be done. Then I smiled.

Read more

Upgrading a D-Link DIR-615 to DD-WRT

Last year I bought my mother in law a D-Link router, an oddball DIR-615 revision E1 that was only sold at a few stores. It was supposed to be a Fry’s exclusive, but I bought hers at Micro Center. It worked for a while, then gave her trouble, so this year I was working with it again, and when I was setting it up, I noticed it had some security vulnerabilities–remote code execution, UPnP vulnerabilities, and who knows what else.  So that got me some practice upgrading a D-Link DIR-615 to DD-WRT.

DD-WRT’s track record and attitude towards security research could be better, but I’d rather trust my mother in law to DD-WRT’s B+ security than D-Link’s F.

Read more

Optimizing dynamic Linux webservers

Linux + Apache + MySQL + PHP (LAMP) provides an outstanding foundation for building a web server, for, essentially, the value of your time. And the advantages over static pages are fairly obvious: Just look at this web site. Users can log in and post comments without me doing anything, and content on any page can change programmatically. In my site’s case, links to my most popular pages appear on the front page, and as their popularity changes, the links change.

The downside? Remember the days when people bragged about how their 66 MHz 486 was a perfectly good web server? Kiss those goodbye. For that matter, your old Pentium-120 or even your Pentium II-450 may not be good enough either. Unless you know these secrets…

First, the simple stuff. I talked about a year and a half ago about programs that optimize HTML by removing some extraneous tags and even give you a leg up on translating to cascading style sheets (CSS). That’s a starting point.

Graphics are another problem. People want lots of them, and digital cameras tend to add some extraneous bloat to them. Edit them in Photoshop or another popular image editor–which you undoubtedly will–and you’ll likely add another layer of bloat to them. I talked about Optimizing web graphics back in May 2002.

But what can you do on the server itself?

First, regardless of what you’re using, you should be running mod_gzip in order to compress your web server’s output. It works with virtually all modern web browsers, and those browsers that don’t work with it negotiate with the server to get non-compressed output. My 45K front page becomes 6K when compressed, which is better than a seven-fold increase. Suddenly my 128-meg uplink becomes more than half of a T1.

I’ve read several places that it takes less CPU time to compress content and send it than it does to send uncompressed content. On my P2-450, that seems to definitely be the case.

Unfortunately, mod_gzip is one of the most poorly documented Unix programs I’ve ever seen. I complained about this nearly three years ago, and the situation seems little improved.

A simple apt-get install libapache-mod-gzip in Debian doesn’t do the trick. You have to search /etc/apache/httpd.conf for the line that begins LoadModule gzip_module and uncomment it, then you have to add a few more lines. The lines to enable mod_gzip on TurboLinux didn’t save me this time–for one thing, it didn’t handle PHP output. For another, it didn’t seem to do anything at all on my Debian box.

Charlie Sebold to the rescue. He provided the following lines that worked for him on his Debian box, and they also worked for me:

# mod_gzip settings

mod_gzip_on Yes
mod_gzip_can_negotiate Yes
mod_gzip_add_header_count Yes
mod_gzip_minimum_file_size 400
mod_gzip_maximum_file_size 0
mod_gzip_temp_dir /tmp
mod_gzip_keep_workfiles No
mod_gzip_maximum_inmem_size 100000
mod_gzip_dechunk Yes

mod_gzip_item_include handler proxy-server
mod_gzip_item_include handler cgi-script

mod_gzip_item_include mime ^text/.*
mod_gzip_item_include mime ^application/postscript$
mod_gzip_item_include mime ^application/ms.*$
mod_gzip_item_include mime ^application/vnd.*$
mod_gzip_item_exclude mime ^application/x-javascript$
mod_gzip_item_exclude mime ^image/.*$
mod_gzip_item_include mime httpd/unix-directory
mod_gzip_item_include file .htm$
mod_gzip_item_include file .html$
mod_gzip_item_include file .php$
mod_gzip_item_include file .phtml$
mod_gzip_item_exclude file .css$

Gzipping anything below 400 bytes is pointless because of overhead, and Gzipping CSS and Javascript files breaks Netscape 4 part of the time.

Most of the examples I found online didn’t work for me. Charlie said he had to fiddle a long time to come up with those. They may or may not work for you. I hope they do. Of course, there may be room for tweaking, depending on the nature of your site, but if they work, they’re a good starting point.

Second, you can use a PHP accelerator. PHP is an interpreted language, which means that every time you run a PHP script, your server first has to translate the source code into machine language and run it. This can take longer than the output itself takes. PHP accelerators serve as a just-in-time compiler, which compiles the script and holds a copy in memory, so the next time someone accesses the page, the pre-compiled script runs. The result can sometimes be a tenfold increase in speed.

There are lots of them out there, but I settled on Ion Cube PHP Accelerator (phpa) because installation is a matter of downloading the appropriate pre-compiled binary, dumping it somewhere (I chose /usr/local/lib but you can put it anywhere you want), and adding a line to php.ini (in /etc/php4/apache on my Debian box):


Restart Apache, and suddenly PHP scripts execute up to 10 times faster.

PHPA isn’t open source and it isn’t Free Software. Turck MMCache is, so if you prefer GPL, you can use it.

With mod_gzip and phpa in place and working, my web server’s CPU usage rarely goes above 25 percent. Without them, three simultaneous requests from the outside world could saturate my CPU.

With them, my site still isn’t quite as fast as it was in 2000 when it was just serving up static HTML, but it’s awfully close. And it’s doing a lot more work.


A DOS-style editor for Linux

I keep seeing “someday someone will write a DOS edit clone for Linux”-type longings in Linux publications. These are pointless, because someone already did, years ago.

And no, its name isn’t vi or emacs. It’s a true blue (it really is blue) DOS-like editor that uses a lot of the same keystrokes as the Microsoft tool we all learned to tolerate, if not love, in the early ’90s. Hey, it wasn’t very powerful or fast, I know, but it was easy to learn and a whole lot better than edlin.

This one’s called SETedit, it’s from Argentina, and it’s just as easy to use but a whole lot more powerful. It’s also been ported to Win32, if you want to run it in more than just Linux.
Read more

Some goodies for your CD burner

This is the coolest thing I’ve seen in a really long time! Everyone and his uncle who has no clue how CD burners work wants a networked CD burner, for some reason. But it’s not as easy as just throwing the drive in a server and sharing it out with write access, as you probably know.
Leave it to someone else to think of combining the power of CGI scripts and the Unix command line to create a Web-based networked CD burning solution. So with this and a minimalist PC (any Pentium with 24 megs of RAM and a 1-gig hard drive ought to be more than enough) and a Linux-compatible CD burner, you can give controlled access to a CD burner to anyone on your network with a Web browser. It’ll even burn bootable CD-ROMs for you.

So now I’m half tempted to permanently install my 2X CD burner in my 486 so that any of my computers can use it, any time.

Speaking of bootable CDs… I’ve mentioned Bart’s way to create bootable CD-ROMs before, but it warrants another mention. Bookmark it. Bart Lagerweij has a great collection of boot floppies as well, and some good utilities, including low-level SCSI utilities.

Windows CD burning software. So you got a great deal on an OEM CD-R or CD-RW only to find it didn’t include software? What to do? You re-use the copy of Easy CD Creator that came with your old CD-R, that’s what. And then you’ll upgrade Windows and you’ll really regret that–Easy CD Creator is one of the most finicky programs I’ve ever seen about Windows versions. Upgrade Windows, you’ll have to buy a new version of Easy CD Creator. So if you’re smart, you’ll tell Roxio where to go and what to do with itself and buy Nero Burning ROM.

If you’re smart and cheap, you’ll pay this site (watch out for the annoying popups and popunders, sorry) a visit. It’s free CD burning software for Windows, based on GNU tools. It comes with dated versions of cdrecord, so you’ll want to download a newer version of CDRTools (current version as I write is 1.10; v1.11 is pre-release code so you use it at your own risk) and extract it to the directory you installed the front-end.

It’s not as flashy as the commercial tools and it doesn’t necessarily have all the features you’ll find in a retail shrink-wrap package, but it’s functional, and some people will find it easier to use. It happily runs on any 32-bit Windows. You can make as many copies of it as you want and install it anywhere you want. It’s legal, and much less invasive than the commercial tools. Good deal.

A remote administration Unix trick

OK, here’s the situation. I had a Linux box running Squid, chugging away, saving us lots of bandwidth and speeding things up and making everything wonderful, but we wanted numbers to prove it, and we liked being able to just check up on it periodically. Minimalist that I am, though, I never installed Telnet or SSH on it. And besides, I haven’t found an SSH client for Windows I really like, and Telnet is horribly insecure.
Sure, I could just walk up to it and log in and look around. But the server was several city blocks away from my base of operations. For a while it was a good excuse to go for a walk and talk to girls, but there weren’t always girls around to talk to, and, well, sometimes I needed to check up on the server while I was in the middle of something else.

So here’s what I did. I used CGI scripts for the commands I wanted. Take this, for example:

echo ‘Content-type: text/html’
echo ”
echo ‘‹pre›’
ps waux
echo ”
cat /proc/meminfo
echo ‘‹/pre›’

Then I dropped those files into my cgi-bin directory and chmodded them to 755. From then on, I could check on my server by typing into a Web browser. Boom, the server would tell me what processes were running, how much memory was in use, and even more cool, how much memory was used by programs and how much was used for caching.

Here’s how it works. The first two lines fake out Apache and your Web browser, essentially just giving them a header so they’ll process the output of these commands. The next line tells it it’s pre-formatted text, so don’t mess with it. This isn’t necessary for all commands, but for commands like ps that output multicolumn stuff, it’s essential. Next, you can type whatever Unix commands you want. Their output will be directed to the Web browser. I echoed a blank line just so the memory usage wouldn’t butt up against the process info. The last line just cleans up.

I wrote up scripts for all the commands I frequently used, so that way when my boss wanted to know how Squiddy was doing, I could tell him. For that matter, he could check it himself.

But if I knew there were going to be girls around, I went ahead and made an excuse to walk that direction anyway. Some things are more important than remote administration, right?

One way to defeat spammers

Ever since Brightmail closed up their free filtering service, I’ve been thinking a lot more about spam because I’ve been getting a lot more. I know where these losers are getting my e-mail address. It’s right here on my Web page. But I need to post that so people can contact me. Fortunately, I found a trick. Look at this:

That’s just an e-mail link, right? It works just like any other, right? Well, here’s the HTML code for that:

See what I did? I obscured the @ sign with an ASCII code (64), along with the dot (46) and a couple of other characters like the colon. Most automated e-mail address harvesters don’t decode the HTML, so their search routines, which look for things like @ signs and dot-somethings will blow right past that.

So if you run a site, obscure your e-mail address. If you don’t remember your ASCII codes, hopefully you’ve still got QBasic on one of your machines. In QBasic, the command PRINT ASC(“A”) will give you the ASCII code for the letter A. Substitute any letter you like. Or you can remember that A is 65 and lowercase a is 97. A is 65, B is 66, and so on.

When a Web site asks you for an e-mail address, you can see if it’ll let you obscure parts of it. Unfortunately, my forums flag illegal characters, but I may be able to modify that. Some Web sites aren’t that smart.

Obviously this trick won’t work in e-mail, unless you always send your mail in HTML format, which I (along with about half the world) really wish you wouldn’t–it’s annoying. And even if you obscure the mail you send, if I copy and paste your mail to my site, it’ll go up there unobscured. So this advice is mostly for webmasters.

Anyway… On to other things.

We’ve moved, if you haven’t noticed. These pages should be at least a little bit faster. The forums will be several times faster. And the forums are goofy. I haven’t figured out exactly why, but posts are missing and user files are acting up. If you’re having problems (Steve DeLassus just told me he can’t post because it tells him his .dat file can’t be accessed), go ahead and re-register. If you want your post count raised to its previous level, just let me know. I can change that. (Hmm, I wonder if Gatermann would notice if I set his post count to a negative number…?) I’d have preferred to move everything intact, of course.

Anyway. Go play in the forums. See what breaks. If I don’t know it’s broke, I sure can’t fix it. (I may not be able to if I do know, but hey, I can give it my best shot.)

Update: It’s 5:45 in the p.m., and you’re watching… Wait. That’s something else. The forums seem to be working properly now. Lack of uniformity between Linux distributions bites me again… It wasn’t the location of the files YaBB was objecting to, nor was it permissions. It was ownership. Under Mandrake, Apache runs as a user named “apache” and thus files created by CGI scripts like YaBB are owned by “apache.” Under TurboLinux, Apache runs as user “nobody,” and thus files created by CGIs are owned by “nobody.” And when you just tar up your Web site and move it to a new box like I did, those files remain owned by their old owners. Since Linux assumes you know what you’re doing, it happily handed those files over to a non-existant user. So when YaBB came knocking, Unix security kicked in and said, “Hey, nobody, you don’t own these files,” hence those error 103s everyone was getting.

How to get mod_gzip working on your Linux/Apache server

My research yesterday found that Mandrake, in an effort to get an edge on performance, used a bunch of controversial Apache patches that originated at SGI. The enhancements didn’t work on very many Unixes (presumably they were tested on Linux and Irix) and were rejected by the Apache group. SGI has since axed the project, and it appears that only performance-oriented Mandrake is using them.
I don’t have any problem with that, of course, except that Mod_Gzip seems to be incompatible with these patches. And Mod_Gzip has a lot of appeal to people like me–what it does is intercept Apache requests, check for HTTP 1.1 compliance, then compress content for sending to browsers that can handle compressed data (which includes just about every browser made since 1999). Gzip generally compresses HTML data by about 80 percent, so suddenly a DSL line has a whole lot more bandwidth–three times as much.

Well, trying to make all of this work by recompiling Apache had no appeal to me (I didn’t install any compilers on my server), so I went looking through my pile-o’-CDs for something less exotic. But I couldn’t find a recent non-Mandrake distro, other than TurboLinux 6.0.2. So I dropped it in, and now I remember why I like Turbo. It’s a no-frills server-oriented distro. Want to make an old machine with a smallish drive into a firewall? The firewall installation goes in 98 megs. (Yes, there are single-floppy firewalls but TurboLinux will be more versatile if you’re up to its requirements.)

So I installed Apache and all the other webserver components, along with mtools and Samba for convenience (I’m behind a firewall so only Apache is exposed to the world). Total footprint: 300 megs. So I’ve got tons of room to grow on my $50 20-gig HD.

Even better, I tested Apache with the command lynx and I saw the Apache demo page, so I knew it was working. Very nice. Installation time: 10 minutes. Then I tarred up my site, transferred it over via HTTP, untarred it, made a couple of changes to the Apache configuration file, and was up and going, sort of.

I still like Mandrake for workstations, but I think Turbo is going to get the nod the next few times I need to make Linux servers. I can much more quickly and easily tailor Turbo to my precise requirements.

Now, speaking of Mod_Gzip… My biggest complaint about Linux is the “you figure it out” attitude of a lot of the documentation out there, and Mod_Gzip may be the worst I’ve ever seen. The program includes no documentation. If you dig on the Web site, you find this.

Sounds easy, right? Well, except that’s not all you have to do. Dig around some more, and you find the directives to turn on Mod_Gzip:

# [ mod_gzip sample configuration ]

mod_gzip_on Yes

mod_gzip_item_include file .htm$
mod_gzip_item_include file .html$
mod_gzip_item_include mime text/.*
mod_gzip_item_include mime httpd/unix-directory

mod_gzip_dechunk yes

mod_gzip_temp_dir /tmp

mod_gzip_keep_workfiles No

# [End of mod_gzip sample config]

Then, according to the documentation, you restart Apache. When you do, Apache bombs out with a nice, pleasant error message–“What’s this mod_gzip_on business? I don’t know what that means!” Now your server’s down for the count.

After a few hours of messing around, I figured out you’ve gotta add another line, at the end of the AddModule section of httpd.conf:

AddModule mod_gzip.c

After adding that line, I restarted Apache, and it didn’t complain. But I still didn’t know if Mod_Gzip was actually doing anything because the status URLs didn’t work. Finally I added the directive mod_gzip_keep_workfiles yes to httpd.conf and watched the contents of /tmp while I accessed the page. Well, now something was dumping files there. The timestamps matched entries in /var/log/httpd/access_log, so I at least had circumstantial evidence that Mod_Gzip was running.

More Like This: “/cgi-bin/search.cgi?terms=linux&case=insensitive&boolean=and”>Linux

Integrity and fiction on the Web

I had thoughts that I thought best not shared, but then I read Frank McPherson’s excellent take on the hoax, so maybe I have something more to share after all. I’d really rather let the topic die, but since it appears there are still things for us to learn, let’s learn. Take consolation in that we can learn without me ever saying that name that begins with “K.” OK?
Here’s Frank:

For most of the last three or four months Dave Winer has been promoting the idea of amateur journalism. His point being that today’s mainstream media cannot be trusted, and does nothing but lie. Dave feels he can’t trust writers of BigPubs because they could be bought out by some person or company. He questions their integrity.

I’m trained as a professional journalist. I’ve seen the corruption from the inside. But I also know the source of the corruption, and that individuals inside can rise above it. I have classmates and former colleagues all over the place. CNet’s Troy Wolverton was in my New Media class. MSNBC’s Bob Sullivan taught my Editing class. The Associated Press’ David A. Lieb was my first editor in college. The AP’s Justin Hyde entrusted a newspaper column to me at the tender age of 19. My mentor as a columnist was Andrew Blasko, now a writer/pr contact/editor (strange combination) at The Heritage Foundation. USA Today’s Elizabeth McKinley was in my Editorial Writing class.

Those are just the people whose bylines I’ve seen recently, or who I remember for one reason or another.

I trust these people. I don’t always agree with them. I trust their ability to get the facts straight, partly because some of them were among the people who helped me learn to get the facts straight, and all of them learned to get the facts straight from the same people I did. Plus I spent a lot of time with them. I know they have integrity because I saw it. Not only do I trust them to get the story straight, but I wouldn’t think twice about tossing my car keys to any of them.

I also believe in amateur journalism. As far as I’m concerned, Mike Royko was the greatest journalist of all time. You know how Royko learned journalism? He went to the Chicago Public Library, grabbed every book on journalism and newswriting he could find, and spent a weekend reading them. He learned the principles and ethics of journalism, combined that with a God-given knack for writing that he may or may not have realized he had, and became a legend.

The key word Frank McPherson brought up is integrity. The individuals I mentioned have integrity. The National Enquirer lacks integrity. NBC’s Dateline lacks integrity. Debbie Swenson lacks integrity. Corporations are inherently no more and no less capable of integrity than individuals.

But corporations may have a slight edge in ability to maintain their integrity, because of accountability. Corporations, being made up of individuals, have a certain amount of accountability built in. Individuals can get accountability or they can reject it. I know if I say something that makes people wonder if I’ve been smoking crack, Dan Bowman or Dustin Cook or Pete Moore or a host of others will call me on it. They’ll chime in with their twenty bucks’ worth (that’s the price most people put on my words, and theirs should be worth what mine are), the truth will come out, and we’ll all be the better for it.

And that doesn’t just apply to my writing. When I teach a Bible study, there are usually two masters’ students among the audience. Those guys are slumming. While there are many preachers who have less formal training than I have, Matt and John know far more than I do. I have no idea what they can learn from me. But I appreciate them being there, because if I’m wrong, I know they’ll speak up, and they know I expect them to.

Integrity and accountability aren’t so much something you get so much as they are something you live. And yes, you should look for them, and if someone appears to lack them, then no, you shouldn’t trust them, not even for the sports scores. Don’t give them the eyeballs the advertisers look for.

Now, Frank brought up Bo Leuf, who brought up the question of fiction. Bo observed that when fiction writing first appears in a new medium, it looks like fact, and outrages people. And some people still can’t tell the difference years later. Having lived next door to people who truly believed the X Files were real, I know this firsthand.

Personally, I love the idea of a fictional weblog. We’ve been trying for years to figure out ways to exploit the unique capabilities of the Internet, and the weblog lets us do that.

The idea hit me as I read the end of this Oklahoman article. “I think [Swenson] wanted to tell a story. But she should have written a book or something.” Those were the words of Julie Fullbright, the local hero who unknowingly gave her face to the fictional character whose name I promised not to mention. That’s the kind of quotable quote a journalist lives for. I read those words just before I left work for the day, and I’ve been thinking about them all night.

I thought she was right then, and I think she’s still right now.

Now, having written a book, and having fallen victim to a publisher’s whims, I know what it’s like to try to write a book. I know what it’s like to try to get someone to publish it. And I know what it’s like to try to get someone to buy it. The difficulty increases with each step of that wretched process.

So, I’m sitting here with a novel about half-written, and no desire to have anything to do with a publisher until I’ve managed to acquire some clout. Now I don’t know for sure what having clout feels like, but I’m pretty sure I’ll know it when I feel it. But I can set up another weblog. I’m comfortable with that. I can give it the following subhead: “A work of fiction by David L. Farquhar.” The novel occurs in the past. That makes life easy. I just put it on its own server, with the clock set back. When today’s entry is dated 1992 or whenever, that makes it look a whole lot more like fiction.

Besides, Murel, my cubicle neighbor, has been telling me for months that I’d end up writing my novel in pieces here and one day I’d just have to tie it all together. I think he was on to something.

I won’t make any money, but that’s OK. I didn’t make any money off the one and a half books I wrote either. At least this time I won’t go in there with that expectation. If something happens that makes it profitable down the road, fine. End aside.

The character can be no more compelling than the author. That was the problem I ran into when I initially wrote the novel. I was trying to write about a 19-year-old, but I wasn’t finished being 19 myself yet. I’m not certain that at 26 I have enough perspective. But I have more than I did then.

And yes, sometimes life is better than fiction. But fiction intertwined with life kicks royal booty. The best thing about The Great Gatsby is that Jay Gatsby’s fears and insecurities were F. Scott Fitzgerald’s fears and insecurities. Jay Gatsby made his money by running drugstores that sold other stuff out the back room. F. Scott Fitzgerald made his money peddling words. But Jay Gatsby was all of F. Scott Fitzgerald’s worst fears wrapped into a character. He was vulnerable and honest. Vulnerable and honest people are compelling. Heck, vulnerable and dishonest people can be compelling.

So do I launch another weblog? I’m severely tempted. This isn’t the time to do it. I need to get my server in order and start getting content migrated to this site from its predecessors and make sure everything’s working smoothly. That’ll take a while yet.

But I know the formula. I have the plot, and the plot’s captivated everyone I’ve tossed it out to. I have some characters, and they’re far more compelling than the characters in the 1994-95 draft because I’ve spent the past seven years getting to know them. A few pieces still have to come together. But I think I really want to try this experiment.

More Like This: “/cgi-bin/search.cgi?terms=weblogs&case=insensitive&boolean=and”>Weblogs Fiction Personal

Short takes

AMD. According to the latest rumors on Ace’s Hardware and The Register, the Palomino core, when released, will be known as the Athlon 4. This is a marketing move; the Palomino is a less radical change to the core and the architecture than Thunderbird was. I think it’s a good marketing move, but it won’t do anything to make people less confused.
Tech support story of the day. A user one of my colleagues supports received an LS-120 superdisk in the mail. This user had no LS-120 drive, only a floppy drive. So my colleague went up to look at the disk and locate an LS-120 drive to read the disk. When he hunted down an LS-120 drive, he stuck in the disk, looked at it, and found a single file on it–a Word document. The file size? 32.5K!

But I guess it could have been worse. At least it wasn’t a 4K file…

Discussion groups. I’m not the least bit happy with how they look, and the performance isn’t so grand (an upgrade next week should help that), but I’ll go ahead and open up my forums. They’re at At the moment they’re totally open. I’ll bolt them down if spam, flame wars, or other things become a problem. I tend to be very open until that openness is abused, then I become a dictator.

I believe you can register without giving a true e-mail address, so you can use a spam filter if you’re afraid of that. Cookies are just used for automatic login and for timeouts–they’re good cookies.

The board is powered by YaBB, a free bulletin board written in Perl. Some things about it I like better than UBB, which is what most forums out there seem to use. I don’t like its color handling, but I’ll sacrifice that to gain other features.

Go ahead, take a look around, start posting stuff, and offer suggestions.

Why the forums when we’ve got comments? Well, I assume people want to talk about more than just what I talk about on a given day. This is preferable to e-mail because I have more options for reading it and it’s already online. Plus there’s always the chance someone else could pipe in with an answer.

More like this:AMD YaBB

WordPress Appliance - Powered by TurnKey Linux