Network infrastructure for a small office

We talked earlier this week about servers, and undoubtedly some more questions will come up, but let’s go ahead and talk about small-office network infrastructure.
Cable and DSL modems are affordable enough that any small office within the service area of either ought to get one. For the cost of three dialup accounts, you can have Internet service that’s fast enough to be worth having.

I’ve talked a lot about sharing a broadband connection with Freesco, and while I like Freesco, in an office environment I recommend you get an appliance such as those offered by Linksys, US Robotics, D-Link, Netgear, Siemens, and a host of other companies. There are several simple reasons for this: The devices take up less space, they run cooler, there’s no need to wait for them to boot up in case of power failure or someone accidentally unplugging it, and being solid state, theoretically they’re more reliable than a recycled Pentium-75. Plus, they’re very fast and easy to set up (we’re talking five minutes in most cases) and very cheap–under $50. When I just checked, CompUSA’s house brand router/switch was running $39. It’s hard to find a 5-port switch for much less than that. Since you’ll probably use those switch ports for something anyway, the $10-$20 extra you pay to get broadband connection sharing and a DHCP server is more than worth your time.

My boss swears that when he replaced his Linksys combo router/100-megabit switch with a much pricier Cisco combo router/10-megabit switch, the Cisco was faster, not only upstream, but also on the local network. I don’t doubt it, but you can’t buy Cisco gear at the local office supply store for $49.

For my money, I’d prefer to get a 24-port 3Com or Intel switch and plug it into a broadband sharing device but you’ll pay a lot more for commercial-grade 3Com or Intel gear. The cheap smallish switches you’ll see in the ads in the Sunday papers will work OK, but their reliability won’t be as high. Keep a spare on hand if you get the cheap stuff.

What about wireless? Wireless can save you lots of time and money by not having to run CAT5 all over the place–assuming your building isn’t already wired–and your laptop users will love having a network connection anywhere they go. But security is an issue. At the very least, change your SSID from the factory default, turn on WEP (check your manual if it isn’t obvious how to do it), and hard-code your access point(s) to only accept the MAC addresses of the cards your company owns (again, check your manual). Even that isn’t enough necessarily to keep a determined wardriver out of your network. Cisco does the best job of providing decent security, but, again, you can’t buy Cisco gear at your local Staples. Also, to make it easier on yourself, make sure your first access point and your first couple of cards are the same brand. With some work, the variety pack will usually work together. Like-branded stuff always will. When you’re doing your initial setup, you want the first few steps to go as smoothly as possible.

I’d go so far as to turn off DHCP on the wireless segment. Most wardrivers probably have the ability to figure out your network topology, gateway, and know some DNSs. But why make life easier for them? Some won’t know how to do that, and that’ll keep them out. The sophisticated wardriver may decide it’s too much trouble and go find a friendlier network.

Why worry about wireless security? A wardriver may or may not be interested in your LAN. But that’s one concern. And while I don’t care if someone mooches some bandwidth off my LAN to go read USA Today, and I’d only be slightly annoyed if he used it to go download the newest version of Debian, I do care if someone uses my wireless network to send spam to 250,000 of his closest friends, or if he uses my wireless network to visit a bunch of child porn or warez sites.

Enough about that. Let’s talk about how to wire everything. First off, if you use a switched 100-megabit network, you can just wire everything together and not give much thought to anything. But if you’re using hubs or wireless to connect your desktops, be sure to put your servers on 100-megabit switch ports. The servers can then talk to each other at full speed if and when that’s necessary. And a switch port allows them to talk at full speed to a number of slower desktop PCs at once. The speed difference can be noticable.

The low-end server

Here’s a good question: What should a small operation do when it gets fed up with its network and is tempted to just chuck it all and start over?
Well, my advice is to start over. But I don’t agree that starting over requires one to chuck everything.

We’ll start with the server. Chances are, these days, you need one. If you’re doing Web and e-mail, you absolutely need one. But to a lot of people, servers are a mystical black box that costs more money than a desktop PC but runs a similar operating system. And that’s all they know.

Here’s what you need to know: A corporate server is built to stricter tolerances than a desktop PC and sometimes uses higher-quality parts (common examples are ServerWorks chipsets instead of Intel chipsets, SCSI instead of IDE, and error-correcting memory instead of the cheap nonparity stuff). You also often get niceties like hot-swap drive cages, which allow you to add or replace hard drives without powering down or opening the case.

They’re generally also better tested, and you can get a support contract on them. If you’re running an enterprise with hundreds or thousands of people relying on your server, you should buy server-grade stuff, and building your own server or repurposing a desktop PC as a server ought to be grounds for dismissal. The money you save isn’t worth it–you’ll pay more in downtime.

But a dozen people won’t hit a server very hard. This Web site runs on a Dell OptiPlex Pentium II/450 workstation. A workstation is a notch above a desktop PC but a notch below a server, in the pecking order. The biggest difference between my Optiplex and the PC that was probably sitting on your desk at work a year or two ago is that my Optiplex has a SCSI hard drive in it and it has a 3Com NIC onboard.

A small office can very safely and comfortably take a reasonably powerful name-brand PC that’s no longer optimal for someone’s desk (due to an aging CPU) and turn it into a server. A Pentium II-350 or faster, outfitted with 256 MB of RAM, a SCSI host adapter and a nice SCSI hard drive, and a 3Com or Intel 100-megabit Ethernet card will make a fine server for a couple of dozen people. (My employer still has a handful of 200 MHz Pentium Pro servers on its network, serving a couple hundred people in some cases.)

This server gets hit about as hard as a typical small business or church office server would. So far this month I’ve been getting between 500 and 550 visitors per day. I’ve served about 600 megabytes’ worth of data. My average CPU usage over that time period is in the single digits. The biggest bottleneck in this server is its 7200-rpm SCSI disk. A second disk dedicated to its database could potentially speed it up. But it’s tolerable.

Hot swappable hard drives are nice to have, but with an office of a dozen people, the 5-10 minutes it takes to power down, open the case, swap drives, and close the case back up and boot again probably doesn’t justify the cost.

A business or church office that wanted to be overly cautious could buy the very least expensive sever it can find from a reputable manufacturer (HP/Compaq, Dell, IBM). But when you do that, you’re paying for a lot of power that’s going to sit there unused most of the time. The 450 MHz CPU in this box is really more than I need.

Jeremy Hendrickson e-mailed me asking about whether his church should buy a new server, and whether it really needed two or three servers, since he was talking about setting up a Samba server for file serving, Apache for Web serving, and a mail server. Running file and Web services on the same box won’t be much of a problem. A dozen people just won’t hit the server that hard. You just make sure you buy a lot of disk space, but most of that disk space will go to file serving. The database that holds all of the content on this site is only a few megabytes in size. Compressed, it fits on a floppy disk with lots of room to spare. Yes, I could realistically do nightly backups of my Web server on floppies. If floppies were at all reliable, that is.

I flip-flop on whether e-mail belongs on the same server. The security vulnerabilities of Web servers and mail servers are a bit different and it would be nice to isolate them. But I’m a lot more comfortable about a Linux box running both being exposed on the ‘Net than I am a Windows box running one or the other. If I had two boxes, and could afford to be paranoid, I’d use two.

Jeremy said his church had a P3-733 and a P2-450, both Dells, due for retirement. I’d make the P3 into a file/print/Web server and the P2 into a mail server and spend the money budgeted for a new server or servers to buy lots of disk space and a nice tape backup drive, since they’d get lots of use out of both of those. A new $1200 server would just buy lots of CPU power that’ll sit idle most of the time and you’d still have to buy disks.

As far as concern about the reliability of reusing older systems, the things that tend to wear out on older PCs are the hard drive and the operating system. Windows deterriorates over time. Server operating systems tend not to have this problem, and Linux is even more immune to it than Microsoft server operating systems. So that’s not really a concern.

Hard disks do wear out. I read a suggestion not long ago that IDE hard disks should be replaced every 3 years whether they seem to need it or not. That’s a little extreme, but I’ve found it’s hard to coax much more than four years out of an IDE disk. Dropping a new SCSI disk or two or three into an old workstation before turning it into a server should be considered mandatory. SCSI disks give better performance in multiuser situations, and are generally designed to run for five years. In most cases, the rest of the PC also has several years left in it.

Later this week, we’ll talk about Internet connectivity and workstations.

Linux network diagnostics

I was doing a little research for Gatermann about Linux networking. I didn’t find what I was looking for, but I found something interesting: a pair of tools co-written by Donald Becker called mii-tool and mii-diag.
The source code for it is available at scyld.com but Debian includes a package for it (mii-diag). It allows you to force your network card to re-negotiate its speed with your hub or switch, which is useful if it’s constantly negotiating the wrong speed. In Windows you can usually open the network control panel and force duplex operation and speed. In Linux, that requires playing around with module options, which aren’t always consistent across drivers (because they’re not all written by the same people) or, if you compiled your driver into the kernel, passing boot parameters. Either way, you’re forced to reboot.

Run mii-diag to find out the status of your card (and commentary on the situation from the authors, in some cases). You can run mii-tool -r to force a renegotiation nicely, or run with the -f parameter to force it to a certain speed (if you’re interested in forcing a speed, you’re probably chasing 100 megabit, full duplex).

If your system is mysteriously not connecting, like my Web server was yesterday after I moved it, this tool can be useful in fixing it. I wish I’d known about it yesterday. I eventually solved the problem by rebooting until it worked right. (I don’t think my server’s 3Com NIC likes my Linksys router/switch much.)

So if you want to change your network’s speed for any reason without rebooting, this is the tool to do it (and it doesn’t make you hunt the Web and Usenet for the module parameters).

More wireless networking

Well, I took the plunge. What good is credit when you don’t use it, right? I didn’t want to run CAT5 Ethernet cable everywhere and I didn’t want to spend hours playing with Linux drivers for phone-line networks that have been in beta for a year. Especially not with what few Usenet posts mention those drivers also mentioning kernel panics. No thanks.
Dan Bowman pointed out that JustDeals had good prices available on wireless gear. So I picked up a plain-old access point for $70 (I don’t want a combo access point/router/switch because I want something I can turn off when I’m not using it–can’t beat that for security) and a PCMCIA NIC for $29 and a pair of USB NICs for $29. That’ll let me put a computer in the front room and a computer in the spare room and it’ll let me wander around with my work laptop.

Dirt-cheap prices, no rebate hassles. Gotta love it. CompUSA’s prices on Netgear kit are good, but there are rebates involved, which is always a pain.

My plan for security, besides powering off the access point when I’m not using it, is to turn off DHCP, hard-code it to my NICs, turn on 128-bit WEP, use obnoxious passphrases, and place the access point as far from the outside wall as possible. That should give me acceptable security, especially considering the physical location of my house. Neither of my next-door neighbors has a wireless LAN, and I seriously doubt the neighbors behind me do either, and they’re pretty far back and might even be out of range anyway. I’m at the end of a street deep in a residential area, so most wardrivers probably won’t bother. And if they do, I’ll be home and I’ll probably see them.

One thing I learned today, which reveals my ignorance yesterday, is that most wireless NICs accept the “Any” parameter that we used to get a Linksys NIC talking with a 3Com access point so we could configure it. But your documentation may or may not mention it.

Let’s talk wireless networking

When I was at church tonight looking at a power supply they asked me to help them set up a wireless network. I didn’t go about securing it just yet because I was paranoid about locking myself out.
I learned enough anyway.

The first thing I learned was that mix-and-matching your stuff for initial setup isn’t the best of ideas. We had a 3Com access point, a D-Link PCMCIA NIC, and a Linksys USB NIC. The D-Link and the 3Com didn’t want to talk to each other. Differing SSIDs turned out to be the culprit. The 3Com’s SSID was “3Com”. The D-Link’s SSID was “default”. The Linksys’ SSID was “Linksys”. But the Linksys setup program hinted that if you changed the SSID to “Any”, it would work with anything. It was right. It linked right up to the 3Com access point, while the D-Link just kept blinking away, looking for something. So we used the Linksys to configure the 3Com access point and changed the D-Link’s SSID. We had to reboot a couple of times before it kicked in, but then the D-Link connected up and held a link.

So the moral of that story is to make sure your access point and at least one of your cards match. And if you can’t match brands, get one Linksys, since you can set its SSID to “Any” and it’ll connect to anything. (I couldn’t figure out how to make the D-Link do that; maybe if I’d set it to “Any” it would have found the 3Com too.) Of course the only way to find out the 3Com’s SSID was to connect to it, so if we hadn’t had that Linksys, we’d have been up a creek.

So now I just have to figure out how to secure the network and they’ll be set. The plan is to only break the wireless stuff out during events, so it’s not like they’ll become much of a wardriving target, but I’ll still feel better if it’s secure. I’m a little bit afraid to just connect to the access point, enter a passphrase and turn on 128-bit encryption, because I couldn’t figure out how to give the cards themselves the passphrase and I didn’t want to take the chance of whether it’ll ask for it upon initial connection.

Time for more research.

And I think I’ll be getting some wireless stuff for myself soon. I’ve thought about phone networking, but Linux support is spotty. Wireless is less secure and more expensive, but it’s a whole lot easier. And it’ll be nice to be able to take a laptop anywhere I want and still be connected. CompUSA has their wireless gear on sale right now.

A semi-easy firewall

A single-floppy firewall mini-distribution can be a quick and easy way to save yourself some money if you’ve got an old PC in a closet not doing anything, assuming you stumble across a combination of hardware that works right.
If you don’t stumble across a combination of hardware that works together, you can just as easily spend a weekend and accomplish nothing but uttering strings of four-letter words in combinations never before heard by mankind.

In case you came here looking for hardware that works, here are a few hints. A 10-megabit PCI NE2000 clone in combination with virtually any 10/100 PCI card ought to work fabulously. A pair of 10/100 PCI cards based on the RealTek 8139 chipset, which includes the majority of today’s inexpensive cards, probably will not. If you’re buying new stuff and want ease of use, get a 3Com card and a cheapie. If you want cheap and a little inconvenience, get a Netgear FA311 or 312 and a Realtek 8139-based card, such as a D-Link DFE-530+ or a Linksys. You’ll have to hunt down and install the natsemi.o module to get the Netgear working; most other inexpensive cards on the market will work with the rtl8139.o driver.

Freesco doesn’t supply a driver for the Intel EtherExpress Pro series out of the box. If you’ve got an EEpro, you can make it work by downloading the module and copying it to the floppy, but don’t rush out to buy one. And yes, the 3Com and Intel chipsets are high-performance chipsets, especially compared to the 8139, but remember, routers are machines that pull packets out of a 1.5-megabit pipe (if you’re lucky) and shove packets down an even smaller pipe. In this application, a $40 big-brand card doesn’t give you any advantage over a no-name card that costs $6 at Newegg.com

While these firewalls will technically work fine even on a 386sx/16, trying to make them work with ISA cards can be a long, difficult road. Used Pentium-75s are dirt cheap (and Pentium-60s and 66s are even cheaper, when you can find them) and they’re a lot less trouble because PCI cards don’t require you to rejumper them or hunt down a plug-and-play configuration disk to find out its IRQ and address. I’ve had the best luck with Pentiums that used an Intel Triton chipset or newer (the 430FX, HX, VX, or TX). I’ve tried a couple of boards that had a SiS chipset of 1995 vintage or so, and I could get one network card or the other working, but not both. I don’t want to generalize and say that based on two isolated incidents that all Taiwanese chipsets are junk for this application–for all I know, the problem could have been the BIOS on those boards–but I’ve done this on a handful of Triton-series boards and done well on all of them, and on two SiS boards and failed. Your mileage will probably vary.

How much memory do you need? 16 megs is sheer luxury.

Once you put all this together, the question becomes whether you use a floppy distribution or a full-blown distribution. If you want peace and quiet and cheap, the answer is pretty easy–use a floppy and pull out whatever hard drive was in there.

A full-out distribution like Red Hat or Debian will give you more versatility. You can run meaningful Web and FTP servers if you want (and your ISP allows it). You can run a caching nameserver to speed up your Web browsing. If you feel adventurous, you can even install the Squid caching proxy and speed up your browsing even more (but either use a SCSI drive or put in a bunch of extra memory and run Squid’s cache out of a ramdisk–Squid’s performance on IDE is, to put it mildly, terrible).

I’m having a hard time finding the documentation on how to set up a second network interface quickly. I believe it involves the file /etc/interfaces and the files /etc/sysconfig/ifconfig.eth0 and .eth1, but I don’t have a Linux box handy to investigate at the moment.

Anyway, I like Debian for this application (of course) because I can easily fit a minimal Debian on a 100-meg hard drive.

Once you get your network cards all working and talking to each other, you can build your firewall using this online tool. I just copy it, then Telnet into my Linux box using PuTTY, fire up a text editor, and right-click in the window to paste.

If you want versatility and quiet and don’t mind spending some cash, pick up a CompactFlash-to-IDE adapter and a CompactFlash card of suitable size. Don’t create a swapfile on the CF card–you’ll quickly burn it up that way. Your system will recognize it as a small IDE drive, giving you silent and reliable solid-state storage on the cheap.

A home Linux server? $1200?

ExtremeTech has an article about building a home Linux server. They’re recommending high-end P4s for the task. And I say, get real.
If what you want is a simple file/print server, anything that’ll take a 100-megabit NIC and has room for some good-sized hard drives will do great. You want a machine that’s running its PCI bus at 33 MHz, so a Pentium-133 is a better server than a Pentium-120, or, believe it or not, a Pentium-150. If the machine is marginal, get something other than an $8 D-Link 10/100 card or another card with the RealTek 8139 chipset. A pricier 3Com or Intel card will conserve CPU cycles for you.

Remember, too, that Linux doesn’t use the BIOS, so if a machine refuses to recognize that 200-gig hard drive you just bought, set the drive type to “none” in the BIOS and keep another, smaller drive in the system to boot from. Linux will pick up the monster drive and use it.

SCSI is much better for servers than IDE, but when two or three people (or one person) will be using it, the only advantage SCSI really offers is being better-built.

And the video recommendations in the article are absolutely ridiculous. You don’t need a GeForce 4MX 420. Dig around in your parts closet and find that 1-meg PCI video card you bought back in 1995 and haven’t used in five years. We’re talking a system that’s going to be using text mode. Or buy the very cheapest OEM AGP video card you can find to save a PCI slot for something useful–last time I looked, Newegg.com had a cheap AGP card based on an old ATI chipset for 18 bucks.

So don’t listen to those guys. If you want to build a Linux server and all you’ve got to work with is a Pentium-100, go for it. It won’t perform like their aging 1.13 GHz P3 (the slower machine in their benchmarks) but for a home network, it’s plenty. Keep in mind this Website is running off a P2-450. I’ve watched it under heavy traffic. There are two bottlenecks when it’s serving files to someone on broadband: My DSL connection, and the Web browser on the other side. The only time I’ve ever seen CPU usage on this box top 50% for more than a few seconds is when someone loads that giant GPS thread (the post with more than 200 comments).

Just be aware that some Linux distros aren’t too wild about older BIOSes. I’ve got a P133 that won’t boot the Mandrake 7.2 CD (yeah, it’s old–that’s how long it’s been since I used Mandrake) or the Debian 3.0 CD, but Debian 2.2 works fine. So be aware that you might have to experiment a little.

The ultimate DOS boot disk

A little over a year ago, someone issued me a challenge: Make a boot disk containing the Microsoft network client and CD-ROM drivers. The problem is that the network client, plus the DOS boot files, plus a CD-ROM driver and MSCDEX almost always takes up more than 1.44 megs.
So I zipped up as much of the junk as I could and made a boot disk that extracted the Zip file to a ramdisk and connected to the network. I had tons of space left over. So I added some niceties like doskey and a mouse driver. I still had space left over. So then I started hunting down every network driver I could find so that one disk could service the mismash of NICs we’ve bought over the years.

It worked, but adding new drivers was beyond the ability of a lot of my coworkers. And I wanted to add a Windows-style network logon and TCP/IP configuration. I started coding it and some of it worked, but eventually I ran out of time so I abandoned it.

Meanwhile, someone else was doing the same thing, and his results were a lot better.

From the guy who brought you Bart’s Way to Create Bootable CD-ROMs, there’s Bart’s Modular Boot Disk.

To get a disk like mine, all you do is make a bootable floppy on a Windows 9x box, then download Bart’s network packages, including whatever NICs you want to support. Then pop back over to the modboot page and grab all the CD-ROM stuff. I made a disk that supports all of the CD-ROM drives Bart had drivers for, plus a half-dozen or so NICs from 3Com, Intel, and SMC, along with mouse support and doskey. I still had over 100K to spare.

If you find yourself just a little bit short of space, you can use the freeware fdformat to format a disk with just 16 root-directory entries and a large cluster size. Use the commmand fdformat a: /d:16 /c:2. The space that would normally go to the bigger root directory and FAT ends up going to storage capacity instead. But don’t try to run fdformat in Windows–find a Win98 box and boot it in DOS mode.

To make life easier on yourself, you might make the disk, then image a blank and keep the image around for when you want to format a maximum-capacity 1.44-meg disk.

WordPress Appliance - Powered by TurnKey Linux