I had a discussion with a client last week that brought up the topic of out of band networks. Out of band networks are a good security measure for reducing risk. But what is an out of band network, and what can it do for you?
An out of band network is a separate network, separate from your main network that carries production data. It is a good practice to put management interfaces such as IPMI on an out of band network and require separate authentication to access the network. This allows you to provide access to necessary functionality while reducing the chances of people misusing or abusing it.
Why you would want to use an out of band network
Infrastructure professionals like out of band networks because they improve reliability. The management traffic doesn’t have to compete with the business, which improves reliability. Segregating the management traffic also reduces the possibility of interference. This improves reliability and performance. This alone can make an out of band network worth the additional hassle and cost.
Security professionals like me like out of band networks because they let you quarantine off necessary evils. My personal favorite example is IPMI, a technology you may recognize by the trade names of Dell iDRAC or HP iLO.
Why segregate IPMI onto an out of band network?
System administrators love IPMI because it lets them connect to servers remotely and power cycle the device and do other functions that would normally require physical access. Since servers increasingly live in remote data centers, IPMI solves a real problem. If a server crashes halfway across the globe, I either need an operator in that data center at all times to power cycle the server, or the system administrator needs to get on an airplane. But both of these things are drains on profits. IPMI avoids all of that.
IPMI saved my butt a few years ago when a decommissioned server came back from the dead. That server was trying to use a user account my system relied on, but it didn’t have the current password. So this zombie server was locking my service account and breaking my log collection. Once we knew what was going on, a coworker connected to the server via HP iLO and shut it down. That let us unlock the account and bring my log collection back to life, ending the production outage. Finding the problem took half the day, but fixing it took about three minutes.
Even when I used to sit in the data center surrounded by servers, I usually couldn’t walk up to a rack, locate a server, and power it down much faster than that.
Why security professionals don’t like IPMI
Security professionals like how IPMI allows companies to quickly recover from outages. But IPMI makes security professionals very nervous. IPMI is a second CPU sitting on a system motherboard with its own access to the main system’s memory and PCI bus.
If I have a system running a completely up to date operating system and database and all of my user accounts are properly secured with no more and no less access than they need, IPMI can bring all of that crashing down. An attacker can use a flaw in IPMI to dump the contents of memory, and the main operating system will have no idea any of it ever happened.
So vulnerabilities in IPMI are very dangerous indeed. IPMI itself provides zero inherent value, since it doesn’t store any of its own data. But the data it can access is priceless. What’s worse is that while almost everyone knows about Patch Tuesday and the need to update Microsoft and Adobe software, very few people talk about iLO and iDRAC.
How to handle risky technologies on an out of band network
The simplistic solution is to not plug in the management port and live without the convenience. But that kind of answer usually doesn’t fly in the real world. Plus, the IPMI functionality doesn’t always have a dedicated network interface anyway.
What you do instead is create a separate network, in its own address space and on its own VLAN. The remote management interfaces and all of the management traffic live there, instead of the main network. Routers, switches, and any other necessary network management devices connect via dedicated network interfaces. Users access this separate out of band network via a jump box, which is a machine that has a network connection on both networks. Other than the jump box, though, these two networks exist independently.
This adds inconvenience for system administrators. It means they have to log into the jump box any time they want to use IPMI. But it makes life almost impossible for an attacker. With IPMI on a separate network, the attacker first has to locate the jump box, compromise an account on the jump box to gain access to IPMI, then find a way to get the tools they need in order to subvert IPMI onto that out-of-band network.
That’s a lot of extra steps. And each of those steps provides an opportunity to make mistakes and get caught. Each of those steps also increases the likelihood that the attacker will give up and move on to someone else. When you have your risky technology on an out of band network, you’re like the house with lots of lights on motion sensors, cameras, and a dog. A would-be burglar doesn’t want to get past all of that in addition to a lock. They’ll probably move on to a house that doesn’t have those three things.
Another example of an out of band network
In a way, the serial ports on network devices that provide access to their command line interface is an out of band network. Requiring a network administrator to connect to the device via a null modem cable and a serial port provides an extra security control, effectively keeping would-be attackers and other prying eyes away from the configuration, in addition to providing a way to get into the device even when a misconfiguration keeps it from communicating on the network.
Related questions
Why is FTP called out of band?
By unfortunate coincidence, FTP is called out of band, which can lead to misunderstanding. After all, out of band networks are good, right? They’re secure. So that can lead to people either concluding that FTP is secure because it’s out of band, or that out of band networks are insecure because FTP is insecure.
FTP is out of band because it uses one port, port 20, to transmit data, and port 21 for control information. But since both ports are unencrypted, FTP isn’t a secure protocol. Being out of band doesn’t do anything to make FTP more secure, only more efficient.
What is the management port?
The management port on most servers looks like an ordinary Ethernet jack. It’s not. It connects to a second system on the motherboard, controlled by its own RISC processor and usually running some form of embedded Unix or Linux. The server’s main operating system cannot see or utilize this port.
What is in band and out of band?
You can think of in band as normal production data. That’s the chatter between computers during their normal course of business. Out of band is data that occurs outside of the production data and is separated from that traffic in some way. This can be a logical separation, like running on a separate TCP port like FTP does. Or it can be a complete physical separation, such as the serial port providing access on a network switch to its command line interface.
David Farquhar is a computer security professional, entrepreneur, and author. He started his career as a part-time computer technician in 1994, worked his way up to system administrator by 1997, and has specialized in vulnerability management since 2013. He invests in real estate on the side and his hobbies include O gauge trains, baseball cards, and retro computers and video games. A University of Missouri graduate, he holds CISSP and Security+ certifications. He lives in St. Louis with his family.