My baby at work is a centralized logging tool. That means my system has to touch every other system in this large company’s large network, which is kind of cool. Not many projects deal with that many different things, and I’m seeing some things I haven’t seen since college–and never expected to see in the real world, actually.
A week or two ago, we had some trouble pulling the logs in from a highly specialized system. That happens. Unix is easy, Windows is almost as easy–yes, the world of logging is a little bit upside down–but the one-off systems that don’t fit into neat categories take a lot longer to bring into the fold.
The problem was that the user account my tool uses kept getting locked out.
The transfer worked just fine for eight days, then locked. The sysadmin checked the logs I didn’t have–yeah, we still need to work on that–and showed me that the account tried to log in unsuccessfully 13 minutes and 17 minutes after its successful login on Saturday, then repeated the same thing on Sunday, locking the account.
Was my tool forgetting its password? Not likely. The last time this happened, we got the account unlocked–which is about as hard as configuring XFree86 was in 1994–and the log collection started working again. And I didn’t think it was my tool. When my tool encounters an error, it waits about 5 minutes and 30 seconds, then tries again. So trying again 13 minutes later is out of character, as is trying again four minutes after that.
So I asked what IP addresses the failures were coming from.
After some digging, the sysadmin found that information for me. These weird attempts weren’t coming from my system.
I looked up and spotted one of my coworkers, Chris, standing in my aisle. I asked him if he had a second. He said yes.
I showed him what I saw. I said I was concerned, and asked who I should ask. Setting up an Outlook reminder to get the account unlocked every Monday isn’t the right answer.
Chris said I was right to be concerned–it might be someone trying to get into the system and pivot. He urged me to tell my supervisor.
My supervisor immediately recognized the system. It was an auditing tool. So then he fired off a message to the administrator of that tool. Guess who that was?
Chris.
Now, normally I wouldn’t care. Go ahead and try to guess that password. It’s six feet long. No, the password isn’t “sixfeetlong,” it would be six feet long if you printed it in 12-point type. It’s probably a memory dump out of an emulator when someone was playing Galaga or something goofy like that. Nobody’s going to guess it.
But there’s a peculiar thing about this system. It locks accounts after three failures, consecutive or non-consecutive. Three strikes and your out, jump through a bureaucratic process to get the account reinstated again.
So, in effect, Chris was unintentionally DoSing my system, and neither of us knew it.
I got a good laugh out of it.
David Farquhar is a computer security professional, entrepreneur, and author. He started his career as a part-time computer technician in 1994, worked his way up to system administrator by 1997, and has specialized in vulnerability management since 2013. He invests in real estate on the side and his hobbies include O gauge trains, baseball cards, and retro computers and video games. A University of Missouri graduate, he holds CISSP and Security+ certifications. He lives in St. Louis with his family.