How to conduct a technical interview

I’ve conducted a dozen or so technical interviews in my career. All of them panned out, and some of them ended up being exceptional. And during that same time frame, I’ve seen others fail, sometimes spectacularly. In this blog post, I’ll explain how to conduct a technical interview, including how to weed out charlatans and identify talent with a high upside.

How most people interview

Almost everywhere I’ve been, my coworkers conducted interviews differently than I did.

I have also noticed during my own job hunting that few people who interview me conduct interviews the way I do. All too frequently, the interview consists of a bunch of CISSP-style questions, as if they are trying to figure out if the authorities who certified me got it right or not. One time, someone asked me in an interview how to evade an IDS. I found out later the place didn’t even have an IDS, so it’s not like that modicum of knowledge could have helped them.

Sometimes I get an offer as a result of these interviews and sometimes I don’t. But it usually doesn’t matter, because I have to assume that’s how they interview everyone. Hiring someone based on the number of nuances they catch in your clever test question isn’t how you build a good team. When I take a job, I want to join a good team.

Interviewing starts with knowing what you’re looking for

Years ago, I read a magazine article on interviewing. I’ve never been able to find it again, so I have to work from memory. But the article started with the premise that you were hiring a hospital director. The article stated that a hospital director is, first and foremost, a medical doctor. So the author of the article argued that the main goal of the interview process is to determine if the candidate is a good doctor.

The author then described how doctors interview one another. They don’t ask trap questions like so many technical interviews do. They don’t try to re-evaluate four years of medical school in 60 minutes. Instead, they talk shop. They get a feel for how the other doctor evaluates the situation they are in and how they go about solving problems.

The author went on to say the reason hospitals have an easier time finding competent leadership than IT organizations do is because they interview differently. I’ll also add that this trick doesn’t just work for leadership positions. It also works splendidly for hands-on practitioner roles.

It occurred to me that I’ve been conducting technical interviews the same way as doctors interview since the 1990s. Part of that may be because I grew up hearing my dad have conversations with doctors and hearing his frustration when non-doctors tried to trap him with bad-faith questions. But I think another part of that is because of the way my first two interviews went.

Ron, the assistant department manager at Best buy

One of the best interviews of my career happened in the employee break room at Best Buy when I was 19. I’m dead serious.

It was a question and answer interview, so it could have gone badly. But they were good faith questions. As long as a manager didn’t go off script, the formula had predictable outcomes. They were precisely the types of questions somebody selling computers at Best Buy in the 1990s would encounter. What’s faster, a 486 or a Pentium? What type of monitor provides the highest resolution?

I remember those two questions specifically because I read too much into both of them. I answered both questions correctly, after a long pause. But then I asked Ron if they were supposed to be trick questions, because the fastest 486 at the time was faster than the slowest Pentium, and SVGA doesn’t always mean the same thing.

Ron could tell from my tone that I wasn’t trying to be a smarty pants, so I got the job.

The non-interview interview

My second interview got straight to the point. It mainly consisted of handing me a couple of different types of computer and asking me to take it apart, take out the RAM, hand it to him, and then put the computer back together.

My job was going to be unboxing 400 computers, installing memory and a network card, then plugging each one in, booting it up, and letting an automated build process take over. It wasn’t so much an interview as a five-minute evaluation of how I work. It got the job done.

How I prepare for interviews

I can’t just hand someone a computer and ask them to take it apart and call it a job interview. But I don’t have to.

The first thing I do is read the candidate’s resume and try to ascertain how well their experience fits the job opening. Usually, by the time the resume lands on my desk, enough other people have looked at their resume that I can assume they are a viable candidate. But I might have five resumes and one job opening, in which case, part of my job is recommending which candidate to offer the job to, and which candidate to offer the job if the first one declines.

I will also look to see if the candidate has a social media presence. I’m not looking for drunk college pictures to use to disqualify them. But if they have anything work-related posted on their socials, which is very common for security professionals, I can get a lot of insight before the interview about how they think and work. It will help me ask better interview questions.

This is an increasingly common practice. I recorded a ton of YouTube videos for one of my former employers. The last time I changed jobs, all of my interviewers went and watched my videos before we interviewed. So they had a very good idea how I think and how I speak even before they had scheduled the interview. They even said they felt like they knew me already. They knew going into the interview we weren’t going to be wasting each other’s time.

The kinds of questions I ask

Unfortunately, I have run into candidates who claim knowledge and experience they don’t have. I’ve run into new hires who got through the interview process claiming knowledge and experience they don’t have. I still remember a guy named Bill from 25 years ago. If our then-employer had let me spend just five minutes with him in the interview process, we would have saved a lot of trouble.

Here are three questions I ask that may not sound technical, but they often tell me all I need to know about a candidate’s technical acumen.

Tell me about a time you solved a tough problem with your favorite technology

I like to start by picking out one of the technologies they list on their resume and ask the candidate to tell me about a time they solved a tough problem with that technology. I don’t care if the technology in question was the problem or the solution. If you have the experience, you have stories. Ideally, if I’m conducting an interview, I know those technologies well enough to be able to tell if the story they are telling me is something plausible or something they are making up on the spot.

If they are nervous, they’re probably making it up. But if they have the experience, this kind of question probably puts their mind at ease, because they were expecting me to ask some gotcha question involving useless trivia they haven’t needed to know since the last time they were job hunting. So the question serves as a good icebreaker, in addition to giving me really useful information. I learn infinitely more from this single question than I will from 30 minutes of asking questions like how you evade an IDS, or how you evade antivirus, or what the difference is between hashing and encryption.

Which of two or more rival technologies do you prefer?

If they list two or more technologies that compete with each other in their experience, I ask which one they prefer. I don’t care if they prefer the same one I do. I care about the reasoning. In my area of specialty, there are three technologies that 90% of the population use. I like one of the three a lot less than the other two. If someone prefers that third one, it may be because it’s the one they know best. That’s fine. It may also be that they know something I don’t. If that’s the case, that’s even better. Nobody knows everything.

That brings up another important point. I am not afraid of hiring someone smarter than me.

What’s your least favorite thing about your favorite technology?

The next question I ask is what their least favorite thing is about their preferred technology. This tells me if the person can be objective when looking at the technology. Depending on the annoyance, it can also tell me how advanced of a practitioner they really are. Because some of the things candidates have brought up when I ask this question are things you don’t encounter until you’ve been using the product for a number of years.

And yes, objectively, any technology has something annoying about it. Amiga is the greatest computer of all time and second place isn’t close. But what annoys me about Amiga? From the moment you turn the thing on, the floppy drive clicks endlessly unless you keep a disk in it.

See what I mean? If you’ve never used an Amiga, you don’t know that about it. But if you have, you’re hearing that sound right now.

If you really know a technology, you know the good and bad about it. Even the best technology has something it could improve. When I’m building a team, I look for technologists, not fanboys.

And the good thing about finding a technologist is that if their preferred technology falls by the wayside, they can learn another one.

One more thing I look for in a technical interview

There is one more thing I look for. Ideally, the interview process confirms this fairly quickly. But I am usually interviewing somebody who is going to be part of a team. I don’t want a team full of clones. I try to determine if the candidate brings something new to the team. It could be a new skill, it could be some unique experience in their background, it could be a different way of looking at things.

Early in my career, I worked someplace that had a very recognizable bias against Generation X. What I learned from that experience is that different generations will look at the same problem and see different things.

That’s a really good thing, especially on my current team, where I have teammates from 3 generations that are not Gen X. That gives us four different perspectives looking at a problem, and when everything goes right, it means we can solve the problem four times faster.

Advantages to interviewing this way

Another advantage to interviewing this way is you find raw talent. Developed talent is expensive, and the demand for it is high. If you can identify people who are capable of quickly learning the skills you need, you have a much easier time finding qualified candidates whose salary expectations fall in line with your budget.

When I find someone who knows at least one relevant technology to the job, including what’s good and bad about it, and is good at solving problems, I’ve found a jewel. They may need to learn a few specifics before they can be a superstar in a given environment. But I can also be confident in their ability to do that.

How to conduct a technical interview: in conclusion

I guess one more thing is worth mentioning. If you aren’t technical yourself, leave the technical interview to someone who is. You can evaluate lots of other useful things about the candidate. Play to your strengths. Don’t try to be something you aren’t.

But when it comes to conducting a technical interview, think about how interactions happen in the real world, outside of the interview process. You know which of your current and former co-workers are competent and which ones are not. You didn’t interview most of them. You worked with them so you saw how they approach a problem and how they work their way out of it.

There is a cliche on social media that every company expects every security professional to be MacGyver and to be willing to work for $40,000 a year. Salaries are in the toilet right now, but they aren’t in that specific toilet yet, thankfully. But I can help you with that MacGyver part. MacGyver was really good at abstract thinking. You are not going to find someone who is good at abstract thinking by orally giving them the CISSP exam. But you will find them by asking questions that tell you how they solve problems.