Why Your Company Should Adopt Structured Behavioral Interviews

One of the most important pieces of research into hiring practice predictiveness, ever, was the 1998 Schmidt & Hunter study. In their findings, they presented meta-analysis and validity scores for 19 different hiring assessment practices - from psychometric tests, to reference checks, to interviews, etc. Meaning, they sought out to explain which hiring practice(s) would be most accurate in predicting future performance.

What they essentially laid out is that, taken alone, no single hiring practice in isolation will ever get much above a .54 predictive validity. Carefully devised, structured interviews, in particular, have a .51 predictive validity meaning that 51% of the time, there is a correlation (predictiveness) between the interview and future performance. You might think 51% is really bad, but compare that to, say, basing hiring decisions solely on someone's reference checks (26%). And it doesn't mean that 49% of your hires are going to do inherently bad - it just means that, with that practice adopted, you're essentially giving up ~49% of your hires to chance.

Schmidt & Hunter also highlighted a couple more powerful combinations of methods that will net above .60 validity. Specifically: evaluations that assess an individual's general mental ability (GMA, eg. intelligence test) coupled with either a work sample, integrity assessment, or the structured interview.

Structured Behavioral Interviews vs. Unstructured Relational Interviews

Schmidt & Hunter also found evidence that behavioral assessments have strong predictiveness (#6 among the 19 assessment techniques measured). In 2008, there was a study done at the Mayo Clinic that looked at the efficacy of behavioral interviews in admissions to health professions. Again, behavioral interviews were empirically shown to be an accurate predictor of future performance when coupled with a GMA evaluation.

Putting aside other assessment techniques (psychometric tests, reference checks, etc.), we know that structured, behavioral interviews are more predictive than unstructured, relational interviews. What's the difference?

• Structured & behavioral: the interview is planned in terms of the questions asked and they don't vary from candidate to candidate. Eg. each panel member has their pre-determined questions and they are aligned from the start. The non-functional questions are behavioral questions that assess pre-determined behavioral competencies of importance for the role or team (eg. communication, collaboration, assertiveness, ability to manage change, operation under pressure). The questions, too, typically evaluate their performance under a challenge. Some examples here.
• Unstructured & relational: the interview is not well planned. Eg. it's likely that the panel members switch up the questions, or the panel changes often, and the questions themselves are not linked to any predetermined success factors. The questions themselves are more about candidate opinions or views on something rather than how they operated in a past example. It's how they are "relating" to the world given your question. Some examples here.

How to Adopt Structured Behavioral Interviews

Now, in that last link, you might find yourself saying, "But those are great questions that help you better understand the candidate." You might also wonder why, given the research, behavioral interviewing is not more engrained (you might have been asked a relational question in your last interview). The challenge is the mixing of questions according to the different assessment needs. Outside of intent (i.e. aligning on candidate understanding of the job vs. hiring manager understanding), the two main things you want to assess in each candidate is their functional and behavioral competence. Functional: do they have the core functional competencies to do the job? Eg. are their coding chops really up to par based on their resume? Behavioral: do they have the core behavioral competencies for the job. I.e. do they map to our values, culture, etc.? This is the "would I want to work with this person" element.

The muddling typically happens because you're moving fast, circumstances change, and you want to get a lot accomplished in the handful of occasions you talk with the candidate (lest you force them to go through 15+ interviews at your company, which is a different topic). A typical arrangement of meeting the different needs would look like this:

• First call (either candidate and Recruiter or candidate and Manager) - Intent. Figure out if they understand the job as you know it. Validate that they are interested. Validate that they meet a lot of your basic requirements on functional experience eg. basic technical ability. This is not an evaluative stage but, rather, a gate. Validate that there are no big obstacles to moving forward. Build a bit of rapport and comfort with each other.
• Second call or onsite (candidate and Manager and / or panel member) - Functional expertise. Assess their functional capability. If you're hiring an Engineer with a specific domain expertise required, you might walk them through a whiteboard evaluation (or shared-screen coding assessment, paired-programming test, etc.). This is purely their functional chops though - don't muddle this and say it's also a cultural assessment. There should be a clear evaluation metric for this portion (eg. 1-5 scale with explanation of the scale).
• Second onsite (candidate and Hiring Manager and panel) - Behavioral assessment. Assess their behavioral competence against the structured, behavioral interview plan with the panel. There should also be a clear and exclusive evaluation metric for this separate portion, ideally using the same scale as the functional evaluation but with clarity on what 1-5 means.

You and your panel then score the interview in isolation and then calibrate as a group, going through the functional, behavioral, and thus total assessment of the candidate(s). Over time, you should aggregate enough data to measure the correlation of your interview scores to future performance and, where possible, when individuals did not work out. You might find that behaviors, and not functional competence, is the piece of the hiring puzzle you most want to get right. Meaning, it's rare that you'll hire an Engineer who doesn't know how to code, but it's increasingly likely you'll hire someone who doesn't have the behavioral competency you need absent adopting a structured, behavioral interview practice. I'd further go out on a limb and say that a reliance on relational (or weak functional) questions like these are what builds homogenous, less diverse cultures.

Still Not Convinced?

If there are any lingering doubts about fully embracing behavioral interviews from top-to-bottom throughout your organization, consider one of the major pieces of research from Google's People Analytics function. Recall that this team, within the past couple years, ruled out things like GPA scores and brain teaser questions as worthwhile candidate evaluation measures. Google SVP of People Operations Laszlo Bock further described some of their findings as such:

"...what works well are structured behavioral interviews, where you have a consistent rubric for how you assess people, rather than having each interviewer just make stuff up. Behavioral interviewing also works — where you’re not giving someone a hypothetical, but you’re starting with a question like, “Give me an example of a time when you solved an analytically difficult problem.” The interesting thing about the behavioral interview is that when you ask somebody to speak to their own experience, and you drill into that, you get two kinds of information. One is you get to see how they actually interacted in a real-world situation, and the valuable “meta” information you get about the candidate is a sense of what they consider to be difficult."

- D