Reference Checks: Just Another Unstructured Interview

As I understand it, reference checks are a vital component of the usual personnel selection process. Since I have never been an HR person, I don’t know what kind of pressure they must feel to make sure they use every available tool to make sure job offers are extended to only the best-quality applicants. I presume it is pretty significant. What I do know, is that HR people are human beings, and human beings are seldom the great decision-makers we believe ourselves to be. So when I read this blog post about the importance of reference checking, I started to think about what is really happening from a psychology stand point. Blame it an academic training that emphasized theory busting, but my initial agreement quickly became doubt.

The problem with reference checking as part of the selection system is that we must make a number of assumptions about them and the quality of information they provide the selection agent.

  1. We are competent at obtaining information through interviews- ask any HR person (actually, pretty much any human being) and you will find that they are all very good. How did humans all become such great interviewers? Was it training of some sort, or is it more likely that interviews seem like conversations and we are all pretty good at those, right?
  2. We obtained the right information- not only did we ask the right questions, but the reference provided us with the correct information. We didn’t let any bias seep into the formation of the question, our tone of voice was neutral when we delivered the question and we faithfully encoded the information the reference provided. Additionally, the reference accurately recalled the information you sought, had no agenda of their own (both pro- and anti- candidate are very prevalent) and actually said whet they intended to say with no encoding or transmission errors.
  3. We are capable of ignoring inappropriate information- Turns out not all information is fair game for selection purposes. I know this runs counter to popular opinion (there’s a reason it’s popular, have you met those other people?), but we should not be looking for every bit of information we can, we should only be seeking information that is relevant to the candidate’s knowledge, skills and abilities (KSAO’s). You know, the important results from the job analysis- I just KNOW you have a job analysis for this job.
  4. We can objectively integrate this information into a high-quality selection decision- It may sound easy enough, but it isn’t. We must combine reference information with all other bits of information we have on this candidate. We have cover letters, resumes, telephone screens, at least one interview (probably unstructured too), each of those provide us with countless data points to evaluate and weigh against similar data points from other candidates. How much does each data point contribute to the overall selection decision? If you don’t know the answer to these, you are probably using clinical synthesis. Without going into specifics, this method usually involves making what amounts to a random decision from a wealth of potentially valuable set of data.

What we have here is a system with what engineers might call “multiple points of failure.” Only, if it was a true point of failure the system would stop. In our case, the decision is still made, so the system continues past the points where it failed and should have stopped. So now that I have established that reference checking gives us information of unknown quality and is combined with other information in such a way that we normally cannot determine the impact of its inclusion on the quality of the selection decision, what should we do? Well, the answer is not as simple as “don’t do reference checks.” While that’s probably the most cost-effective answer, I suspect people will sooner give up unstructured interviews (Actually, no. people will never give up unstructured interviews). Check back later this week, for a way to include high-quality (or at least higher quality) reference checks in your personnel selection system.



~ by George Guajardo on October 21, 2010.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s

%d bloggers like this: