The Executive Selection Problem

Bryan B of HR Tests brings up a very interesting discussion about the role Industrial/Organizational Psychology plays in executive selection. The short of it, is that we don’t do such a great job. This is an area that is very difficult for us to tackle as rigorously as other areas. Sadly, it is also an area that we desperately need to address.

It is certainly true that we treat executive selection differently that we do other forms of selection. The reasons for this are manifold; to my knowledge, the principle factor in this difference in approach is that the most predictive of our selection tools fail us at this level in the organizational hierarchy. Cognitive ability tests are the “gold-standard” where selection methods are concerned. However, most candidates vying for executive level position are among the best and brightest individuals the organization has to offer. The cognitive ability among this group of people just doesn’t vary enough for researchers to identify a statistically meaningful difference between them.

To fill this gap, practitioners of Individual Assessment fill make use of a variety of additional objective and subjective measures. Personality tests, Interest Inventories, Assessment Centers play a role and occasionally projective tests (like the TAT) do as well. How well do these work? You’re going to have to ask someone else. Neither executive selection, nor individual Assessment is my areas of expertise. What I can say, is that using a character approach is not likely to be much more help than cognitive ability tests. Why? I’m glad you asked!

The underlying premise of assessing a candidate’s “character” is to identify traits that enable us to predict future behavior. This assumes a few things:

1)      Traits are relatively stable

2)      Traits predict behavior

3)      Available trait (personality, character, values, etc.) assessments can actually measure executive’s traits

What does research tell us about these three assumptions? The Magic 8 Ball says, “Reply hazy, try again.” First off, we don’t have a lot of research about the stability of traits. What is out there suggests some measure of temporal stability, but it depends on what time spans we are talking about. I also suspect recent events and cognitive salience play a larger role in trait-based test responding than we normally acknowledge… call it a hunch.

The assumption about behavioral prediction is more problematic. Trait tests tend to be fairly general and the executive behaviors we are trying to predict are pretty specific. Work from the literature on attitudes suggests this is the worst possible combination. We also have evidence suggesting that external information (versus internal) influence behavior in many situations [Note: check out this neat summary of the impact of contextual information in behavior prediction].

The final assumption also gives us cause for worry about the role of trait tests on executive selection. In the best of times, personality tests are prone to faking. While there are some things we can do to reduce faking (I won’t give this away, if you don’t already know it. We need some tricks in our tool bag, after all), faking is a legitimate concern. Many people can make fairly good guesses about how to “fake good” on personality tests. I have reason to suspect that the best and brightest candidates will have little problem with this.

So where does this leave us. I suppose if I knew the answer, I would be paying somebody else to type this out for me while I wrecked yet another jet ski. The best I can say is that the status quo is not a great place for the scientific selection of captains of industry. We are long in questions, and short on answers. The good news is that now is a fantastic time for a brilliant person to find (and market) the solution. In the mean time, try not to buy any expensive snake oil; if you must, I can assure you that at least my brand is delicious (tastes suspiciously like a Venti Americano).


~ by George Guajardo on July 7, 2009.

4 Responses to “The Executive Selection Problem”

  1. I did some contract work for a woman who used the Birkman test as a recruiting tool, which I think was illegal in the state of Texas (because it is basically an intelligence test and not necessarily directly related to the ability to do the job? can’t remember for sure). She was looking for a new HR person, so I was screening people for the job by giving them the Birkman test. She would only interview people who got over a certain (very high) score.

    What was really crummy was that I knew as soon as I had graded the tests (while they were waiting!) whether someone would be continuing in the process but she wouldn’t let me tell them. She wouldn’t even let me write a letter. I hated her entire process.

    • @ Class I am pretty sure cognitive ability tests for personnel selection are not illegal. They have to be reasonably tied to job performance. Research has demonstrated that they predict performance across a pretty wide swath of jobs. However, they have the nasty habit of producing disparate impact, so it is best to have some type of validation study under your belt when you are using them.

      Having said that, the other elements of her selection process were pretty insensitive. I can see how you would feel uncomfortable. I have no idea why people forget to respect other people in situations like these.

  2. It’s my understanding that recruiters are more and more interested not just in the candidates’ traits, but in personal background and upbringing to compliment perceived experience levels.

    The only thing is, when interviewing reasonably intelligent people at all levels, hiring is still a crap shoot.

    • @RJ Hall: I agree. Discrimination relies on meaningful differences between choices. Absent those differences, choosing among alternatives because way too subjective. Bio data seems to be the best weapon in our arsenal at this time. The idea is that what you have done in the past is a good indicator of what you will do in the future. It works pretty well, but seems insufficient for people that will be compensated with the national treasuries of small island nations. I think assessment centers work pretty well and can be much less subjective that personal histories and values.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: