Beware The Shortcomings Of Adjective-Based Assessments

By Dana Borowka, MA

Pop quiz for business leaders. Complete these statements:

A. If something seems too good to true, it ________________.
B. You get what you _______________________.
C. Haste makes ______________________.
D. Quality takes _______________________.

Many companies now use very simplistic adjective-only personality assessments. These quickie assessments are used for hiring, promoting employees, and improving team dynamics.

On average, some of these assessments only take only six to ten minutes to complete.

Typically, assessment takers receive two lists of adjectives. The makers of the tests claim the simple assessments measure four complex behavioral drives: Dominance, Extraversion, Patience, and Formality.

But a 2014 article in the Harvard Business Review raised a serious red flag regarding the adjective-based simplistic tests. In the article “The Problem With Using Personality Tests For Hiring,” Whitney Martin reported the widespread use of Four Quadrant (4-Q) personality tests for hiring is especially problematic. According to the article.

Generally speaking, 4-Q tools consist of a list of adjectives from which respondents select words that are most/least like them, and are designed to measure “style,” or tendencies and preferences. While they can seem highly insightful — not to mention being widely available and inexpensive — they have some severe shortcomings when used in high stakes applications such as hiring. For one, they tend to be highly transparent, enabling a test taker to manipulate the results in a way that they feel will be viewed favorably by the administrator.

Another popular test measures how much an individual matches to several personas. Though there are several variations of the assessment, a standard quiz presents statement prompts and asks respondents to rate the degree to which they agree. Assessment takers will also receive a diagram that represents how the personas react within the psyche.

Why You Need To Be On Guard

Not to alarm you, but don’t take choosing a personality assessment lightly. There are a multitude of assessments available out there, and the industry is totally unregulated.

So be wary. Sometimes cheap and fast is not always the best with so much on the line.

Today there are approximately 2,500 cognitive and personality tests on the market. To understand how to choose from the cornucopia of assessments, it is helpful to understand the origins of these instruments.

The story begins with a University of Illinois professor, Raymond Cattell, who was able to use the first electronic computer, the Iliac I, to do a large-scale factor analyses of his personality testing theories.

The 16 Personality Factor Questionnaire (16PF) was first published by Cattell, Tatsuoka, and Eber in 1949. Since then, there have been more additions.

The questionnaire is designed to measure normal behaviors and can be used for career development, employee selection and managing employees. The 16PF measures: warmth, reasoning, emotional stability, dominance, liveliness, rule-consciousness, social boldness, sensitivity, vigilance, abstractedness, privateness, apprehension, openness to change, self-reliance, perfectionism and tension.

Still in use today, in-depth assessments for screening candidates and assessing a team using 12-16 scales with 60-120 questions with more than 164 questions can take 30-90 minutes to complete.

There has been a 60-year quest to find a shortcut.

In 1963, W.T. Norman verified Cattell’s work but felt only five factors shape personality: extraversion, independence, self-control, anxiety and tough-mindedness. Dubbed the “Big Five” approach, this has become the basis of many of the modern personality tests on the market today.

There have been hundreds of studies validating the approach.

Different tests use different terms for the five factors. Some other terms include ambition, agreeableness, likeability, prudence, conscientiousness, adjustment, openness and intellect.

“This topic that’s been researched to death by the field of industrial and organization psychology,” said Wharton professor Peter Cappelli to Inc. magazine. “The amazing thing is how few companies take it seriously. It’s kind of mind-boggling that they would undertake such huge investments and not pay attention to what we know about how to pick out people who are going to be the best.”

But let’s say you take it seriously, then how do you pick the best assessment?

How To Evaluate Assessments

Here are seven factors to use to evaluate assessments:

1. Interpreters. How much training or degrees are required to interpret the results? Data interpretation is the most important factor when reviewing results.

2. Scales. Are enough scales used to cover the human personality? Lighthouse Consulting believes a minimum of 12 primary scales are needed to get a complete picture, with 16 scales being optimal in order to get the clearest picture of the individuals. This is why we only offer an in-depth work style and personality assessment. Anything less can lead to costly mistakes in the hiring process.

3. Validation. Is the test properly validated and on what basis? Just saying it is derived from the “Big Five” approach is not enough.

4. Reliability. Is the test reliable and on what basis? Test reliability refers to whether the test is consistent in measuring personality.

5. Legality. Is the test legal? Has it been reviewed for ADA compliance and gender, culture, and racial bias?

6. Impression. Would the test leave a negative impression with job candidates? If it is too easy it might send the wrong message.

7. Versatile. Is it proper for both hiring and managing? The information needs to be detailed enough to measure a candidate and improve manager/team member communications.

Our Point Of View: Faster Is Not Better

We believe the best tests require someone with comprehensive psychological training or degrees for proper interpretation of the data. Weekend training programs can be problematic since testing and human behavior is a highly complex subject.

Secondly, the more personality scales, the clearer the picture of the individual’s personality and work style.

Using 12 or more primary scales is the more cost-effective method since the personality assessments can be used for both screening candidates and for team building. We feel this offers the best return on investment for a manager because they can first have their existing team of employees tested, and then use the data to best judge how new hires will work with the existing team.

Here was the bottom from for Martin in the Harvard Business Review:

When using any assessment, managers need to step back and ask themselves one basic question before giving it to a potential employee: Is this test predictive of future job performance? In the case of 4-Qs, probably not. They can provide tremendous value for self-discovery, team building, coaching, enhancing communication, and numerous other developmental applications. But due to limited predictive validity, low test-retest reliability, lack of norming and an internal consistency (lie detector) measure, etc., they are not ideal for use in hiring.

Just to be clear, the pop quiz questions at the article’s beginning are called business truisms for a reason. If something seems too good to true, it probably is. You get what you pay for. Haste makes waste. Quality takes time.

To learn more about how you might take advantage of services offered by our Testing Division and our Business Consulting for Higher Productivity Division, please email [email protected].

Dana Borowka, MA, CEO of Lighthouse Consulting Services, LLC and his organization constantly remain focused on their mission statement – “To bring effective insight to your business”. They do this through the use of in-depth work style & personality assessments to raise the hiring bar so companies select the right people to reduce hiring and management errors. LCS can test in 19 different languages, provide domestic and international interpersonal coaching and offer a variety of workshops – team building, interpersonal communication, stress & time management, sales & customer service training and negotiation skills as well as our full-service Business Consulting Division. Dana has over 30 years of business consulting experience and is a nationally renowned speaker, radio and TV personality on many topics. He is the co-author of the books, “Cracking the Personality Code,” “Cracking the Business Code” and “Cracking the High-Performance Team Code.” To order the books, please visit www.lighthouseconsulting.com.

Lighthouse Consulting Services, LLC

Testing Division provides a variety of services, including in-depth work style & personality assessments for new hires & staff development. LCS can test in 19 different languages, skills testing, domestic and international interpersonal coaching and offer a variety of workshops – team building, interpersonal communication.

Business Consulting for Higher Productivity Division provides stress & time management workshops, sales & customer service training and negotiation skills, leadership training, market research, staff planning, operations, ERP/MRP selection and implementation, refining a remote work force, M&A including due diligence – success planning – value creation and much more.

Permission is needed from Lighthouse Consulting Services, LLC to reproduce any portion provided in this article. © 2023