Before committing to a psychometric tool, employers should get a peek under the hood and ensure that they have a basic understanding of how personality tests work. The assessment industry is unregulated, which means it’s flooded with tests that promise more than they can deliver. Being informed can help you avoid introducing the psychometric equivalent of snake oil into your organization. This article will explain the key areas employers should focus on when trying to understand the nuts and bolts of personality tests.
Types of Personality Tests
First, employers who want to know how personality tests work must know the difference between subjective and objective personality tests.
Subjective personality tests are used in clinical and court settings.1 They require the test taker to offer subjective responses to stimuli, which are then analysed by a psychologist.1 The Rorschach inkblot test is a well-known example of this type of test.
Objective personality tests have a wider range of applications and are particularly suited to workplace settings. They require participants to take a standardized test and answer various multiple-choice questions about themselves.1 Their answers are then scored against an objective classification system.1
Conceptual Framework
The second thing employers must know when asking “how do personality tests work?” is what a conceptual framework is. Put simply, it is the accepted theory by which a tool is developed.2 A sound conceptual framework gives psychologists a better chance of developing a tool that works. It is parallel to how a scientist developing tools to mitigate climate change must first have a firm understanding of, and work within the context of, Earth’s atmosphere. If the first step is flubbed, then what follows is bound to be bunk.
Most personality tests follow one of three conceptual frameworks. The first and arguably most well-known of these is psychodynamic theory. Inspired by clinical psychology, it attempts to explain the origins of psychopathology.2 It assumes that everyone is somewhat neurotic and attempts to identify the source of participants’ neuroses.2 Since neuroses are evenly distributed across the population instead of universal, this conceptual framework is now mostly defunct.2
The next conceptual framework is trait theory, which is currently the most widely used framework in personality science. It divides personality into recurring behavioural tendencies and tries to measure these traits based on the five-factor model.2 Participants listen to or read a question, compare the question with their self-view, and then self-report based on that comparison.2 While the five-factor model is useful, newer lower-order variables have been shown to predict outcomes more accurately.2 Another issue is that this model assumes that self-reports are a trustworthy measurement of what participants actually feel, think, or do.2 This assumption is shaky because self-reports are likely skewed to serve the interests of the participant, especially when they are aware that their answers will be used for a hiring decision.2 Moreover, this approach assumes the disposition to introspect about one’s actions is universal rather than distributed throughout the population.2
The last conceptual framework we will cover is socioanalytic theory, which is the bedrock of Hogan Assessment Systems. The theory posits that all people live and work within groups and that those groups are structured in status hierarchies.3 This suggests three primary motives in life: getting along with other people, achieving status, and finding meaning.3 People solve these problems during their careers and individual differences in personality drives their career success or failure.3 This framework is rooted in pragmatism and does not view participant data as self-reporting but rather as self-presentation.3 Instead of trying to predict why participants say the things they do or whether they are true, this framework simply focuses on how responses will predict behavior and performance.3
Scoring
The third thing employers must know when asking “how do personality tests work?” is how a personality test scores data. Because each personality test follows a slightly different process depending on its conceptual framework, we will focus on Hogan’s approach to scoring. Per socioanalytic theory, we differentiate between participants’ identity and reputation.4 While identity is how participants see themselves and cannot be scientifically measured, reputation is how others see them and can be measured.4 Therefore, we score participant’s data from the view of the observer to gauge their reputation.4
This scoring approach means that when we are scoring a participant’s answer to the question “I read 10 books a year,” we are not actually looking at if the participant identifies as a reader.4 Rather, we are asking if the participant self-represents as a reader to observers.4 How participants answer this question will translate into reputational data about how observers perceive the participant, and this has career implications.4 We can predict the participant’s reputational status because we have reputational data for millions of participants in local and global populations that have already answered these questions in similar ways.4 These are called norms and reveal how participants compare to the local and global population.4
In a nutshell, our scoring system focuses on measuring reputation by asking people about themselves and finding the statistical relationship between matching groups and observer descriptions.4 By organizing reputational descriptors, the scores predict tendencies and indicate likely workplace reputation.4 It is important to keep in mind that these tendencies indicate probability, not certainty, which brings us to our next point.
Validity and Reliability
The fourth, and arguably most important, thing employers asking “how do personality tests work?” must know is how tools confirm that they are accurately and consistently assessing the things they claim to. Personality tests can and do claim to assess all sorts of things, but only the ones with published evidence of validity and reliability can back up their claims. This is important for more than just predicting performance — it’s also critical to ensure your organization can make equitable employment decisions. Well-validated and reliable personality tests do not discriminate on age, gender, race, sexual orientation, religion, or ethnicity, but unvalidated and unreliable tests have the potential for adverse impact, so it is essential that employers check the validity and reliability of personality tests when shopping around.4
Validity comes in different forms but the most consequential for personality testing is predictive validity, which ensures that the test measures what it’s supposed to measure.5 Predictive validity is typically measured with a coefficient between -1 and 1, which is called the Pearson correlation coefficient.5 The closer to 1, the more predictive the assessment. Hogan’s validity coefficient in predicting workplace performance is .54. This means that Hogan’s assessments predict 29% of variance in applicant’s future job performance. To compare, ibuprofen’s validity coefficient in treating headaches is only .14. This means that usage of ibuprofen explains merely 2% of differences in pain relief symptoms versus a placebo.5
There are many ways to determine the predictive validity of personality tests. At Hogan, we take a multipronged approach and validate our tests against external data on job performance and peer ratings.4 The results clearly demonstrate that our tools accurately predict participants’ performance outcomes.4 Even though it is expensive and time-consuming, we continuously evaluate our personality tests against external job performance data. In contrast, tests such as the Myers-Briggs Type Inventory (MBTI) and Enneagram have low or unknown validity for predicting workplace performance.4,5
Reliability is equally important because it certifies that the test’s results aren’t a fluke and that the test will measure the same thing each time.4 Reliability can be measured through internal consistency and test-retest reliability.4 Reliable personality tests contain multiple questions measuring the same idea, or construct, to ensure that it has been accurately measured.4 When these questions score similarly to each other, the test has high internal consistency reliability, a key metric of a personality test’s quality that is measured with a coefficient called Cronbach’s alpha.4 Test-retest reliability is measured by using the Pearson correlation coefficient to compare test takers’ scores from two separate testing periods.4 Personality is stable and unchanging, so scores that are closer to 1 have high test-retest reliability.4
In addition to looking for published evidence of validity and reliability, employers should also verify that a personality test has been evaluated by an unbiased party, such as a psychological council or association.4 Appearance in peer-reviewed academic journals is also a good indicator of a tool’s quality.4 As part of peer review, panels of impartial experts conduct reviews of manuscripts to ensure scientific accuracy.4
Global Applications
Depending on the nature of the organization, a final consideration employers should look at when asking “how do personality tests work?” is a tool’s global application. The goal of personality assessment is to measure human universals that cross geographic boundaries.4 This means that personality tests must be carefully translated by fluent speakers in a way that is sensitive to the nuances of language and adapted to local cultures and norms.4 Studies show that we are more alike than we are different, so a valid and reliable tool that is carefully translated and measured against local norms should be highly predictive. 4
Well-Constructed Personality Tests Do Work
So, how do personality tests work? Objective psychometric tools are powered by strong theoretical foundations, a sound scoring system, proven validity and reliability, and a capability to measure human universals across geographic boundaries. Equipped with this savvy, employers will have no problem finding a well-constructed personality test to improve their talent acquisition and development processes.
