Introducing Personality Tasks
Personality tasks provide a different way of assessing personality to personality questionnaires. Utilising recent advances in technology, personality assessment can be revolutionised so that one no longer needs to rely solely on questionnaires, which routinely suffer from distortion such as social desirability responding. (Birkeland et al 2006).
Objective Personality Tests or OPTs – builds on the research of Cattell (e.g. Cattell and Warburton 1967). OPTs rely on measuring task performance – but it is the way people complete each task, rather than how well they do on it, that reveals their personality traits. Participants are unaware of which aspects of their personality are being assessed. The tasks are typically short e.g. 5-12 minutes, completed online and resemble puzzles or challenges that are applicable to all ages and levels of seniority.
We present the results of our own research into the validation of multiple OPTs using self-report ratings of personality and biodata.
Personality assessment is widely used in organisational settings e.g. the Business Insider UK (2015) report that 80% of Fortune 500 companies use some form of personality assessment. There have also been many studies into the validity of personality scales in predicting a wide range of organisational behaviour e.g. Barrick & Mount (1991) and Hurtz & Donovan (2000).
Whilst nearly all personality assessment within organisational settings currently utilises self-report questionnaires, there has been a long history of concerns about the distortion associated with this type of assessment e.g. Mueller-Hanson et al (2003). Paulhus and Reid (1991) argue that distortion consists of 2 factors: self-deception (lack of self-awareness) and impression management (faking). Both are clearly relevant in organisational settings.
Many psychologists and HR professionals like to believe that the problem of distortion is not widespread, and in any case is largely controlled for e.g. via ipsative scaling or social desirability scales. However attempts at faking amongst job applicants in high stakes selection settings appear to be quite widespread. For example, Birkeland et al (2006) found in a meta-analysis of 33 studies that job applicants scored significantly higher than non-applicants on Extraversion, Emotional Stability, Conscientiousness and Openness to Experience.
However, given the validity of personality in predicting work performance, is there an alternative to questionnaires? Is a revolution in personality assessment possible? In this paper we argue yes to both these questions, and present reliability and validity data on an alternative form of personality assessment: Objective Personality Tests (OPTs).
OPTs were first proposed by Raymond Cattell (e.g. Cattell & Warburton, 1967) and in more recent years, researchers in Europe have designed OPTs for use in both academic research and organisational settings e.g. Ortner et al (2006). Please see Ortner & Schmitt (2014) for a recent review of the OPT field.
OPTs use “real time behaviour”. That is participants’ reactions to the tests are collected as they complete them and these are then objectively scored without the person being aware of what is being measured or even in many cases without being able to control or distort their responses. There is no self-report element, no questions to answer about oneself. Instead, a task is presented, and various measures of the participant’s behaviour are collected and scored for personality traits e.g. reaction times, memory performance, error rate, omissions, decisions made and so on. The approach makes extensive use of technology, and may involve an element of gamification. It is important to point out that no “ability testing” is involved – the tasks are typically quite straightforward to complete. Please see the Results section for examples of OPTs.
We have researched and developed a range of OPTs (under the brand name Mosaic) each of which measures a separate personality trait relevant to the world of work. In this paper we present early research into the reliability and construct validity of these OPTs. We prefer the term “tasks” to “tests” in the sense that each task is a short (typically less than 10 minute) activity completed online in the form of a puzzle or challenge. These do not rely heavily on gamification or video game material, and are likely to be acceptable to a wide range of employees. In practical terms, organisations will be able to use these OPTs to assess personality instead of having to rely on personality questionnaires and their attendant problem of distortion.
Are OPTs a reliable and valid alternative to self-report questionnaires as a measure of personality – that is, do they actually “work” in a psychological sense and measure the constructs they claim to? Could they be used in organisational settings to select and develop employees?
To date we have results from 74 volunteers who have completed 4 of our OPTs. The volunteers also completed a NEO personality questionnaire (ipip 300 item version) plus 2 specially designed biodata questionnaires, one measuring personality and one measuring risk taking behaviour.
In attempting to “validate” OPTs, one has the challenge presented by having to rely on the same self-report questionnaires that OPTs are aiming to improve on in order to validate the OPTs. Despite this we still expect to see modest correlations in the expected direction with relevant personality scales, although the size of the correlations may be limited by the problems of distortion (intentional or unintentional) in our research volunteers.
The biodata questionnaires, which ask for actual past examples of behaviour, are an attempt to get around the problems of validating OPTs using exclusively self-report measures. The OPTs themselves will be described in the Results section.
We examined the construct validity of each OPT via correlations with relevant NEO and Biodata scales.
Initial validity results for 4 of our OPTs (the Quality Inspector, Shapes / Shades, Colour Reactions, Investment Task) show that they appear to measure the ‘Big 5’ personality constructs of Conscientiousness, Openness to Experience and Neuroticism.
“The Quality Inspector” OPT (based on Hernandez et al 1999) is a simple checking task completed over multiple short rounds. The task is relatively easy, and participants simply have to spot errors amongst a number of objects, most of which do not contain any errors.
The “Shapes / Shades OPT (based on Lommen et al 2010) is a simple discrimination task. Participants are first introduced to shapes of various shades of white, grey and black on screen. They are instructed to avoid (by pressing a button) the shapes that are white or very light grey and to accept (by not pressing any button) shapes that are other shades of grey or black. They then practice the task before beginning the assessment proper. For the actual assessment, each shape they choose to accept earns them a point, and they accumulate points as the task progresses. However if they choose to accept a white or very light grey shape (which they have been instructed to avoid), they will lose the points they have accumulated so far. To increase the “pressure” of the task, some shade discriminations are very fine and in all cases only 4 seconds is given to make a discrimination decision.
The Colour Reactions OPT is based on the well-established Emotional Stroop paradigm, and performance on the task has been found to correlate significantly with Neuroticism and Anxiety (see a meta-analysis by Bar-Haim et al 2007). Participants are instructed to identify the type/font colour of a word as quickly as possible whilst ignoring it’s meaning e.g. TERROR, answer = purple. Neutral and anxiety-provoking words are used, and reaction times / error rates recorded.
The Investment Task is based on the work of Rubio et al (unpublished). Participants are asked to make investment decisions. They are given some fictitious money and a range of investment options with different returns and odds of success or failure. They are told that they are competing against other online participants, and that they need to achieve a certain “league table” rank to progress to the next round of the competition, 4 rounds in all.
18 separate behavioural measures from 3 OPTs combined to produce a measure of Big 5 Conscientiousness which correlated 0.63 with the NEO Big 5 Conscientiousness scale (self-report). The OPTs were also able to provide facet scores for Orderliness (correlated 0.54 with NEO Orderliness), Self-Discipline (correlated 0.54 with NEO Self-Discipline), Self-Efficacy (correlated 0.52 with NEO Self-Efficacy) and Achievement Striving (correlated 0.54 with NEO Achievement Striving).
4 separate behavioural measures from 1 OPT combined to measure Big 5 Neuroticism which correlated 0.41 with the NEO Big 5 Neuroticism scale (self-report). 2 separate behavioural measures from 1 OPT combined to measure Big 5 Openness to Experience which correlated 0.53 with the NEO Big 5 Openness to Experience scale (self-report). At this stage these Big 5 areas are covered by a single OPT only. We are developing more OPTs to cover these areas more fully, and also developing OPTs to cover the remaining aspects of personality not yet covered.
Because of potential problems with distortion on the NEO questionnaire because it is self-report, we don’t expect perfect correlations with the OPTs. Indeed, we found that the size of the gap between OPT scores and personality questionnaire correlated 0.52 with social desirability, suggesting that “faking” on the NEO questionnaire is partly responsible for any score differences between the OPTs and the questionnaires.
Whilst we have much more research planned, our results to date are already beginning to show that brief OPTs completed online can produce valid measures of personality scales relevant to the world of work. However at this stage our sample size is small. We also need to validate our OPTs against wider information than just volunteers’ self-report questionnaire data e.g. peer-ratings of personality.
OPTs are likely to be of considerable benefit to the practitioner and Human Resource teams alike. They are significantly more distortion-resistant than conventional personality questionnaires. Approximately half of our volunteers replied to a survey asking them to indicate what they thought our 4 OPTs were measuring – hardly any guessed correctly. For high-stakes selection settings in particular, but also for important personal development situations, OPTs may well be an important supplement to self-report questionnaires, or indeed a replacement altogether.
Where particular personality attributes are vital in the successful job applicant, OPTs can be employed as an efficient sift mechanism at early stages of selection. Further, the contrast between what an individual reports about their own personality and their OPT personality results may provide useful developmental insight.
The OPTs we have trialled so far show considerable promise in being able to validly measure aspects of personality. As the range of OPTs available expands, clients will be able to assemble their own configurations of OPTs to assess the areas of personality most relevant to them, or alternatively to use standard packages of OPTs that assess the full range of Big 5 Personality without having to rely on self-report questionnaires and their distortion problems.
Bar-Haim, Y., Lamy, D., Bakermans-Kranenberg, M.J., and Van IJzendoorn, M. H. (2007) “Threat-Related Attentional Bias in Anxious and Non-Anxious Individuals: A Meta-Analytic Study”. Psychological Bulletin 2007, Vol 133, No. 1, 1-24.
Barrick, M. R. and Mount, M. K. (1991). “The Big Five personality dimensions and job performance: a meta-analysis.” Personnel Psychology, 44 1-26.
Birkeland, S. A., Manson, T. A., Kisamore, J. L., Brannick, M. T. and Smith, M. A. (2006). “A Meta-Analytic Investigation of Job Applicant Faking on Personality Measures”. International Journal of Selection and Assessment 14 issue 4 317-335.
Cattell, R. B. and Warburton, F. W. (1967). “Objective Personality and Motivation Tests.” Urbana, University of Illinois Press.
Hernández, J.M., Santacreu, J. & Rubio, V.J. (1999). Evaluación de la personalidad: una alternativa teórico-metodológica. Escritos de Psicología, 3, 20-28.
Hurtz, G. M. and Donovan, J. J. (2000). “Personality and job performance: the Big Five revisited.” Journal of Applied Psychology 85 869-879.
Lommen, M.J.J., Engelhard, I.M., van den Hout, M.A. (2010). “Neuroticism and avoidance of ambiguous stimuli: Better safe than sorry?” Personality and Individual Differences, 49, 1001 to 1006.
Mueller-Hanson, R., Heggstad, E. D. and Thornton, G. C. (2003). “Faking and selection: Considering the use of personality from select-in and select-out perspectives.” Journal of Applied Psychology, 82 (2), 348 – 353.
Ortner, T. M., Kubinger, K. D., Schrott, A., Radinger, R. and Litzenberger, M. (2006). “Stress Resistance Assessment: Computerized objective test battery – German Version.” Frankfurt / M: Harcourt Assessment.
Ortner, T. M. and Schmitt, M. (2014). “Advances and continuing challenges in objective personality testing.” European Journal of Psychological Assessment 2014: Vol 30 (3) 163-168.
Paulhus, D. L. and Reid, D. B. (1991). “Enhancement and denial in socially desirable responding.” Journal of Personality and Social Psychology 60 307-317.