The effects of medium and sequence on personality trait assessments in face-to-face and videoconference selection interviews: Implications for HR analytics

Recent years have witnessed an inexorable shift towards new technology adoption and data driven decision-making in human resource management (HRM), through Artificial Intelligence (AI or machine learning), transparently emerging experiences (augmented reality), and digital platforms such as 5G. Today, technology companies provide computer software to help identify, recruit, and manage employees. A fundamental issue concerns how new technology may distort data gathering and processing outcomes. This was particularly salient during the COVID-19 pandemic when online platforms and AI were used extensively in candidate selection, performance management, training and monitoring (Ryan, 2020).

Recent years have witnessed an inexorable shift towards new technology adoption and data driven decision-making in human resource management (HRM), through Artificial Intelligence (AI or machine learning), transparently emerging experiences (augmented reality), and digital platforms such as 5G)

AI and Personality Traits in HR selection

A key application of AI during candidate selection involves assessing candidates’ personality traits. Recent AI-driven, pre-hire analytics applications, such as HireVue, Cammio, Vima, Pymetrics, myInterview, and Retorio offer speedy and low-cost automated computer assessments of candidates using pre-recorded (asynchronous) video interview data. Candidates answer standardised questions within a given time frame delivered by a virtual interviewer. The visual and text data is then ‘fed’ into an AI system and matched with human analysis to train the machine learning algorithm. For instance, HireVue (2021) and Vima  (2021) analyze the reliability of their algorithms by correlating multiple human raters’ assessments with algorithm results. Beyond reducing costs and increasing the speed of assessment, the system is designed to minimize personal bias with the interview process. Although there are indications that AI has the potential to identify similar personality traits to humans using facial and text data collected via computer-mediated video interviews, an unresolved issue concerns the validity of the data source at the heart of recruitment algorithms. This is the case because machines lack specific human abilities such as intuition and context-based judgement. There have already been instances where the use of HR analytics fell short of desired outcomes and had to be supplemented with traditional methods of assessment involving human judgements. For example, Amazon’s pre-hire screening model matched gender data patterns in candidates’ CVs with gender data from the company’s hiring practices to consistently recommend male candidates as the preferred choice. However, the algorithm failed to recognize the industry’s history of male dominance.

Candidates answer standardised questions within a given time frame delivered by a virtual interviewer. The visual and text data is then ‘fed’ into an AI system and matched with human analysis to train the machine learning algorithm.

New communication technologies in HR selection: Risks and unintended consequences

This study contributes to address concerns related to the use of analytics for machine learning, especially whether new communication technologies may have unintended and possibly undesirable consequences on selection decisions. Specifically, we compare two common selection interview modes — Face-to-Face (FTF) and videoconference interviews (i.e., Skype) — and their sequence, to evaluate their impact on personality trait assessments. In order to investigate this topic, we evaluated selection interviews across different employment offers. Personality traits were assessed using the Big Five personality traits instrument—openness, conscientiousness, extraversion, agreeableness, and neuroticism (John & Srivastava, 1999).

This study contributes to debates concerning the use of analytics for machine learning, especially whether new communication technologies may have unintended and possibly undesirable consequences on selection decisions.

You cannot suddenly replace 300000 years of evolution: Our contributions

We contribute in four distinct ways to address concerns centered on the effects of new technology adoption in the employment selection process and its impact on HR analytics. First, we extend our understanding of media richness and media naturalness theories by showing that the relationship between media richness and communication outcomes may be spurious in the context of a selection interview. Second, this paper contributes to social presence and emotion regulation theories. We expand previous conceptual frameworks and practice by demonstrating the potential mediating role of technology affecting the capacity of individuals to regulate and interpret emotions in certain conditions, especially when behavioral expectations are present in a formal selection interview. Third, this study extends prior research on memory theory. We note that memories of prior interactions (interviews) influence evaluations of personality traits which, in turn, are moderated by videoconference technology. Fourth, our findings contribute to the literature on HR analytics by demonstrating the potential for new technology to impact the validity of data used for analysis and AI algorithms.

We expand previous conceptual frameworks and practice by demonstrating the potential mediating role of technology affecting the capacity of individuals to regulate and interpret emotions in certain conditions, especially when behavioral expectations are present in a formal selection interview.

Methodology

We used a 2 x 2 mixed model design with interview medium (face to face versus videoconference) as a between-subjects factor and interview medium sequence (commencing with either face to face or videoconference) as a within-subjects factor. The idea was to capture the effects of videoconference technology in the interviewers’ first impressions and consequent memories of candidates’ personality. Each candidate was interviewed twice. One group of candidates was interviewed via videoconference followed by a face-to-face interview while another group of candidates was interviewed in person followed by a 2nd videoconference interview.

Applications and beneficiaries

This study has several implications for HR management practices. Our findings point to significant distortions in judgements of candidates’ personalities, which can be attributed to the way videoconference technology interferes with human perceptions and memories of earlier encounters. Our data clearly suggest that distortion increases with videoconference. Our results, therefore, point to a preference for FTF followed by videoconference if a multi-stage approach is required. HRM selectors should also receive training on how to correct distortions associated with videoconference interviews. Another key issue is related to the growth of HR analytics solutions offering AI- enabled computer assessments of personality traits, relying on audio-visual asynchronous interview data. Thus, whenever possible, algorithms used to evaluate candidates using pre-recorded, asynchronous interviews should be developed from FTF interviews rather than a videoconference.

Reference to the research

Michelotti, M. , McColl, R. , Puncheva, P. , Clarke, R. , McNamara, T. (2021) The effects of medium and sequence on personality trait assessments in face-to-face and videoconference selection interviews: Implications for HR analytics. Human Resource Management Journal

Consult the research paper

Link to media

Puncheva, P. La perception d’un candidat par un recruteur est plus favorable quand la rencontre se tient en présentiel. Le Monde 16 February 2022.
Puncheva, P. Les chances de recrutement sont moindres si la seule video est utilisée. Entreprise & Carrières 1558 janvier 2022: p.12
Puncheva, P., McColl, R. Face-to-face versus online recruitment: which is the best scenario for the candidate?  AMBA online, 10/05/2022
McColl, R., Puncheva, P. Vorstellungsgespräch: Lieber persönlich als online. PT Magazin fur Wirtschaft und Gesellschaft, 20 July 2022