Universities should not use personality or ability tests in recruitment

NTNU is now preparing to appoint several new heads of department. Members of selection committees will have noticed that NTNU still allows, and in fact uses, personality and ability tests in recruitment.

According to NTNU’s brief information page on the topic, such tests are used “as a supplement to the traditional recruitment process, with interviews and reference checks.” Critiques of personality tests have been expressed before, but apparently they have not managed to convince many to abandon the practice. 

On the contrary, ‘personal qualities’ are now considered as relevant as qualifications for appointing new candidates, particularly in leadership roles. This makes the problem of how to assess those qualities more pressing.

The same NTNU page states that the purpose of these test is to “provide a broader factual basis when hiring new employees.” But can these tests provide any ‘factual basis’ about the applicants, and can their results be used to “predict work performance”? 

It is impossible to summarize specialist discussions in this short space. On balance, the literature suggests we should be skeptical. It is not difficult to develop measures or tests for almost any aspect of cognition or behavior. But once introduced, a new test or measure is not going to go away easily. 

It will be far easier to find correlations with other variables, like ‘work performance’, than to prove those effects are real and robust, let alone meaningful. Meanwhile, the test is among us, ready for use. That is the story of IQ tests, personality tests, and many others.

The personality tests that universities are buying and using rely on self-report: you basically describe yourself in the answers you give in the test. As Simine Vazire has put it, “personality tests can only tell you what you tell them”. That is why they are liable to manipulation. A new industry is flourishing, coaching applicants to impersonate the presumed ‘desired candidate’. One could say that an institution must assume the candidate’s honesty when taking a test.

But some applicants might feel legitimized to consider responding strategically, if they believe the institution is not transparent about how it will use data from such tests, or why the tests are being administered. This is how easily (and how soon) mutual trust can be undermined.

NTNU is not communicating clearly about how such tests may be used, or how they are used in its recruitment procedures: it does not communicate this clearly either to candidates or to committee members. There has not been enough debate about whether such tests should be adopted or how they should (not) be used. Are they used to determine whether an applicant has particular personality traits that are considered desirable for the position, or to find out the personality traits of those it intends to hire anyway? Is it acceptable to use mental ability and aptitude tests to appoint PhD candidates and postdocs?

Narratives are circulating about ‘the ideal candidate’, ‘the ideal leader’ etc. Personality tests could become a way to crystallize such narratives into procedures that can be applied to choose applicants according to certain desired traits, creating a bottleneck that could reduce diversity in our academic communities and among our leadership. 

To the extent that the tests lack scientific validity or lose it in their contexts of application, they can introduce more uncertainty and opacity in academic hirings, which are already notoriously ‘noisy’ processes. Academics hold each other to high standards of quality and integrity in their work, particularly in research. It is only fair that the methods we use to hire our colleagues are held to the same scientific and ethical standards.

The fact that heated debates still exist around these tests should make us pause. We have wisely agreed as a society that current AI models are ‘black boxes’: we pour data in, we get some answers, but we may not understand why we got those answers, and what goes on in the model that produces those answers.

Personality and ability tests are no different: we let the candidate’s responses in, we get a profile, but any intervening steps remain obscure. The rule ‘garbage in, garbage out’ applies to such tests, too: even if a test is valid, fake answers, or answers given casually in what some may experience as a boring task, will produce unusable results.

In reality, the test is one black box, which is embedded into a larger black box (we do not know much about the companies that provide these tests, apart from the lush profits they make), which is embedded into an even larger black box (how an institution uses in its hiring decisions the interpretations of test results that the company advises it to adopt). 

We should be as suspicious about these tests as we are of AI: both have the credentials of bad technological aids to decision making. We must also be critical of the thoughtless automation of hiring, whether that uses computer algorithms or human-administered tests.

Some Danish organizations are experimenting with a ‘dialogical reframing’ of personality tests. Regardless of how personality traits are identified, in tests, dialogues etc., the problem remains: that the results may be used to screen or select applicants based on preferred personality traits.

Personality and ability tests have become staples of recruitment in organizations, but they defy economic logic: they have high monetary and time costs, dubious benefits, and several likely harmful side effects. If personality tests are really only used to flag certain ‘extreme’ personality traits, what is the evidence that a test, which can be hacked by a malicious applicant, can achieve that goal better than aggregate human judgment?

A university is the people it recruits. NTNU has ‘science’ in its name and ‘knowledge for a better world’ as its motto. It would be an unfortunate slip, contradicting our values, if we let hiring decisions be affected by tools with a weak scientific basis and significant potential for misuse. There is still time for NTNU’s top to advise Faculties against the use of personality and ability tests in selecting the next generation of department heads and colleagues.