Validity and Utility of Psychometric Assessments
Why is the test not the risk. Misuse is.
A question we hear often is:
“Do psychometric assessments actually help us distinguish between high and low performers?”
But for most leaders, the real concern is sharper:
“Can we trust these tools to improve hiring decisions without increasing legal, performance, or reputational risk?”
That’s the right question, because psychometric assessments are not inherently good or bad.
They are only as useful as the role clarity, decision logic, and professional rigour around them.
If this is you, this article is for you
You’ll find this useful if:
Hiring feels noisy , too much “gut feel” and inconsistent interviewer opinions
You’ve made a costly mis-hire and want a more defensible selection process
You’re worried about fairness and compliance under South African employment law.
You want structured selection, but you don’t want to add bureaucracy that slows you down.
What the evidence actually says (in plain terms)
Concerns about whether tests can predict job performance are not new. This has been studied for more than a century.
A landmark meta-analysis by Schmidt & Hunter (1998) reviewed decades of research and showed that:
General mental ability measures are among the strongest single predictors of job performance across many roles
Personality and integrity measures can add predictive value when they are job-relevant and interpreted correctly
Structured methods consistently outperform informal selection approaches
The practical takeaway isn’t “tests guarantee performance.”
It’s this:
Evidence-based selection reduces noise, especially when combined with other structured methods.
A quick scenario that explains the real problem
A company hires a senior leader. On paper, they’re impressive. In interviews, they’re confident and polished.
Six months later, the organisation is dealing with:
Rising conflict between teams
Decisions slowing down
Escalating tension and turnover risk
“Performance issues” that are hard to pin down
At that point, leaders ask:
Was this a capability gap, a behavioural fit issue, or a role/structure design problem?
This is where psychometrics are often misused, not as part of diagnosis, but as a way to “explain” outcomes after the fact.
A good assessment doesn’t replace thinking.
It strengthens the accuracy of your thinking, if the system around it is sound.
Why “validity” isn’t enough
Many organisations stop at:
“Is the test valid and reliable?”
That’s necessary, but not sufficient.
Two additional questions matter just as much:
Is the test job-relevant for this specific role and context?
Is it being used in a way that is fair and defensible?
Validity ≠ fairness
A tool can be technically valid and still create unfair outcomes if:
It is applied without local evidence or appropriate norms
It is used as a blunt filter rather than a structured input
The role itself is poorly defined, making interpretation arbitrary
Fairness requires more than a good instrument.
It requires correct design, application, and governance.
“But candidates can fake these tests…”
This concern is common, especially with personality assessments.
In practice, deliberate faking is less effective than most people assume because:
Profiles are built from many interrelated items, making distortion hard to sustain
Candidates often don’t know what “profile” the organisation is looking for
Modern assessments include validity indicators and response consistency checks
More importantly:
Faking becomes a bigger problem when you treat psychometrics as a standalone gatekeeper.
When assessments are used as one input within a structured process (role clarity + structured interviews + evidence), the risk drops significantly.
The bottom-line question: does this materially affect performance?
Yes, because performance variance is expensive.
In most organisations:
Strong performers often produce significantly more output than poor performers
Leadership mis-hires create second-order costs: instability, rework, conflict, attrition, and leadership bandwidth loss
Even in operational roles, selection quality affects productivity, errors, and supervision load
The cost of a wrong hire is rarely “just the salary.”
It shows up as friction and friction compounds.
Why using a registered Industrial Psychologist matters (South Africa)
In South Africa, selection practices are governed by the Employment Equity Act (No. 55 of 1998). In practice, this means the employer must ensure that assessments used:
have been shown to be valid and reliable
can be applied fairly
are not biased against any individual or group
measure constructs that are demonstrably related to job performance
This is where untrained or informal use becomes risky.
For example:
Rejecting a candidate because of a personality trait without evidence that the trait is an inherent requirement of the job can be discriminatory.
In addition, administering and interpreting psychometric assessments is a psychological act and should be handled by appropriately registered professionals (e.g., HPCSA-registered psychologists/psychometrists), with proper ethical safeguards.
Common mistakes that make psychometrics fail
Psychometric assessments usually “fail” when organisations:
Use tests before clarifying what success in the role actually requires
Treat assessments as a shortcut instead of part of a system
Pick instruments based on popularity, not job relevance
Interpret results without professional competence
Rely on scores without integrating behavioural evidence from interviews and references
The issue is rarely the tool.
It’s the surrounding decision system.
What good looks like (without turning this into a checklist)
A defensible, high-utility assessment approach typically includes:
Role clarity first: clear outcomes, decision rights, and behavioural expectations
Evidence-based tools chosen for the role and context
Structured interviews that test for job-relevant behaviour
Integrated interpretation: assessments inform the decision, they don’t make it
Fairness and governance built in from the start
This is how you reduce mis-hires without turning hiring into bureaucracy.
A final reassurance
If psychometric assessments have felt confusing or disappointing in the past, that doesn’t mean they don’t work.
It usually means:
the role wasn’t clearly defined
the wrong tool was used for the job
results were interpreted without sufficient rigour
or assessments were asked to compensate for a system problem upstream
You don’t need to become a psychometrics expert.
You need a clearer, more defensible selection system.
The next right step
If hiring currently feels high-stakes, inconsistent, or legally risky, start here:
Clarify what “good performance” actually means in the role, then choose assessment inputs that match that reality.
If you want support, Ernestco offers a focused selection diagnostic to help you:
identify where your selection system is introducing risk or noise
confirm which assessment inputs are appropriate (and which are not)
strengthen fairness and defensibility without overcomplicating the process
No one-size-fits-all tools.
Just clearer decisions, made the right way.
