A practical test for a technical vacancy is particularly worthwhile if the candidate needs to master a specific, critical skill from day one and you cannot properly assess that skill during an interview. Can someone learn that skill on the job, or is a shortcoming in that area not a serious issue? In that case, a trial assignment often takes more time than it yields in terms of information. A good case study or a structured interview is usually sufficient.
In this article, you’ll find out how to make that judgement. We’ll cover how to compare different test results fairly in a separate section in compare candidates fairly using different tests. The issue here is the decision that precedes it: do you use a practical test or not?
When does a practical test add value to a technical vacancy?
A practical test gives you something that an interview doesn’t: you get to see the work for yourself. A welder might say he produces neat welds, and a mechanic might say he systematically troubleshoots faults. In a test setup, you can see whether that’s actually the case.
The US Office of Personnel Management (OPM) describes in its explanation of work trials and simulations when such a test pays for itself. The return on investment is high if you need candidates who already possess specific, critical competencies upon joining the organisation. Can those competencies be learnt on the job, or are they not particularly critical? In that case, according to OPM, the return on investment is clearly lower.
Three conditions from the same source
OPM states that work trials are most suitable for roles where:
- the competencies assessed are key determinants of success in the role;
- the number of candidates you are testing is limited;
- but only a small proportion of the potential candidates are expected to have the necessary skills.
Please note: this is a US government source on recruitment in general, not specifically about technical roles in the Netherlands. However, the criteria are still useful as a framework for consideration.
What this means for technical roles
If you translate those conditions into technical terms, you end up with roles in which professional skill yields an immediately visible result and a mistake has consequences for safety or quality. Think of welding according to a specific procedure, connecting an electrical installation or setting up a machine. This is a line of reasoning, not a fixed rule: for each role, consider what could go wrong if someone has not yet mastered the skill.
Are you recruiting career changers or entry-level candidates whom you train yourself? In that case, a trial period mainly serves to identify what they are not yet capable of. Learning ability, motivation and safety awareness are often the better selection criteria in such situations.
How much does a practical test cost, and what are the risks involved?
A practical test is not a free extra step. OPM is clear on this: work trials can be costly to develop, in terms of both time and money, and they can be time-consuming and expensive to administer. Someone must be present to observe the performance and, in many cases, to assess it as well.
Weigh up the costs and risks before you make a decision:
- Development time. You can’t put together an assessment that really says anything about the role in an afternoon. You need materials, a task and assessment criteria.
- Maintenance. According to OPM, a work sample test sometimes needs to be updated if the work changes. If your team is working with new machines or systems, an old test setup will assess the wrong skill.
- Reviewers. One of the skilled workers on your team won’t be working during the test. Make sure to factor those hours in.
- Individual purchase. OPM notes that work trials are often conducted on an individual basis and are not always suitable for groups. When there are several candidates, the time required can quickly add up.
- An additional hurdle for the candidate. A technician who is already in employment often has to take time off work to sit a practical test. Whether this leads to candidates dropping out varies from case to case. Check within your own recruitment process whether candidates are more likely to drop out after receiving an invitation to the test than at other stages.
- False security. A test without fixed criteria is still just a subjective judgement, albeit with a workbench thrown in.
If you want to know where candidates in your recruitment process are dropping out, a candidate journey analysis to work that out before you add a step.
When is a case study question sufficient?
A case study question is sufficient if you’re primarily interested in seeing how someone thinks and makes decisions, rather than how they work with their hands. How does a mechanic tackle a recurring fault? What does he do if production needs to continue whilst he hears an unusual noise? You can clearly highlight this sort of work behaviour through a realistic scenario during the interview, without the need for a trial day. You can read about how to structure such questions in Designing case study questions for technicians in 8 steps.
| Situation | A case study question is often sufficient | Consider a pilot scheme |
|---|---|---|
| What would you like to see? | Approach, prioritisation, safety decisions, communication | Carrying out a specific professional procedure |
| Can it be learnt on the job? | Yes, you train them yourself | No, it has to go well right from day one |
| Consequence of a mistake | Minor or easily rectified | Significant risk to safety or quality |
| Number of candidates in the final round | It’s a bit of a stretch; testing everyone individually will be a real challenge | Small-scale; the test is easy to plan |
| Is the evidence already available? | Recent certificates or evidence of comparable work | Little other evidence of his skill |
This table is a tool to help you organise your thinking; it is not a measured distribution. If you’re unsure, start with a case study question and, after making a few assumptions, check whether you’ve missed anything that only a practical test would have revealed.
How do you combine a practical test with a structured interview?
The two methods complement each other, as they measure different things. OPM describes in the an explanation of structured interviews that such an interview can assess competencies that are difficult to measure using other methods, such as interpersonal skills. All candidates are asked the same pre-determined questions in the same order, and all answers are scored using the same assessment scale and criteria.
A practical test shows whether someone is capable of carrying out the work. The interview reveals how someone works with others, communicates and has dealt with situations in the past. Here’s how to combine them without duplicating effort:
- Set out the criteria in advance. For each criterion, determine which method is used to assess it. Professional competence in the test; teamwork and communication in the interview.
- Keep the test short and to the point. Test only the critical skill, not everything a person does in the role.
- Use the same bowl. Mark the test and interview on a single scale using fixed criteria, so that you can combine them later.
- Discuss the test during the interview. Ask why the candidate made certain choices. That will provide context for what you observed.
- Choose a logical order. It is best to schedule the practical test only for candidates who have performed well in the interview, so that you only use the most demanding stage where it is necessary.
If you don’t yet have a set interview format, start there. In How to conduct a structured interview in 8 steps Find out how.
Five questions to answer beforehand
Go through these questions with the hiring manager before including a practical test in your selection process:
- Which skill should someone have mastered by their first day at work, and why?
- What happens if someone hasn’t quite got the hang of that skill yet?
- Can we also assess that skill using a case study, a certificate or a reference?
- Who administers and marks the test, and how many hours does that take per candidate?
- How can we keep the extra step for the candidate to a minimum, for example by scheduling the test and interview on the same day?
If you are unable to give specific answers to the first two questions, a trial set-up is likely to be little more than a waste of time.
Would you like to review your shortlist for technical roles? FosFor handles the pre-selection and guidance, meaning that clients, on average, only interview the 2 to 4 best candidates per month. Find out how it works in our data-driven approach, or discuss your vacancy with us.