Recruitment errors scorecard: common mistakes

Why scorecards still go wrong in practice

A scorecard is intended to make discussions shorter and decisions easier to justify. In practice, however, the opposite often happens: the scorecard becomes a mere form-filling exercise that gives the impression of objectivity, whilst the assessment is still based on impressions and isolated arguments.

If you hear hiring managers say “everything was fine” or “I’m not sure, because I can’t really explain it”, it’s rarely the candidate’s fault. It’s almost always a problem with the scorecard: the criteria are vague, not measurable or weighted incorrectly.

This article brings together the most common recruitment scorecard errors and how to solve them, without making the process more complicated than necessary.

The 8 most common scorecard mistakes (and how to fix them)

You’ll come across the mistakes listed below in virtually every sector, from manufacturing and engineering to sales and the maritime industry. If you recognise two or three of them, your scorecard is probably already due for a redesign.

1) Criteria that are, in fact, opinions

“Fits into the team”, “has a good vibe” or “comes across as mature” seem useful – until you have to score them. That’s when things become arbitrary, because everyone means something different.

Fix: Translate such concepts into observable behaviour within your context.

• “Fits in with the team” becomes: welcomes feedback, shares information upon transfer, seeks to resolve conflicts.
• “Strong communication skills” becomes: can explain an incident step by step, asks clarifying questions, checks understanding.

2) Everything is equally important

Many scorecards have 8–12 criteria, all of which carry equal weight. The result is that your team will still end up prioritising the most enjoyable conversation or the most convincing story, because the scoring system makes hardly any distinction between them.

Fix: Choose 2–3 key criteria that truly predict success and give them greater weight. For an operator, safety is often more important than “presentation”; for sales, discipline may carry more weight than charm.

3) Too many criteria, too little evidence

A long list gives the impression of thoroughness, but during an interview you have limited time. If you have to tick off 12 points, you’ll end up listening superficially and failing to note down enough concrete evidence.

Fix: Keep it short and to the point.

• 6–10 criteria is usually the maximum; 6–8 is often better.
• For each criterion: 1 set question or case study component through which you gather evidence.
• For each criterion: one field for “evidence in a single sentence”.

4) No clear scoring reference points (1 to 5 means nothing)

A scale without a description encourages responses such as “I felt it was a 4”. One interviewer might give a 3 as ‘satisfactory’, whilst another might give almost everyone a 4. In that case, the scores are not comparable.

Fix: For each criterion, write brief key points for 1, 3 and 5, tailored to the role. This is precisely what makes a scorecard useful.

Do you want a foundation that’s already proven to work in practice? Take a look at the approach Create a selection scorecard for hiring managers and adjust the anchors according to their function.

5) Criteria that you don’t really check

A classic scenario: “problem-solving skills” are listed on the assessment form, but no case study is presented and there are hardly any follow-up questions. As a result, you end up being assessed on your language skills or self-confidence, rather than on the skill itself.

Fix: Link each criterion to a measurement method. If you can’t test it, remove it.

• Vakskill: mini-assignment, portfolio, practical question, certificates.
• Behaviour: Use STAR questions, probing further until the outcome and your own role are clear.
• Contextfit: strict preconditions (shift work, travel distance, safety requirements) as “knock-out” criteria.

6) Enter scores after the interview

If you only assess the situation after the event, your brain fills in the gaps with emotion and the “halo effect” (one strong point colours the rest). The 60-minute conversation is then reduced to a few highlights.

Fix: Make notes immediately after each key question or after each section, using brief, factual notes. A maximum of one sentence per criterion is sufficient, provided that sentence is specific.

7) No pre-determined decision rule (the bar is adjusted accordingly)

Teams often say, “We’ll just see who turns up.” That sounds flexible, but it makes your process vulnerable: you end up having to justify, after the fact, why someone is or isn’t eligible, depending on availability or personal preference.

Fix: Agree in advance on what is non-negotiable and what can be worked out.

• Minimum score for safety or quality.
• Disqualifying factors: driving licence, certification, shift work, language proficiency (if that’s really necessary).
• What could a 3 represent if someone scores a 5 on key criteria?

8) No calibration between interviewers

Without a brief discussion to clarify, “3” means something different to everyone. This leads to arguments afterwards, not because people are being difficult, but because the scorecard is ambiguous.

Fix: Hold a short calibration session of 30–45 minutes for each job category. Have two sample answers marked and discuss the differences.

This ties in with the idea of a structured approach to the interview itself. In How to conduct a structured interview in 8 steps Can you see how scorecards and standard questions work together to eliminate noise?.

A practical checklist for fixing your scorecard

If you want to improve your score as early as tomorrow, use this checklist when reviewing your scorecard. The aim is not to increase the amount of paperwork, but to reduce the need for interpretation.

• Can anyone else understand your criteria? without an explanation?
• Does each criterion a standard question or case study that you use to gather evidence?
• Are there any anchors at 1/3/5? with observable behaviour?
• Are 2–3 key criteria given greater weighting? than the rest?
• Is there a decision rule? before the first candidate (knock-outs + minimum scores)?
• Is there scope for risks? and a 90-day area for attention, without the “gut feeling” overriding the score?

Why this also relates to fairness and privacy

A scorecard isn’t just a useful tool for speed. It also helps to explain the selection process more clearly and make it more consistent, which is important if you’re working with several interviewers or if candidates ask for feedback.

Please note that scorecards often contain recruitment data. Keep them in a safe place and do not retain them for longer than necessary. The Explanation from the Dutch Data Protection Authority regarding job application data provides guidance on handling and retention periods.

Bias does not disappear of its own accord simply because of a form, but structure does help. For background information on how to recognise and reduce unconscious bias in recruitment, see the NICE guidance on workplace equality and bias a useful, systematic resource.

A practical next step for hiring managers

Choose one role that is difficult to fill and tackle its scorecard first. Keep it concise, measurable and carefully weighted, and schedule a single brief calibration session with the interviewers involved.

If you’d like to discuss criteria, weighting and a scorecard tailored to your blue-collar, sales or maritime vacancies, FosFor can often help most quickly by taking a look at the work itself and at data from previous campaigns. Please feel free to get in touch if you’d like a scorecard that’s actually used in practice, rather than just being filled in.