How to align a panel of judges the day before the tournament

The judges' meeting before a tournament is usually a read-through of the rules and a list of reminders. It is useful, but it aligns nobody: everyone nods, and the next day each one draws the line where they always drew it.

Aligning a panel is something else. It means scoring the same performances together, seeing where they split, and agreeing on the borderline cases before there is a medal at stake. This guide suggests a session of about 90 minutes to do it.

Before the session

Fix the rulebook and the version

It sounds obvious and it is not. If half the panel learnt to score patterns starting from 10, and the tournament uses version 2026-1 of the ITF's rules, which starts at 6 and adds up to 4 at the end, the panel is split before it starts. Say it out loud: which rulebook, which version, and which local decisions the tournament makes.

Pick 8 to 12 videos

The selection makes the session. A good mix:

• Two or three clear ones, to warm up and so everyone sees they agree when it is easy.
• Four or five borderline ones, where an experienced panel splits: a hesitation that may or may not have reached 2 seconds, a kick that may have landed on the arm, a hand-and-foot combination.
• Two or three on one specific mistake you want to unify: the kihap, the return to the starting spot, the foot used to return, the difference between 2 and 3 points.

Real competition footage, from the table's angle if possible. If you have an expert judge's scoring for those videos, better still: that is your reference. If not, the reference will be the panel's median.

Set up the room

A projector or a large screen, a good connection and each judge's phone. If they will score on paper, printed score sheets, one per video.

The session, in six steps

1. Session rules (5 minutes)

The same as at the table: one pass, no pause, no rewind and no talking while the video runs. The rules already forbid corner umpires from talking during a performance. Here the reason is different: whoever speaks first drags the rest along, and then you are measuring nobody.

2. First round, blind (25 minutes)

Everyone scores each video at once, each on their phone or sheet. Nobody sees anybody's score until everyone has finished that video.

3. The reveal (per video)

Show all the scores together: the median, the spread and who landed furthest out. In sparring, how many picked red and how many blue.

4. Discuss only the disagreements (30 minutes)

Do not discuss the videos where everyone agreed: they teach nothing. Discuss the ones that split the panel, and discuss them at the exact second where they split. The useful question is not "what did you give it" but "what did you see at that moment, and why did you count it or not".

The goal is to leave with concrete decisions: "a hesitation like that, we deduct", "that kick does not score if it touches the forearm". Write them down.

5. Second round with new videos (15 minutes)

Three or four videos the panel has not seen, of the same kind as those that caused discussion. If the spread went down, the session worked. If not, the decisions in step 4 were not concrete enough.

6. Close (5 minutes)

Read the written decisions out loud. Keep the results of both rounds: they are the baseline for the next session.

What to do about the judge who drifts

There is always one. It is not a punishment or a show. It is information:

• If they drift always in the same direction, it is a bias. Knowing is often enough: it frequently corrects itself in the second round.
• If they drift in different directions, it is noise, and it is not fixed in an afternoon. Put them in a ring with steady judges and keep measuring after the tournament.

Individual results are for each judge and for the organiser. Showing them to the group by name turns a learning tool into a public trial, and next time people score so as not to look bad.

On paper or with a tool

The session works with score sheets. What costs is time: collecting, adding up and showing 12 scores per video, 10 times, eats half the session. And paper does not keep the second at which each judge tapped.

With a live scoring tool, each judge joins from their phone, scores aggregate on their own and the reveal is immediate. What you save on counting, you gain in discussion.

How it is done with an Anchor live class

On Anchor's web platform, the organiser builds a session with the chosen videos and opens a live class on it. Each session has one way of scoring, so patterns and sparring go in two classes. Judges join by scanning a QR code with their own phone, with nothing to install. If the class has an email list, only those called up get in: a QR code gets forwarded and photographed, a list does not.

The organiser controls playback, enables and locks the clickers and resets the whole room's counters. Results aggregate in real time on every screen, with the red or blue winner and the vote breakdown. Measured against the production server, 100 judges at two taps per second saw a p95 latency of 252 ms, so a whole panel fits with room to spare. Afterwards, attendance and results export to CSV.

Frequently asked questions

How long does it take to align a panel of judges?

About 90 minutes is enough for two scoring rounds and the discussion of the disagreements, with 8 to 12 videos.

Which videos should a calibration session use?

Real competition footage: a few clear ones, several borderline ones where judges tend to split, and some chosen for one specific mistake the tournament wants to unify.

Should each judge's results be shown to the group?

Better not by name. Show the group the spread and the median. The individual result is for each judge and for the organiser.

Do judges need to install anything for an Anchor live class?

No. They join from their phone's browser by scanning a QR code, and the organiser can restrict the class to an email list.