The honesty test for a grade — how we score it

Late in 2024, after a few too many small disputes about grades, we sat down as a team and wrote out something we'd been doing implicitly for years. We called it the honesty test, and it now hangs printed on the wall above the grading benches.

The test

The honesty test for a grade has three questions. Each question gets a yes or no. A grade only ships if all three are yes.

1. Would I be comfortable buying this box at this grade if I were the buyer, sight-unseen?
2. If this box arrives at the customer's dock, will the buyer's dock manager say "yep, that's a B" without hesitation?
3. If we had to grade this box again in three months under different lighting, would we come up with the same grade?

If any of the three is a no, we either grade down or send to refinishing.

Why we wrote it down

The team had been operating on these principles for years, but they'd been informal. New hires sometimes drifted toward more optimistic grading because they were trying to be helpful, or because they hadn't fully internalized that an optimistic grade is a future customer dispute.

Writing it down — and pinning it on the wall — made the principle explicit. New hires read it on day one. Experienced graders quote it back to each other when a grade is borderline.

The downstream effect

Our internal grading drift — the variance between graders on the same box — has been measurable and improving since the principle went up on the wall. We do periodic sample audits where two graders independently grade the same set of 50 boxes, and the agreement rate has gone from about 86% to about 93% in the year since the test was written down.

Customer-side disputes have also dropped, though that's harder to measure cleanly because dispute volume depends on a lot of other things. The trend is in the right direction.

The broader principle

Most teams have implicit standards. Writing them down doesn't change them — it makes them transmissible. The standards that work get reinforced. The ones that don't work become visible and can be revised. The standards that nobody can articulate keep changing silently as the team changes.

The honesty test for a grade is on the wall above the bench. The next standard we write down might be about something else. But the practice of writing things down — that, we're keeping.