The 25% We Guess At: Inside the Character Column

Editor’s note, September 23, 2026: The counts here — 39 hand-graded races of 5,004, with 4,965 scoring nothing on character — described a smaller index. The 39 graded races are unchanged, but the index now holds 7,000, so 6,961 races score nothing on character rather than 4,965. On September 22 we found that RunSignup’s race search returns races in alphabetical order and stops at 5,000 results, so every race we had ever synced was named with a number or a letter from A to F. Searching state by state found 16,101 upcoming US races; the index now holds 7,000, the most our hosting can serve. The argument is unaffected: a character grade is still something we assign by hand, to almost nothing.

Three of the four columns in our composite score are arithmetic. Field size is a number someone published. Longevity is subtraction. Credentials are a checkbox a governing body ticked or didn’t.

The fourth column is character, a quarter of the weight, and it is the one where we are the ones making the call. This is the piece of the formula most worth being skeptical about, so here is exactly how thin it is.

4,965 races used to share one number

Character runs off an editorial scenery grade from 1 to 5. Until today, a race without one defaulted to the midpoint, which came out to 60 of 100.

That default was the single worst thing in our formula. Of the 5,004 races in our index, only 39 carry a hand-assigned grade. The other 4,965 were being handed a number that reflected no judgment about them at all. Because the other three columns are usually empty on a synced race too, that invented 60 was frequently the entire score. Thousands of races sat at an identical 15.0 out of 100, and every digit of it came from a grade nobody had given them.

So we removed it. A component we have not measured is now left out of the arithmetic entirely and the remaining weights are renormalized over what is actually known, rather than the gap being quietly filled with a midpoint. A race we have not graded now shows character as not measured, which is the true statement. Where that leaves too little measured to rank a race at all, the race is listed without a score instead of carrying a fake one.

There is a second problem, and it’s ours. The scale runs 1 to 5. Here is how the 39 grades actually break down:

Nothing has ever been graded below a 3. In practice we have been running a three-point scale used only to promote, never to demote. That is what grade inflation looks like when you write it down.

The six races graded a 3 are worth naming, because they are not being criticized: the California International Marathon, Houston Marathon, Mesa Marathon, Indy Mini-Marathon, Austin International Half, and Broad Street Run. They are fast, flat, superbly run, and built for a personal best rather than a view. A 3 is the scale saying so.

Where character stops being a judgment call

For trail races, it isn’t one. When we have an elevation figure, character becomes 60% scenery and 40% measured vertical gain, scaled against an 18,000-foot ceiling. Eight races in the index qualify:

Look at the last row. The JFK 50 is graded a 4 on scenery, which is above average, and it still scores the lowest character of any race we have measured, below all 21 of the fives and all 12 of the fours. Its measured vertical pulled it under. A 50-miler that drops off the Appalachian Trail onto a flat towpath is a genuinely different proposition from a mountain race, the elevation profile says so, and the column reports it.

That is the whole argument for measurement over taste, and it is the argument that killed the default. Evidence can score you down. A vibe never will: a midpoint handed out for free could only ever hold a race up.

What we did about it

We never hid the old default. It was in plain sight on every race page, and it was still wrong. “We would rather show a race at the midpoint than invent an opinion” sounds like restraint until you notice that the midpoint is the invented opinion: 60 out of 100, asserted about a race nobody looked at. On most of our index it was the whole score.

We didn’t replace the guess with another one. A race page now tells you which of the four components we measured and which we didn’t, the score rests only on the former, and a race with too little measured gets no score at all rather than a flattering one.

The rest of the fix is to measure more inputs instead of grading more races. Elevation data already does real work on the trail rankings; it should be doing the same on the road. Directors can claim a listing and put an elevation profile and a course description on the record for free. Under the one rule that never bends here, doing so cannot buy a better rank. It can only replace a blank with a fact.

For 99.2% of American races, character now says plainly that we don’t know: silence, rather than a number.

Rich Shea runs BestRaces. Spot something wrong? Tell me and I'll fix it.