When Australia’s Kaylee McKeown touched the wall ahead of her US rivals in the 100 metre backstroke on 31 July, she secured herself back-to-back Olympic gold medals – the second woman to achieve the feat in the event. Australia’s sixth gold at the Games put the country third in the standings.
But not in America.
Unlike practically everywhere else in the world, the US likes to rank Olympic achievement by the total medal count – all the gold, silver and bronze accrued by a country’s athletes. The official Olympics website, Google and practically every publication covering the Games outside the USA goes by total gold medals.
So, yes, McKeown had won gold in her event, but the silver and bronze positions went to Regan Smith and Katharine Berkoff of the USA. By its own metrics, the USA roundly defeated Australia in the event, two medals to one. And those same metrics put the US on top after three days of competition – and saw it roundly mocked across social media and in the press.
“Congratulations to the US beating Australia 2 medals to 1 in the Women’s 100 metre backstroke,” one cheeky tweet read.
Assessing a country’s performance at the Olympics is a tough task, given the differences between nations. Focusing on gold medal counts or total medal counts has typically favoured the largest countries with the biggest populations and greatest resources. Since the Sydney Olympics in 2000, only the US and China – in the top three in the world for population and the top two countries by GDP – have topped the table.
At the other end of the spectrum is medals per capita, a method advantaging smaller nations. Take San Marino, the microstate within Italy. It has a population of about 33,000 and at the Tokyo Olympics in 2021, it won three medals. In a medals per capita ranking, it was clearly the winner, earning one medal per 11,000 people. The US finished on that table in 59th place, China 78th. For China to get the first spot in per capita rankings, it needed to win 126,763 medals – an impossible task, given the Olympics hands out fewer than 1,100 medals.
But there may be a better way than either of those methods: a more nuanced, statistically rigorous ranking system that doesn’t favour large or small countries, but instead works on probability. It was developed by two friends with a keen interest in marathon running: Robert C Duncan, a retired astrophysicist from the University of Texas, and Andrew Parece, a strategy consultant and vice-president of Charles River Associates in Boston, Massachusetts.
“I’ve always loved watching the Olympics,” says Duncan. “And I just thought, you know, there’s a way to make the whole thing more exciting.”
The Goldilocks model
Duncan and Parece published their ranking method in the Journal of Sport Analytics just before the Games kicked off. Officially, they call the method the “probability-adjusted national rankings”. The New York Times dubbed the creation the “Duncan-Parece model”, for obvious reasons. But because it strikes a balance in the rankings that favours neither large nor small countries, it’s more like the Goldilocks model.
This ranks countries according to how improbable their medal counts would be if all people in competing countries worldwide had equal propensity per capita for winning medals. Therefore, the expected number of medals a nation is expected to win scales with population size. For instance, because the US population is about 13 times larger than Australia’s, the US is expected to win 13 times more medals at the Games.
“The way I describe it is how many medals did you expect the country to win if the only thing you knew about the country was its population?” Parece says.
The reference model is used to rank nations by determining two things. First, how many medals a country is expected to win, and second, the improbability that it will win as many medals as it actually won. To determine the latter it uses a simple maths calculation known as “binomial probability” – the same calculation used for determining the chance of flipping heads five times in eight coin tosses. In short: the more improbable a nation’s medal wins, the higher it ranks.
The Tokyo Olympics provides an example.
At the end of those Games in 2021, the official tally had the US on top in both gold medal count, with 39, and total medals, with 113. China was next best, with 38 golds and 89 total.
But the Goldilocks model put Australia in first place. Cue Aussie, Aussie, Aussie!
How did Australia top the table? There were 1,080 medals awarded during the Tokyo Games and the total population of all medal-winning countries at that time was 7.23 billion. Australia’s population was 25.92 million. Plug these numbers into the Goldilocks calculator and Australia is expected to win 3.87 medals. It won 46.
This is considered an extremely improbable result for a nation of this size. Australia edged out Great Britain, the Netherlands, New Zealand and Hungary, while the US finished in sixth place. The US was expected to win 50 medals in Tokyo. It grabbed 113. This was also improbable based on the model – but not as improbable as Australia’s effort.
‘There’s no absolute correct method’
Throughout the Paris Olympics, Duncan and Parece have been providing daily updates to their ranking system at their website.
Host nation, France, had a dominant start. By day three, it topped the Goldilocks rankings, and it held on to that lead until Day 11, when Australia snatched first place.
“This is the right way to do it if you want to do it meaningfully,” Duncan says. “However we understand there are people that will like other methods better.”
China would probably opt to continue using gold-only counts as its ranking of choice – for one, Duncan notes, Chinese media has focused on this tally since Beijing, when the nation topped that table. With the Goldilocks model, it would be harder for China to land in the top 20 nations, having to earn more than 100 medals. The same applies to India, which also has about one-fifth of the world’s population.
David Frazier, a statistician at Monash University, says there’s nothing wrong with the methodology of the Duncan Parece ranking system, but the key assumption makes it unrealistic: not all countries will field athletes in every event and some nations will pump resources into specific events which can alter the underlying ability to medal in those sports.
“If we don’t account for this change in probabilities of winning across events, this will bias our calculations on the expected number of medals that the country is likely to win,” he says.
Different models have been proposed by scientists and sports enthusiasts over the years in an attempt to balance or explain the standings. Economic status, political status, team size and the level of cultural difference between the competitor country and the Games host nation have all been considered. The International Olympic Committee, however, takes no stance on which method is the best to use. Neither do Duncan and Parece.
“There’s no absolute correct method,” Parece says.
For the pair, this isn’t an exercise in trying to determine who won the Olympic Games; that would go against the spirit of the competition. Instead, it’s a mechanism to get people interested and excited. Mid-sized nations can turn to the ranking system and step out from the long shadow cast by China. They no longer have to accept the US sitting atop the medal table (in gold and all medal tallies).
“We want to make people happy and get people engaged everywhere,” Duncan says.
