Are Polymarket and Kalshi as reliable as they say? Not quite, study warns

Are prediction markets really the future of forecasting?

A new study by researchers Joshua Clinton and TzuFeng Huang at Vanderbilt University suggests that booming prediction markets aren’t nearly as accurate at predicting the future as their proponents claim.

Researchers examined 2,500 markets with $2.5 billion in volume across leading platforms such as Polymarket, Kalshi, and New Zealand-based PredictIt.

They defined the accuracy of a market by how closely a market’s odds lined up with what ended up happening — the better the match, the more accurate the market.

The conclusion?

Polymarket got only 67% of markets right, while Kalshi hit 78%, and PredictIt scored 93% accuracy.

Despite being the largest exchange, Polymarket “has the least” amount of accuracy, researchers wrote. Kalshi followed, and then PredictIt.

Other examples of inaccuracy the researchers identified as well, included, for instance, contracts for “Dem wins by 6% to 7%” and “GOP wins by 6% to 7%” — mutually exclusive outcomes — occasionally moved in the same direction simultaneously.

Prediction markets are exchanges where people buy and sell contracts that pay $1 if a future event occurs, with the contract’s price reflecting the market’s implied probability of that outcome.

They work by allowing traders to bet based on their information, beliefs, or analysis, theoretically aggregating diverse knowledge into a single price.

Proponents argue that because real money is at stake, prices should incorporate all available information and therefore offer a more accurate, real-time forecast of an outcome’s viability.

Booming prediction markets

Prediction markets have stormed onto the scene.

Kalshi announced on Tuesday that it has raised $1 billion at an $11 billion valuation — a massive leap for a company that launched barely four years ago.

The same day, the firm struck a deal with CNN and a day later with CNBC, putting real-time prediction data across their channels starting in 2026.

Polymarket, meanwhile, reached a $9 billion valuation in October. Just two weeks later, Bloomberg reported that the company was looking for funding at a $15 billion valuation.

All of this comes as the stakes get even higher.

Kalshi CEO Tarek Mansour recently said that his company’s long-term vision is to “financialise everything and create a tradable asset out of any difference in opinion.”

Concerns abound, however.

If prediction markets struggle to price presidential elections efficiently — among the most information-rich events in the world — the prospect of financialising everything raises questions about what happens when markets expand to topics with even less available information.

Kalshi vs. Vanderbilt

Not everyone accepts the findings.

Jack Such, who leads media relations for Kalshi, told DL News that the study’s methodology “completely misunderstands prediction markets, and shouldn’t be taken seriously.”

The researchers’ methodology included analysing every political prediction market traded on Kalshi, PredictIt, and Polymarket during the final five weeks of the 2024 US election.

They measure accuracy using industry-standard forecasting metrics such as log-loss and Brier scores, which penalise markets that are confidently wrong and reward well-calibrated probabilities.

Such argues that accuracy should be measured by calibration: whether markets at 20% odds resolve “yes” 20% of the time, at 70% resolve “yes” 70% of the time, and so on.

“When measuring calibration, the true way to measure accuracy, Kalshi, is almost perfectly accurate,” said Such, pointing to the company’s open source data as a source of truth.

But the researchers say the critique misses the point entirely.

“Although many jumped on this ‘accuracy’ point, I think the point of our paper was not really about accuracy at all,” Clinton told DL News.

The real finding, Clinton reckons, is that similar markets — like the presidential winner-take-all market — were “not only consistently priced differently, but also that the changes in daily closing prices were largely unrelated.”

This suggests that market activity was based on “within-market pricing dynamics rather than a reaction of traders to new political information,” Clinton said. “A lot of the volatility was due to within-market actions and reactions.”

In other words, traders weren’t reacting to political reality.

They were reacting to each other.

The whale problem

Polymarket’s accuracy woes boil down to the firm’s basic structure.

Since the platform allows near-unlimited stakes with minimal friction, it attracts large, aggressive, and risk-seeking speculators — a “different class of traders” from PredictIt or Kalshi, according to the researchers.

Because the platform doesn’t cap positions, one player can move entire markets, producing prices that reflect individual beliefs rather than collective wisdom.

The researchers say Polymarket’s design — near-unlimited stakes, crypto-based access, and a high concentration of large traders — makes it more susceptible to outsized influence by single actors.

Polymarket did not reply to a request for comment from DL News.

Take the 2024 “French Whale.”

The trader, identified as Théo, placed $42 million in outstanding election bets across four different accounts, all riding on a Republican win. On just one bet, he stood to make $47 million if Trump won.

If he lost? Say bye-bye to $26 million.

At one point, Théo held more than 20% of all “yes” shares, expecting Trump to win the election.

For comparison, the largest Harris shareholder held only 7% of the company’s shares.

Negative correlations

Even for the same outcome, Polymarket’s daily price movements barely correlated with Kalshi or PredictIt.

When new political information emerged, Polymarket often didn’t react or reacted in contradictory ways.

Worse, 58% of Polymarket’s national presidential markets showed negative serial correlation. A price spike one day was typically reversed the next.

That’s a textbook signal of noise trading and overreaction, not informed forecasting, researchers said.

Inefficiency actually increased in the final two weeks before Election Day — when information was most abundant, and prices should have converged.

In the end, Polymarket and Kalshi “encourages herd behavior driven by visibility and hype rather than news,” the study concluded.

Pedro Solimano is DL News’ Buenos Aires-based markets correspondent. Got a tip? Email him at psolimano@dlnews.com.