New Numerai tournament changes (October 23, 2019)


Numerai is implementing some significant tournament changes since its July iteration. While constant improvement is a good thing, it's not immediately clear (to me) what problems the new staking system is addressing. Here's the announcement from its Community forum:

October 22, 2019 3:39 PM. Exciting news everyone! We are improving the way staking works this next round (R183). In short: a single continous [sic] stake, daily scores and payouts based on average correlation across overlapping submissions. A new reputation bonus based on rank. And a new website redesign. No change in data or submissions. I have updated the docs here https://docs.numer.ai/tournament/learn to describe the new rules. I will followup with a changelog / migration doc tomorrow. New website will be up by Friday. As usual, feedback is welcome!

The new changes seem to have gotten rid of "confidence" in its staking mechanism -- a nod to the fact that confidence has been completely broken after Numerai's July changes. "Confidence" was meant to function like the "second-price" in a second-price auction -- meant to force logical stakers to reveal their beliefs about their models' forecast ability. In the earlier iterations of the tournament, Numerai pointed out that this will allow it to select the good models to guide its hedge fund trading. It's a shame that the recent changes have broken this mechanism. Other possible interpretations: 1) stakers are not logical, 2) stakers have no idea whether their models are any good and so the confidence information Numerai got is useless.

The only positive I can think of at the moment is its effect of reducing the volatility of the scoring correlation. One big issue with the current Numerai tournament is that the scoring correlation (that determines whether a staker gets a payout) is volatile from week to week. The volatility likely has an effect on discouraging stakers. If the new scoring correlation is less volatility, it might encourage staking.