From the forums
Hi everyone! A couple big changes are coming soon: New/better/much-more data, a single new regression style target/tournament, a new correlation based scoring metric. All of this is scheduled to go live in two weeks on July 13th for round 168. Expect the new training data to be released in the next few days along with Richard's own notes and modeling tips. And finally, we will be holding a fireside chat next Tuesday July 2nd 9am PT to go over the new data/tournament, and review the work this quarter (reputation, retro bonus, compute, auto-staking). Looking forward to chatting with you all then! In the meantime, post your questions and feedback here: sli.do/numeraiSome drastic changes will come to the Numerai tournaments soon -- changes that are unknown at the moment. I was going to discuss the liveAUC data and how it correlates with the stock market a bit more -- as one way to outperform the average performance is to try to find external correlations and one way to make money from the tournaments is to stakes strategically. With the changes, it's unclear whether that information will be useful. Therefore, my short series on the Numerai tournament will be on hiatus for now, but I want to conclude with some suggestions to Numerai.
Numerai should consider setting its tournament up with a specific start and end date -- similar to many of Kaggle's contests. If after one tournament and Numerai wants to make changes, they can. If after one tournament and no changes are needed, then setup a new tournament with the same rules. Both new and old participants of Numerai are somewhat frustrated by these occasional and sometimes drastic changes, and that frustration can be tempered by adjusting expectations.
Take this current announcement as example, it was announced June 29th and will go into effect on July 13th. New participants recently started is annoyed that the past weeks of effort in doing good forecasts might be wasted. Old participants are annoyed because small and big changes keep coming up and require model tweaking.
Finally, having set tournaments allow Numerai to award prize money at the end of the tournament for top performers. As of now, to earn payoff participants need to put in their own money, and the standard payout is cap at 1:1. I.e., by buying 100NMR (about 700USD as of now), you might win 100NMR if your forecast is good. And as I have noted in a prior post, the average Numerai forecast is not good. The risk-reward ratio is bad, and it plays a role in the lack of stakes. This is also why Numerai has been forced to come up with other incentives to boost that 1:1 cap to 1.55:1. Having set tournament where at its conclusion addition prizes can be won will boost that cap further, and it can be setup to avoid the p1p or sybil attacks to a large extent.