There is a shift toward more communication outside of the Numerai chat channels, and that's great. But more work is needed because information is scattered all over the place now. Some of these changes are highlighted in the Fireside Chat on April 14, 2020. You can find the Fireside Chat video on Youtube.
Here's some of the upcoming changes relevant for fellow data scientist participants.
More validation data
Starting on May 2, 2020, the tournament file will have more data to help validate your model better (or overfit even more). Information on that is on the Numerai forum here: https://forum.numer.ai/t/validation-2-announcement/166.
From what I can tell after checking the posted example file, no code change is needed. One thing that can go wrong though is if Numerai goes overboard and doubles the size of the datafile from the current 2.7GB size to 5.4GB. The last time they upped the filesize led to a lot of complaining and code crashing because many data scientists don't write efficient code to manage and use memory.
MMC roll out
Starting on May 31, 2020, MMC-based payout will be rolled out. (Note also that Numerai already got rid of the daily payout early April.) More information is on the Numeari forum here: https://forum.numer.ai/t/mmc-payout-details-and-analysis/220. You can also find some additional information here in a Google doc: https://docs.google.com/document/d/1o3-J8qFyo7aQQZ8BIja0Lmw7Gr81sPVYazq6v41-ENI/edit?usp=sharing.
I anticipate that MMC will eventually take over completely. (Or at least the use of rank correlation for reputation will be eliminated.) For now, you have the option to be ranked via MMC or rank correlation. Good for them to roll out changes slowly.
Leaderboard bonus gone
This is bad. Starting on September 9, 2020, the leaderboard bonus will be discontinued. This is mentioned in the same post with the MMC changes. You can see lots of complaints in the Numerai chat about this because everyone recognized this is a net negative for participants wishing to profit from the tournament.
One reason Numerai gives on moving away from the bonus is because of the p1p exploit. From the post, Numerai's words are:
"There are a few reasons why we are opting not to keep the leaderboard bonus in place permanently. A primary reason is simply that it is susceptible to some certain attacks in which a bad-actor could game the tournament and guarantee high payouts for himself with minimal risk."Given that this is a known exploit (people in Numerai chat commented about this when the current system is rolled out), draw me a Pikachu face and color it surprised when Numerai caught someone actually doing it and is using it as one reason to discontinue the leaderboard bonus. You can find more on that here: https://forum.numer.ai/t/leaderboard-bonus-exploit-uncovered/200.
The current payout system is not sustainable in the long run without a hard cap (as noted in my prior blog post on concerns about their payout system), and the exploit is a convenient reason for killing off the bonus. Numerai damaged its credibility when the bonus was rolled out and stated in the forum that the cap is there for safety only -- and that Numerai will adjust the cap upward as staking increases. Staking did increase and the cap was hit, and they are now coming up with reasons to justify getting rid of the leaderboard bonus now.
I think they can implement the cap and prorate the top X participants using a function based on staking and rank that guarantee the top X participants get something. But I don't know the ins and outs of the company, so there might be some business reasons to justify killing the leaderboard bonus for good and risking the potential consequences. What consequences? My guess is that we will see staking fall over time if this is the new steady state. The incentive to staked has changed for the worse for data scientists. Then again, I'm not sure how much a drop in participation matters for Numerai in terms of profitability and attracting investors. It might not matter at all.