Advance Rewards Submission
We are delighted to share that our first Advance submission for our Neural Net Optimizer Challenge has been made public today (round 111)!
The algorithm, Nova Prime FGD, is now open for the community to review.
Please see here: Advance Evidence Form.
The code submission that embodies the method described above: Code Submission.
You are invited to explore the evidence and code and prepare to cast your vote in the token-weighted vote on the submission’s eligibility for Advance rewards. Voting will open at the beginning of the next round (round 112) and remain open until the end of that round.
Optimizer for Neural Network Training
An optimization algorithm for neural network training is the engine behind modern artificial intelligence. It works by iteratively adjusting a model’s millions of parameters, guided by gradients, to minimize a loss function. The optimizer is the algorithm that decides how those adjustments are made.
Optimizers capture the dynamics of learning itself — how fast to move, when to slow down, and how to escape bad regions of the loss landscape. Built on variants of Stochastic Gradient Descent (SGD), they efficiently navigate extraordinarily high-dimensional, non-convex loss surfaces to find parameters that generalize well to unseen data.
Why It Matters
Training neural networks underpins virtually every modern AI system. Optimizers are crucial for:
- Training at Scale: They must handle millions to billions of parameters, navigating loss landscapes where brute-force methods are completely impractical.
- Powering Real-World AI: Optimizer speed and quality directly determines the cost, energy, and feasibility of training models for language understanding, autonomous driving, medical imaging, and scientific simulation.
Neural network training consumes entire data centers running continuously. Even marginal improvements in optimizer efficiency compound into enormous savings in compute, energy, and cost — and can unlock capabilities previously out of reach.
This submission represents a potential step forward in one of the most fundamental problems in computer science. Your participation in reviewing, discussing, and voting will help determine whether it qualifies for Advance Rewards. For more information on the Challenge, check out our technical paper