The Sigma Update

Protocol Enhancements
1

The Sigma update is the next step in the evolution of the protocol. It incorporates many of the improvements we’ve identified internally as well as recommendations from the community.

For Benchmarkers → A reworked deposit system:

  • TIG’s deposits are now integrated solely within the Optimizable Proof-of-Work mechanism, and are no longer part of cutoff.
  • Deposits are split into self-deposit and delegated-deposit, giving benchmarkers more control and strategic options.

For Innovators → A more realistic and flexible framework to develop algorithms.

  • Innovators now control algorithm inputs through hyperparameters.
  • Innovators manage an algorithm’s output in case of running out of fuel or the nonce crashing.

In addition, this update will include the introduction of minimum and maximum frontiers, as well as, the hiding of baseline related fields and verify_solution from algorithms. These major enhancements will bring TIG in line with real-world development.

The following post outlines each of these six key changes in more detail:

  1. Cutoff.
  2. Separating Deposit Factors.
  3. Hyperparameter Inputs.
  4. Saved Solution Output.
  5. Minimum and Maximum Frontiers.
  6. Hiding Baseline Fields and Verify Solution.

1. Cutoff

Cutoff will no longer be linked to deposit. This prevents the protocol from becoming a proof of stake and this lowers the barrier to entry for benchmarkers.

Each benchmarker i is attributed a cutoff, denoted \text{cutoff}_{i}. Benchmarkers are capped by their \text{cutoff}_{i}, that is benchmarker i can’t have more than \text{cutoff}_{i} qualifiers for challenge x. A benchmarker’s cutoff is recalculated every block:

\text{cutoff}_{i} = 1.2 \times \min_{x} ( \text{num_solutions}_{i,x})

where \text{num_solutions}_{i,x} is the number of active solutions benchmarker i has for challenge x.

2. Separating Deposit Factors

This update will introduce two separate deposit factors, a self-deposit factor and delegated deposit factor. Both self‑deposit and delegated‑deposit will now feed into the influence metric, so benchmarkers must balance their deposits as well as their challenge output.

Because the deposit required is determined by the market rather than synthetically fixed, there’s no mandated minimum deposit. Moreover, since we weight the deposit factors as well as cap them we ensure that Proof of Work remains the dominant anti Sybil mechanism.

Details on how deposit factors are derived from TIG deposits are explained in detail in this forum post.

3. Hyperparameter Inputs

This update redefines how hyperparameters are handled in innovator submissions. Hyperparameters are tuning parameters for a given algorithm and will now be set as inputs for TIG algorithms.

Currently, innovators often submit multiple versions of essentially the same algorithm, one tuned to return high quality solutions say and another optimised for speed, even when the only difference is a change in the chosen values of the hyperparameters. This duplicates code and allows competing innovators to resubmit another’s algorithm with potentially just optimised choices for the hyperparameters rather than genuine innovative improvements.

Details of the Update

  • Innovators can now submit a single algorithm with variable hyperparameters. The base algorithm itself remains the same but hyperparameters, eg: the stopping condition: maximum number of iterations, are now variable.
  • Benchmarkers, when running the algorithm on a challenge, choose the hyperparameter input values.
  • All chosen hyperparameters are fully transparent to other benchmarkers.
  • Innovators may publish recommended hyperparameter values or ranges for particular problem-size regimes, but the ultimate choice lies with the benchmarker.

Benefits

  • Reduces unnecessary duplication in the system.
  • Encourages focus on genuine innovation, not trivial resubmissions.

4. Saved Solution Output

This update will introduce a save_solution function which innovators can call any number of times during the algorithm. If a nonce runs out of fuel or crashes, the last saved solution will be used as the solution, rather than returning nothing.

Currently, if an algorithm runs out of fuel before returning a full solution, it returns no value at all. This wastes partial progress.

With the new mechanism:

  • Innovators can save their best-known solution as frequently as wished during the runtime of their algorithm.
  • If evaluation halts prematurely, verify solution is called on the last saved solution rather than returning a non-solution for crashing or running out of fuel.

Innovators will need to incorporate this function into their algorithms to save their best-known solution periodically.

Formalisation

Assume the challenge is a maximisation problem, let f(s) denote the objective function. An algorithm generates a sequence of candidate solutions (s_1, s_2, \dots, s_t). At time period, t, chosen by the innovator, the save solution function keeps track of the current best-known solution:

s^*_t = \arg\max_{1 \leq j \leq t} f(s_j).

If execution stops, due to running out of fuel or the program crashing, at time T, the outputed solution is s^*_T. Verify solution is then called on s^*_T.

5. Minimum and Maximum Frontiers

We’re updating the current fixed minimum and maximum difficulty points to dynamic difficulty frontiers. These frontiers will be defined for each challenge based on testing results and algorithmic performance data.

  • A minimum frontier: Prevents trivial or spam submissions. A solution must exceed a minimal level of computational effort or improvement relative to a given threshold dependent on the difficulty parameters.
  • A maximum frontier: Prevents gaming by filtering for instances where the baseline algorithm performed exceptionally poor.

Following the implementation of difficulty frontiers and the save-solution capability, sub-instances will be removed from all challenges where they are currently used. An upcoming protocol update will focus on mitigating variance in the baseline algorithm.

6. Hiding Baseline Fields and Verify Solution.

This update will hide baseline related fields and verify_solution from algorithms. This is achieved by making those fields/functions private to the tig-challenge crate, i.e. inaccessible from tig-algorithms.

For TIG challenges which include a better_than_baseline parameter, this update will remove the better_than_baseline parameter as an input for algorithms.

In both literature and real-world applications, algorithms tend not to have access to the optimal solution or any metric that gives them an indication of how far from the optimal they are. This input allowed algorithms to shortcut complete exploration of the search space S. This input therefore artificially reduced realism, and could have unintentionally biased results toward algorithms that leveraged better_than_baseline as a tuning crutch.

This update will impact innovators who used better_than_baseline as a stopping condition. The goal is for hyperparameters and fuel to now dictate the stopping condition of an algorithm, aligning more with real world use cases.

1 Like