Optimizer Demystified

Though you’ve already optimized your first strategy, you may still have open questions about the inner workings of the optimizer. How does it work? What is “population size”? How are the weights applied?

These questions, and many others, shall be answered on this page.

Basic Principles

Naxbot’s optimizer works by employing an algorithm named Simulated Annealing  - a probabilistic technique for approximating a global optimum of a given function (the function being your strategy). It works like this:

1. In the beginning, the optimizer generates a set of random configurations. The number of configurations generated is equal to your population size parameter. It generates that amount of configurations for every thread that is running, so the total number of configurations the optimizer operates on is $K_\text{PopulationSizePerThread} \times K_\text{NumThreads}$
2. For every iteration, the optimizer changes one thing about every configuration and generates a “mutation”.
3. It then compares this mutation’s backtest results to that of the original configuration.
4. If the new mutation performs better, it replaces the original configuration. If it performs worse, there is still a random chance that it will replace the original configuration regardless, though this chance decreases as the number of iterations increases.
5. Steps 2-4 are repeated until the number of iterations has reached $K_\text{TotalIterations}$

This all happens in parallel, depending on how many threads you’ve allocated to the optimizer. Since this algorithm doesn’t require data synchronization between threads (apart from keeping track of the overall best configuration), it can happen that some threads are completing tasks faster while others take longer, depending on what other programs are running on your machine and how your OS allocates CPU resources. This does not impact the optimizing process at all however.

Isn’t this just brute force?

No! With brute force, you would go through every set of possible configurations and pick the best one. However, in more complex strategies, this is simply not viable, as going through every single configuration could take literally billions of years with today’s computing power.

While there are some elements of randomness at play, especially in the beginning, over time the optimizer always converges toward an optimal solution (note: not the optimal solution). With brute force, you would have no guarantee of obtaining an optimal, or even a good solution.

Scoring

As mentioned before, for each iteration the optimizer compares the new configurations’ performance against the previous ones. How does it do that though? For this, it makes use of a score that is composed of:

• total profit
• hit rate
• target trade rate

The exact formula is: $0.01 \times (\frac{T_\text{Success}}{T_\text{Total}} W_\text{HitRate} + \text{Min}(\frac{T_\text{Total}}{N_\text{Markets} \times N_\text{AvgKlinesPerMarket} \times K_\text{TargetTradesPerKline}}, 1) W_\text{TradeAmount} + \frac{P_\text{Total}}{T_\text{Total} \times K_\text{RiskPercent}} W_\text{TotalProfit})$ where $T$ is the number of trades, $P$ is profit, $K$ is the input, and $W$ is the scoring weight.

Selecting appropriate scoring weights for your strategy to be optimized is crucial, as placing too much weight on any one element is very likely going to result in undesirable configurations.

For example, imagine tuning all weights except for $W_\text{HitRate}$ to 0. This will get you a configuration that results in an astronomically high hit rate. So why is this bad? Because, a strategy that makes 1 successful trade (and no other trades) in the entire history of crypto would achieve the maximum score according to those criteria.

The optimizer is like that annoying kid in primary school that always “technically” follows the teacher’s instructions when told off, but not to any reasonable degree of common sense. So, it is your responsibility to be as clear and concise in your instructions as possible, and leave no room for misinterpretation. If you do this, the results you can achieve with the optimizer will be phenomenal.

Finding the right weights

When composing a strategy, it is often not a given as to what the best scoring weights are. You will have to experiment. For this, it is usually a good idea to perform a few optimizer runs with smaller population sizes and fewer total iterations, to get a general idea of what kinds of configurations the optimizer is trying to present to you.

Once you’re somewhat confident you’ve dialed in your weights, you can begin incresaing population size and total iterations and perform a “proper” optimizer run.

Threads, Population Size, and Iterations

When it comes to threads, more is always better, up to the number of your CPU’s logical cores. Increasing the number of threads directly increases the number of computations Naxbot can perform in parallel in order to find the best configuration for your strategy at no additional time cost.

When it comes to population size, more is also generally better, but you do pay a penalty with regards to the amount of time it takes to complete an iteration. Remember that the total population size is $K_\text{PopulationSizePerThread} \times K_\text{NumThreads}$

Lastly, for iterations, the same holds true as for population size, where more is generally better, but it comes at a cost of increasing the time it takes to complete the run.

FAQ

When starting to optimize, the current best config changes quite often, but stops changing after a while. Why?

When just starting out, the bar is set quite low when it comes to accepting a new best config. For example, initially, the “score to beat” is 0, so literally any config will do. As the optimizer progresses and better configurations are found, these configurations also become increasingly harder to beat, which means that the best config is not going to update as often anymore.