Asset allocation receives much of the focus in the literature for optimization. Whether it is Minimum-Variance, Risk-Parity or Mean-Variance, we think of these tools as suitable primarily in the context of asset allocation. While these are key concepts to understand to be successful, they represent the tip of the iceberg in the quest for efficient alpha (arguably beta) generation.  The underlying logic and mathematics of these algorithms are behind many of the classic forecasting methods such as linear and non-linear regression. Since the goal of all trading should be to either efficiently react or predict the time series in question, the concepts of portfolio optimization should be useful at all levels of an intelligent trading machine.

What does that mean? Consider a host of simple “Level 1” examples: there are free parameters in any asset allocation optimization model, and in practice many of us will knowingly or unknowingly optimize these free parameters based on our backtesting observations. Why not let the same optimization tools be used for these purposes as well– certainly we don’t attempt to backfit the portfolio selection over time, so why try to do the same for say the lookback length of the free parameters of the asset allocation model. I could use Mean-Variance to find the best free parameters (eg 20-day ,60-day, 120-day lookbacks etc) and weight them accordingly. In a more counter-intuitive example I may wish to use Risk-Parity to create a “parameterless” model that distributes the risk of the different lookback lengths  for a Risk-Parity approach. In doing so, I create an arguably more unbiased and more robust portfolio. Taking this a step further, I could use Risk-Parity to allocate to different moving averages strategies for a single asset to create one composite and unbiased signal (or you could use mean-variance as well).

The latter example is akin to a form of “derivative optimization” where the goal is to efficiently use a set of derivatives -indicators on the underlying time series- to either create a forecast or trade the time series itself. From a system developer’s standpoint this sounds a lot like strategy allocation, and truthfully this is somewhat the case but there are nuances involved. Derivative optimization (my own terminology) considers the broadest range of alternatives and focuses on simultaneously synthesizing predictors for a time series. The input to derivative optimization is a set of sample parameters and indicators, and the output is a final predictor. In contrast strategy allocation often represents the tactical allocation to a collection of a set of “mined” strategies that have also been refined into the final form of a trading system. These systems can be either simple or very complex, and can possibly trade dozens of markets or potentially even one market in isolation. The latter is vastly more difficult to combine effectively since there are substantial non-linearities in complex strategies, plenty of data-mining bias, a greater tendency towards over-fitting , and more insidiously there is the impact of information loss. In contrast, derivative optimization offers the promise of being able to control for some of these weaknesses and give greater insight into the drivers of final performance.

The benefits of derivative optimization are substantial– one can theoretically improve every area of a given model by segregating it into the  unique process inputs that can become adaptive. At the limit, such a model becomes increasingly self-aware and can change or modify itself to increase the chances of performing well over time. An efficient model will operate as a parallel network and share information accordingly to accomodate the interaction between process inputs. It will also use the broadest range of ingredients and mathematical transforms/indicators possible to create unique predictors. Perhaps it can eventually perform the latter tasks on its own…….. The use of derivative optimization is a key step towards taking the “dumb” and “naaive” out of quantitative models and moving closer towards the “Watson” and “Big Blue” of the investment world.