Equity Curve Lights and Best Performing Indicators
This has been long overdue—-but a necessary addition to the suite of tools required to manage strategies and systems. Please comment to indicate your interest. If the interest level is high enough, I will make this a regular addition along with the Livermore on a weekly basis for the time being.
Every Friday we will provide equity curve lights on the blog for several major indicators. They will be quite simple: red means stop, orange means you should take a small position, yellow means reduce exposure, green means you can take a full allocation. Furthermore we will also provide a list of the top performing indicators by category, as well as a category relative strength measure looking at both trend and mean reversion indicators.
Why Is This Important?
The rationale is simple: the best we can do is determine what has worked best in the recent past, and also in the intermediate term, and then apply a safety mechanism or “stoplight” to manage the strategy exposure. There is no magical metric that can tell you whether a system will continue to work in the future no matter how careful your measurements are. Performance evaluation is most certainly not an exact science–if it was then it would be a lot easier to predict which mutual funds or hedge funds perform best in the future using past performance (hint: most methods fail to achieve their goal out of sample) . What you need is something that that also indicates whether your initial hypothesis was correct in assessing performance and a means to adjust your position otherwise. Drawing concrete conclusions and persisting with these beliefs is a perilous path that many professors/practioners have taken. These same individuals preach about effects that were validated using extensive testing, and then they go out and start a mutual/hedge fund and everything miraculously stops working. They stubbornly protest and defend their studies when they should be focused on trying to determine whether they should stop using the strategy and proceed to trading another. You should not care what variables or what indicators happen to be the best, I would trade anything that is logical and and is working well, and I would stop using my favorite strategy if I felt that it was no longer relevant —the real goal is to maximize risk-adjusted return and achieving that goal is a matter of structured portfolio management and dynamic asset allocation .
Bringing System Testing Out of the Dark and Into the Future
Its about time we open our minds a little more and learn to change with the tide. Human beings are victims of the inherent need to have strong and persistent belief systems. In an ideal world most of us would slip into reliable routines and rituals that ensure progress. No one wants to consider the prospect of having to move jobs and/or houses every year any more than we want to change our methods for trading. Our secret burning desire for regularity and unquenchable thirst for too much information in the face of uncertainty makes the trading game difficult to succeed as a pro. Step 1 is that we all have to limit our enthusiasm and comfort with the 60-year+ backtests–what worked decades ago has virtually no relevance for short-term strategies, especially since commissions were in excess of 1% per trade each way back then. Nothing you could find would even be tradeable net of commissions–there is no free lunch– the structure of market created an illusion of profitable anomalies or effects. These long data histories are valuable primarily to display the ability to learn from data on a walk-forward test and thats pretty much it. Step 2 pertains to avoiding futile attempts at unconstrained mass parameter optimization without considering degrees of freedom or the false discovery rate. Even worse is the tendency to assume that such ridiculously over-fitted strategies will work for years to come after 5 minutes of optimization (I guess even worse than that is trading these “systems”). Step 3 pertains to becoming too exuberant about trying to make a strategy “robust” –we have to limit painful and meaningless exercises in trying to establish robustness by verifying that a short-term S&P500 strategy that must also work on the Euro or the Solar Industry ETF to be considered valid. Guess what, markets are different–even the same index traded using different instruments or time zones can behave very differently (great post here) http://davesbrain.blogs.com/mindmoneymarkets/2010/05/mean-reversion-etfs-timeframes.html. The only way to survive is to adapt to a particular market, and adapt to the times, and trade the best effects/anomalies you can and be willing to hop off the train when things stop working —anything less is asking for trouble.