Preface: Optimizer’s Anonymous
There are some valuable concepts to be taken from the “trend-following” research. Whether its the “turtles,” Ed Seykota, Larry Hite, or Michael Covel, a key concept that is emphasized is “robustness.” If you are a hard-core mathematician type, you tend to scoff at the homely, grandfatherish advice being espoused by the trend-following crew. They spin simplistic tales of creating systems that work across markets, and finding strategies that work over long periods of time. They preach about fat tails, and the unknowable future. The fact is that these guys are really on to something. The field of robust statistics emphasizes a lot of these principles, and truthfully PhD trained economists, physicists and finance geeks tend to ignore them at their own peril. The who’s who in the history of hedge fund blow ups is littered endlessy with ivy league quant gurus. I dare you to show me a single hedge fund that blew up trading the 200-day moving average across asset classes.
The colorful introduction is meant to grab your attention, and shake you up. Deep down in the depths of human psychology we really crave certainty and consistency. Accepting that we can’t control the future is a battle we have to fight every day. Sadly the smarter you are the harder this is to accept. Tell me you haven’t fallen prey to the dreams of creating endless riches by optimizing your favorite indicator. I remember dark moments myself many moons ago trading e-minis dreaming of Freedom 35. I would change my indicator settings daily, desperately trying to find the magic setting that would prevent me from getting whipsawed nearly every day. When i started trying to design stock screens for the first time, i went crazy backtesting every possible combination. I was hoping to find the one combination of P/E ratios or EPS growth that would always win. If this was you, there is hope, and I would like to share with you my story. The first step in Optimizer’s Anonymous is recognizing that you have an optimizing problem!
Fortunately, I had a tremendous advantage over the rest of you: the software i learned on only kept 5 years of historical data and rolled forward every week. As I furiously tested away, I got to see how the screens that i built performed using a new start date. As time rolled on, i got to see how my screens performed in real life as well as their equity curves before and after. It was an incredibly valuable experience that everyone should try for at least a few years. I started realizing though the course of testing that the more specific my parameters were, the noisier they performed out of sample. This problem was made even worse as i expanded the number of parameters. Selecting a price-earnings ratio of 7 or lower was unstable enough, but it was made worse by choosing stocks with an ROE higher than 15, and earnings growth greater than 20%. This led me on the path of using percentiles which i have already become notorious for on this blog—and with my professor friends. Another advantage was that i always tested across sectors simultaneously, under the assumtion that the underlying screen as a concept was more important than the idiosyncracies of a given industry. I tested like this for years even before receiving a formal education, and having that backround allowed me to design models that actually worked in real-life.
Trading strategies should be designed based on concepts, relationships, and logical theories. A theory that is not generalizable is more likely to be valid that one that is highly specific. The greater the number of unique environments that a backtest can survive–the more likely it is to be robust. Building upon these observations are my prescriptions for Optimizer’s Anonymous:
1) Test a given technical trading rule first on at least 5 distinct instruments—try to incorporate commodities, bonds, stocks, countries, sectors, currencies etc.
2) Try to find the parameter that works best on the greatest number of instruments.
3) Map the parameter on a grid to find the broad area of highest and most consistent profitability (local density point)
4) Take the median or average setting
5) Test this setting on specific instruments that you haven’t tested
6) Test this setting out of sample, noting the new area of local density
7) Determine the degree of overlap between the intial local density and the area of local density out of sample.
8) If the overlap is too small you will need to broaden the parameter selection area out to include more combinations.
9) Modify and re-test