Skip to content

Quick Thoughts on Drawdowns and The Importance of Understanding Return Distributions of Strategies

November 24, 2009

One of the most important components of a good system is to have a low drawdown relative to its average return. Most traders make a concerted effort to pursue strategies with low drawdowns. In fact, people are so focused on drawdowns, that it is often detrimental. To me one of the most important things when considering drawdowns is WHEN they occured. Drawdowns that typically start after periods of strong profitability are far more desirable than equivalent drawdowns that start during flat or even poor periods. Few traders consider this, and that is why few people pursue trend-following because of the high perceived drawdowns. What is not mentioned is that most of the time these drawdowns occur AFTER high profitability. In contrast, short term strategies can “spontaneously combust” especially if they are poorly conceived. Even worse is that fact that commissions and bid-ask spreads can crush these strategies to bits right from the start. You will be down 50% with no chance for recovery from the get-go even though the strategy curve had low drawdowns in retrospect.

The often irrational desire for ultra-low or no drawdowns is the main reason why the Madoff scandals of the world occur in the first place. No one seems to be able to resist a straight line equity curve even at the expense of lower returns. The straighter the equity curve the less questions people ask about how the returns were generated. Many strategies secretly contain non-linear payoffs such as risk arbitrage or selling options, or ponzi schemes. Another silent but deadly killer in this area are “overfitted” strategies with more than 3 rules and les than 20 trades. The reality of statistical significance is that there will ALWAYS be rule combinations that peformed phenomenally well with 100% successful trades. The chances of them repeating their performance in real life depends on how and when you did your testing. These strategies appear wonderful until they blow up and never recover.

The bottom line is, even if you see a perfect equity curve, always ask yourself why a strategy should continue to perform the same way in the future. Also ask yourself whether certain conditions would cause your strategy to have non-linear payoffs. Solid human judgement in this area is absolutely unavoidable.

Advertisements
8 Comments leave one →
  1. Alex permalink
    November 24, 2009 8:02 pm

    Word. Great point on drawdowns. Makes 1987 easier to live with in the backtesting world. 🙂 $20 says you’ve quantified this equity management rule. 100 trade ROE / 25 trade ROE? 100%, 50%, cash, -50%, -100% depending on the level?

    • david varadi permalink*
      November 24, 2009 8:10 pm

      hi alex, nice to hear from you. lol about 1987—interestingly enough the trend-followers didn’t suffer-and many thrived in that decline, while the short-term strategies based on selling puts etc got massacred. The difference is even if trend-followers lost during that period, structurally speaking they should recover since they have more constant risk. Nonlinear strategies that employ hidden leverage blow up and never recover in sudden market shocks which are bound to occur—so are traders that use “magical strategies” and take big risks because the backtests shows the system “never lost”.

      As for you other comment, I have found some robust methods that I will obviously keep to myself–although nothing can ever be perfect. A false discovery rate application is certainly important.

      cheers
      dv

  2. bgpl permalink
    November 25, 2009 1:02 am

    hi David,
    re. your comment about overfitting.. Is there a measure of overfitting ?
    For instance, your example about more than 3 rules with 20 trades does look like an overfit.
    But would (hypothetical) 25 rules with 3000 trades in the last 10 years still be an overfit ?
    Especially if the strategy were stable with respect to time frames..
    look forward to any thoughts, especially if there were a way to quantify this..

    • david varadi permalink*
      November 25, 2009 1:09 am

      hi badri, please look into the concept of “degrees of freedom.” as for the answer, there is no fixed concrete method, however there are some approximations and rules of thumb. with respect to 25 criteria, it is a well-known concept that the more you input into neural networks or regression, the better the “fit” will be. However you are effectively “super-fitting” the past and no longer generalizing to how the market is likely to behave. To me 4 criteria is about the limit that I would recommend for any system. I would further recommend that you choose 4 different yet complementary criteria, and try to avoid specific parameters where possible. Try sticking to averages, standard deviations or percentiles—ie DV2< 30 etc, and not DV2=56 etc.

      cheers
      dv

  3. November 25, 2009 8:19 am

    There is a bit in Ralph Vince book (that one: http://www.automated-trading-system.com/Handbook-Portfolio-Mathematics-Vince) that shows that higher volatility is a necessity to higher profits and that system drawdowns “can be” a good thing (ie as it is definitely necessary for high performance)… not to say they are good in and of themselves!

  4. bgpl permalink
    November 25, 2009 10:32 am

    hi David,

    thanks for the tips.. I will think about that and use it..
    Just to share and get your comments on what i have been doing:
    – a backgrounder: i find that very quickly 10 parameters seem to spring up.. for example if one wants to only account for short, medium, long term trend, short and medium term mean reversion, and for both-long and short, that is 10 parameters. Maybe it is not 10 degrees of freedom as some may be related. Which led to my initial question when i saw your note on 3 parameters..
    I wonder if trying to determine if there is an overfit using some criteria such as
    (a) number of trades / year as compared to the number of parameters
    (b) number of years over which the trades have been run
    (c) the simplicity of the individual concept for which there is a parameter
    (d) how the sharpe ratio varies from year to year (i.,e if the sharpe ratio itself remains within reasonable ranges)
    (e) control analysis such as for instance the ratio of wins to the number of trades in the last ‘n’ bars
    etc., (a few other things such as ratio of MAE/MFE *over time*)
    I am pretty sure others as doing this as well, so I just wondered if they could share..

    thanks for your insights !

    • david varadi permalink*
      November 25, 2009 1:06 pm

      hi badri and excellent list. what i was referring to was “screening” criteria. this could come in the form of: 1) stock is above 50 day average 2) RSI14<70 which I consider to be two parameters in this case. As for your comments, I will try in a future post to create a rough formula that people can apply. The variability of the Sharpe ratio itself is certainly a subject that few people talk about. I personally look at the variability of the DVR adjusted for the differences in returns expected under different volatility conditions. This avoids making incorrect conclusions.

      best
      dv

Trackbacks

  1. Wednesday links: Sharpe experience Abnormal Returns

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: