Skip to content

Market States: Regimes and Waves

May 12, 2010

There are many different types of market states–the longer term that are more aptly called “regimes”, and the short-term shifts in the market  that I call short-term “waves.”  Different types of waves can exist within the context of a regime. For example, we can have a brief wave of volatility bursts or a wave of “chop” within the context of a regime that is best described as an up trend with low volatility. Waves can be very brief–lasting from a few days to a week or more, while regimes often last a month and can last up to a full year. What I find interesting is that market states are always in the state of flux–one can even measure the likelihood of a regime or wave ending on the basis of historical precedent. This type of methodology is a part of what I like to call “state-based classification”, and the goal is to partition strategies according to their expected performance in both waves and regimes. Accurate state-based classification is extremely valuable, because if we know how a particular type of strategy behaves in a given market state we can “anticipate” how to adjust our portfolio allocation or position size. For example, it is well-known and logical to state that mean-reversion performs well during conditions of high volatility. In this case, if we can predict volatility, we can predict when to allocate more or less to a mean-reversion strategy.

Unlike conventional “swarm” or rotational strategies for adaptation–which reacts rather than anticipates—a clear sense of how to adjust for market states permits smoother transitions. You trade off one form of “error” for another: 1) “state adjustment error“:  Swarm adaptation will be pushed towards what is working well with a fair degree of lag, and thus will adjust to “waves” incorrectly since they are transient. however, it should be able to adjust to regimes correctly as long as they persist for longer than one month. 2)”strategy classification error“: in contrast “state-based classification” faces the risk of incorrectly identifying the strategies most likely to perform well during a given wave or regime. I like to call this “strategy classification error” since you are making a judgement based on historical performance in advance and the actual strategy may not perform according to expectations during a given regime. This type of error is unlikely to be made by swarm/momentum type strategies since they will move towards what is actually working.

Again, it is logical to combine both procedures to enhance results. Using a swarm or relative-strength algorithm to confirm expected strategy performance within a given regime (or wave) is the penultimate form of “hypothesis testing.” You build a theory based on historical testing of a given strategy in a given state, and then once that state is confirmed to be present, you observe to see if the strategy is performing according to expectations in “reality.” Using anticipatory state-based strategies take this to the next dimension where you are predicting what states are most likely to occur and then looking to get earlier evidence from strategy performance prior to confirmation that you are in a given regime/wave. You can even bet a small amount in advance, and wait for confirmation so to speak.

Even if you are not creating an adaptive algorithm, these concepts are highly valuable for discretionary trading. After all, an algorithm is often designed to capture how a good trader with the objectivity and tools of a scientist would make a given decision. Thinking about these topics is a little less abstract for me because they relate directly to my experiences playing high level  No-Limit Texas Holdem. Everything you do to be successful at a high level involves anticipation of how a given hand will play in a certain situation, a calculation of possible outcomes, betting in advance, and waiting for confirmation before going “all-in.” However, even for highly intelligent people—or perhaps most dangerously for intelligent people, they wish to trade or invest (and in poker too!) in some deterministic universe where the proper course of action is always clear-cut and logical. On the other hand the average person wants to always be right, and desperately seeks a formula that is reliable with an unreasonable degree of accuracy. They spend most of their time seeking new gurus or indicators that will solve their problems, and unfortunately tend to select what has worked best in a regime that is already past its prime. As a consequence, they are always behind the curve, and lose with an incredibly high frequency. Thinking about market states is a perilous but highly profitable path for the un-initiated. However, it is worthwhile considering how these concepts will inevitably affect your trading.

7 Comments leave one →
  1. May 12, 2010 7:35 am

    Nice post, taking regime in consideration not only gives better results during a particular regime, it helps me reduce my drawdowns significantly during the regime shifts as well.

  2. May 12, 2010 11:01 am

    “Using a swarm or relative-strength algorithm to confirm expected strategy performance within a given regime (or wave) is the penultimate form of ‘hypothesis testing.'”

    Wasn’t this illustrated in “Zone Report – Best Indicators: S&P 500, Oil, Gold”? “Indicator or Zone exit” had a better or equivalent DVR (or Sharpe) compared to “Indicator Exit”. The “Best” indicators are regime specific and the ARO matrix rotates into the top X strategies depending on the regime? I just want to make sure I’m connecting the blog dots correctly.

  3. Larry P. permalink
    May 12, 2010 9:16 pm

    David, I enjoy your posts and particularly the “Adaptive time machine”.

    Questions (1) Would Neural Nets, with some regular re-optimization perform the equivalent task of “adaptation of strategies”?

    (2) I never understood how you came up with 50 strategies – with runs of 5, and being able to go long and short all I can create are 32 possible five bar combinations (of up or down) and multiply by 2 (buy or sell) for 64 systems. You mention a “five by five grid” but I am afraid my lack of recent math and stats courses (pre-med stuff) have left me unable to decipher your meaning. Any chance you could draw it in crayon for me? (and what about bars that are “unchanged”??)

    Please continue to share your thoughts and if I am able to contribute in any way I’ll send it your way.

  4. May 13, 2010 12:53 pm

    I like the approach of classifying a market regime, and then applying a trade system best suited to that regime upon the market. This is where adaptive systems can come in handy, in my view. Not necessarily predicting the future price action, but more importantly identifying the current market state.

    Do you have some classification neural networks that you are using for this purpose? If a network can identify the number ‘6’ as the number six, it can perhaps do something similar in the world of identifying market regime ‘x’ as the regime x.

  5. prazor permalink
    May 14, 2010 4:12 am

    David, Thanks for sharing your ideas!

    Not sure how relevant this or not, but…

    One aspect when switching between strategies is the actual
    switching cost. This cost affects the overall performance of a strategy.
    You have a cost of not trading the strategy soon enough – missed opportunity.
    And there is cost of trading the strategy too long – lossed money.

    Both of these costs are almost never taken into account for when comparing
    and designing strategies and systems. We have a lot of benchmarks for performance
    and risk. But traditionally we don’t include switching costs.

    Can the switching cost be used as a target itself when designing strategies?

    Knowing, or rather quantifying, the swithing cost – can it help us select better
    strategies and can it help us manage them better?

  6. Carl permalink
    May 17, 2010 7:47 am

    With a state-change approach, you now have another expectancy statistic to collect which is the likelihood of going to state 1.1, 1.2, 1.3, 1.x from state 1.0. This would help in a probablistic allocation during the “transition” from one state to another.

  7. Carl permalink
    May 17, 2010 7:48 am

    I would add that you may discover a meta-regime based on how the state transitions statistics are clustered.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: