Skip to content

An Intermarket Ensemble Model for the S&P500 Using ETF Rewind’s “Sentiment Spreads”

September 19, 2010

David Varadi with Henry Bee

An “ensemble” is a term borrowed from the field of computational intelligence to describe the process of using multiple inputs or variables in combination to create a prediction, or a composite signal. Typically a good ensemble will be superior than any of the individual components, or at the very least be much more stable and robust out of sample. This ensemble approach is the most powerful and appropriate way of combining technical indicators or fundamental variables (or both). However, this method is even more important for intermarket analysis because of the instability and transient nature of such relationships. I decided that exploring intermarket analysis was a valuable exercise since most people focus on technical analysis and seasonality. However, I required a broad yet compact set of intermarket variables to work with. In browsing many of my key sources, one of the most interesting intermarket indicators that caught my eye as being potentially valuable was on ETF Rewind– a nightly product that I use which is a wealth of information and also contains several valuable tools. http://etfrewind.blogspot.com/ .

The indicator is called “sentiment spreads” and represents the sum of the difference in 1-day returns between 8 different pairs of markets. It also nicely captures a wide range of intermarket relationships (or variables) and thus was a compact set to work with for further analysis. There are only 7 spreads listed excluding the 8th spread- VXX-VXZ – due to insufficient historical data.  The indicator according to Jeff Pietsch (who is the creator of ETF Rewind) is most valuable for intraday-trading as an indicator that captures the market’s sentiment towards risk assets. A positive spread or positive differential return implies that the market is willing to take risk and thus likely to go higher. By extension, the more spreads that are positive, or the greater the sum of the spreads, the more likely the market will go up and vice versa. However, the spreads also happen to represent a good overall base input for creating an intermarket model to predict the S&P500. First lets look at how the spreads perform used as a composite indicator–in this example we include the 5-spreads that have the longest data history (the first 5 listed) used to predict the SPY exactly like the sentiment spreads indicator. We trade the SPY long if the sum of spreads is positive and short if the sum is negative.

As you can see, the base spread model outperforms the S&P500 (SPY) with higher risk-adjusted returns. Even though their intended use is not for predicting next day performance (it is best for day-trading), the indicator does a reasonable job nonetheless. One of the problems with the indicator is that it is static and does not change with market conditions. Since intermarket relationships change far more rapidly than technical relationships, it would make sense to incorporate something that can adjust to different regimes. Furthermore, the simple aggregation process itself is prone to error : 1) some spreads are more volatile than others and thus have more contribution weight even if this was unintended 2) some spreads will invariably be more important than others in determining the S&P direction 3) the spreads will vary in terms of correlation to one another during certain periods and thus redundancy can exist within using certain spread signals.  The natural question is whether using the model components with a dynamic allocation and an adaptive process could improve upon this result. This is a good task for the Kelly Optimzation model where the inputs are each spread as a trading system to trade the SPY and the final exposure is aggregated or summed just like the sentiment spread model. The new model represents a simple ensemble model and is updated each day and tested out-of-sample to demonstrate the ability to adapt over time.

As you can see the model does substantially better than the non-optimized version. The Spearman model that uses a non-parametric correlation is superior to the Pearson model which uses a linear correlation. Overall the results are quite good. In fact, the returns and risk-adjusted returns  rival many of the technical indicators tested on the blogosphere. What is important to point out is that a turnkey ensemble intermarket model such as the one presented is uncorrelated to say the DV2 or to RSI2.  Therefore it can add value within a larger composite model that takes multiple factors into account.  While I will leave this subject for another day, lets take a look at a shorter sample that includes more of the sentiment spreads to see if performance is improved by including more information. As it turns out this is in fact the case:

This shows that using the extra two pairs- quality spreads and the yen-euro relationship- increased returns as well as risk-adjusted returns. Recent performance was also improved. It stands to reason that including even more uncorrelated spreads would substantially improve the model as well as its robustness over time. Adding different time frames beyond just a 252-day lookback would also strengthen results. There are more complex methods that can be used such as boosting or bagging, but this is a very intuitive and less computationally-intensive approach that is arguably more robust.

18 Comments leave one →
  1. Mike permalink
    September 20, 2010 10:35 am

    This is really great work. If I understand Kelly optimization correctly, the weights that are returned can be far greater than 1. In these examples, are any of the calculated leverage amounts determined by the kelly optimization unreasonable large or small ie far greater (or less than) 1 or 2? Thanks a lot.

    • david varadi permalink*
      September 21, 2010 1:56 am

      hi mike thanks for the kind words. the actual kelly weights were not used in this case in the method you are suggesting—we used the net signal provided by multiplying the kelly weights times the suggested direction of each spread and then used a 100% allocation long/short to trade the spy. thus the model presented is unlevered.
      dv

  2. Ben B permalink
    September 20, 2010 4:57 pm

    Excellent post David. Only suggestions I would have are:

    1) For item #6, it might make sense to HYG/LQD or JNK/LQD rather than HYG/AGG. I say this because (a) AGG is already used in previous spread, (b) using LQD rather than AGG is “purer” because it limits to the corporate space, and there is no overlap (whereas there is overlap between HYG and AGG).

    • david varadi permalink*
      September 21, 2010 1:57 am

      hi ben, i completely agree with you—I preserved these spreads on account of them being used on REWIND. one added advantage is that there is more data history as well.
      good comment.
      best
      dv

  3. September 20, 2010 9:13 pm

    Hi David,

    Thank you for continuing my work, which was in turn built upon a series of posts by Dr. Steenbarger several years back. Here is additional reading on the topic for your readers:

    http://etfprophet.com/market-sentiment-system-revisited-part-i/

    Note that we have also built an intraday price target based on multiple spread regressions, here:

    http://etfprophet.com/mrkt_end-of-day-model/

    Perhaps I’ll use this post as good motivation to say more about that model. And, what can you tell us about improving performance yet again with weak learner methodologies?

    Best, Jeff

    • david varadi permalink*
      September 21, 2010 2:03 am

      hi jeff, yes these are excellent links and I am happy to highlight your good work (and the doctor’s). this model is of course distinct from what we are doing but useful nonetheless. i imagine the fit is substantially more accurate intraday. i would be pleased to hear your added comments. as for you last point,there is no question that there are superior methods to the simple model presented. however that said, this model is superior to regression for a variety of reasons due to the lack of constraints and nonlinearity that can exist as well as the fact that covariances are explicitly considered in the model. in fact there is an entire field dedicated to this task called “ensemble learning” and I will cover some key articles and concepts down the line—good suggestion.
      best
      dv

  4. September 20, 2010 9:14 pm

    (Oh, and Mr. Bee too!)

  5. kostas permalink
    September 21, 2010 12:45 am

    Hi David and Henry, any chance you guys can walk us through the Kelly optimization process application in a step by step fashion (excel?) as it pertains to any of your own examples? I think it would be tremendousl useful for the blog…
    Thanks for your creativity!
    K

    • david varadi permalink*
      September 21, 2010 1:58 am

      hi kostas, thanks for you kind words. i will put up a couple links but it is nearly impossible to walk through a portfolio allocation that includes covariance matrices on the blog. this cannot be done efficiently in excel but rather must be done in code. i suggest you use matlab or R to implement this.
      best
      dv

  6. Shivar permalink
    September 21, 2010 5:14 am

    Hi David,

    Very interesting post. The idea looks smart with a lot of room for improvement !
    Two questions:
    1) which covariance matrix are you considering? Covariance matrix of spreads themselves or covariance matrix of returns if each spread was used as a single predictor for S&P return?
    2) Kelly weights calculation requires a drift matrix. Which one are you using?

  7. September 21, 2010 10:01 am

    This sounds fascinating because I use intermarket analysis in my work. I’m having trouble understanding exactly what you are doing though. Are you trading each spread or just the SPY? It sounds like you are allocating a portion of the portfolio to each of the various spreads? Is this induction correct, so you could have up to several positions open at any time?

    • Aristotle permalink
      September 21, 2010 11:25 am

      He’s trading the SPY using the spreads as a predictor

  8. September 28, 2010 9:35 am

    Excellent, thought provoking post! Thank you. I’m looking forward to reading your other posts.

  9. Nicolas permalink
    October 14, 2010 5:06 am

    Hello David,
    First thks for the nice work done through your site
    Regarding this post, you could add the UUP / DBC pair to the ensemble.
    All the best
    Nicolas

    • david varadi permalink*
      October 15, 2010 12:17 pm

      thanks nicolas—good suggestion.
      best
      dv

  10. January 5, 2012 2:32 am

    Fantastic goods from you, man. An Intermarket Ensemble Model for the S&P500 Using ETF Rewind’s “Sentiment Spreads” CSS Analytics I have understand your stuff previous to and you are just extremely excellent. I actually like what you have acquired here, really like what you are stating and the way in which you say it. You make it entertaining and you still take care of to keep it sensible. I can not wait to read far more from you. This is actually a great An Intermarket Ensemble Model for the S&P500 Using ETF Rewind’s “Sentiment Spreads” CSS Analytics informations.

  11. January 28, 2012 3:49 am

    Wonderful goods from you, man. An Intermarket Ensemble Model for the S&P500 Using ETF Rewind’s “Sentiment Spreads” CSS Analytics I’ve understand your stuff previous to and you are just extremely excellent. I really like what you have acquired here, certainly like what you are stating and the way in which you say it. You make it entertaining and you still care for to keep it smart. I can not wait to read much more from you. This is actually a terrific An Intermarket Ensemble Model for the S&P500 Using ETF Rewind’s “Sentiment Spreads” CSS Analytics informations.

Trackbacks

  1. Market Rewind’s Sentiment Spreads Remix « Quantum Financier

Leave a comment