Considerations When Making Predictions With Short-Term Factors
Our thoughts on time-frame integration are often helplessly binary, while prediction error rates are anything but. Most people think in terms of rules without understanding the “why” or specifically what they are trying to accomplish. A common example was reflected in the previous posts done by Enn showing the performance of a short-term indicator as moderated by a moving average rule. The real gem in those posts which many readers may have caught is that the longer you hold a trade in the direction of the trend whether you started out short or long and have the distribution dicates that eventually you will end up with a positive expectation. The more subtle implication is that if you use a short-term indicator to predict next day returns and you are wrong, you face a major penalty that increases with time while you wait for the signal to revert.
Effectively the decision to use a short-term indicator to forecast direction carries different penalties depending on other time frames. Thus, there is effectively a tradeoff we must consider: what is our marginal accuracy in being able to predict up versus down days in different situations, and also what our marginal average daily return benefit to doing so. In effect, what is most important is the return and winning percentage spread between up and down days as conditional upon different situations. Thus it is not important per se in an absolute sense whether and up or down day has a high expected return and winning percentage, but rather the conditional effect that a given variable has in being able to separate the two. Going one step further, we would also test any variable’s effect on up or down day performance in relation to up or down day performance in isolation. Thus assuming we wanted to test a variable’s effect on daily follow-through, it would have to pass the null hypothesis test that the variable does not moderate daily follow-through in a statistically significant way. This prevents us from considering more specific conditions or situations that take away from our statistical power and therefore add a high degree of noise. This type of offense is highly prevalent in trading system design–multiple rules are taken in sequence on the basis of viewing an increase in performance without knowing the marginal impact in statistical power versus a simpler version.
Getting back to the subject at hand, there is a quick way to investigate whether it is worthwhile using reverse follow-through as a means of market prediction in different situations. Because this is a “thought” article, I have not gone to the time/trouble of running statistical tests–however, one simple method you can employ in this situation is to look at the spread between the winning percentage and the average daily return in different environments as defined by a given variable. Below is a table that shows the next day spread depending on whether the 50 day moving average is above the 200 day moving average on the SPY. As you can clearly see the spread in both winning percentage and average daily returns is much lower when the 50 day moving average is above the 200 day moving average. In fact, at an absolute level, returns are positive and w% is also fairly high following up days when the trend is up. Since the spread appears statistically insignificant, this begs the obvious question as to why you would want to bother including reverse follow through as a factor in a prediction model at all in this environment. Not only is the next day spread seemingly irrelevant, but as we mentioned before–the penalty is severe and increases as a function of time. Thus we face the prospect of compounding errors in such a situation. This does not per se imply that you should not use a reverse follow through model, but rather that you should significantly reduce your bet size or model contribution stemming from this factor in a continuous manner.
Reverse Follow Through Down Minus Up Day Spread (2000 bars SPY) | ||
50ma>200ma | 50ma<200ma | |
Daily Return | 0.060% | 0.30% |
winning% | 2.27% | 5.69% |
Do you suggest testing every incremental rule for statistical significance to the DVSharpe ratio (forget what it’s called – DVR?) while building a system (group of rules). Taken to the extreme, you end up with an “ecosystem” of rules (targeting various aspects of a market like trend/chop, length of period) and continuously pick the best combinations out. Very DVDarwin 🙂
What do you mean by “next day spread”?