Filtering White Noise

Most asset return processes can be characterized as containing a primary trend, along with mean-reversion around that trend, as well as a certain amount of random noise.  Econometricians classify these elements using a Hurst Exponent as either : 1)black noise (trending/positive autocorrelations- Hurst>.5) 2) pink noise (mean-reverting/negative autocorrelations- Hurst<.5) or 3)  white noise ( no trend/mean-reversion, low/insignificant autocorrelations- Hurst=.5). Intuitively traders wish to capitalize on either the trend or mean-reverting behaviour- often at different time frames since they are part of the same unified process (trends tend to occur at longer time frames, and mean-reversion around that trend at shorter time frames). The key obstacle for both styles is to eliminate or minimize the impact of white noise on indicators that are used to measure either trending or mean-reverting behavior. The failure to do so results in poor trading results due to false/random signals.

Consider two charts of the same time series (from Jonathan Kinlay’s good blog) one is a black noise process while the other contains a white noise process:

In the first chart- with a black noise process- it is easy to see how profitable it might be to use a simple moving average to trade the underlying–there is very little noise to speak of that is not self-reinforcing (trending). In the second chart- a white noise process- you can see the similarity to real financial time series. There appears to be a fair amount of random noise, and it would be more difficult to trade for example with a moving average. The chart below shows a pink noise process, and looks familiar to those who trade pairs as a log of the ratio of two asset prices that are cointegrated (ie like one sector ETF versus the same sector from a different ETF provider).

Notice that this process appears to have a stationary mean and predictable negative autocorrelation. It would be impossible to trade this series using a moving-average based trend strategy. However, this would be an ideal dataset to trade using runs (ie buy on a down day short on an up day). In practice, time series data contains elements of all three types of noise and thus what we want to do is to filter out the white noise which is less predictable and obscures otherwise predictable asset behavior.

A recent paper was written by a colleague- George Yang-  that sheds light on how to go about filtering random/white noise elements and also shows the practical impact on trading system profitability. The paper recently won a prize in the prestigious Wagner Award competition which is run through NAAIM. Mr. Yang shows that one can filter out “insignificant” data using a rolling or historical standard deviation threshold and extend indicators to use only “significant” data. For example, if one were to use a 200-day moving average on the S&P500, you might stipulate that market moves between .25% and -.25% are too small to be considered significant in defining the trend. That is, a small up or down day (or series of small days) may cause a trade which will not signal a true change in the underlying trend. This can also be translated for example as a fraction of a rolling standard deviation unit. To calculate the true 200-day moving average in the first case, one would eliminate all insignificant days from the data set and count back in time until there were 200 days of significant data to calculate the moving average. The results in the paper demonstrate that this type of filtering is effective increasing the signal to noise ratio and improving trading results across a wide range of parameters. The paper also shows the same technique is effective at improving a short-term mean-reversion system using runs. This highlights the potential of applications that can filter white noise from the data.

There are multiple extensions to improve this concept, many beyond the scope of this post. However, one seemingly obvious method would be to also filter insignificant days as also requiring trading volume to also be insignificant– presumably volume that is below average would signify a lack of conclusive agreement on the current market price. On the flip side a seemingly small market move accompanied by very heavy trading volume could be a warning sign. Another method could look (on George’s suggestion) at the high to low range for the day in relation to the past (ie like DV2). Presumably a tight daily range implies insignificant movement, while a wider range is more informative. One can picture using multiple filters to enhance the ability to identify truly significant from insignificant trading days. This would in turn significantly improve trading signal performance or forecasting ability.

Minimum Variance Algorithm Comparison Snapshot

The Minimum Variance Algorithm was compared to several standard optimization methods and algorithms in a recent set of tests done by Michael Kapler of Systematic Investor.  Michael created a webpage for MVA to review some details of these tests and also to summarize some of the background information.  We plan to release a whitepaper on MVA with some additional material in the coming weeks. Below is a summary of testing done across multiple data sets contained in the MCA paper.  We used a standardized score (the normsdist of the z-score) of the performance of each method versus other methods using three metrics: 1) sharpe ratio (higher is better) 2) volatility (lower is better) 3) portfolio turnover (lower is better). These factors were weighted equally to create a composite score. We tested across a wide range of data sets– stocks, ETFs and Futures. The Minimum Variance Algorithm (MVE in the chart below) scored the highest of all methods across datasets- outperforming standard minimum variance and also the minimum correlation algorithm.

 

The following acronyms are defined below.

MVE:  Minimum Variance Algorithm (MVA) in Excel

MCE: Minimum Correlation Algorithm (MCA) in Excel

MC: Minimum Correlation Algorithm (MCA)– Whitepaper/R Version

MC2: Minimum Correlation Algorithm 2 (MCA)

MV: Minimum Variance – standard minimum variance using a quadratic optimizer long only

MD: Maximum Diversification-standard maximum diversification using a quadratic optimizer long only

EW: Equal Weight

RP: Risk Parity- basic version inverse volatility weighting

 

Minimum Variance Algorithm (MVA) Excel Sheet

The link below contains a spreadsheet example for computing MVA weights from multiple times series. Next week, I will try to show some different applications.

Minimum Variance Algorithm

 

Minimum Variance Algorithm (MVA) Test Drive

The Minimum Variance Algorithm (MVA) follows much of the same logic as the Minimum Correlation Algorithm (MCA) and differs primarily in the objective function which is to minimize portfolio variance versus correlations. Both are “heuristic” algorithms that seek to approximate the results of more complex methods that require employing quadratic optimization. In a recent whitepaper, Newfound performed various simulations and came to the same conclusion that I have shared for a long time:   in the case of uncertainty in the parameter inputs such as returns, correlations and volatilities, simple heuristic methods achieve results that are equivalent to more complex optimization methods. It is therefore feasible that good heuristic methods can exceed the performance of their more complex counterparts especially if they are designed to be less sensitive to parameter uncertainty.

The core principle of both MVA and MCA is to use proportional allocations to generate weightings because they are more stable than using discrete selection of both assets and weights. This principle is supported by information theorists, and is used frequently in technological applications. Cover also covers this principle in his work on Universal Portfolio Theory. A good summary article is presented on Ernie Chan’s blog. Another aspect of both MVA/MCA is that they use a gaussian transformation  to normalize the relative average correlations/covariances. MVA is very similar to “mincorr2″ (see the whitepaper for more details) and simply finds the average covariance of each asset versus all other assets -including its own variance- and then converts the average value for each asset to a cross-sectional distribution using normalization. This is used to proportionately weight each asset to find an initial set of weights. The final weights are derived by multiplying each initial asset weight by its inverse variance and then releveraging to sum up the weights to a total of 100%.  The result is that weights reflect both the asset’s own relative variance and also average covariance to the universe of assets. However, the weights are less dependent on correlation estimates (which are critical in complex minimum variance but are noisier than volatility estimates) and do a better job of distributing risk since allocations are made to all assets in the universe.

Below is a backtest of the MVA on eight highly liquid ETFs used for the original MCA tests since 2003. The variance-covariance matrix uses a 60-day parameter with weekly rebalancing. The benchmark used is equal weight:

As you can see the MVA achieves a high sharpe ratio (higher than MCA) and achieves slightly superior returns to an equal weight portfolio with less than 50% of the volatility. The benchmark analysis shows that MVA is simply a means to efficiently reduce downside relative to an equal weight portfolio, and this comes at the cost of some upside performance. MVA captures 75% of the upside in bull markets for the equal weight index, and only 50% of the downside in bear markets using a continuous distribution measurement. The actual results of this one test are not meant to be conclusive, but I have done a large range of tests on different universes with both long-term tests on index data and using recent ETF data and have found similar results. While there is nothing magical about MVA, it supports the point that a heuristic method can be very effective-especially with noisy time series data. For the sake of practicality, it can be implemented easily in just about any platform and like MCA can also be computed very quickly for large datasets. There isn’t really a good case to employ quadratic optimization to minimize variance unless you need handle different constraints. While I haven’t done much in the way of comparisons between the two, I would imagine that MVA would perform at least as well across a wide range of universes.

 

Minimum Variance Algorithm (MVA)

Often readers ask about methods for approximating minimum variance portfolios. In practice the minimum variance portfolio can be calculated in closed form only for long-short portfolios, and requires a quadratic optimizer to solve for long-only portfolios. Source code and examples for  long-only minimum variance can be found at Systematic Investor - a very good blog that also has a toolkit for a lot of standard optimization methods. Michael Kapler (the man behind Systematic Investor) and I wrote a whitepaper about an algorithm for finding minimum correlation called the Minimum Correlation Algorithm (MCA), which was meant to approximate maximum diversification portfolios. The primary benefits of the algorithm versus the conventional optimization were: 1) speed of computation and ease of calculation 2) greater robustness to estimation error and 3) superior risk dispersion.  Testing results across a wide array of universes also demonstrated the superiority of MCA in terms of risk-adjusted returns versus its maximum diversification counterpart. The Minimum Variance Algorithm (MVA) is a close relative to MCA and shares the same benefits versus conventional minimum variance optimization. In testing, MVA showed superior risk-adjusted returns to MCA across most universes. While I have not yet conducted comparisons versus conventional minimum variance, preliminary results are very competitive. This is encouraging considering that MVA is very simple to calculate. Later this week I will present the logic and also post a spreadsheet for calculation along with some test results.

Deciphering Dynamic Asset Allocation: Lessons From Perold and Sharpe

The recent popularity of “tactical” investment strategies has given rise to a dizzying array of new terminology and strategy descriptions. Most investors and investment professionals lack a deeper understanding of the core nature of such strategies. They can hardly be faulted for all of the marketing material floating around that often obfuscates the difference between a separate brand and a truly separate strategy.  In reality, most “tactical” strategies are very similar and have predictable payoff profiles even if their returns are not predictable. The class of tactical strategies are essentially part of the broader class of “dynamic asset allocation”strategies (DAA). The opposite of DAA is to employ “strategic” or policy-based asset allocation that contains a constant mix (like 50/50 or 60/40).The premise of DAA is that through active shifts in portfolio weightings, one can add value versus buy and hold or stategic/constant mix portfolio allocation.  One of the best articles that helps provide a solid grounding in Dynamic Asset Allocation was a classic paper by Perold and Sharpe.

There are essentially three core strategies compared in the paper (excluding those that are option-based): 1) buy and hold–yes this is a strategy 2) constant mix (CM)- this is like a policy weighting that is constantly rebalanced such as 60/40 stocks/bonds  3) constant proportion (CPPI)- this is essentially synthetic portfolio insurance generated by using a dynamic allocation between stocks and t-bills.  In CPPI, you would buy stocks as they are rising with an increasing proportion, and sell stocks when they are falling with a decreasing proportion until you hit a “floor”- which is essentially akin to a stop loss. These three strategies are  demonstrated in the paper to have very different payoff profiles as a function of market conditions. The table below provides a useful “cheat sheet” that also helps clarify the differences. The best performer in a given market regime is ranked #1 while the worst performer is ranked #4:

 

Here is a table that relates the different  strategies above as closely as possible  to more commonly used investment strategies or products (note that due to the multi-asset composition of these vehicles/strategies the linkages are not quite perfect):

The first takeaway is that there is no uniform winning strategy in all market conditions. Each strategy has a particular regime in which is it likely to shine. Bull Markets tend to favor Buy and Hold (BAH) unless one is able to successfully employ a CPPI strategy on the underlying asset with leverage (this may or may not be possible .Other things being equal, the degree to which CPPI-L will be able to beat a BAH is proportional to the degree of market noise; as the market becomes noisier, the CPPI-L will have more difficulty matching BAH. With more predictable/trending behavior the CPPI-L will easily beat BAH.Without the use of leverage, it is impossible for CPPI or CM to keep up with BAH in rising markets. This is the case for most tactical strategies- especially if they hold assets other than the equity market. This under-performance can also be compounded by market noise. CM also has difficulty in bull markets because it is constantly “taking profits” via re-balancing and inherently reducing the delta to the market.

In Sideways Markets, both BAH and CPPI struggle  due to a lack of return and a greater abundance of noise. This is where CM shines since it is like Shannon’s Demon- the optimal strategy for capitalizing on entropy. Unlike BAH and CPPI, it is possible to make money with CM even if the market does not produce a positive return.  CPPI is most vulnerable in sideways markets because it is the most sensitive to noise and can get “whipsawed.”

In Bear Markets, CPPI strategies shine because they have a fixed maximum total loss defined by the floor that is always set at a % that is greater than zero. The degree of protection that is guaranteed will be proportional to the reciprocal of the slope (1/m). The protection will also be a function of whether the floor is ratcheted as the asset rises, and also whether the floor is periodically reset- like rolling call options.  BAH obviously does the worst, as it is fully exposed to any losses that incur. CM falls in the middle as it has inherently less exposure through re-balancing, and also tends to capture some of the market volatility.

In general, CPPI-type strategies are most related to trend-following and momentum or relative-strength investing. The broad class of “tactical” or “active” strategies are likely to have a payoff profile very similar to CPPI. In contrast CM-type strategies are either more similar to “balanced” type passive funds or ETFs, “strategic” asset allocation methodologies that are static, or represented by strategies that attempt to capture mean-reversion.  BAH-type strategies represent virtually any passive holding- I would include most equity or bond mutual funds/etfs in here simply because they all seek to have low tracking error and make minor bets on individual holdings in an attempt to outperform on a relative basis.

Perhaps the most important takeaway from the paper was defined as a theory for which strategy will be likely to outperform/underperform:

“the fact that convex and concave strategies are mirror images of one another tells us that the more demand there is for one of these strategies, the more costly its implementation will become, and the less healthy it may be for markets generally. If growing numbers of investors switch to convex strategies, then markets will become more volatile, for there will be insufficient buyers in down markets and insufficient sellers in up markets at previously “fair” prices. In this setting, those who follow concave strategies may be handsomely rewarded. Conversely, if growing numbers of investors switch to concave strategies, then the markets may become too stable. Prices may be too slow to adjust to fair economic value. This is the most rewarding environment for those following convex strategies. Generally, whichever strategy is “most popular” will subsidize the performance of the one that is “least popular.” Over time, this will likely swell the ranks of investors following the latter and contain the growth of those following the former, driving the market toward a balance of the two.” Perold and Sharpe, “Dynamic Strategies for Asset Allocation.”

The theory is simple and is supported by recent market history- if everyone wants for example market upside with protection–ie tactical type strategies- then buy and hold and constant mix will probably outperform. The opposite is true if everyone wants  a more passive approach. Remember the 1990′s when everyone was switching to index funds? That massive shift in demand gave rise to one of the most profitable decades for tactical/active management- 2000- 2008. The spectacular success of tactical strategies in 2008 especially gave rise to tremendous demand for tactical which subsequently underperformed in a big way in 2009. Renewed market problems in 2010 and 2011 and the prospect of a sovereign debt crisis produced good performance for tactical strategies and naturally tremendous demand given all of the renewed “fear mongering”. Naturally 2012 was not kind to tactical strategies–especially those that sought to minimize volatility to protect against a crisis. As investors begin to pile into equity markets and passive investments again, it is possible that tactical products will eventually outperform.

The key lesson is that markets do not exist in a vacuum– the relative peformance of a strategy is inversely proportional to the general demand for that strategy:  when it is least popular to be tactical, it is likely that tactical will outperform. In contrast, when it is least popular to be passive or hold a strategic asset allocation it is most likely that these will outperform. Basically, if it hurts to invest in a given strategy and you have a lot of company in feeling that way it is probably a good idea to invest! The problem is that most investors and advisors want to go with what is working now as this is the easiest sell and also the most comfortable to invest in. On the choice of which dynamic strategy to use in the long run, again Perold and Sharpe had some wise words:

“Which dynamic strategy is demonstrably the best? The goal of this article is to emphasize that “best” should be measured by the degree of fit between a strategy’s exposure diagram and the investor’s risk tolerance (expressed as a function of an appropriate cushion). Ultimately, the issue concerns the preferences of the various parties that will bear the risk and/or enjoy the reward from investment. There is no reason to believe that any particular type of dynamic strategy is best for everyone (and, in fact,only buy-and-hold strategies could be followed by everyone). Financial analysts can help those affected by investment results understand the implications of various strategies, but they cannot and should not choose a strategy without substantial knowledge of the investor’s circumstances and desires.”

 

The Performance of the “All-Weather” Sector Portfolio Using Fidelity Funds

In the last post, we introduced the “All-Weather” Sector Portfolio which was developed using data from Fidelity Asset Allocation Research. I created a heuristic approach to integrate a variety of factors (length of  stage, sector performance ranking by stage) in order to create the final portfolio allocation. It is obviously very interesting to examine the performance of this static/strategic portfolio allocation over time. For testing comparisons, I used the Fidelity Sector mutual funds total return series and also the S&P500 total return cash index. The time period for testing was chosen to include all active
sector members to ensure a fair comparison.  Three time series were created: 1) “All-Weather” Sector 2) Equal Weight Sector 3) S&P500 Total Return Index. Rebalancing was conducted on a monthly basis.The graph and table below depict the results:

The results are promising– both higher returns and risk-adjusted returns than both an equal weight benchmark and the S&P500.  The All-Weather Sector also has the lowest risk of the three portfolios. Transaction costs and turnover are likely to be negligible in this case–especially if one were to stay within the minimum holding period for the funds. The broad diversification across sectors and limited “tilting” of sector weights makes this version of the All-Weather Sector Portfolio a desirable core equity holding for investors. The All-Weather Sector Portfolio is arguably a superior theoretical index construction than equal or market cap weightings, with  low tracking error,  tax-efficiency,  and good results to back up the concept.  In subsequent posts I will show some alternative formulations and weighting schemes that have superior performance.

Building an “All-Weather” Sector Portfolio

The central concept of the “All-Weather” portfolio is balance: having an allocation that will perform equally well across different economic regimes.  The original portfolio balances portfolio risk and performance with broad asset classes to be neutral to changes in  economic growth and inflation.This basic concept can be extended to create an “All-Weather” equity sector portfolio. One of the traditional ways to look at sector rotation  is to use the economic “business-cycle” to determine which sectors are the most favorable. In this view, the economy goes through four distinct phases that progress in sequential order and repeat in a multi-year cycle: 1) early 2) mid 3) late and 4) recession. In this context, economic growth is highest at the earliest stages and the rate of change declines as the business cycle progresses. Inflation is low at the earliest stages and builds over time to the point where there is “over-heating” at the late stage prior to a recession where inflationary pressures cool down.   Fidelity has conducted a study spanning over 50 years to determine the sectors that perform best in each stage. Below is a graphic (Source: Fidelity Asset Allocation Research) that depicts the business cycle and the expected relative sector performance by stage:

The results of the study indicate some very robust and significant differences in sector performance. The problem with applying this approach is that there is considerable uncertainty as to both which stage the economy is at in a given point of time and how long the current business cycle will last (they can vary from 6 months to several years). It therefore makes logical sense  to create a sector portfolio that is effectively neutral to the business cycle– an “All-Weather” Sector portfolio. This can be accomplished by generating four different portfolios that perform the best in each stage of the cycle and then weight them to account for differences in stage length and performance. The resulting portfolio should perform very well over time with greater consistency than a more naaive allocation.

To create the “All-Weather” Sector Portfolio, I used a simple scoring system to capture relative differences in sector and stage performance. The choice of  using a ranking/scoring model in favor of the actual data avoids the noise associated with using past sector returns and also the historical stage of cycle lengths which can be more difficult to extrapolate in terms of raw magnitude. Theoretically, the relative favorability and length of cycle should be more stable. Both the length of stage and relative performance by stage are ranked (highest to lowest). The cumulative weight is generating by multiplying the length ranking by the relative performance ranking. This determines the relative weighting of each of the four portfolios (early, mid, late and recession). Within each portfolio, each sector is assigned a score according to relative favorability. A score of 3 is given to the historically best performing sector for a given stage, a score of 2 is given to sectors that have outperformed significantly, a score of 1 is given to sectors that have neutral performance (match the market) and a score of 0 is given to sectors that have historically under-performed the market. I have compiled a All Weather Sector Worksheet to show the breakdown of the calculations in greater detail. Below is the final composite and allocation breakdown of the “All-Weather” Sector Portfolio:

The All-Weather Portfolio: Static or Dynamic Risk Allocation?

The All-Weather Portfolio was designed by Ray Dalio (and clearly influenced by Harry Browne of the Permanent Portfolio) as a robust static allocation that can be used by investors to deliver consistent performance over time. The logic of the portfolio construction is to be neutral to risk/uncertainty with respect to inflation or economic growth–the two primary factors considered to explain all asset returns.  The allocations are a function of the long-term expected sensitivity of each asset to the change in these factors- based on whether they are rising or falling substantially in relation to historical norms.

We know that the “Static” All-Weather Portfolio (using the method above) has a very good long-term track record to back up the story. The more interesting question is whether a dynamic risk allocation can outperform using the static method. Theoretically, risk inputs- especially standard deviations- should be easy to model in a dynamic context since they are fairly predictable. Furthermore, we do not necessarily need to pre-specify the relationships between assets because we can observe their changing relationships via clustering. Since the All-Weather approach has often been considered interchangeable with Risk Parity, it is interesting to see if the purely mechanical and dynamic approaches to risk parity perform in comparison using the same assets. Michael Kapler of Systematic Investor, ran the following tests in R using different risk parity variants. We also show for comparison the more sophisticated “Cluster Risk Parity” (Kapler, Varadi, 2012) which removes the universe bias from portfolio allocation and delivers a more precise risk allocation. The assets used below to represent the different asset classes are a combination of funds and ETFs to maximize data history:

 

The relative risk-adjusted performance of the Static All-Weather Portfolio versus the dynamic variations is presented below.

 

We can see that all dynamic methods perform better than the static method by a fairly substantial margin in terms of risk-adjusted returns. This suggests that the changing risk and correlations of each asset class already reflect expectations for changes in the economic factor risk to both inflation and economic growth. Furthermore, these changes can be predicted by looking at recent historical data. In addition, we also can see that more complex versions of risk parity (ERC and Clustering variants) slightly underperform the simplest version of risk parity that ignores the correlations between securities and only uses the risk information. This potentially implies either a constant correlation between assets, or that the careful choice of these different assets already reflects an embedded static clustering method (which would make the correlation information much less useful than risk in a dynamic context). Since previous tests demonstrate the superiority of clustering methods (both static and dynamic) to basic risk parity, this implies that the universe chosen is a good static clustering approach. In conclusion, the results at least suggest that dynamic risk allocation is a valid way to create an effective “All-Weather” Portfolio. In practical terms, using cluster risk parity with a diverse and large asset pool is the easiest way to capture this profile while avoiding a lot of pre-specification.

Static versus Dynamic Clustering on Multiple Asset Classes

In the last post we looked at the performance of static versus dynamic clusters on Dow 30 stocks. It is also logical to look at the same comparison on multiple asset classes. Michael Kapler of Systematic Investor ran the same set of tests on major market asset class ETFs for comparison.  To avoid distortion in static versus dynamic clustering, the starting point for the test data was set at the point when all ETF data for each asset class was available. We used the “common sense” method for static clustering, which is typically how investors and traders categorize assets:

The ETFs chosen cover a broad range of asset classes. For dynamic clustering, we again used the  principal components clustering method which is referred to as “hcluster” in “R”. Note that Cluster Risk Parity refers to using dynamic clustering with risk parity allocation both within and across clusters–ideally with risk parity-ERC, or equal risk contribution.The test comparisons are presented below:

While this is not a long backtest, we see that the results are consistent with prior results on the Dow 30 tests and also with what we would logically expect: 1) Cluster Risk Parity is the best performer in terms of risk-adjusted returns (and also annualized returns in this case) 2) dynamic clustering outperforms static clustering in terms of both returns and risk-adjusted returns 3) static clustering outperforms non-clustering and all clustering methods outperform non-clustering in terms of returns and risk-adjusted returns. To further break things down, we also see a logical rank progression based on the risk allocation method: 1) All risk parity variants outperform equal weight in terms of returns and more importantly risk-adjusted returns 2) risk parity-ERC outperforms the more basic risk parity methods- which do not make use of the covariance information. In this dataset, all of the rankings show a greater separation in terms of magnitude than on the Dow 30 tests, which can be expected since assets are less homogenous than stocks.

In general, the purpose of these tests is to show the importance of dynamic clustering and also more precise risk allocation methods in portfolio management. The combination of these two methods leads to a superior risk control and risk-adjusted performance than either in isolation. While the performance improvements are somewhat modest, they are fairly consistent and also more importantly make the portfolio allocation process less sensitive to unfavorable variation arising from universe specification. In fact, it is possible (with some refinement in these methods) to avoid having to carefully pre-select a universe in the first place. This leads to backtest performance that is less likely to be inflated in relation to out of sample results. In a perfect world, we would want to input a large universe of liquid tradeables and have a self-assembing optimization and allocation process with multiple layers based on a set of pre-specified constraints.