Skip to content

FTCA2

December 5, 2013

The strength of FTCA is both speed and simplicity. One of the weaknesses that FTCA has however, is that cluster membership is determined by a threshold to one asset only at each step (either MC or LC). Asset relationships can be complex, and there is no assurance that all members of a cluster have a correlation to each other member that is higher than the threshold. This can lead to fewer clusters, and potentially incorrect cluster membership assignments. To improve upon these weaknesses, FTCA2 uses the same baseline method but computes the correlation threshold to ALL current cluster members rather than just to MC or LC. In this case, the average correlation to current cluster members is always calculated to determine the threshold. It also selects assets in order of the closest correlation to the current cluster members.

The pseudo-code is presented below:

 

While there are assets that have not been assigned to a cluster

  • If only one asset remaining then
    • Add a new cluster
    • Only member is the remaining asset
  • Else
    • Find the asset with the Highest Average Correlation (HC) to all assets not yet been assigned to a Cluster
    • Find the asset with the Lowest Average Correlation (LC) to all assets not yet assigned to a Cluster
    • If Correlation between HC and LC > Threshold
      • Add a new Cluster made of HC and LC
      • Try adding each of the remaining assets that have not yet been assigned to a Cluster in order of highest correlation to the current cluster if correlation of the asset is > the average correlation of the the current cluster.
    • Else
      • Add a Cluster made of HC
        • Try adding each of the remaining assets that have not yet been assigned to a Cluster in order of highest correlation to the current cluster if correlation of the asset is > the average correlation of the the current cluster.
      • Add a Cluster made of LC
        • Try adding each of the remaining assets that have not yet been assigned to a Cluster in order of highest correlation to the current cluster if correlation of the asset is > the average correlation of the the current cluster
    • End if
  • End if

End While

9 Comments leave one →
  1. stefan permalink
    December 6, 2013 4:32 am

    Hi David,
    Thanks for updating FTCA. I modified my code to implement this. It is indeed generating more clusters, even at a Threshold of 0.5, vs the older FTCA at a higher threshold.
    However, some clarification are needed for the pseudo code above.
    Here is what I did, where there was no specification:
    1. I calculated “correlation of the asset with the current cluster” by averaging the correlations of the asset with each cluster member.
    2. I calculated “average correlation of the current cluster” by adding all the elements of the cluster members correlation matrix, except the diagonal, and dividing by the number of pairs (ie N*(N-1) ) .
    3. There is a limit condition when the cluster has only 1 member. That’s how they all start. In this case I used the threshold as “average correlation of the cluster” to qualify the asset with the highest correlation to the cluster.
    I would appreciate if you would validate/correct these assumptions.

    • stefan permalink
      December 6, 2013 5:05 am

      Clarifying point 3 above:
      For a cluster of one, I check if the correlation between the asset and the cluster (member ) is > threshold.
      If >=2 members in the cluster, I use the FTCA2 pseudo-code logic.

  2. Pierre permalink
    December 6, 2013 10:09 am

    Hi David and Stefan,
    I am waiting for the answer of David too.
    But I have a question related to the general philosophy of the algo. In case you start processing two assets which have a very high level of correlation between them, will it not raise the threshold for a third asset to join this cluster to a level which may be too high compared to a situation where the two first assets may be lower correlated.
    Wouldn’t it make more sense to compare the average correlation of the new asset with all the assets in the existing cluster to the threshold instead of the average correlation of assets already in the cluster to have a process which is independent of the order you process the assets ?
    Thanks,
    Pierre

    • stefan permalink
      December 6, 2013 4:51 pm

      Hi Pierre & David,
      Here are my observations:
      Using FTCA2 with my specs above, over an all weather set of assets classes, I see that in general clusters have 2 members, in normal market conditions (risk-on times). They are clusters made of the usual suspects: TIP & TLT, GLD & DBC, SPY & IWM or SPY & QQQ, etc. However, in more euphoric times, I see clusters grouping 3 or 4 and sometimes 5 risk-on types of assets like SPY, foreign markets, real estate, etc. And, finally, in risk-off times (like end of 2008) I see one big cluster of 6 or more risk-on assets.

      • Pierre permalink
        December 7, 2013 8:03 am

        Hi Stefan,
        I have implemented the two methods. According to my tests, it seems that the way David described and you have precised produces a larger number of clusters, as the effective threshold for a product to join a cluster is much higher than the displayed threshold. Matched with a risk parity allocation between clusters and an equal weight among clusters, the results do not seem to show significantly lower turnover of products neither improvement of performance.
        The other interpretation I suggested, based on the goals described by David in his introduction, and which compares the average correlation in the cluster formed by the products already in the cluster, and the new candidate, to the threshold and not to the existing average correlation, produces much less clusters, as the threshold remains basically unchanged.
        Results produces by this implementation are closer to the former one.

  3. david varadi permalink*
    December 7, 2013 3:49 pm

    Hi Stefan and Pierre, sorry for the confusion–it was a quick and dirty post. Pierre’s last comment is actually the intention–the average correlation of cluster members is not considered. Rather, it is the average correlation of the target asset to all current members of the cluster. It does not need to be done in order–and is only done so for subsequent applications. It does in fact have similar results and turnover to the original version. It was not assumed to be an empirical improvement, but rather a theoretical one that also highlights some of the factors worth considering in clustering. A wide variety of complex calculations and steps can be taken to enhance clusters–and these tend to mirror existing standard methods of clustering. Unfortunately I haven’t seen anything that is materially better than the very simple FTCA empirically, hence my attraction to the method.

    thank you both for your excellent comments. perhaps the confusion inspired some new ideas rather than wasting valuable coding time 🙂
    best
    david

    • Pierre permalink
      December 7, 2013 4:12 pm

      Hi David, Thanks very much for this precision and for sharing as well. Best,
      Pierre

    • stefan permalink
      December 9, 2013 4:57 pm

      Thank you as well. Basically, what you say is that FTCA2 is FTCA with the modification that cluster membership is validated if the candidate’s average correlation with the cluster > threshold.
      It does indeed produce very similar results to FTCA. The key is the threshold value.

      It is interesting to see this clustering algo applied to a set of diverse closed end funds. The CEFs portfolios are not so transparent but seeing them grouped in clusters reveals the portfolios concentrations in different asset classes.

      For Pierre: could you tell more about the asset universe you are applying cluster risk parity to?

      All the best!
      Stefan

  4. stefan permalink
    December 9, 2013 11:03 pm

    One more note about FTCA2: The operation of adding a new member to the cluster should be done in order of the highest correlation to the current cluster, as per David’s pseudo-code above.
    This is key because, as the cluster membership increases, the average correlation of a new candidate with the cluster is affected by the current membership. Else, if the candidates for the new cluster are selected in random order, the average cluster correlation with a new asset can vary randomly as well and the generated clusters may be different.
    So, for FTCA2 to generate stable results, the highest correlation criteria above should be used.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: