Skip to content

The Future is “Theory-Free”

July 20, 2012

The dusk is setting in on the world of factors and economic theory. If my views seem dramatic or extreme, consider that the markets did not make sense from a CAPM world view during the best (and simplest) of times. Today, the world markets are a strange and dangerous place. We are in the midst of the era of dominance for computerized trading (think Skynet from the Terminator). Floor trader jackets will soon become museum relics or sold as collector’s items on E-Bay. Most of the retail investors and individual traders have left the market entirely–outclassed, outgunned and tired of poor returns and being unable to compete. We are simultaneously in the throes of perhaps the greatest financial mess of all time. Our pithy attempts to control the uncontrollable with the aide of simplistic economic theory, political idealism, and crony capitalism has put the world in a perilous situation. The stress in financial markets reflect unprecedented levels of global debt, government intervention, unexpected shifts in sovereign risk, and banks that can lose billions for no obvious reason seemingly overnight.

The complexity of the modern world, and the degree to which it is interconnected, has rendered simplistic linear theory almost meaningless. Human behavior- a feature once predictable in its hybrid version of rationality and cognitive bias- no longer manifests in markets as a fluid mass of “greed” and “fear”. Instead, game theory and behavioural finance now share equal importance as players engage in Keynesian beauty contests with greater than six degrees of anticipation. (not to be confused with six degrees of separation) . The highly complex interaction between governments, banks, hedge funds, institutions, wealthy individuals, and high frequency firms is a multi-dimensional high stakes game where the rules change by the day. Players must interpret or anticipate others actions, the perception that other players have of their actions, and how rules might change and how this may affect their actions and perceptions. And this is a simplistic description……………..

The future of investing must be theory-free and focus on creating algorithms that can adapt to change and adequately capture the nature of complex systems. Ed Seykota once correctly observed that a surfer does not need to know the theory of fluid mechanics to learn how to surf a wave in the ocean– he needs only to be able to make probabilistic estimates and adjustments. This example puts the financial economist at odds with the scientist or the engineer. The future favors the practical problem-solvers, the inventors, the gamers, and the deep thinkers. Investing is becoming like the evolution in mixed martial arts– no single style or background is likely to win, nor is a stubborn resistance to learning about other disciplines. I predict that the theory police will resist this transition and be left standing naked before their disciples when the tide goes out. It is much easier and more comforting to explain the world in deterministic terms- replete with graphs of marginal utility and “proofs.” Of course it is also more comforting to play chess than engage in a high stakes game of “liar’s poker”.

It is important for quants to override the desire to be fed theoretical pablum and avoid trying to seek academic approval from high priests of a defunct religion. There is nothing wrong with learning finance, economics or econometrics- in fact it is crucial to understand these disciplines at a high level. But it is time to start thinking for yourself, and to stop being a sheep. Instead one should favor asking hard questions, learn to think independently, collaborate effectively, and most importantly spend their waking hours thinking about how to solve high-dimensional classification and game-theoretic problems.

17 Comments leave one →
  1. July 20, 2012 3:13 am

    Best blog post I have read in a long time, I couldn’t agree more!

    • david varadi permalink*
      July 21, 2012 2:54 am

      thanks theo, appreciate the kind words.

  2. July 20, 2012 8:41 am

    Right on, David. My only disagreement would be I think we hit this point years ago. Few realized it…as is still the case today. Great post!

    • david varadi permalink*
      July 21, 2012 2:56 am

      hi jerry, thank you for the kind words—i agree that this has been going on for a while and the magnitude of the change is not properly appreciated.

  3. Chris C permalink
    July 20, 2012 9:46 am

    Great post. Though, I’m curious to see what follows; deep down I’m hoping this is a prelude, of sorts, into an article about combing multiple signals/aggregate indicators.

    My only real comment coinciding with your thoughts is that for the past year or so, technical/light quantitative techniques have only worked, for me, when combining multiple instruments, and their derivatives, in order to generate a buy/sell signal. I’ve implemented levels of complexity into my strategies that previously would’ve been useless. Using a simplistic approach to trading/creating our indicators, overall, seems far too crowded. I suppose that this is all part of the game, and that this proves long term success is more than a just matter of intelligence, but ones ability to adapt.

    Hope all is well, David. Glad to see you writing again!

    • david varadi permalink*
      July 21, 2012 3:02 am

      hi chris, thank you for the kind words. i agree that the post you are requesting is certainly a subject high on everyone’s list. it will take me some time to fully cover some of the existing topics that are a prelude to addressing that at the proper level. i also agree that using a modest degree of complexity while avoiding overfitting is critical to preserve alpha. chasing simple effects are much more likely to lead to sudden reversals, but chasing overly complex or specific strategies can lead to random performance. adaptation is indeed critical– of course, over-adapting is also a pitfall.

  4. July 20, 2012 11:19 am

    Theory-Free… and surfing… and gaming ! There is life in it, David, juicy life ! Whao, I imagine a great life of yours on the Pacific coast surfing in the day, playing poker in the night… and theory-free trading in the meantime – the inputs arriving to your brain are theory-free processed ! And the success is there by tapping the original source of your tacit insights, your intuitions.. just like the lucky surfer do !

    Ok David, it sounds great but I am sorry, I don’t buy your enthusiasm. Theory-Free is just a dream, not at all adaptive in the darwinian way. Being based in Europe, “lateral thinking” has for ages existed on our continent but sooner or later, it needs to evolve in a new emerging theory.

    It is a key moment called by academics the “serendipity pattern” (by sociologist Robert K. Merton, Social Theory and Social Structure) and the occasion to observe the emergence of a new theory.

    On the contrary, if one is unable to cope with this emergence and stay “theory-free”, he won’t avoid the Blind Spot (see Blindspots analysis) and, just like the surfer, will very probably fall in the water !

    Theory-Free is not Adaptive, David, please give me your feedback.

    • david varadi permalink*
      July 21, 2012 3:21 am

      I certainly wasn’t envisioning a life of surfing and late night poker while writing this post—though that doesn’t sound too bad either…… I think that you are taking my comments to the extreme– I am not suggesting that 1)inferences 2) hypotheses 3) sub-theories are not useful. In fact the latter are important in the context of adaptation, and until machines can completely replace humans, intuition and creative thinking will be required to develop such algorithms. My comments relate to grand macro theory–with economics being the primary target. I dislike the concept of simplifying a tremendously complex adaptive system like the world economy into a neat and convenient theory that requires a laundry list of assumptions. I think it is possible to make intelligent inferences and derive plausible scenarios, but it is not possible to create a theory that describes how the world economy works that will hold true throughout time–especially factoring in the influence of technology. However, my comments could for example extend to grand theory of the universe as well. I think human beings are currently not smart enough to conceptualize nor reduce something like the world economy or the universe to a theory that most intelligent individuals can grasp. The problem with conventional science is that we are looking at a few factors at a time, while a complex system may contain the interaction of hundreds of variables with nonlinear patterns. That explains the failure of most drug research, and also the inability for financial theory to cope with what actually happens in markets. As a sidebar, I’m not sure how theory-free is not adaptive since the world has evolved along with numerous species without a particular theory. Darwin merely observed and explained a theory for how this might occur. Nonetheless, adaptation occured prior to this key observation without the benefit of his theory. Scientists now apply his insights using genetic algorithms and genetic programs which can evolve new equations or inventions, and even derive scientific laws without prior knowledge. If that isn’t theory free, im not sure what is.

      I appreciate the insightful comments and welcome any debate.

  5. July 31, 2012 6:59 am

    Hi David,

    Great post but I find myself not to agree with all you said…. while I agree that the future is “Theory-Free”, I believe that so was the past.
    The world, and particularly its socio-economics side, has always been a complex system resulting from interactions between governments, banks and institutions.
    For sure from a quantitative trading prospective, a lot of “simple” inefficiencies disappeared, but I wouldn’t necessarily link this fact with a global view on the way the world works or with a macro trading point of view.
    In the past players had been rewarded for a great macro intuition on a consequence of entangled situations (eg see Soros and his bet on GBP), and so are they now (eg somebody could have made a fortune, and probably some did, on BTPs) and so they will in the future.
    I am not saying that human beings are able to fully grasp the complex world dynamics (nor that CAPM or any economic model is able to give a clear macro picture)… but rather that humans never needed to do this. It is enough to be able, at one particular time, to make a single correct prediction of what next little step will be (somewhat like in the surfer example by Ed Seykota). Not saying it’s easy, just saying I think is pretty much as difficult as it has always been.

    Especially because I don’t see reasons for it to be otherwise. What has changed in last years compared to, say, 20th Century? The advent of High-Frequency firms? While they surely have affected intraday traders, by definition they play a totally different game than macro traders.

    So while I couldn’t agree more with your invitation to think independently and with the importance you attribute to game theory and behavioural finance, I think this approach is as valid now as it was 20, 40 or 100 years ago.

    Having said this, if you think that the global economy can’t carry on this way, that capitalism is a flawed concept, that debt levels are too high and that inequalities among people are becoming too big…this is probably another story🙂


  6. July 31, 2012 6:31 pm

    Hi David,

    I particularly enjoyed the surfer analogy because it describes what the automated investment tool SectorSurfer is all about: Investment Strategies to Ride Sector Waves, just as Ed Seykota says. Checkout SectorSurfer if you have not already seen what it can do.

  7. August 1, 2012 12:11 pm

    Hi David – Great post. Thanks.

    Just like there are beautiful concepts in quant that TA/FA folks can use, I think there are good concepts quant methodologies can make use of from other fields. It is not that easy though as often playing well in markets require strong belief and confidence in once own methodology. But a side effect of that is the tendency to go into mental rigidity and silo themselves unless one has humility and makes a concious effort.

    I think some attributes like integrity, humility, mental flexibility and comfort with uncertainity are needed irrespective of which methodology (FA/TA/Quant) the practicioner uses. Problem is no course will teach these (or may be there are courses). I suspect just like in TA/FA, in quant models also one major driver of complexity in the system is lack of comfort with uncertainity by the practitioner. That is not bad but there needs to be a balance.

    I don’t have lot of quant experience but enough trading experience. From the post it seems herd behavior and guru worship seems to be prevalent in quant field just like in other fields. One aspect my views differ from post is 2nd paragraph of the post. I agree there are many causes and the number of these causes plus interconnections between them are growing rapidly. Also they are morphing as your post eloquently puts it. On other hand, the other side of the tunnel is fixed and doesn’t change i.e., net result of all these causes is still same i.e., the price moves only in 3 directions – up, down and side ways. So the way I see is a tunnel with one complex causes and interactions and other end always acting withing fixed constraints. I think one reason Meb Faber TAA system is simple because his paper does hedging of assets based on the result instead of the cause.

    On another note, I like your blog and views. Would it be ok if I include RSS feed to your blog on my website? You can see your latest post as sample under “Quant Currents” tab. The read me section describes as an example how I use the site for my own trading.



  8. August 1, 2012 2:57 pm

    Forgot to mention…if any of your readers like the site but prefer different contents, they can do so with iGoogle homepage and feeds. They can see the details of the feeds they are interested. from my site. I modeled the site and layout after my iGoogle home page except it can be accessed by others.

  9. August 15, 2012 8:19 am

    Reblogged this on ThEcOnomics.

  10. Andrea Malagoli permalink
    August 29, 2012 4:00 pm

    I randomly stumble on this blog, and I must say I am impressed. It is the first really innovative piece on quant modeling I have read in a long time. I guess I like it because it resonates with my personal crusade against ‘quant myopia’. Too often quants and academics fall in love with their models because of the fancy math they can showcase, nothing to with trying to understand what is going on.

    Furthermore, it is the case that the orthodox financial theories are flawed to the core (think of CAMP, MPT, and the whole stochastic equilibrium nonsense). While finance tried to imitate science, it failed to understand that data come before theory in science. Finance instead chose to select a model based on its logical beauty and mathematical consistency (fancy math = more valid model). This is has been a tremendous fallacy that has had practical repercussions in the ‘real’ world in the form of flawed investment models. Now we see the consequences …

    Above all, I fully support the call to return to a more “intellectually honest” and humble approach to quantitative financial modeling (and an approach which is more empirically driven).


  1. Friday links: managing money is a full-time job | Abnormal Returns
  2. Quote of the Week • Systematic Relative Strength • Dorsey Wright Money Management Systematic Relative Strength
  3. Quantitative trading and Macroeconomics | Math Trading

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: