View from Crystal Bay: Using Machine Learning to Build Portfolios

Investors aren’t limited to only investing in the S&P 500 and the Dow Jones Industrial Average. Every week we share the market trends we are following. We are interested in whether the trends in those markets are continuing or if they are experiencing a temporary or complete reversal.

When we identify trends, we are only concerned about the price data and what it says about any given market. We don’t need to know why a trend has formed to invest, but our human nature wants to understand what is driving them. Each week we try to offer some perspective on what we think the most substantial moves are and what critical drivers are behind them. Here we look at what is going in our globally diversified, non-correlated Crystal Bay Ubitrend strategy.

Last week’s continuing trends:

  • Iron Ore
  • Soybeans
  • Aluminum
  • Chinese Yuan
  • European Maze

Last week’s reversing trends:

  • US Government Bonds
  • UK Government Bonds
  • Gasoil
  • Palm Oil
  • Russian Ruble

What we are taking note of: 

The field of machine learning and artificial intelligence has created wonderful breakthroughs in many fields. We now have cars that can drive themselves, cell phones that understand spoken commands, and programs that recognize human faces. The field of finance, however, has not seen much success with artificial intelligence techniques.

There are many possible reasons why finance has been a tough nut to crack for artificial intelligence. The main ones include scarcity of data (we only have a few decades of daily asset returns, compared to billions of images and messages) and a lot of noise in the data. Techniques that work well for image recognition fail utterly when faced with the cacophony of the markets.

In the last five years or so, finance practitioners have honed in on a subset of machine learning techniques that show promise in financial markets. In general, they tend to be simpler tools with a specific purpose. They may not be as flashy as self-driving cars, but they are starting to prove their usefulness.

One such tool is hierarchical tree clustering. The idea behind the technique is fairly intuitive. Imagine that we look at a subset of the animal kingdom, containing chimpanzees, gorillas, cats, lions, walruses, and sharks. We want to figure out how related these species are to each other. A simple technique is to collect data about each species across several categories, called features. The features could include the number of legs, whether the animal has fingers or paws or fins, whether it breathes air, whether it gives live birth, and whether it eats fish. We can then define a “distance” between two species as the percentage of features they share. The distance between a chimp and a gorilla would be small because they have lots of common features. The distance between a chimp and a shark would be large because they don’t share many common features. We would quickly notice the existence of a few clusters: chimps and gorillas are clustered together, and so are cats and lions. Walruses have some similarities to both, but they don’t quite belong with either. Chimps, gorillas, cats, lions, and walruses would form a weakly related supercluster. Sharks would remain apart, not clustered with any other animal. We could draw a tree that illustrates these clusters. Starting from a common root, sharks branch off to the side, and then walruses, then the remaining branch splits into two smaller branches with cats and lions as the leaves on one of the branches and chimps and gorillas on the other branch. We have discovered a hierarchy of species, defined by their similarities to each other.

The beauty of this technique is that it can be applied to any collection where we can quantify the features and define a measure of distance. It does not require lots of data and isn’t overly sensitive to noise: it works quite well in financial markets, where it is used to classify stocks into sectors and assets into asset classes.

A recent paper by Robert Stock of SBV Research called “Asset Allocation Via Clustering” uses hierarchical tree clustering to investigate diversification benefits of asset classes and hedge fund strategies. The features in his models are historical returns; assets with similar returns are clustered together, while assets with different returns are separated. The distances between asset classes measure the diversification benefits we would get from combining them in a portfolio. The more dissimilar or distant from each other they are, the higher the benefits of diversification. Robert Stock also looks at how the diversification benefits evolved over time. His results contain both expected and unexpected findings.

When looking at asset classes, Stock finds that traditional style boxes (growth versus value, or large versus small stocks) don’t provide meaningful diversification benefits anymore. Starting in 2002, all of these equity categories merged into a single cluster. Even foreign equities add only a small diversification benefit. Only MLPs and, to a lesser extent, mining stocks stood out from the rest of equities.

Bonds maintained a distinct internal structure. Treasuries, investment-grade bonds, and municipals remained separate categories. High yield bonds tended to join the equities cluster and not behave like other bond classes.

The truly surprising results came from examining hedge fund strategies. Since 1995, most hedge fund categories started to take on more and more equity characteristics, until they just became equity bets. Categories as diverse as convertible arbitrage, market neutral strategies, volatility trading, statistical arbitrage, distressed credit, and special situations all merged into a single equity cluster. There is no point in spreading exposures across these categories. 

Astonishingly, discretionary macro strategies have become equity strategies, too; discretionary traders couldn’t resist the pull of the stock market.

The only diversifying hedge fund categories are now systematic macro (commodity/currency) and merger arbitrage. The paper concludes with an examination of optimal asset allocation. It shows that adding systematic macro and merger arbitrage strategies to traditional stock and bond portfolios increases returns and lowers volatility.

Conventional asset allocation techniques based on rules of thumb miss the implications of this evolution. It is time to ditch them and bring machine learning tools to the table.