The Man, The Machine, The Legend: AI In Hedge Fund Management

I Know First Research Team LogoThis article was written by the I Know First Research Team.
Virginia Marie Rometty

“Some people call this artificial intelligence, but the reality is this technology will enhance us. So instead of artificial intelligence, I think we’ll augment our intelligence.”

Virginia Marie Rometty, head of IBM.

Robots In The Workplace

As the world embraces the power of artificial intelligence and machine learning, whole industries are transformed to reflect this new reality. AIs are now helping taxi companies identify spots that would see lots of ride requests coming in. They could also drive themselves soon enough, seeing how the self-driving car industry pus the pedal to the metal. IBM Watson, a US-based supercomputer complete with sophisticated AI software, helps with patient diagnosis, albeit at times with questionable results. AIs are transforming many aspects of manufacturing, from optimizing the production pipeline to enhancing quality control with advanced machine vision software.

AI In The Workspace

In other words, the age of robots is upon us, and there is no turning back.

This fact is not lost on the hedge fund industry, which has seen its ups and downs in the recent years. The traditional investment strategies utilized by hedge funds, such as going market-neutral or utilizing leverage, have been bringing in questionable returns lately. Furthermore, traditional hedge funds are now facing off with the tech-savvy quantitative funds, those utilizing advanced computer algorithms and troves of unstructured data to make market predictions. Is there a future for the human in trading, or will high-paid hedge fund managers find themselves in the ranks of those whose jobs fell prey to the machines? Let us find out.

What Are Hedge Funds, Anyways?

Hedge funds are limited partnerships including a general partner – the manager calling the shots, – and limited partners, or, simply put, investors. Not any investors, of course: the world of hedge funds is picky and will only open its doors to those who can boast an income in hundreds of thousands. The more, the better, of course.

AI In Hedge Fund Management

What these high-value individuals get in return is a promise of high profits on the money they bring in. While hedge funds work with a huge variety of assets these days, from stocks to cryptocurrencies, the overall idea behind them is roughly the same. They are dealing with less regulation than mutual funds and tend to go for riskier plays (or, to be more precise, those falling under the more or less generous risk threshold of the fund) in a bid to generate the highest returns. In doing so, they utilize a wide variety of strategies (often in a mix), shorting stocks they expect to lose value, longing stocks that they think will go up, bringing in leverage etc.

The history of hedge funds goes all the way back to late 1940s, the era of long positions. This was, however, about to change as Alfred Winslow Jones published his article on hedging in the Forbes. Hedging, in investment terms, means making an investment to offset the risk from another investment. In other words, going long on the stocks of Company A while shorting the shares of its key competitor, Company B, in hopes that you win regardless of whoever does well.

This is exactly what Alfred did as he created his own hedge fund, establishing some of the staples of the industry as we know it today. He took a 20% fee for the revenues and was a major contributor to the fund himself. This would ensure that his interests were aligned with those of his partners. Alfred’s endeavor has been wildly successful, easily outperforming mutual trusts, which has led to a boom of hedge funds.

Today, however, the history of hedge funds is as rife with failures as it is with shining successes. As managers take risky gambles, one small mistake can lead to the fund imploding. In some cases, people’s fortunes can end up being wiped out even by something as benign as reports that the winter was going to be mild. Other notable examples include the Crusader Fund, which held $1.5 bln. at its height and fell prey to the 2008 crisis, or the all-time legend of Long-Term Capital Management, the hedge fund led by Nobel Prize winners and Wall Street highflyers, which once had $120 bln. in assets. After a wild initial success, however, it was decapitated as it placed its bet on the Russian government bonds and was hit hard when Moscow defaulted.

For investors, all this risk comes with the added negative of the fee scheme employed in most hedge funds. Known as Twenty and Two, it includes both a 2% fee and a further 20% fee on the returns. But now that the era of AIs is upon us, some are preferring to turn to a cheaper alternative – AI-driven funds relying on huge amounts of data to build sophisticated investment strategies. However, before we go on to those, we should first say a few words on what exactly the whole AI buzz is all about.

Artificial Intelligence: It’s All About The Data

While the recent advances in technology, or, more specifically, the rising computational power of CPUs and GPUs was the factor that enabled the rise of the AIs, the mathematics behind the technology has been around for a while. In fact, one could argue that a simple linear regression model is a basic type of machine learning. In its simplest shape, with only one predictor, all the machine does is fitting a line through the scatterplot of the data in a way that minimizes the distance between the data points and the line. This line is then used to calculate the response for predictor values outside the dataset.

AI In Hedge Fund Management

Now, this is arguably the most basic thing to bring up here, but it does illustrate the fact that, in its current shape, the AI industry works with some very complicated statistics rather than building hordes of Skynets that would one day become self-aware and wipe all of us off the face of the Earth. It is all about data and patterns, in this example – a simple line.

Now, let us dig into the more exciting stuff.

In our first example, we explicitly told the machine that predictor A defines response B, and that we expect their relationship to be linear. This is supervised learning in its finest, with humans telling the machine what to look for in the data it is being handed. With unsupervised learning, however, things go differently: the machine is expected to look for patterns in data all by itself. In its most advanced form, this is exemplified by the so-called Deep Learning. Here, the computer is fed a gargantuan dataset and is given the green light to process it, relying on artificial neural networks, complex mathematical functions emulating the work of a human brain. An important thing to note is that data does not have to be structured, or presented in a way that would make it easily readable to the machine. With Deep Learning, you can have your computer process financial reports, tweets and news articles alike.

This technique has been famously used by Google as it taught its DeepMind AI to play Go, an ancient Chinese strategic board game. After learning from thousands of games played by humans, AlphaGo was able to beat the best Go player in the world in 2006, claiming four games out of five. Furthermore, some of its moves were so unorthodox that they opened a completely new page in the history of the most studied board game in the human history.

If that is not impressive enough, consider what happened next. A new version of the AI, called AlphaGo Zero, which was taught only on playing the game against itself, came in to wipe the floor with its predecessor AlphaGo, claiming all games of 100.

Such tremendous machine learning capacities are, of course, applicable not only to games, as we noticed in the introduction. Among the examples cited there, we could also have mentioned finance and fintech. The last point is of specific interest to us, because this is exactly where we come to the proverbial robots creeping in to take the jobs of the unfortunate hedge fund managers.

Robots And Money: The Disruption Of Traditional Finance Industry

In broadest terms, the hedge fund landscape of today can be split into two large groups. The first one is, for lack of a better word, classical hedge funds relying on fundamental analysis to pick up the undervalued and overvalued assets. The second comprises quantitative, or quant, funds that build sophisticated computer algorithms to predict the fluctuations of the stock prices.

AI In Hedge Fund Management

Of course, using computer models in investment is not necessarily new. In fact, the aforementioned Long-Term Capital Management bet on the Russian debt was courtesy of the fund’s computer models. But what separates today’s AI-driven quant funds (and let’s note that not all quant funds are AI-driven) from their past peers is the scope and complexity of the operation. With Deep Learning, they can process humongous troves of various data, looking for signals on what happens with the stock next.

Are customers unhappy about the new product? AI can pick that up through sentiment analysis from what they post on the social media. Is the debt-to-equity ratio getting dangerously high up? AI will take that into account as soon as it scouts out the latest financial statement. Or maybe even earlier, as it scrapes through the news reports on company X issuing debt. Absolutely spectacular, right? Unfortunately, equally spectacular are some of the quant funds’ failures.

The most curious one, perhaps, is the case of a robot hedge fund launched by Tyndaris Investments, a Hong Kong-based company. As the robot wasted million upon million, Hong Kong real estate tycoon Samathur Li Kin-kan, a limited partner in the hedge fund, grew increasingly irate. After sustaining particularly heavy losses, he withdrew his funds and hit Tyndaris with a $23 million lawsuit. In a twist worthy of an Isaac Asimov or Philip K. Dick novel, this begs the question – can you sue a robot?

The list of failing quant funds, unfortunately, is not over here. Sentient Investment Management, a high-tech quant fund and, until recently, an absolute media darling, shut its doors after just two years of operations. As an another example, March 2018 saw the Ai-driven quant funds take a beating on the back of a market selloff.

So what is the way to go? Should a machine run your finance, or is there nothing better than a competent human investment manager running the show? The future, it seems, is somewhere in the middle.

Seeking Singularity: AI-Driven Decision Enhancement Tools For New Era Of Hedge Fund Management  

Humans and robots should go hand in hand – this motto, it seems, resonated quite strongly among the guests of the recent Milken Global Conference. One of the major matters brought up there was how to enhance the decision-making capabilities of human investment managers with the use of AI tech. A similar line of thinking can be noticed among some investment giants, such as Morgan Stanley, which sees AI as a helper to its analysts, not their replacement.

AI In Hedge Fund Management

This synergetic approach seems to offer the best of both worlds. On the one hand, machines bring in the capacity to process huge troves of data in the blink of an eye. Sometimes, they can also offer moves that no human would ever think of, as with the game of Go. Finally, they can help automate some routine tasks, giving humans more time to do the work that really matters. On the other hand, human managers offer an approach rooted in broad experience and theoretical knowledge as opposed to statistical methods, as sophisticated as those could be. They can also act on their gut, if necessary, and will not make the mistakes that an AI can make.

And if you are looking for an AI-based solution that would bolster your own investment decision-making capability, look no further than the proprietary AI developed and trained on a dataset covering 15 years of trading by the Israeli-based I Know First company. Its design includes elements of Deep Learning, while also drawing upon genetic algorithms technology. What this means is that the algorithm is aware of its own successes and failures, re-calculating its models and beefing up its accuracy with each running cycle.


Scraping the Web for daily quantitative market data, it models and predicts the price dynamics for over 10,000 financial tools, including stocks, ETFs, currencies, commodities and cryptocurrencies. In doing so, it utilizes elements of chaos theory, viewing the markets as dynamic systems where one small event can produce a major fallout. Furthermore, all weights in the deep neural networks that the algorithm relies upon can be re-configured to simulate various events and scenarios.      

The output is delivered as an easy-to-interpret heatmap with two key indicators: signal and predictability score. Signal indicates the relative performance of each asset on the forecast against other financial tools on the forecast, and predictability score shows how well the algorithm predicted the stock before. The time horizons for forecasts range from 3 to 365 days, covering short, mid- and long-term outlooks.

The algorithm boasts a high accuracy rate that makes it possible to profit even from fully automated trading, relying on a bot that simply buys and sells the stocks with the highest expected returns and predictability. In fact, a recent evaluation demonstrated that a portfolio built on I Know First predictions beats the one based on the S&P 500 index.