Machine learning in hedge fund investing
Alternatives for uncorrelated income
- Artificial intelligence and machine learning are prevalent in all aspects of everyday life and play an ever-increasing role in investing.
- Machine learning investment strategies aim to deliver persistent, uncorrelated alpha streams while adapting to changes in market conditions—without the human input required in other quantitative investment approaches.
- Applying machine learning techniques to financial markets is not easy. Diligent manager selection and portfolio construction, informed by deep industry expertise, are critical to capturing the potential benefits of machine learning strategies and improving portfolio outcomes.
Artificial Intelligence (AI) attempts to mimic the immense decision-making capabilities of the human brain. Machine learning (ML) is a subset of AI utilized to build predictive rules based on the identification of complex patterns. Today the impact of ML is pervasive, from movie recommendations to medical diagnoses. Greater and cheaper computing power, increases in the availability of global data, cloud technology and advances in techniques have propelled ML into hedge fund investing as well.
Quantitative hedge fund managers are increasingly turning to AI and, more specifically, ML to meet investors’ needs for new and diversified sources of return. However, the inherent “noise” in financial markets makes quantitative investing one of the most challenging applications of ML. We believe that, applied correctly, ML can offer a differentiated edge in consistently generating uncorrelated alpha.
Investors too recognize the potential of AI and are gaining comfort with its investment applications.1 As these strategies populate the hedge fund landscape, investors want to know:
- What differentiates these strategies from more traditional quantitative approaches?
- What are the potential investment opportunities and risks?
- How can these strategies be most effectively implemented within portfolios?
Machine learning and the evolution of quant investing
Quantitative investing encompasses the universe of strategies in which managers use computer programs to trade systematically. Traditional quant investing relies on investment teams to identify pricing signals, constantly monitor their efficacy and actively intervene if the signals falter.
Early quant models based their forecasts on trading philosophies like “less expensive securities tend to outperform” (the value factor) or “markets exhibit trends” (the momentum factor). As more managers traded using these same factors, signals became crowded. This culminated in the “quant quake” of August 2007: A three-day period of dramatic losses occurring when managers had to sell similar positions in their quant books to cover margin calls from other portfolio losses.
Despite the events of 2007, some managers continued to trade based on these signals, which became a commoditized, low fee approach to gaining broad exposure to these market factors (or risk premia). Other managers developed more complex predictive rules using a greater variety of signals. Sophisticated players identified signals that were less known and thus slower to become crowded, enabling them to generate a higher quality return. These managers continue to deliver alpha but need to closely monitor their signals and use discretion to adjust predictive rules should their risk-adjusted returns begin to wane.
The machine learning difference
Machine learning algorithms digest reams of data to identify patterns and build a predictive rule that constantly evolves as it adapts to continuous feedback (EXHIBIT 1). ML’s application to quantitative investing took hold when quant managers realized that, just as their traditional models seek to systematize what fundamental active managers do, ML could systematize more of their own quant processes, including autonomously:
- finding predictive relationships and signals
- monitoring the environment for change and noticing when a source of return is fading or a new signal is emerging
- adapting to change on an ongoing basis—for example, by using more appropriate signals, reducing risk or shifting signal weights and allocations
Potential investment opportunities …
For investors, ML’s broader, deeper, faster analysis and, most importantly, its ability to continuously adapt investment processes give it the potential to deliver:
- More diversified alpha streams: The range of ML methods that quant investing can employ (from Bayesian processes to genetic algorithms and neural networks)2 and the varied quant strategies (e.g., directional strategies or market neutral strategies like statistical arbitrage), markets (equities, futures, fixed income, commodities, options) and investment horizons to which ML can be applied mean its alpha streams are likely to be less correlated to traditional equity, fixed income and quant strategies as well as to one another.
- More persistent alpha: Unlike traditional quant investing, in which signals are essentially fixed—i.e., unresponsive to changing market environments—ML systems decipher change and can even adapt the time frames of their measurements and price predictions to potentially enhance alpha generation across different market environments.
- Added value at multiple stages of the investment process, as seen in Exhibit 1.
Machine learning can automate, evolve, broaden and deepen the quant investment process
EXHIBIT 1: APPLYING MACHINE LEARNING TO THE QUANT INVESTMENT PROCESS
What’s more, processing power is estimated to double every two years,3 while global data, including alternative data sources,4 is projected to grow fivefold from 2018 to 2024.5 This strongly suggests ML’s predictive accuracy will become ever more pronounced over time.
… And potential risks
As with any investment process, there are risks inherent in quantitative hedge fund strategies that rely on machine learning.
- Crowding is perhaps the primary risk on investors’ minds, given the history of quant investing. Fortunately, the defining characteristics of ML models help address these fears: The diversity of techniques suggests less risk of crowding and more opportunity to spread risk across investment styles, while the ability to recognize and adapt to changing patterns— including those associated with crowding—creates the potential to guard against and even benefit from these shifts.
- Leverage is often employed to amplify returns in processes designed to remove market risk and isolate idiosyncratic alpha. Leverage should be analyzed and incorporated when stress-testing portfolios, as it can also magnify downside risk.
- Overfitting can occur in ML processes when models are so finely tuned to identify past patterns that they fail to accurately predict future price movements. More robust predictions can be created by designing models that solve for a distribution of outcomes vs. a specific forecast for a single point in time.
- Exogenous shocks: Algorithms may not be able to identify patterns in scenarios that they have not previously experienced, challenging performance until systems catch up. Longer and deeper datasets can reduce these risks, as can trading a greater array of lowly correlated instruments.
Realizing machine learning’s potential
The use of ML does not guarantee investment success. High single-digit returns may be a reasonable expectation for a diversified portfolio of top-performing ML quant strategies, but many newly launched ML funds will fail, and it can be challenging to identify the successful managers early on. Diligent manager selection and portfolio construction based on deep industry expertise are critical to mitigating risks and realizing the potential of ML in quant investing.
Some key questions for investors to ask when seeking to add machine learning managers to their existing portfolios:
- Is machine learning playing a truly meaningful role at the core of the alpha generation process?
- Does the manager have both expertise in machine learning and an experienced understanding of financial markets?
- Is there clear transparency into the investment process? This is more critical than position-level transparency in a process where the drivers of return can change every day.
- How does the manager address risks such as crowding, liquidity, overfitting and dealing with exogenous shocks?
- What are the capacity constraints? To what extent will the fund’s market impact degrade returns?
- What is the degree of human intervention? This is critical to an understanding of the risk management process and its implication for the risk-return profile of an investment.
How can investors best integrate and size ML strategy allocations within portfolios? Quant strategies, and ML strategies in particular, can deliver attractive returns that have little to no correlation with investors’ existing allocations. Often, long-term investors will pair ML managers with more conventional quant managers to enhance net of fee returns while diversifying equity beta risk. Importantly, diversification across different ML managers, forecast horizons and styles can potentially improve risk-adjusted returns, provide greater resilience across market cycles and decrease exposure to managers’ individual capacity constraints.
The importance of manager diversification, careful due diligence and skilled portfolio construction point to multi-manager offerings as an effective approach for accomplishing investors’ portfolio objectives through machine learning strategies.
Significant growth in the use of machine learning to trade financial markets is making ML difficult for investors to ignore— especially those looking for new sources of uncorrelated alpha. While risks and challenges remain, investors able to combine judicious manager selection with robust portfolio construction have the opportunity to tap into the full power of machine learning to improve portfolio outcomes.
1. In its 12th annual Global Alternative Fund and Investor Survey, November 2018, Ernst & Young (EY) reports that over 40% of hedge fund investors say it is critically important that their managers use AI in their investment processes.
2. See, for example, “Big data and AI strategies: Machine learning and alternative data approach to investing,” J.P. Morgan Securities LLC, May 2017. 3. Gordon E. Moore, “Progress in digital integrated electronics,” Intel Corporation, 1975. 4. “Alternative data sources” (in finance or investing) generally describes data, other than that generated by a company (e.g., SEC filings, press releases, etc.), and used in assessing its investment potential. 5. John Gantz, David Reinsel, John Rydning, “Data Age 2025: The digitization of the world from edge to core,” IDC paper #US44413318, November 2018, sponsored by Seagate.
Arbitrage strategies are highly complex. Such trading strategies are dependent upon various computer and telecommunications technologies and upon adequate liquidity in markets traded. The successful execution of these strategies could be severely compromised by, among other things, illiquidity of the markets traded. These strategies are dependent on historical correlations that may not always be true and may result in losses. Investors should consider a hedge fund investment a supplement to an overall investment program and should invest only if they are willing to undertake the risks involved. A hedge fund investment will involve significant risks such as illiquidity and a long-term investment commitment.