Applying Artificial Intelligence to Trading

In the past few years it has been hard to escape the ubiquity of Artificial Intelligence. Although everything from your phone to your car or even your refrigerator might be endowed with AI, what is actually considered AI will vary widely from person to person. It turns out that even researchers in the field of AI don’t have a universal definition, but one (of unclear origin) is:

AI is any device that perceives its environment and takes actions that maximize its chance of successfully achieving its goals.

HRT is an algorithmic trading company, where models and code drive all our trading decisions every (nano)second of the day. How does this definition of AI map to our trading problems?

First, let’s consider our “environment.” One view of HRT is that it provides services to the world like price discovery (ensuring securities are fairly priced relative to each other) and liquidity provision (if you want to buy or sell something, HRT stands ready to take the other side). We need to integrate large amounts of data in order to provide these services at lowest cost, and so in some sense “everything” is our environment! One of the most integral forms of data is “market data” like prices and volumes for exchange-traded products like stocks, cryptocurrencies, commodities, options, and currencies. We can then add “alternative data” like news reports, company filings, and even more exotic forms of information like Tweets, satellite data – even the opinions of people on your favorite stock-trading subreddits.

Next, we need to define our “actions.” Exactly how we define this is probably going to be dependent on exactly what we want to achieve, but let’s simply go with buy, sell, or hold any given security or instrument we are considering.

Finally, what is our goal? Maximizing the net trading revenue (profitability of our trades less trading costs like commissions) would be sensible – although, we need to consider when that revenue will be realized. For example, if we said something like “maximize revenue over the next hour” then we might expect some strange effects like accumulating a large position over the next hour which looks profitable on paper but can’t actually be sold to realize the profit because the position is too large.

We can put this all together into an optimization problem. One thing we haven’t yet discussed is the constraints on our trading AI. For example, we might want to maximize our trading revenue but without exceeding risk (which can take many forms). We definitely don’t want to violate any regulations or laws in the markets we trade in, but we want to go even further: some actions may not violate the letter of any law, but we want to be good market participants, so want to avoid certain behaviours.

After all the difficulty of precisely defining our problem, now we have to actually optimize our AI to achieve its goals. Before we embark on that long journey, we can try to map our problem onto one where AI techniques have been successful. If we can make that map without too much fudging, then we might be able to use the same techniques to solve our problem.

One of the classic success stories of AI has been board games; for example, IBM’s Deep Blue (for Chess) and DeepMind’s AlphaGo (for Go). In both regimes AI demonstrated convincingly super-human performance (although human performance at trading isn’t very good, so super-human isn’t much of a bar for us!). Let’s discuss some of the key aspects of the success of AI for board games, and contrast them to the trading problem:

• Two-player “zero sum”: Chess/Go are played by two players, and only one player can “win” or they draw. That is, for one player to win, the other player must lose, and we both want to win. This doesn’t map very well to financial markets: while “cash” is roughly conserved, markets aren’t zero sum in terms of the utility or benefit to the participants, which partially explains why anyone trades at all. For example, when I put money into my 401(k), I want to purchase stock now at minimal execution cost, but I’m somewhat insensitive to the exact price since I won’t be selling for 40+ years. The person selling to me might be selling to hedge some risk they have from some completely unrelated-to-me trade. Both of us can be happy with the trade – we both win because we both got what we wanted. One crucial implication is that we cannot use HRT’s AI as a proxy for the other “players”, which is a key component of how AlphaGo works. Instead, we would need to model a wide variety of different types of “players” to simulate the interactions that happen in real markets.

• Knowing the rules: AlphaGo makes use of full knowledge of the game rules in its tree search procedure, and the immediate outcome of every action is fully known and deterministic. Then, the other player’s subsequent response is well-modelled by just doing what we’d do, as we described above. In trading we partially have this property: market behaviour in more developed markets like national stock markets is usually well-defined, but we do not have the property of knowing exactly what will happen if we send an order. A simple example of this is if another “player” sends their order at the exact same time, the outcome is random.

• Fully observing state: Board games like chess and Go don’t require knowing past states of the world, outside of some corner cases – everything you need to know is visible on the game board. In markets, the past is critical for forecasting the future. A simple example is the idea of reversion: if the price of a stock moves suddenly, it is more likely than not to “revert” to its long-run average price. The past is important, but any given instant holds a lot of hidden information: the identities and positions of all other market participants is an obvious one, but at a fine-grained level, consider “hidden” order types that don’t appear until you trade with one.

• Infinite data: We can play as many games of chess against ourselves, or a different AI, as we want. Each of those games is “real” and the only shortcoming is that the diversity of that generated dataset might not capture all possible ways of playing the game. In contrast, in trading we have only one reality, and acquire new data in real time – i.e., very slowly. We can attempt to produce data by simulating many different AIs against each other, but it is a poor proxy for the rich complexities and interactions of the real world, for reasons described above. Having a large amount of data is critical for training the large neural networks that were used by AlphaGo’s winning AI.

Based on all that, one might be tempted to think we can’t apply AI to trading – there are just too many obstacles! However, there are plenty of ways to work around these problems. One of the best is to decompose the big problems into smaller subproblems using our human understanding of the overall structure. If we put the solutions to these subproblems together, we can get a good but imperfect solution to the original question, and working on closing that gap is one of the problems our researchers actively work on.

Let’s work through applying this to a style of trading and think about how to decompose it. In this problem our objective is to maximize our net trading profit over multiple days, subject to a limit on the total risk of our portfolio. Every minute, we can choose to buy or sell each of the securities in our portfolio to maintain the balance between owning those we think will increase in price, and reducing the risks of our position. Naturally, there are many aspects of this problem that are hard for an AI, from identifying which securities will increase or decrease in price, to managing risk, to determining the best time to buy or sell them.

Rather than trying to train one AI to do it all, we can use mathematics and intuition to break this up into three subproblems:

• Prediction. The job of this AI is to simply predict the future! For example, we could say “given the observations of the world, predict the price of each security at the end of the next five days.” This is still a very hard problem, but much simpler than the original. The AI doesn’t need to worry about how other market participants will react, or even decide on an action. Instead, it just needs to solve this “supervised” learning problem. We have many techniques to attack this problem, from hand-crafting “signals” from data and combining them, to letting a complex black-box model use all the data any way it can. Many of our researchers spend most of their time on this subproblem.

• Optimization. The job of this AI is to take the predictions of the first AI and convert them to an overall allocation of our investable money to each of the securities. It will put more money into securities that we think will increase in value, but will attempt to spread the money around so we don’t go all-in on one company – a much riskier position. Financial mathematics has many solutions to this problem already, so we don’t have to take an AI perspective here at all necessarily, but there is plenty of room here to make our solutions more and more sophisticated.

• Execution. Given the target allocation from our optimization, how do we actually buy or sell to reach that target? For example, let us suppose our AI is sure at 10am that Apple stock will be worth more tomorrow than today, and after adjusting for risk it is sure it wants to buy an additional 9,000 shares over the next 5 hours. Should it buy them all at once? You’d be sure to reach the target, but there might not be enough people selling right now, and we might pay a high price. If we buy 30 shares every minute, we’d definitely get enough, but maybe some minutes have better prices than others – based on a very short-term forecast. But if we don’t buy in some minutes, we have to catch up later. Trading off all these concerns is a complex problem, but simpler than taking them all together, and one we can try to train an AI to do. Still, this problem is in many ways the hardest as it hits the issues we described above – our actions can have immediate consequences that are hard to reason about, unlike in a game.

HRT has been working on these problems for many years, and will be working on them for many more to come. These are rich, complex, and difficult considerations, and there is no book or academic paper that you can read to find their solutions. Instead, our researchers must draw on insights from related fields, their own intuition and creativity, and a rigorous scientific approach to make progress. If that sounds interesting to you, apply to join our team!