Machine Learning Taking over HF research analyst roles in near future?
I read an interesting article recently regarding machine learning taking over research analyst roles over the next 5 years or so (https://qz.com/1113633/wall-street-analyst-jobs-a…).
I also spoke to the CFO of a multi-billion dollar pension fund recently, and he also supported this idea, that in the next 5 years, a majority of research analyst positions on Wall Street would simply be taken over by computers, putting them out of work (at least in public securities). What is your perspective on this? Machines can obviously crunch numbers faster and spot patterns/trends far more effectively.
Do you believe that the era of the research analyst will come to an end by 2030? Does it make any sense at all now to enter asset management or the hedge fund world (on the public securities side)? If so, where should someone interested in fundamental research look into for opportunity?
I see this stuff as 99.9% breathless hype from people who know neither finance nor comp sci. "Analyst" isn't some single skill that can be brute forced. A computer that could perform the wide variety of tasks required would need to be near generalized AI, and that is nowhere close
No, it will not make them irrelevant. It may make certain types of strategies irrelevant (or at least less relevant) but I am fully confident that human investors will learn and adapt faster than any AI we will create by the year 2050.
I can only speculate but I think the amount of reading/research will be heavily reduced. natural language processing algorithms might be able to identify the important aspects of 10Ks and 10Qs. Maybe AI/machine learning will be able to reduce the number of companies that an analyst would want to consider based upon some predictive machine learning algorithm that isolates the top N companies in a particular industry that will do well (or poorly) over some time horizon.
Once the investing horizon becomes too long, I don't think ML will be of much help. Maybe there will be certain tools that will become helpful but I don't think any significant aspects of a long-term value investment approach would be replaced by machine learning. Perhaps there could be some sentiment analysis tools on earnings transcripts to see how 'risky' the language a CEO is using?
There's a lot that people could try doing but I don't think the primary investment decision making process could be replaced by an AI/machine learning algorithm in the near future for the strategies that you mentioned.
I'm a data scientist at a BB and I write algorithms . . . that's all I have to say about that . . .
I think that the general perceptions that are being expressed here are true. Algorithms may not be as advanced as people make them out to be and there's definitely all kinds of gaps (knowledge gaps, data gaps, hardware gaps, etc).
However, where all of you should be concerned is the speed at which ML can learn and adapt to new information. THAT is where algorithms will truly take off and make an impact in all kinds of industries. The key to any algorithm is the depth & variety of data that you can feed into it. Sparse data leads to weak learning, deep data leads to better learning. And when an algorithm has a lot of data, watch out, because it can learn faster than any human can hope to achieve.
If you really want to get your mind around what ML can do, go find the whitepaper written by the Google researchers that solved the game of Go last year. No, don't read the fucking CNN article on it, read the actual whitepaper. The researchers talk about how they gave their algorithm very little seed information and then set it loose on Google hardware to play millions & millions of games in very little time. This kind of ML development is where the world is headed - small algorithms, given enough time and data to learn, become big algorithms.
So, coming back to the OP, it may be easy to poo-poo algorithms in their current state, I get that. But once they have enough data, they will grow and become something huge almost literally overnight.
Ok yes Google creating an algorithm that beats the best Go players is nothing short of amazing. But there is a HUGE caveat here that you are aware of I'm sure and that's the fact that Go is a perfect information game. What is a perfect information game? It is a game where you have all of the relevant information directly in front of you that you need to make the optimal move or decision.
Unfortunately, real life isn't like board games. The information you have is imperfect and you don't have all the information relevant to you. This is where ML/AI truly falls apart in its current state. The human brain (and other animal brains too) are phenomenal at modeling unknown/hidden information as well as imperfect information and making accurate inferences nonetheless.
It is super cool that the researchers at Google gave their algorithm very little a priori information on how to play the game (unlike IBM's Deep Blue in the 90s). However, the fact that Go and most other board games are perfect information games makes the feat pretty much incomparable to the problems that humans solve on a daily basis at work.
Right. The DGP basically makes poker an unrealistic comparison. You can simulate everything. Excellent point on non-stationarity
I generally agree with what has been posted here. There's a ton of hype about machine learning, but I believe it's highly unlikely algos will "take over" investing.
The issue is the overwhelming scarcity of useful data for a computer to use. The reason machine learning dominates short-term stock-market trading (HFT, intra-day, etc.), is because the relevant data for figuring out what moves stocks over short time horizons are things like stock momentum, correlations across stocks, order book data, trade sizes, etc. There are billions (trillions?) of relevant datapoints available, in well-formatted data-sets, for quants to use to develop trading strategies around. It's a near-ideal environment for machine learning approaches to investment. But it does not generalize at all to longer time horizons. And even in short-term trading, approaches tend to be about gaining "51/49-style" edges on the next move in a stock, not predicting big winners/losers.
For longer-term investments, there are definitely useful quant signals, arbitrages, etc. (such as the classic "value" and "momentum" strategies), but most information that is actually relevant for how a stock should be valued/should perform over time is sparse and poorly structured. Think about what tends to matter for stock market investors (and what moves stocks): financial releases, earnings calls, meetings with management, management presentations at conferences, diligence calls with industry experts, etc. None of this data is well set up for machine learning. Firms release 4 (badly formatted, inconsistently presented) financial statements each year. Huge amounts of relevant data is not included in the actual financials, but in commentary on the earnings call, non-GAAP company-specific operating measures in the MD&A of the 10-Q/K or the press release. And then there is the way management's tone/attitude changes from meeting to meeting, the industry color you get from your diligence calls, things like that. And there are also only a couple thousand public companies releasing such data points, which isn't really all that much from a machine learning perspective.
One can imagine a computer synthesizing such information, yes, but you're getting into the realm of science fiction and true AI, not machine learning algorithms as they're seen today. That being said, there's lots of room for AI to simplify things for analysts: generating financial models from reported financials, highlighting changes to risk factor in the 10-K, creating a readable summary of the key portions of a 10-K, identifying "risk phrases" in earnings calls. A lot of this stuff already exists, though I can easily imagine improvements. But this isn't the sort of thing that replaces analysts in the foreseeable future.
Taking a step back here, there are near-constant articles in the press about how machine learning is going to take over everything. I have a framework I like to fall back on to quickly assess the plausibility of such claims.
If the answer to these 4 questions is yes, then there is reasonable likelihood that the task can be automated. But always keep in mind that what we call "machine learning" isn't at all like "AI" in the popular imagination. The algorithms tend to do something akin to optimized trial and error. They throw variations of predictive models at a huge data set until they come up with something that looks like it "works." It's "dumb learning," in some sense. To get any sort of result requires incredible amounts of data, and as the complexity of the problems rise, the amount of (relevant) data required for a useful model rises in a non-linear fashion (read: rises massively).
In college I did a fair amount of machine learning coursework, though by now I've forgotten pretty much all of it. However, I do remember one thing that struck me was that human intuitions of what is "hard" or "easy" to do don't line up well with what is "hard" or "easy" for a computer to do. And I've noticed that this tends to lead journalists, commentators, and hypesters to point toward some success of a computer at one task to suggest that another success is right around the corner, even if the logistical issues involved in gathering and using the relevant data are radically different.
Put another way: imagine if you gave a digital calculator from the 1970s to a man from the 18th century, and explained to him that it was a form of machine intelligence. Imagine his shock as it performed calculations instantly that would take him hours, and performed them flawlessly. What would his understanding of machine intelligence be? He might well assume that machines in our era could easily perform any tasks that humans could do, or that such a development must be right around the corner, given the impressive power of this machine. He would have no intuition at all about its limitations, or the limitations of the era's technology. Today's public commentary on machine learning feels a bit like this to me.
Fundamental analysts will never go away given the structural complexities and messiness of financial data, information, companies, corporate events, people, etc. There will always be a need for analog thinking, where a purely digital approach wouldn’t suffice
However, what I think will happen, and what is already happening, is that funds and strategies that use purely fundamental perspectives and techniques will be at a distinct disadvantage. The fundamental space will become more quantamental, as computing power can be used as a powerful tool to augment fundamental analysis
I think it was Garry Kasparov who said that the best chess player isn’t going to be a computer or a person, but a combination of the two working hand in hand