First DS at small oil trading firm – what ML actually helps physical/paper desks?
Just joined a small commodity trading firm as the first Data Scientist/Analyst.
Background: ~7+ years in ML (mainly NLP, but comfortable with time series / forecasting & deployment). The firm is mostly physical oil trading, but my manager wants to grow paper/spec trading and use Data to support it
We have vendor that already provides a lot of analytics. There are no clear internal requirements – my mandate is basically: look at the data, find useful use cases, and build something that will be useful
I’m much stronger in ML/models than BI. I can build dashboards, but would rather focus on models / tools that support trading/business decisions.
Questions for people who’ve seen this work in practice:
- What in-house ML/analytics have you seen that genuinely helped physical or paper oil traders?
- Where would you start as the first DS on such a desk, given vendors already cover generic analytics?
- What are the career prospects for someone entering oil trading from the DS side?
Based on the most helpful WSO content and insights from similar discussions, here’s a breakdown of how you can approach your role and make an impact as the first Data Scientist at a small oil trading firm:
1. ML/Analytics That Genuinely Help Physical or Paper Oil Traders
2. Where to Start as the First Data Scientist
3. Career Prospects for a Data Scientist in Oil Trading
Additional Tips
By starting with impactful, trader-focused tools and building trust with the desk, you can establish yourself as an indispensable part of the team. Good luck, and welcome to the world of oil trading!
Sources: AI in fundamental investing, Is there a future for non-quant hedge funds?, Machine Learning in fundamental HF?, Physical Energy Trading And Logistics, https://www.wallstreetoasis.com/forum/hedge-fund/machine-learning-taking-over-hf-research-analyst-roles-in-near-future?customgpt=1
writing here now: OP is not going to last more than a year
why? I've seen how most DS efforts fail at physical shops unless it's a DE/DBA role
Which efforts have failed in particular? Maybe the have before LLM adoption, now it should be faster to provide some internal ML product thanks to faster iteration.
OLS is the only useful "model." I am not joking. You will not get traction on trading ideas with anything more complex and even the OLS should not have more than 2-3 variables in it which themselves are easily explainable and pass the smell test on having an impact on your target. Transformations on the data will also make people hesitant. Simple variables that be explained with a simple scatter plot is the only way. If you manage to print a bunch of money doing that then you can think about more complex models. Odds are most people at a physical oil shop don't know what standard deviation really is. They will nod and make you think they do but they don't.
Sounds like my guy is about to attempt to teach rocket science to a bunch of second graders.
Best of luck.
Why did you join without a clearly defined mandate? In my experience DS initiatives tend to fail when the roadmap and expectations are not clearly defined and aligned prior?
Anything you build doesn't have to be tradable, it can be things that make traders lives easier. I would spend a significant amount of time trying to understand what the traders do, what data they interact with and why etc to get an understanding of what might be useful and generally build some intuition
Sorry, but agree that you will probably quit soon.
You are the first data hire in a space that is difficult to systematize at a seemingly small company which is data illiterate. And there are way bigger institutions before you which have tried a lot harder with a lot more money, tools, data scientists and also more experienced traders to guide them, and mostly to no avail.
They hired you as a low downside gamble. Traders will secretly think that the things they ask you to check are probably a dead end, but they will do it anyways because you are going to be the one doing the work, wasting time, getting frustrated, not them. Prepare to hear the words “can we look into” something that sounds conceptually plausible that no one can seem to get to work.
Your most achievable deliverables will be things that are mechanically functional. Can you download data on a schedule, put it into a dashboard, notify people if there are big changes. It’s a living but it’s not glamorous.
They indeed are data illiterate. But I can try to bring some data culture into the firm, make them data-literate, can't I?
So what should I do? Do you know any low-hanging ML fruits in oil industry?
Are there any career opportunities for me outside this firm, such as moving to a bigger trading company?
Ya I think the low hanging fruit is just going to be operational in nature- getting updated data, showing it. That type of thing.
There’s not been much progress in ML for any type of oil trading that I’m aware of.
There are very simple regression type things that you can do- what is this price does it cause this pipe to flow more or less. But nothing super fancy, and as mentioned before, fanciness might result in less buy in.
And if they gave you a “vendor” that “does all their analytics” like energy aspects, I’m not sure mining that is going to result in a ton. And is also a red flag to me- like if they don’t even do their analytics I’m not sure they are anywhere close on the data culture front to being productive with anything more advanced.
I would say build your own CTA models, momentum based & try trade if off based there. Agree with others comments, seems you’re a low investment/high reward profile, but it’s high reward for you as well personally as a career. I wouldn’t try to jump immediately since they let you tinker around & trade. I think a lot of guys would be willing to do what you’re doing in order to get to a trading seat. I also think the trading seats this year are lesser than before & expected to stay the same as it’s been a challenging year for oil as a whole
Vortexa/Kpler have other functions you may like
Energy Aspects has a quant subscription you may be keen on enrolling
As for seasonality, you can look into RBOB & HO & ICE gasoil… though it’s not going to be easy & it’s incredible volatile, you have been warned. Look out for futures positioning for starters & see if you can derive anything useful
As for Brent/TI, it’s highly macro right now & maybe an OPEC discount against physical at the moment, so you’ll need to pull all macro variables & see what suits best
Other thing is… only you will know… is your company downsizing physical.
that may also be the case why they decided to hire you as a gambit, but potentially as a spec replacement to another phy trader. As mentioned, this year physical for some of streams, are just horrible & every 3 months we see someone exiting or downsizing. so back to original equation, low risk gambit, but high reward for the company & yourself if you figure it out ;)
If the company is physical and been around a long time - here is a use case - lots of times various grades and not published well but rather send out daily via broker quotes in PDF, excel sheets etc. scrape this data using NLP and build a “exotic” grade pricing data base and then try to analyze how various grades trade relative to each others. Then the next step would be to try and build a refinery model that based on grades and freight you can optimize what kind of slate it should be running …you can move on from here
Sunt cumque enim fugiat. Commodi ratione et animi non eos itaque est. Voluptas modi et similique amet occaecati. Cumque velit reiciendis doloribus magnam omnis sapiente voluptatem.
Impedit error id non occaecati accusantium ex. Molestias rerum consequatur aut aperiam est ad. Sed amet velit saepe repellat quidem unde.
Maiores blanditiis pariatur nesciunt dolore voluptatem aut nisi. Sed doloremque reiciendis accusantium voluptatem rerum optio perspiciatis odit.
See All Comments - 100% Free
WSO depends on everyone being able to pitch in when they know something. Unlock with your email and get bonus: 6 financial modeling lessons free ($199 value)
or Unlock with your social account...