AI still cant do work it wasnt trained to do that an average middle schooler is capable of
Summary: AI still can't figure out how to do easy work related tasks even if its primarily language/words related, which a LLM should be good at.
Proposal: transform advanced tasks into programming problems, then use current model programming ability to solve it, then transform it back into the original context.
Recently I asked a flagship AI to read through several thousand notes compiled over several years to extract all instances of things like contract interference at work, my boss intentionally sabotaging my work, etc, then include the original notes with dates under each incident.
It basically got lost in an infinite loop for hours spending tokens. I then gave it a basic algorithm and it was then able to quickly complete the task, but it didn't catch any of the incidents. There are roughly 20 for it to catch. When given something specific and well documented like a crime, it can find it well, but looking for general dishonesty etc, it missed everything. I gave it 2 examples and it understood, but then still basically didn't find anything else. I gave it multiple chances across the last 2 flagship models with the same result.
(Note: one example: Rick tells me to change a variable name from state A to state B. The next week Rick tells me to change it from state B to state A. Then the next week Rick tells me to change it from State A to State B and criticizes me for taking so long to complete this task he already asked me to. He then complains to my consulting company about this, and lies about the situation. Later when presented with proof of him doing this same scheme again, he says he doesn't remember and says he must have misspoken.)
AI currently is pretty good at solving programming problems, and most work related complicated tasks can be translated to a programming problem. Years ago I read that even in business law sometimes they do the contract in computer code or at least pseudo code, so this concept is definitely possible. So, develop a model that translates non simple tasks to programming problems, then use the good programming problem solving abilities of AI to figure it out, then translate it back to the original context when finished.
Ex: I had to tell it somethin like: read each note, then determine its topic. For the next note, repeat it but also test it against the current list of topics and add it under one if it fits. Then go back and look for the requested information in the vsrious topics. After I gave it that algorithm, it didn't get stuck and quickly completed the task, albeit without being able to find the incidents.