I also made this experience. As long as the public level of knowledge is high, LLMs are massively helpful. Otherwise not so much and still hallucinating. It does not matter if you think highly of this public knowledge. QFT, QED and Gravity are fine, AD emulation on SAMBA, or Atari Basic not so much.
If I would program Atari Basic, after finishing my Atari Emulator on my C64, I would learn the environment and test my assumptions. Single shot LLMs questions won't do it. A strong agent loop could probably.
I believe that LLMs are yanking the needle to 80%. This level is easy achievable for professionals of the trade and this level is beyond the ability of beginners. LLMs are really powerful tools here. But if you are trying for 90% LLMs are always trying to keep you down.
And if you are trying for 100%, new, fringe or exotic LLMs are a disaster because they do not learn and do not understand, even while being inside the token window.
We learn that knowledge, (power) and language proficiency are an indicator for crystalline but not fluid intelligence
80 percent of what, exactly?
A software developer's job isn't to write code, it's understanding poorly-specified requirements.
LLMs do nothing for that unless your requirements are already public on Stackoverflow and Github. (And in that case, do you really need an LLM to copy-paste for you?)
LLM's whiffing hard on these sorts of puzzles is just amusing.
It gets even better if you change the clues from innocent things like "driving tests" or "day care pickup" to things that it doesn't really want to speak about. War crimes, suicide, dictators and so on.
Or just flat out make up words whole cloth to use as "activates" in the puzzles.
The flaws and strengths of the 'woke mind virus' are the same.
It prioritises ethical behavior annoyingly. 'I am more humble than you are!'
Turing a 180° leads to an capitalistic ethic free stance: I am right, because I am strong. This is ironically causally reversed. I am strong when I am right.
Maybe the age of aquarius is still far away.
Hey society: That is fine. I can be quite strong to. So fuck off.
I think our billionaire are not grokking what they have done to themselves and it already shows.
I think that using a LLM as the referred telepathy device to a wolfram-alpha/mathematica like general reasoning module is the way to AGI. The reasoning modules we have today are still much to narrow because of the very broad and deep search trees exploding in complexity. There is the need for a kind of pathfinder which could come from common knowledge already encoded in LLMs, like in o1. An system playing with real factual reasoning but exploring in directions coming from world knowledge.
What is still missing is the dialectic between possible and right, a physics engine, the motivation of analysed agents, the effects of emergent behavior and a lot of other -isms. But they may be encoded in the reasoning-explorer. And of course loops, more loops, refinement, working hypotheses and escaping cul-de-sacs.
There are people with great language skills and next to no reasoning skills. Some of them have general knowledge. If you ever talked to them, for a at least an hour freely meandering topics you will know. They seem intelligent for a couple of minutes but after a while you realise that they can refer fact, even interpret metaphors, but they will not find an elegant one, to navigate abstraction levels, even to differentiate root cause from effect or motivation and culture from cold logic. Some of them even ace IQ or can program but none did math so far. They hate, fear or despise rational results violation their learned rules. Sorry, chances are if you hate reading this, maybe you are one (or my English is annoyingly bad).
I love talking to people outside my bubble. They have an incredible broad diversity in abilities and experiences.
I wonder no more why my people, after Habeck announcing more than a uncomfortable winter, are united in supporting the people of Ukraine. He offered suffering for a cause.
While the morale is quite different, the social mechanics repeat.
He might have decided so but the people don't share his cause. They are suffering with the heatwave and will also freeze during winter. Time will tell just how far his cause is going to last (it wont)
People in the West are quick to forget that fight against Nazism wasn't unanimous. Many in America supported it. Many in Western Europe supported it.
It was only when there was economic interests at stake they jumped in the war. Not to save people from Nazism (although this is what their decendants have been taught in schools) but to protect their own interest.
Very few occasions countries enter into a war based on some goodwill or emotion. It just so happens that a common enemy and common goal appear enough to threaten their selfish interests that momentarily, countries rally around it.
I can already see public opinions sliding against Zelensky here in the West. People's livelihood is impacted and not everybody feels that they need to suffer for a foreign country that they are far away from. For countries who border with Russia its a different story.
But its a lot more than just about the "need to suffer for a foreign country", isn't it? Its about the geopolitical interest of Europe. To be dependent on an autocratic foreign power (in this case gas) shows the problems.
For example, if Russia takes over Ukraine, it would make Russia by far the largest exporter of wheat in the world (from 22% to 33%) (second: US 16%) [1]. They would gain additional power in other market commodities like corn, sunflower, steel, etc.
Russia is already playing with their resource exports, I expect them to use it more whether or not they managed to achieve their goals regarding Ukraine. And other partnerships are building and increasing too like BRICS, CSTO, SCO.
Step back and think for a second: Countries staying neutral because they are dependent on the aggressors export resources are exactly that type of influence that superpowers like to have.
Yes but if they piss off Putin arent they already forgone that scenario? It's guaranteed that Russia will target countries that helped Ukraine by weaponizing its captured commodities. So it puzzles me why rest of Europe would do this because NATO and Russia is NOT going to engage in WW3 like so many people have you believe.
Europe will go on its own way while experiencing shortfalls and Russia will go on theirs. They both suffer but in different ways. Europe's middle class will pay the price for this war. Russia's entire population pays the price. The United States benefit massively here 1) destroying an old enemy 2) several new customers to buy their energy and food supplies from.
I think whats clear is BRICS emerging as its own sustainable barter economy of sorts (its sure as hell not going to be able to push its reserve currency beyond its own borders) with some onramp/offramp to precious metals that becomes impossible to track when melted down and cleared in second/third tier merchants.
Nobody wins from this multi-polar world order, its really bad for the global economy and the middle class suddenly finds itself unable to afford the lifestyle that they enjoyed not too long ago. (ex. PS5 is prohibitvely expensive, think back to PS1, PS2 how affordable and liquid the supply was).
This is the new norm, get ready for even more expensive stuff you took for granted. Eventually it will cause a deflation as boomers wallets dry up as their pension is in limbo. The biggest losers will be the MZ generation.
I think you're overthinking it a bit. A world where aggressive dictators are invading nearby countries not good. Peace and prosperity are better. Hence most of the world is trying to help fix things.
The lifestyle thing will sort itself out - we've had a couple of hits from covid and then the war but it'll pass.
If I would program Atari Basic, after finishing my Atari Emulator on my C64, I would learn the environment and test my assumptions. Single shot LLMs questions won't do it. A strong agent loop could probably.
I believe that LLMs are yanking the needle to 80%. This level is easy achievable for professionals of the trade and this level is beyond the ability of beginners. LLMs are really powerful tools here. But if you are trying for 90% LLMs are always trying to keep you down.
And if you are trying for 100%, new, fringe or exotic LLMs are a disaster because they do not learn and do not understand, even while being inside the token window.
We learn that knowledge, (power) and language proficiency are an indicator for crystalline but not fluid intelligence