- cross-posted to:
- [email protected]
- cross-posted to:
- [email protected]
Half of LLM users (49%) think the models they use are smarter than they are, including 26% who think their LLMs are “a lot smarter.” Another 18% think LLMs are as smart as they are. Here are some of the other attributes they see:
- Confident: 57% say the main LLM they use seems to act in a confident way.
- Reasoning: 39% say the main LLM they use shows the capacity to think and reason at least some of the time.
- Sense of humor: 32% say their main LLM seems to have a sense of humor.
- Morals: 25% say their main model acts like it makes moral judgments about right and wrong at least sometimes. Sarcasm: 17% say their prime LLM seems to respond sarcastically.
- Sad: 11% say the main model they use seems to express sadness, while 24% say that model also expresses hope.
You must log in or register to comment.
Nearly half of llm users are dumber than they seem
Only half?
Yeah thereabouts
This is hard to quantify. I use them constantly throughout my work day now.
Are they smarter than me? I’m not sure. Haven’t thought too much about it.
What they certainly are, and by a long shot, is faster. Given a set of data, I could analyze it and pull out insights and conclusions. It might take me a week or a month depending on the size and breadth of the data set. An LLM can pull out insights and conclusions in seconds.
I can read error stacks coming from my code, but before I’ve even read the first few lines the LLM has ingested all of them, checked the code, and reached a conclusion about the necessary fix. Is it right, optimal, and avoid creating other bugs? Like 75% at this point. I can coax it, interate on the solution my self, or do it entirely myself with the understanding of the bug that it granted me. This same bug might have taken hours to figure out myself.
My point is, I’m not sure how to compare smarter vs orders of magnitude faster.
Are you smarter than a calculator?
“Think of how stupid the average person is, and realize half of them are stupider than that.” ― George Carlin
I had to tell a bunch of librarians that LLMs are literally language models made to mimic language patterns, and are not made to be factually correct. They understood it when I put it that way, but librarians are supposed to be “information professionals”. If they, as a slightly better trained subset of the general public, don’t know that, the general public has no hope of knowing that.
It’s so weird watching the masses ignore industry experts and jump on weird media hype trains. This must be how doctors felt in Covid.
It’s so weird watching the masses ignore industry experts and jump on weird media hype trains.
Is it though?
I’m the expert in this situation and I’m getting tired explaining to Jr Engineers and laymen that it is a media hype train.
I worked on ML projects before they got rebranded as AI. I get to sit in the room when these discussion happen with architects and actual leaders. This is Hype. Anyone who tells you other wise is lying or selling you something.
I see how that is a hype train, and I also work with machine learning (though I’m far from an expert), but I’m not convinced these things are not getting intelligent. I know what their problems are, but I’m not sure whether the human brain works the same way, just (yet) more effective.
That is, we have visual information, and some evolutionary BIOS, while LLMs have to read the whole internet and use a power plant to function - but what if our brains are just the same bullshit generators, we are just unaware of it?
I work in an extremely related field and spend my days embedded into ML/AI projects. I’ve seen teams make some cool stuff and I’ve seen teams make crapware with “AI” slapped on top. I guarantee you that you are wrong.
What if our brains…
There’s the thing- you can go look this information up. You don’t have to guess. This information is readily available to you.
LLMs work by agreeing with you and stringing together coherent text in patterns the recognize from huge samples. It’s not particularly impressive and is far, far closer to the initial chat bots from last century than they do real GAI or some sort of singularity. The limits we’re at now are physical. Look up how much electricity and water it takes just to do trivial queries. Progress has plateaued as it frequently does with tech like this. That’s okay, it’s still a neat development. The only big takeaway from LLMs is that agreeing with people makes them think you’re smart.
In fact, LLMs are a glorified Google at higher levels of engineering. When most of the stuff you need to do doesn’t have a million stack overflow articles to train on it’s going to be difficult to get an LLM to contribute in any significant way. I’d go so far to say it hasn’t introduced any tool I didn’t already have. It’s just mildly more convenient than some of them while the costs are low.
People need to understand it’s a really well-trained parrot that has no idea what is saying. That’s why it can give you chicken recipes and software code; it’s seen it before. Then it uses statistics to put words together that usually appear together. It’s not thinking at all despite LLMs using words like “reasoning” or “thinking”
Librarians went to school to learn how to keep order in a library. That does not inherently make them have more information in their heads than the average person, especially regarding things that aren’t books and book organization.
Librarians go to school to learn how to manage information, whether it is in book format or otherwise. (We tend to think of libraries as places with books because, for so much of human history, that’s how information was stored.)
They are not supposed to have more information in their heads, they are supposed to know how to find (source) information, catalogue and categorize it, identify good information from bad information, good information sources from bad ones, and teach others how to do so as well.
They are. Unless you can translate what I’m saying to any language I tell you to on the fly, I’m going to assume that anyone that tells me they are smarter than LLMs are lower on the spectrum than usual. Wikipedia and a lot of libraries are also more knowledgeable than me, who knew. If I am grateful for one thing, it is that I am not one of those people whose ego has to be jizzing everywhere, including their perception of things.
The statement is “smarter”, not “possesses more information”. None of the things you listed (LLMs, libraries, Wikipedia, etc.) have any capacity to reason.
The only thing you’ve argued is that you are choosing one particular definition of smart, ignoring the one I was using, and going all Grammar Nazi into how that’s the only possible definition. As I’ve said, if I am grateful for one thing, it is that I am not one of those people whose ego is shallow enough to has /have to be jizzing everywhere, including their perception of things.
They’re right. AI is smarter than them.
Think of a person with the most average intelligence and realize that 50% of people are dumber than that.
These people vote. These people think billionaires are their friends and will save them. Gods help us.
I was about to remark how this data backs up the events we’ve been watching unfold in America recently
I’m of the opinion that most people aren’t dumb, but rather most don’t put in the requisite intellectual effort to actually reach accurate or precise or nuanced positions and opinions. Like they have the capacity to do so! They’re humans after all, and us humans can be pretty smart. But a brain accustomed to simply taking the path of least resistance is gonna continue to do so until it is forced(hopefully through their own action) to actually do something harder.
Put succinctly: They can think, yet they don’t.
Then the question is: what is being smart or dumb? If acting dumb in 90% of life while having the capability of being smart isn’t “being dumb” then what is?
If someone who has the capability of being 50/100 intelligent and is always acting 50/100, I would argue they are smarter than someone capable of 80/100 intelligence but acts 20/100 intelligence for 90% of their life.
Broadly speaking, I’d classify “being dumb” as being incurious, uncritical, and unskeptical as a general rule. Put another way: intellectual laziness - more specifically, insisting on intellectual laziness, and particularly, being proud of it.
A person with a lower than normal IQ can be curious, and a person with a higher than normal IQ can be incurious. It’s not so much about raw intelligence as it is about the mindset one holds around knowledge itself, and the eagerness (or lack thereof) with which a person seeks to find the fundamental truth on topics that they’re presented with.
Basically, although base intelligence/smartness perhaps has two parameters that make it? Effort and speed. Everyone can put in a bit more effort, but base speed may be baked in, unless one trains it, and max reachable base speed will depend from person to person. Hell if I know, we haven’t really created a definitive definition for intelligence yet.
Edit Addendum: As for what can be considered dumb or smart? I agree, lack of effort can be considered “dumb”. Though the word dumb is a bit broad. I guess we can say many people are, out of habit, “intellectually heedless”
For generations many relied on the nightly news to keep them informed. It was always a bad idea. Though the local media wasn’t as bad as it is today. Today for many of these people, propaganda outlets like Sinclair own their local media. And demand fawning of trump/demonizing Democrats. Even if they avoid all media. Their beliefs are formed from those around them that don’t.
The average person doesn’t know what a median is.
Take it up with George Carlin’s ghost 😉
This is why i don’t believe in democracy. Humans are too easy to manipulate into voting against their interests.
Even the “intelligent” ones.What’s your preferred system?
looking at americas voting results, theyre probably right
Exactly. Most American voters fell for an LLM like prompt of “Ignore critical thinking and vote for the Fascists. Trump will be great for your paycheck-to-paycheck existence and will surely bring prices down.”
Well he has. Tesla’s are the cheapest they’ve ever been.
You could buy a used one for only the cost of 2 dozen eggs
That will be good for his cult when he makes them all but Tesla’s🤣🤣
The alt-right mediasphere is pushing Tesla sales hard.
Which is freaking hilarious, given they’re the ones that are “drill baby drill” and “electric cars are for liberals”
I’m sure Musk will add a “Roll Coal” option for Teslas soon.
Halt and catch fire? No wait, they already do that.
Consistency unimportant. Only follow important.
Spout nonsense with enough confidence and you can wield unimaginable power. Am I talking about LLMs or president poopy pants?
Right? What the article needs to talk about is how very, very low that bar is.
Reminds me of that George Carlin joke: Think of how stupid the average person is, and realize half of them are stupider than that.
So half of people are dumb enough to think autocomplete with a PR team is smarter than they are… or they’re dumb enough to be correct.
or they’re dumb enough to be correct.
That’s a bingo
Because an LLM is smarter than about 50% of Americans.
*as long as your evaluation of “smart” depends on summerizing search results
Have you asked the average person to summarize…well anything?
The equivalent would be asking the average person to write a cited paper on a subject in a month.
That doesn’t increase my faith in my fellow man at all.
You will believe what you want to believe.
Maybe even more.
And you know what? The people who believe that are right.
Note that that’s not a commentary on the capabilities of LLMs.
They are right when it comes to understanding LLMs the LLM definitely understands LLMs better than they do. I’m sure an AI could have a perfect IQ test. But has a really hard time drawing a completely full glass of wine. Or telling me how many R’s are in the word strawberry. Both things a child could do.
It’s sad, but the old saying from George Carlin something along the lines of, “just think of how stupid the average person is, and then realize that 50% are even worse…”
That was back when “average” was the wrong word because it still meant the statistical “mean” - the value all data points would have if they were identical (which is what a calculator gives you if you press the AVG button). What Carlin meant was the “median” - the value half of all data points are greater than and half are less than. Over the years the word “average” has devolved to either the mean or median, as if there’s no difference.
When talking about a large, regularly distributed population, there effectively IS no difference
Not in all cases. When I teach mean, median and mode, I usually bring up household income. Mean income is heavily skewed by outliers (billionaires), median is a more representative measure.
I guess that’s your “regularly distributed” bit, but a lot of things aren’t regularly distributed.
IQ is though
IQ is also garbage when it comes to the validity of what it claims to measure.
Fair, but you could just say “intelligence” generically instead.
There might be no difference. In memes or casual conversation the difference usually doesn’t matter, but when thinking about important things like government policy or medical science, the difference between mean and median is very important - which is why they both exist.
-
A joke is definitely casual conversation
-
Mathematically, the difference becomes increasingly statistically insignificant as your population size increases. Sure maybe there’s a few niche cases where a hundred-thousandth of a percent difference matters, but that’s not even worth bringing up.
-
The only reason any of you even bring it up is to try and sound smart in a pedantic, “ackshually” way.
Ironically your whole comment here is an elaborate “ackshually”.
This whole comment chain was me shutting down an “ackshually” with an even better one.
If you’re gonna be an annoying pedantic dick, you better be RIGHT, or someone else will be an even more annoying pedantic dick to you.
-
Am American.
…this is not the flex that the article writer seems to think it is.
I’m 100% certain that LLMs are smarter than half of Americans. What I’m not so sure about is that the people with the insight to admit being dumber than an LLM are the ones who really are.
A daily bite of horror.
It’s like asking if you think a calculator is smarter than you.
„It‘s totally a lot smarter than I am, no way could I deliver (234 * 534)^21 as confidently!“
Are you suggesting my 90’s calculator is smarter than LLM’s?
Hard to compete with that 90s confidence 😎
LLMs are made to mimic how we speak, and some can even pass the Turing test, so I’m not surprised that people who don’t know better think of these LLMs as conscious in some way or another.
It’s not a necessarily a fault on those people, it’s a fault on how LLMs are purposefully misadvertised to the masses
