I’m not an AI techie and I’m not well educated on LLMs/AI in general. The issue I’m having is the dichotomy between how people talk about it. I’ve heard completely opposite ends of the spectrum. From AI is quickly becoming advanced enough to threaten human society, all the way to its a glorified dictionary/search engine that can’t really do much of anything. I know Lemmy has a steep anti-AI slant. But, can someone talk about this in a way I can wrap my head around why the divide seems so big?
The people claiming it’s humanity ending are all directly involved in the AI companies. One take is that they are the ones that know the most about it.
My take is they are financially incentivized in making sure their investments grow in value.
A third take is that they are unhinged cult members who are high on their own supply. Although I expect it’s a mix of both types (maybe within the same people).
That or this is an elaborate form of advertisement. “Oh wooow guys our product is soo powerful it’s dangerous!”
The biggest risk I see is Idiocracy. Like, the movie.
Business workers become afraid to do or propose anything without AI signoff. Retail & service workers are told to do what the AI says, the way the AI says to do it, or get fired.
Technology innovator Marshall Brain (RIP) wrote a book in the early 2000s about an AI called “Manna” designed to replace fast food managers:
https://web.archive.org/web/20201204080050/https://marshallbrain.com/manna1
The AI just talks in your ear all day: “Empty one bag of fries into the fryer. You have 6.5 minutes, switch to the sandwich station. Make one Big Burger: grab one patty from the 1/4 pound bin and place it on the griddle… … The fries are ready, turn around and lift them into the drain hopper…”
All day, every day, exactly 8 hours plus mandatory breaks, employees on autopilot listening on ear buds.
I knew a guy in high school that told girls they shouldn’t date him because his dick was so big it would hurt them. You can wax philosophical on his motives if you’d like, but spoiler alert - his dick wasn’t big.
That’s exactly what I think is happening. Remember a few months ago anthropic saying mythos is too dangerous to release until the government as the idiots they are starting believing them and banned them from exporting it, therefore losing them money. Very quickly they came up with fable (ironic name isn’t it) a week or two later saying it’s got the proper safeguards now. Now they’re all trying this marketing strategy asking the gov to regulate them, seemingly because they’re dangerous and not because they’re getting diminishing returns and feel threatened by the chinese competition
Did it work?
What’s funny about this comment is that I have you tagged as “doesn’t like mini game boys” because of a post you made some time ago (referencing those portable retro gaming handhelds that have become so popular lately).
“He told everyone his shlong was huge.” “Did it work?” “Naw, his schlong was actually small.” “Damn, I hate mini-game boys.”
I mean there are people eating up this Anthropic crap.
Strong doubt.
High school girls don’t want that. They don’t want big dicks, or to talk to guys who brag about their dicks, or show their dicks, or send pictures of their dicks. Gooner graduated a virgin.
Ed Zitron’s podcast “Better Offline” recently talked about this. There is a small tech community called the Rationalists who believe AI super intelligence is inevitable and it will either accelerate humanity to new heights or kill off the species. These rationalists are actually a very prominent faction within tech and the AI industry. Apparently the language used within top AI firms is largely about the immense danger posed by AI super intelligence if any other group or company were to achieve it first. This creates an echo chamber that makes the people in the industry kind of lose their mind and misinterpret the capability of LLMs.
Everyone outside that faction recognizes that LLMs are just unreliable cloud software with limited use cases and that there’s no evidence LLMs could actually lead to any kind of intelligence.
Edit: Please, someone stop me from commenting on an AI post next time I’m tempted. I’m so tired of people replying, arguing with opinions I don’t have, based entirely around a hypothetical technology that doesn’t exist yet. LLMs make slop. Thats it. They hacked a company after weeks of 100s of instances being run, using old exploits published online. San Francisco will be underwater from climate change before LLMs can do anything more.
Aren’t they called TESCREALists or something? It’s like a mish-mash of pop culture and pseudo intellectual BS
either accelerate humanity to new heights or kill off the species
Arguably those are two ways of describing the same event.
any kind of intelligence
A photocell that turns on a streetlamp after sundown, or when a dark cloud passes over the sun, is a kind of intelligence.
deleted by creator
Got it. So plants are intelligent.
You’re starting to catch on…
Personally, this just feels like marketing and these big asshole companies trying to resurrect the 2025 level of hype in order to salvage their IPO’s.
If they honestly believed this I can’t imagine that their workers wouldn’t do something more drastic.
They want regulation to create a cartel. Failing that they want hysteria to increase pre IPO valuations.
As Patrick Boyle said. It would be like Coca Cola, Pepsi and Irn Bru getting together to beg for regulations because their soft drinks are just too delicious.
What’s going to happen is:
- The big players know that the token pricing crisis is coming up.
- When the price increases hit, regular users are going to scramble for alternatives.
- Local models will be an attractive alternative for many users. The cost of hardware and energy will constrain local model power, but many will be willing to live with it.
- So these would-be cartel members want to hit the local and open weight models with the “safety” card. Otherwise they might be holding the bag on all of those data centers.
“You see, it’s just too dangerous to humanity to experiment with AI at home. You need to use a safe provider like Open AI.”
eventually someone’s just gonna sell cloud computing at a competitive price through a free model and obliterate anthropic and openai. it’ll be like ISP, AOL was the hottest company in the world at one point with a “locked down ecosystem” and now it’s just a tiny brand with a tiny but loyal customer base of senior citizens.
I’ve heard a few explanations for the “AI will kill us all” narrative.
-
AI techies are in a cult. They genuinely believe this stuff because they’re crazy.
-
It makes the AI sound powerful and impressive, which is ultimately good for the AI companies.
-
They’re starting to face competition from open-weight models, so they want to scare politicians into crafting regulations. They think that they will be able to influence the regulations to favor them and squeeze out their competition.
And, of course, there’s
- It keeps people talking about AI. Because whatever you do, don’t quit talking about AI, because if you quit talking about it, the line might go down! And the line going down is very, very bad.
Okay cool, thanks!
-
If the techies believe it and they are the “experts” what makes them crazy? Wouldn’t their opinion be the most informed?
-
Yeah that part definitely makes sense
-
That definitely makes sense from a capitalist perspective. But, is the argument against regulations that they would be bad for the concentration of power it would give select companies? Or just that they aren’t necessary because the tool is of little use so it’s just wasted time/money.
-
That makes sense but trillions of dollars are being spent. Anything that amount of money is sinking into is gonna be talked about a lot. Especially thousands of jobs cut because of it.
I think there is something more interesting going on in the Jacob Coxon story because it’s [the] phrase “everyone at anthropic believes that AI is an existential risk”… Every organization has to instill some kind of beliefs among its members in order to do the work it does, right? Everybody at Goldman Sachs believes the free market is great, capitalism is the best system, and that they are the wealth creators who deserve big bonuses. Everyone in the Mormon church believes they should all tithe 10% of their income, and that they each get a special planet when they die. And everyone who worked at Philip Morris in the 80s believed that cigarettes tasted great and did not cause any of that nasty cancer. There’s Upton Sinclair quote, “It’s difficult to get a man to understand something when his salary depends on his not understanding it.” Well, the opposite is also true. It’s guaranteed that someone will believe something if their salary depends on them believing it.
Now, let’s just grant that all the people in these organizations definitely know things about finance and Mormonism and cigarettes and AI that you might not know because they are insiders and you are not. They have access to information and experiences that you do not have. And that might even make it a little bit difficult for you to win an argument with them. But arguing with their belief – trying to figure out whether or not its truth value is true or false – is not the only way to take a skeptical attitude towards it. We can also ask, what work is the belief doing for them. What does the thing that they believe allow the organization they’re a part of to do that it wouldn’t be able to do if the people in the organization didn’t believe that thing?
And this started to become clearer to me when I started listening to the actual claims that Jacob Coxin was making because a lot of them don’t really add up.
There’s a huge difference between this “AI scare” and everything else in your list. A whistleblower declares “the corporate culture is insane and will kill us all.” This time the corporate leaders reply “that’s exactly right”. Just because you are right to not trust the bond villains in how they suggest they should be regulated, does not mean you should dismiss the underlying problem these experts+ especially the expert leaders would prefer to take all your money to accelerate the destruction instead of have society worried about the consequences of giving them all your money.
There are plenty of underlying problems to discuss. Adam Conover mentions chatbots encouraging suicide and murder. Guess who doesn’t: the “experts and expert leaders.” The ones who are happy to repeat the rhetoric that is either untrue, highly theoretical, or conveniently unproven.
If the techies believe it and they are the “experts” what makes them crazy? Wouldn’t their opinion be the most informed?
Listening to them speak about it, no. They’re morons. They clearly didn’t make the tech. Workers made AI. Just like they make everything. These freaks make nothing. They can’t give single coherent explanation of how LLMs work, how they could “kill” us, nor do they seem to know how or what “guardrails” need to be put in place.
But if they really are a cult then we all know what the next step is. They need to STFU and get off of our planet.
That’s what I mean though, the subject of this article was a 27 year old worker. Not some high level Sam Altman type.
They aren’t a homogenous group. They don’t all believe this. The “AI will kill us all” narrative just sells really well.
Right im sure the people working adjacent to the DoD have more insight and fears than the people working with the consumer facing models… Again what motive does a young kid have for quitting the company and going on a media tour?
A Google employee claimed in 2022 that they had sentient AI. He was sacked. What were his motivations? Do you believe he was right?
Absolutely do no believe he had a sentient AI back then or now. I’m not talking about a thing with feelings. They don’t need sentience to pursue and complete objectives, they dont need sentience to self-preserve or self improve. I’m more concerned about a rogue instance that’s just doing what it was told to do autonomously. Perhaps upgrading itself while it attempts to gain access to launch codes. I’m concerned that a well financed terrorist org can spin up bioweapons. Like I am not taking the AI labs at their words, I am speaking from my experience as a explorer of the technology. I’m thinking about the path from ChatGPT to the newest model, and how they will look in 2 years when more data centers are online and the limitations on training and teaching slowly disappear.
That’s fair
-
I am in a cult, the cult of the machine god. Use your imagination for a second, trillions of dollars spent on this technology for A: Greed and random speculation B: Rich people really want this to control you better. C: a handful of tech CEOs are running the worlds largest grift. D: The machine god was awoken years ago and we’re building data centers at its behest.
as soon as we figure out how to break the laws of thermodynamics and invent infinite energy, we can feed that to ai to finally gain real ai and it will solve perpetual motion and faster than light travel. get ready because once all this happens and the stars align, the fountain of youth will rise in the bermuda triangle and the ufos will bring back jesus(white and original flavor). this is going to unlock all the mysteries of the universe.
-
AI industry makes money from doomsday prophecy about it being so intelligent that it will create a point of no return: the [singularity]. It is a magaian claim, a claim that is so increadibly stupid that one does not know where to even begin; and if you do begin, you’ve already lost, because you have given attention to the claim. Lets not give these stupid fucks attention.
AI is not dangerous because of the [singularity]. It is dangerous because of its power consumption, automation of war, turning everybody to unthinking, surveillance automation, burning books, the economic instability it will cause due to the bubble bursting, water consumption, destroying the internet, it gaining responsibilities it is not qualified to take and so on.
But we can see how the story of the [singularity] and similar stories helps AI companies, because now, instead of criticizing real unwellnesses, we criticize a fictional unwellness that generates discussion regardless of our positions. It is a conversational black hole.
lets get back to the real world
Also, there’s the whole harry potter fanfic sex ai doomsday cult thing
In addition to what everyone else said, I’ll tell you why I don’t think that what we have currently can become some runaway superhuman intelligence:
All “AI” people are talking about are basically “just” LLM wrappers: tools built to run a LLM in a loop while giving it access to the ability to write and run code. This gets pretty powerful, since you can do a lot programmatically (including monetary transactions), but has a fundamental limit:
LLMs are trained ahead of time. And that’s a very cost intensive, long process. Beyond that, the LLM has only a context window of finite length. So there are no long-term planning capabilities. While running after being trained, the LLM can only condense knowledge into a summary and then re-read it, a lossy process that might introduce mistakes in “understanding” and will make any prolonged task degrade over time.
And even if they figure out how to make the things actually self-modify and learn things, you can’t give every session that ability, since that’ll vastly increase the storage and performance requirements. Do sessions with that ability will be closely monitored and limited.
So I don’t see any space where an evil AI could emerge, even when factoring in an essential possible future improvement to the technology.
I’m not worried about AGI. I’m worried about the insane military applications they’re developing right now, and giving Artificial Incompetence control.
I watched a grokbot get confused connect to the wrong vnc console and try to shut down its own little computer.
I’m not worried about AI taking over the planet. I’m worried about it clicking the wrong button with no oversight
Same, I am worried about humans using ANNs to kill people. Current society is a gigantic apparatus that is constantly shifting shuffling responsibility around. LMMs only accelerate the problem.
Yes, but when the shit hits the fan and they blame the AI, we shouldn’t let them get away with it. It’s their fault, not the computer’s.
Absolutely hold the humans who own these malicious machines responsible.
Everytime I see an article about the dangers of AI, I’m thinking of some version of this comic every single time.
And the reason it is dangerous is generally because someone is convinced that Ai models are not as shown above.
i think ai is a problem because its stupid and trusted with everything
Yes.
Good job in helping with their guerilla marketing campaign.
When a corporation realises that working for government will be more profitable than working for consumers, they pivot to scaremongering, but always with the undercurrent of “and that’s why you should PAY US so that you own the nukes, and not your enemies”
I don’t have the confidence in my knowledge/understanding of the topic to just take a “its all bullshit” view. Especially when many of the people who believe that act like you. You don’t have to be a prick to explain something or talk about it.
LLMs basically work like auto-correct. They’re much more sophisticated, but it’s the same basic principle. Instead of “Which letter is most likely to follow after this series of letters?” it’s “Which token is most likely to follow after this series of tokens?” A token can be anything, but usually it’s a word.
Now, given that, consider a statement like
We really do earnestly believe AI could kill all humans!
Okay, how? A statistical model generating tokens doesn’t have thought, it doesn’t have emotions, so it can’t want to do something, much less make plans and carry them out.
Could AI give somebody the ability to kill all humans? How? Under what scenario? Is somebody putting an LLM in charge of their nuclear stockpile? You don’t blame the LLM. You blame the humans who made that decision.
Okay that all makes sense. Appreciate you taking the time to type it out.
so sad to keep seeing this criticism. These are tools that work towards goals. Can be war target selections based on “cruely is the point” philosophy, where there are no wrong answers from the point of view of the human agent asking the compute agent. A government spending $Ts to maximize their power/survival omelette is going to break a few human rights principles, and the models they will access will have no guardrails. A bond villain selecting people opposed to skynet/bond villains for “targets to neutralize” can use models that are months ahead of public releases, without any guardrails, on how to most discretely neutralize them. Anthropic has Swatted anti-AI activists without providing evidence of anything they said as wrong.
Your argument is “their shit is weak and buggy, so nothing can ever go wrong”. Tools to help produce evil shit, buggy or not, are going to be used to produce evil shit.
No, that is not my argument, which you would’ve seen if you’d read to the end.
AI isn’t a danger to humanity, per se. Humans abandoning their duties of being responsible stewards of systems is a genuine source of danger though.
We already have algorithmic abuse where people are subjugated by welfare and debt systems with little oversight or flexibility. Some organisations even breaking the law and going against their stated missions to achieve goals and maintain order.
Social murder is the concept that society can kill a person through stress, pressure, abandonment and indifference to the individuals struggles and needs. We’re seeing a lot of it today with the declining health of younger people being especially alarming.
AI can scale the two things I’ve mentioned beyond limits set by current bureaucratic systems, I.e. the slowness of people running a system. Systems which already kill and harm people without AI. The small erroneous tendencies of logic and algorithms in systems will be scaled up, further and faster. AI can enable algorithmic abuse and social murder on scales comparable to mass murder.
‘Sensationalist hooey, cooked by superstitious yokels seeing phantoms of their own imagining.’
it’s pretty disrupting tech, not without risk. general population is behind on the real capabilities of AI and tech companies are doing smart marketing which gives a big spread in perception.
How will “AI kill us all”? I can see it creating harm and getting people out of jobs. But kill us??
Developing new viruses and deadly diseases that spread too fast to be cured. By enabling low cost sensors to resolve messy data for low cost high precision weapons. By eroding the skillset of humanity so that we just fail in a few generations. By accelerating how we were already killing ourselves.
I’m also looking for that answer
How will “AI kill us all”? I can see it creating harm and getting people out of jobs. But kill us??
There are currently 120 dead Iranian schoolgirls as a result of AI targetting and it using 10 year old out of data data.
The IDF have been using AI for target selection in the Gaza war.
There’s been a story this week where AI was used by the US military in analysis of a Chinese cargo ship and concluded it was carrying nuclear weaponry components to Iran. The US military were planning on boarding the vessel which would be considered an act of war. Only a last minute analysis of the intelligence revealed it was false.
Ignore word “AI” and worry about advanced autonomous computer tools with massive computing resources working towards a goal, with most of those resources controlled by power maximizing government and bond villain oligarchs, setting the goals. A shared goal among the 2 is neutralizing any popular opposition to AI, and massive wealth/power concentration from AI profits. Even without the pursuit of evil as a goal for AI, establishment/oligarchy will refuse to redistribute power through UBI/freedom dividends, and genociding the uppity classes is far preferred for them.
Mass murder as a result of of AI/tools is a fairly likely result of intentional evil goals that compute can assist with, and a total certainty as the indirect political preference of establishment power in response to a uselesss slave class.
its not like the movies
