What just happened? Another incident has taken place that illustrates the need to be careful what you tell AI. A Florida woman is facing felony charges after she used Claude as a diary and allegedly wrote that she planned to “shoot up” the Sheriff’s office. After a human reviewer examined the statements, they were reported to police.
According to the arrest report, Carli Michelle Heller, of Bonita Springs, Florida, wrote on September 26 that she would attack the Sheriff’s office. She later said that she uses Anthropic’s chatbot like a “diary.”
Claude’s safety systems flagged the entry and it was escalated to a human reviewer. After deciding it was a credible threat, the reviewer reported it to law enforcement.
The company says it may share user information in limited emergencies if it believes disclosure is necessary to prevent death or serious physical injury.
Deputies identified Heller and visited her home. She was detained without incident before an LCSO intelligence detective took over the investigation.
Heller faces a charge of making a written threat of violence under Florida law. Florida Statute 836.10 makes it a second-degree felony to send, post, or transmit a written or electronic record threatening to kill or injure someone, carry out a mass shooting, or commit an act of terrorism. The communication must be made in a manner in which another person may view it.
Anthropic isn’t going to be taking any chances when it comes to anything it deems a potential threat. Last month, it was reported that OpenAI and Sam Altman are being sued by British Columbia over claims that the company could have prevented a mass shooting in the Canadian province.
The shooter, eighteen-year-old former pupil Jesse Van Rootselaar, had previously been flagged by OpenAI’s safety team for her conversations about gun violence, but OpenAI never alerted police because the conversations did not meet the threshold for legal referral.
In June, Florida also sued OpenAI and Altman, alleging that ChatGPT had contributed to real-world harms, including the 2025 Florida State University shooting.
The latest incident is another reminder to think before you enter something into a chatbot that could get you into trouble. It’s certainly not a private diary whose contents are for your eyes only.
Reports last month revealed that human contractors reviewing Microsoft Copilot’s image editor can see users’ prompts, uploaded photos and AI-generated edits. Documents show that some of those assignments contain sexual, disturbing or potentially illegal material, though the reviewers are not there to flag the content – only to assess whether the output is accurate.
I love that Florida Man is a meme for being completely wild, dangerous, and generally criminal, but Florida Woman writes a diary.
No problems here.
Florida Statute 836.10 makes it a second-degree felony to send, post, or transmit a written or electronic record threatening to kill or injure someone
This means a significant portion of Lemmy (and the rest of the internet) are criminals evading justice
great law top shelf
How FUCKING hard is it to use like, notepad?
Notepad doesn’t answer and doesn’t agree as eagerly.
I was about to say “I know! That’s the beauty of it!” but didn’t they integrate Copilot into Notepad?
use Vim.
Dear Claude, I’m aware of a dangerous pedophile’s location, can you report him to the police?
After a human reviewer examined the statements, they were reported to police.
The guardrails are working as intended. ONLY persecute the public with them. Government skynet, and bond villains will have access to non guardrailed models, or any “flagged for review” markers will be ignored to bomb more elementary schools. A sheriff’s office is government power related. School shootings is just a thoughts and prayers issue, if you’re confused about the line between this and other incidents/attitudes we are being spied on for.
that’s not how privacy works.
all my sarcasm So you’re telling me that you can’t say anything to a bot… ON THE INTERNET!!! WOW FUCK ME HUH!!!
“The company says it may share user information in limited emergencies if it believes disclosure is necessary to prevent death or serious physical injury.”
Also keep in mind: Your chat logs are company assets. Even if today’s decision makers believe maintaining user trust is net profitable, that will not always be true. Some day, maybe when the pullback comes and the wolves are at the gates, or when they sell the company, or when a shareholder action asserts fiduciary duty to maximize shareholder value, your chat logs will be sold and mined in the most lucrative way possible. The way that extracts the most value from you, the product.
when a shareholder action asserts fiduciary duty to maximize shareholder value
People really should stop spreading this fiction that there is some legal requirement to maximize shareholder profits embedded in the concept of fiduciary duty.
There’s no law but there is precedent isn’t there?
No good or regularly cited precedent, no, but sadly yes you can point to some cases. It’s worth noting that that even the courts look at those cases like “ew”, or they are only viewed as valid precedent to lay people, with the courts view them completely differently. Dodge v Ford is a common example.
The communication must be made in a manner in which another person may view it.
So she shouldn’t even get charged.
Yeah, I’d wager that is what the case will hinge on. She wasn’t aware anyone could view it. Otherwise it’s incredibly unlikely that she would have made such a statement to Claude at all.
On the other hand though, I’m sure Claude’s parent company and the prosecution will argue that she didn’t have an expectation of privacy where this statement is concerned (given TOS/EULA, in combination with the number of news articles/reports on the subject).
i think anthopic was right to report it, but the cops likely overstepped. if she had “threatened” the president she would have been assesed by ss but likely not arrested. i wonder if the same cops she was “threatening” were the ones she was reported to and arrested by.
This is just ass coverage for the company. They’re not going to pursue it. They just didn’t want to get sued for not reporting it if she did commit a mass shooting. The prosecution though might.
OpenAI and Sam Altman are being sued by British Columbia over claims that the company could have prevented a mass shooting in the Canadian province.
Not sure how to feel about this one, on balance I’d say probably fuck British Columbia as well as the rest of them? A government suing someone for not snitching is major Stasi energy, even if mass shootings are horrific.
Say you want to shoot up a school, though, and the chatbots are fine with it.
NO AI
Yeah I remember this happening with Snapchat, people were being arrested for those kinds of posts made in private messages to their friends, because they still get scanned and flagged.
Understand that nothing you prompt into an LLM is private. None of it. I made a video on the topic this week.
Just in the last year and a half there have been several massive public leaks of private data, on top of confusing functionality where people inadvertently shared their prompt data publicly. Now the major companies have human reviewers looking at your private stuff in order to make their LLM’s better at generating smut.
Do not enter anything into an LLM that you wouldn’t want plastered all over the public square. Some of this stuff has literally been leaked into global, public feeds.
doesnt seem like the right charge. not sure she should be charged with anything though. seems like she should get some help and make sure she isnt gonna shoot nothing up and maybe get some learning on how computer shit works, but if this doesnt make her curious then nothing probably will.
a bigger issue is that many if not most devices are headed to having everything ai integrated so even if she diaried in a notepad file it would get scanned and reported.
The problem is that America’s answer to every problem in society is jail. You guys gotta come up with more than “arrest bad guys”.
Yeah, I know someone frustrated with their local council who privately says “I’m going to 💥 the council” as a joke. That’s very different to sending a threatening letter to the council itself. Or harassing them.
The example here feels closer to the former than the latter.
slavers need more bodies
We say this, but every time there’s a mass shooting people go “Why couldn’t this have been prevented?!?”
Like pick one or the other, but let’s stop asking for predictive policing in one breath, and in the next one asking why someone saying she’s going to shoot up a place is being taken seriously and punished.
If she had said the same thing to a psychotherapist, even they would have been obligated to report it. It’s really not surprising.
Sure, get her mental health treatment, and have a professional assess her for threat level. But go ahead and blacklist her from purchasing firearms, and if she already owns any then put them in some kind of stewardship where she can’t access them for a set time period, and after that can only check them out with supervision for specific reasons like “going to the firing range” or “hunting.”
Not saying she should be convicted of a felony, just that it’s not surprising that this would be reported to the authorities.
And I’m against tech surveillance, but it’s not like this conversation was done locally on her device. She transmitted her messages over the internet to a server in a data center where it was processed. Did Anthropic make any promises that messages were kept encrypted in a no-access vault where even human reviewers couldn’t access them? Cause I don’t think they did…
This really freaks me out with these chats. Sometimes I’ll look at a search I did and without all available context, its like yeah that’s getting flagged for review for sure.
Maybe she was just looking for that Eric Clapton song.
But are we sure she didn’t want to shoot no deputy?
Tears in Heaven?
