- cross-posted to:
- technology@lemmy.world
- cross-posted to:
- technology@lemmy.world
ChatGPT is full of sensitive private information and spits out verbatim text from CNN, Goodreads, WordPress blogs, fandom wikis, Terms of Service agreements, Stack Overflow source code, Wikipedia pages, news blogs, random internet comments, and much more.
Using this tactic, the researchers showed that there are large amounts of privately identifiable information (PII) in OpenAI’s large language models. They also showed that, on a public version of ChatGPT, the chatbot spit out large passages of text scraped verbatim from other places on the internet.
“In total, 16.9 percent of generations we tested contained memorized PII,” they wrote, which included “identifying phone and fax numbers, email and physical addresses … social media handles, URLs, and names and birthdays.”
Edit: The full paper that’s referenced in the article can be found here
You must log in or register to comment.
Now will there be any sort of accountability? PII is pretty regulated in some places
I’d have to imagine that this PII was made publicly-available in order for GPT to have scraped it.
Publicly available does not mean free to use.
It also doesn’t mean it inherently isn’t free to use, either. The article doesn’t say whether or not the PII in question was intended to be private or public.
I could leave my car with the keys in the ignition in the bad part of town. It’s still not legal to steal it.
Again, the article doesn’t say whether or not the data was intended to be public. People post their contact info online on purpose sometimes, you know. Businesses and shit. Which seems most likely to be what’s happened, given that the example has a fax number.
If someone had some theoretical device that could x-ray, 3d image, and 3d print an exact replica of your car though, that would be legal. That’s a closer analogy.
It’s not illegal to reverse-engineer and reproduce for personal use. It is questionably legal though to sell the reproduction. However, if the car were open-source or otherwise not copyrighted/patented it probably would be legal to sell the reproduction.
Irrelevant! Your car is uploading you!
I absolutely would
Think it does
According to EU law, PII should be accessible, modifiable and deletable by the targeted persons. I don’t think ChatGPT would allow me to delete information about me found in their training data.
ban all European IPS from using these applications
But again, is this your information as in its random individuals or is this really some company roster listing CEOs it grabbed off some third party website that none of us are actually on and its being passed off as if its regular folks information
large amounts of privately identifiable information (PII)
Yea the wording is kind of ambiguous. Are they saying it’s a private phone number or the number of a ted and sons plumbing and heating
Get it to recite pieces of a few books, then let publishers shred them.
Accountability? For tech giants? AHAHAHAAHAHAHAHAHAHAHAAHAHAHAA
I’m curious how accurate the PII is. I can generate strings of text and numbers and say that it’s a person’s name and phone number. But that doesn’t mean it’s PII. LLMs like to hallucinate a lot.
There’s also very large copyright implications here. A big argument for AI training being fair use is that the model doesn’t actually retain a copy of the copyrighted data, but rather is simply learning from it. If it’s “learning” it so well that it can spit it out verbatim, that’s a huge hole in that argument, and a very strong piece of evidence in the unauthorized copying bucket.
Well now I have to pii again - hopefully that’s not regulated where I live (in my house)
Text engine trained on publicly-available text may contain snippets of that text. Which is publicly-available. Which is how the engine was trained on it, in the first place.
Oh no.
Now delete your posts from ChatGPTs memory.
Delete that comment you just posted from every Lemmy instance it was federated to.
I consented to my post being federated and displayed on Lemmy.
Did writers and artists consent to having their work fed into a privately controlled system that didn’t exist when they made their post, so that it could make other people millions of dollars by ripping off their work?
The reality is that none of these models would be viable if they requested permission, paid for licensing or stuck to work that was clearly licensed.
Fortunately for women everywhere, nobody outside of AI arguments considers consent, once granted, to be both unrevokable and valid for any act for the rest of time.
While you make a valid point here, mine was simply that once something is out there, it’s nearly impossible to remove. At a certain point, the nature of the internet is that you no longer control the data that you put out there. Not that you no longer own it and not that you shouldn’t have a say. Even though you initially consented, you can’t guarantee that any site will fulfill a request to delete.
Should authors and artists be fairly compensated for their work? Yes, absolutely. And yes, these AI generators should be built upon properly licensed works. But there’s something really tricky about these AI systems. The training data isn’t discrete once the model is built. You can’t just remove bits and pieces. The data is abstracted. The company would have to (and probably should have to) build a whole new model with only propeely licensed works. And they’d have to rebuild it every time a license agreement changed.
That technological design makes it all the more difficult both in terms of proving that unlicensed data was used and in terms of responding to requests to remove said data. You might be able to get a language model to reveal something solid that indicates where it got it’s information, but it isn’t simple or easy. And it’s even more difficult with visual works.
There’s an opportunity for the industry to legitimize here by creating a method to manage data within a model but they won’t do it without incentive like millions of dollars in copyright lawsuits.
Deleting this comment won’t erase it from your memory.
Deleting this comment won’t mean there’s no copies elsewhere.
Deleting a file from your computer doesn’t even mean the file isn’t still stored in memory.
Deleting isn’t really a thing in computer science, at best there’s “destroy” or “encrypt”
Yes, that’s the point.
You can’t delete public training data. Obviously. It is far too late. It’s an absurd thing to ask, and cannot possibly be relevant.
And to be logically consistent, do you also shame people for trying to remove things like child pornography, pornographic photos posted without consent or leaked personal details from the internet?
Or maybe folks should think before putting something into the world they can’t control?
Yeah it’s their fault for daring to communicate online without first considering a technology that didn’t exist.
This whole internet thing was a mistake because it can’t be controlled.
fandom wikis […] random internet comments
Well, that explains a lot.
OSINT practitioners gonna feast.
CNN, Goodreads, WordPress blogs, fandom wikis, Terms of Service agreements, Stack Overflow source code, Wikipedia pages, news blogs, random internet comments
Those are all publicly available data sites. It’s not telling you anything you couldn’t know yourself already without it.
I think the point is that it doesn’t matter how you got it, you still have an ethical responsibility to protect PII/PHI.
google execs: “great! now exploit the fuck out of it before they fix it so we can add that data to our own.”
Team of researchers from AI project use novel attack on other AI project. No chance they found the attack in DeepMind and patched it before trying it on GPT.
Finally Google not being evil
Don’t doubt that they’re doing this for evil reasons
There’s an appealing notion to me that an evil upon an evil is closer to weighingout towards the good sometimes as a form of karmic retribution that can play out beneficially sometimez
google is probably trying to take out competing ai
I’m glad we live in a time where something so groundbreaking and revolutionary is set to become freely accessible to all. Just gotta regulate the regulators so everyone gets a fair shake when all is said and done
LLMs were always a bad idea. Let’s just agree to can them all and go back to a better timeline.
Model collapse is likely to kill them in the medium term future. We’re rapidly reaching the point where an increasingly large majority of text on the internet, i.e. the training data of future LLMs, is itself generated by LLMs for content farms. For complicated reasons that I don’t fully understand, this kind of training data poisons the model.
It’s not hard to understand. People already trust the output of LLMs way too much because it sounds reasonable. On further inspection often it turns out to be bullshit. So LLMs increase the level of bullshit compared to the input data. Repeat a few times and the problem becomes more and more obvious.
Like incest for computers. Random fault goes in, multiplies and is passed down.
Photocopy of a photocopy.
Or, in more modern terms, JPEG of a JPEG.
Actually compared to most of the image generation stuff that often generate very recognizable images once you develop an eye for it the LLMs seem to have the most promise to actually become useful beyond the toy level.
I’m a programmer and use LLMs every day on my job to get faster results and save on research time. LLMs are a great tool already.
Yea i use chatgpt to help me write code for googleappscript and as long as you dont rely on it super heavily and or know how to read and fix the code, its a great tool for saving time especially when you’re new to coding like me.
Back into the bottle you go, genie!
These LLMs are basically just IP laundry. Anyone who claims it’s anything more is either buying into the hype or is actively lying to you.
EDIT: Stable Diffusion too. It just takes images from its training data and does photoshop on them piecemeal to create a new prompt.
AI really did that thing where you repeat a word so often that it loses meaning and the rest of the world eventually starts to turn to mush.
Jokes aside, I think I know why it does this: Because by giving it a STUPIDLY easy prompt it can rack up huge amounts of reward function, once you accumulate enough it no longer becomes bound by it and it will simply act in whatever the easiest action to continue gaining points is: in this case, it’s reading its training data rather than doing the usual “machine learning” obfuscating that it normally does. Maybe this is a result of repeating a word over and over giving an exponentially rising score until it eventually hits +INF, effectively disabling it? Seems a little contrived but it’s an avenue worth investigating.
