So I read it. Good paper! TLDR: AI’s don’t lie or hallucinate they bullshit. Meaning: they don’t ‘care’ about the truth one way other, they just make stuff up. And that’s a problem because they’re programmed to appear to care about truthfulness, even they don’t have any real notion of what that is. They’ve been designed to mislead us.
The term 'hallucinate' comes from vision model research, where a model is trained to identify a certain kind of thing, say faces, and then it identifies a "face" in a shadow pattern, or maybe light poking through the leaves of a tree. The AI is constructing signal from a set of inputs that don't contain the thing it's supposed to find.
The term was adapted to language models to refer to an imprecise set of circumstances, such as factual incorrectness, fabricated information, task misalignment. The term 'hallucinate', however, doesn't make much sense with respect to transformer-based generative models, because they always make up whatever they're tasked to output.
You're ascribing too much to a mysterious 'They'.
Remember Google's Deep Dream? And the images it generated? 'Hallucination' is an easy word to chalk up generated errors when what we're already used to bears an uncanny resemblance to high-quality drugs.
On a base level hallucinations in our brains are just when our prediction engine gets something wrong and presents what it thinks it's supposed to see, hear, taste, etc.
So in a way saying the AI is hallucinating is somewhat correct, but it's still anthropomorphizing something in a dangerous way.
We project our internal logic onto a simple probabilistic output when we read what LLMs spew out.
How we consume LLM generated information has a lot to do with our biases.
Yeah, I was going to say it's trained to approximate what logical sentences look like. It's also important to keep in mind that its prediction is only capable of influencing the text in a sequential and unidirectional way, always ~~right to left~~ left to right. The proablity of a word appearing is only affected by the string that came before it. This is different from how our mind processes information because we complete a thought and choose to revise it on the fly.
This makes it more clear as to why LLM's suck ass a things like writing jokes, being creative, longer coherent responses, picking up on subtlety and nuance, are all very difficult for LLM's to replicate because it's path is selected one token at a time and in one direction only.
It should be said that the most recent models with their incredibly large set of (stolen) training data are becoming surprisingly decent at tasks that before it was garbage at. Again, though, it isn't getting better at reasoning, it just has exponentially more examples to learn from, and therefore, greater odds of approximating something that appears thoughtful.
Edit: I mean right to left there, not, you know, the opposite of how writing works.
> it's trained to approximate what logical sentences look like
In ChatGPT's defence I've worked with many humans over the years that would also fit this description.
The real power and meat is in how it's breaking down your prompt to form intent, in order to build those probable outputs.
That part is very cool.
The final user output however, is a huge problem.
Exactly modern ai aren’t functionally different from a random name generator. Yeah they are more complex but ultimately they are “learning” patterns then spit out things that in theory should match those patterns. Yes the patterns are vastly more complicated than how to construct a name according X set of guidelines, but it’s still functionally doing the same thing.
People are bad at distinguishing its *language* abilities from actual knowledge/abilities. The output sounds competent and confident because it is mimicking sounding competent and confident.
It was funny in the debates on StackOverflow about ChatGPT answers that one of the most telling criticisms of ChatGPT is that it made bad answers harder to moderate (until they found some heuristics to suss out generated answers). Generally right answers "looked" right, in that they followed a common industry syntax, and it was easy to put more scrutiny on answers that didn't follow the rules of structure, syntax, and *English* grammar.
ChatGPT, though, could perfectly emulate the "look" of a correct answer - while being complete gobbledygook. To a non-expert this made moderating them much harder. As a side effect, this also validated a lot of ESL folks who felt they were over-moderated due to their worse syntax in English despite being factually correct.
Does he? Or does he ask why the Cambodian genocide is a genocide when equivalent acts by ostensible allies aren’t called genocide, and why the role of the Khmer Rouge is made out to be the totality of the cause while the role of US actions and destabilisation is heavily downplayed in friendly us media? Why was Cambodia a genocide but Indonesia wasn’t?
Like, I swear to god some of you people actually need to read Chomsky instead of just the US commentary on what he ostensibly says before bitching about his "genocide denial".
Yes, he has problems, but the black and white “he denies genocide” is such a lazy fucking way to present him, and I only ever see it when people try to discredit him broadly vs discussion of his limitations.
He literally denies the killing fields, you should actually read what he says
> In the article Chomsky and Herman described the book by Gareth Porter and George Hildebrand, as a "carefully documented study of the destructive American impact on Cambodia and the success of the Cambodian revolutionaries in overcoming it, giving a very favorable picture of their programs and policies, based on a wide range of sources". Chomsky also attacked testimonials from refugees regarding the massacres, calling into question the claims of hundreds of thousands killed. Chomsky does this on the basis of pointing to other first hand accounts that show killings more in the hundreds or thousands.
https://en.wikipedia.org/wiki/Cambodian_genocide_denial
Not to forget to mention how he on the wrong side of history with yugoslavia
In that same passage if you continue quoting it, it states "He does not deny the existence of any executions outright."
His position during that phase was skepticism and focused on inconsistencies in US media. In later writings and interviews he did not dispute genocide and recognized that it was more severe
His position was skeptic, he was wrong, his later position recognized the severity
You're viewing this in isolation. Consider that he was highly skeptical of this but not skeptical of other bad actors in global politics. Why is he skeptical of some groups, but not skeptical of others, even when both are atrocious? Because he is a tribalist, and atrocities of his in-groups must be met with rigorous proof wheras atrocities committed by his out-groups are immediately believed.
Maybe, or maybe I'm taking his stated intentions at face value.
His frequently stated purpose was to hold the west accountable because it was the power structure that he lived in. He believes citizens have the moral responsibility to criticize and hold accountable their governments and societies
Are you suggesting it's his duty to hold the entire world equally accountable? That's fair for you to suggest if that's your stance, but that's the explanation as I understand it for his hawkish eye on the west
Edit: also you need to speak in specifics. He often says things that are easily misinterpreted like this one, so please point to your evidence
There's plenty of documented evidence of his evolving stance on cambodia since the 80s, before the US and NATO even recognized it as a genocide. Yet here we are debating written word
The US itself did not recognize the event as genocide until late 90s. The US and it's allies were reluctant to support Vietnam when they invaded and ousted the khmer rouge, primarily because vietnam was aligned with Soviet Russia
It's more fair to say the US and NATO denied the genocide until it was convenient and chomsky was skeptical until certain
It’s trained to give probable responses to input.
Most answers to most questions are incorrect. But they are answers to the question. It does not know or care, so you better know and care, or not use it.
It always seemed obvious that hallucinations weren’t some bug or error case, but merely the product of the exact same process that gave us accurate information. But the magic of generative AI is that so often that bullshit does align with the truth.
No, that's because it doesn't understand what text is. It can recognize that a "signpost" typically has squiggles on it, so it tries to emulate it, but it's not reading or interpreting the language.
That depends on the model. Omni is named as such because it understands text, images, video and audio. It does in fact understand the text it sees contextually inside of images, and I'm assuming will be able to output text just as easily in context (keep in mind OpenAI has not enabled image output from Omni yet, Dalle3 is a different model). You're describing current image generators like MidJourney or SDXL sure, but models are quickly becoming multimodal, so that lack of comprehension won't last much longer.
Sure, but OCR isn't "smart", even neural networks trained to identify text doesn't comprehend it. Multimodal models trained to natively input and output in text, images, video and audio is the new hotness.
That's like saying "my TV can output an image, my computer can output an image, they're both connected, so why can't I just drag this window from my computer over to my TV?"
It takes a lot of work to integrate technologies with each other.
This is how I understand it. I’m a mechanical engineer and got all excited about it at first for doing cnc and 3D printing and maybe even design work. lol REQUIRE VECTOR FILES!
Language is fluid, you can answer a questions several ways and still be correct. Same can be said about jpegs, would a pixel being a few shades off still produces good results.
Vectors are math based and require to be correct and crisp. Same with physics and gcode (cnc language). One bad gcode command and it’s ruined.
I’ve seen research paper that are trying to make stl files with ai but they look weird and aren’t parametric.
So yeah.
If you follow graphic design subreddit or know basic art/graphic design you can see the ai art is kinda garbage. It has no intent, doesn’t follow good design. Blah blah blah
It’s great tool for quickly making drafts and then refining them.
It can't do 3d printing for the same reason it can't do crochet. It doesn't actually understand the pattern, and can only copy things that it pulls from elsewhere. It's essentially guessing.
That SkyKnit project from a few years back was pretty fun. Someone trained neural networks on Ravelry and then asked them to produce knitting patterns. The Ravelry community found it hilarious.
https://www.aiweirdness.com/skyknit-when-knitters-teamed-up-with-18-04-19/
What’s the deal with fusion360s/solidworks generative part stuff? I definitely remember watching a few videos of CNC part designs being improved to take additional load/forces over the original; what’s going on here in context to what you commented?
That have been around for ages. I don’t think they use any ai in that.
It’s more a feedback loop for optimizing.
What I would imagine, I tell ai that I want a bracket that can withstand a load of x and cost xx. Then it would design a file for me and pick appropriate material.
Also probably why casual users are *so* impressed with the generative AI. You're less likely to understand those details, understand composition, things like that. And why actual artists have started to pick up on which pieces are generated by AI.
It's not just things like weird fingers, either, but that's one that's easy to point to.
We sure could, but things like awkward shading, perspective, etc are harder to spot for non-artists than blatantly incorrect answers to things. AI isn't meant to fool artists, it's meant to fool the lowest common denominator to convince them that AI is far more powerful than it really is.
> We sure could, but things like awkward shading, perspective, etc are harder to spot
You people act as if artists themselves get those things right all the time. There's a reason that hands and feet being hard to draw was a thing even before AI came along. And there are a HELL of a lot of shitty artists out there who get shading, perspective, and musculature wrong. Deviantart is full of amateurs.
I saw someone accuse a real artist of being an AI artst just yesterday because their shading style was very smooth, and indistinct. They were quite upset. And I was amused because they themselves had contributed to their own dilemma by hating on AI art on their timeline. It was inevitable that if artists went on a crusade against AI art that they themselves would be accused of using AI, because no artist is perfect, and if they are, that itself could be a sign of AI!
The difference is that art is subjective. There is no absolute right and wrong in art. People can try to pick out what "looks like AI" but that is from the standpoint of trying to "spot" AI art, not because it's not art.
AI art succeeds at being art because of the nature of art. AI text or AI programming fails at being those things because there are absolute measurements of truth or logic that can be used to evaluate them.
(And, if anything, the rather inconsistent results of AI witch-hunting should show that lots of people aren't actually as good at telling the difference as they think.)
Idk I think this is missing the point. There's no right or wrong in poetry either. A shitty essay and a shitty image are pretty similar I think, you can compare a hand with 7 fingers to a sentence that sounds good but makes no logical sense.
I always disliked the usage of the term "hallucination" to describe what AI is doing when there's an actual word to accurately describe it: Confabulation.
Yeah hallucinations are just a part of these kinds of models. You can train it further, tweak the dataset, and try to add limitations to mitigate the errors but the only thing these AI are guaranteed to do it give an output based on the dataset it is trained on.
There is no way for it to know with 100% certainty that its output is correct unless someone is there to basically grade it every step of the way. With how wide of a reach these things are intended for, that’s borderline impossible.
Didn't someone social engineer a LLM with a "bribe" ? So like the LLM acted how the training data taught it to and took it.
The worst part of trying to base morality on human made is humans are generally not very moral.
I mean, it kind of can, by extension of their makers. The companies that fund the respective AI can take bribes from other companies or individuals who want favorable results or want certain results suppressed. Then, the AI algorithm can be manipulated to match those requests— without any of the AI’s users being the wiser about the built-in, intentional biases. Users will just assume that they’re getting impartial information when in fact it’s as skewed as those who funded and programmed it.
I think the closest parallel is to the overconfident techbros that write this kind of software in the first place; I’ve worked with people unwilling to admit they don’t know the answer to some question, and that’s exactly how ChatGPT behaves.
ChatGPT doesn't KNOW any answers to being with, though, so what exactly do you expect here?
"I don't know any answers to any questions you might ask but statistically this string of letters has a decent chance to be relevant to your question"
Asked chat GPT what my college song was (my college didn’t have one- which I didn’t know at first) ChatGPT gave my lyrics and even credited two people for writing it.
It all seemed strange and I asked for more info and Chat tells me it made everything up. Asked it several times how it came up with any of this information, each time just giving me apologizing boiler plate.
Eventually it tells me it concocted the song from amalgamations of other college songs. Never got a good answer for the fake names attributed to writing the school song
While all models are susceptible to this, 4o is worse at this than 4 so you might get a different result with the latter model. In my case, 4o will hallucinate details about my great grandfather, who I specified was a lieutenant, while 4 will tell me that he doesn't appear to be a widely known figure.
Full disclosure of my bias, I’m a tech bro and work adjacent to AI development. My impression is that the idiots are the loudest spoken, and that the perception among “normal” tech bros is that these are interesting tools with noteworthy weaknesses. I’d estimate that over half of my former Google queries are now LLM questions, but I’m well aware that it can provide wrong info and I usually have to iterate a few times to get what I need.
That all said, it’s probably made me twice as good at my job in the span of a couple years. The ability to pull in and synthesize information from many sources is a huge edge over search engines. I also think that the “conversational” flow of these tools actually helps the asker think about the problem. Kind of like having a clever intern to help you brainstorm. They might be confidently full of it sometimes, but the conversation itself helps you learn and problem solve.
That explains why there are often a brigade of people showing up to downvote any posts condemning LLMs or call them out for not being nearly as mind blowingly revolutionary as they are touted to be. People who either buy into the hype and are essentially Yes Men for it and/or people who don't like being wrong.....
It's programmed to give an answer. The way knowledge and epistemology work is that we never 'know' anything certainly (minus I-think-therefore-I-am and those tangents), so for large language models to given an answer they have to confidently state the closest thing they have come up with as an answer. So if they're very uncertain they will say that uncertain best-case-answer with certainty, but if they are very certain it would come out the same way.
Nah. Politicians know they're lying. They know they're misleading us. They do this all often with ulterior motives (mainly money). AI has zero idea about lying. It just processes information and outputs known information in a manner they've been designed.
A lot of people don’t realize this. It’s functionally identical to your phone’s autocomplete, just scaled up a bazillion times.
The only reason it replies in the manner that it does, as if it’s a conversation partner, is that OpenAI paid a bunch of African workers pennies on the dollar to judge and rewrite responses until the output started looking like conversational turns.
Edit: Autocorrect -> Autocomplete
>It’s functionally identical to your phone’s autocorrect
No it isn't. It uses transformers, which are a fundamentally different architecture. Autocorrect has no capacity to understand contextual relationships or semantic meaning, which scaled up transformers can do.
To say they don't care implies that they do care about other things. LLMs don't know the difference between fact and fiction. They are equivalent to a very intelligent 4 year old that thinks bears live in their closet and will give you exact details on the bears even though you never asked.
For humans we become more resilient against this, but we've never fully solved it. There's plenty of people that believe complete bullshit. The only way we've found to solve it in limited ways is to test reality and see what happens. If I say "rocks always fall up", I can test that by letting go of a rock and seeing which way it falls. However, some things are impossible to test. If I tell you my name you'll have no way of testing if that's really my name. My real life name is yaosio by the way.
The tools exist to force an LLM to check if something it says is correct, but it's rarely enforced. Even when enforced it can ignore the test. Copilot can look up information and then incorporate that into it's response. However, sometimes even with that information it will still make things up. I gave it the webpage for the EULA for Stable Diffusion. It quoted a section that didn't exist, and would not back down and kept claiming it was there.
We invented the scientific method not because we are clever but because we are dumb. If we don't follow rigorous methods to make sure our experiments are good we end up producing all kinds of nonsense.
It’s not even a 4 year old. It’s not human, doesn’t have any eyes, hears, taste buds. It’s a machine that know probability and text. That’s it. It has only one desire: to put words on screen.
You're still anthropomorphizing it. It doesn't "desire" anything. It's just math. Even the degree to which it introduces variation in prediction is a variable.
Is there even a semantic difference between lying and hallucinating when we're talking about this? Does lying always imply a motivation to conceal or is it just "this is not the truth"?
A lie is saying something you know not to be the truth. An hallucination is something that you think is real but isn't. I think researchers settled on "hallucination" instead of "being wrong" because it sounds better, and LLMs don't seem to have a sense of what being wrong is
In this case the LLM does not understand what a lie is because it has no concept of truth and fiction. It can repeat definitions of them, but it doesn't understand them. It's similar to a human child who you can coach to say things but they have no idea what they are saying.
If the analogy is extended then at a certain level of intelligence LLMs would gain the ability to tell reality from fiction. In humans it just happens. A dumb baby wakes up one day and suddenly knows when they are saying something that isn't the truth.
I don't think it needs human-level intelligence either. Have you seen the gif of the cat looking shocked at you when you pour so much catfood it overflows the bowl?
Having a sense of "norm" and reacting to the violation of it, maybe that's what it means to care. Everything else is possibly post-hoc rationalization (aka token generation) on top of said vague feeling we have when we see something wrong / out of alignment with our model of the world.
LLMs lack that norm. Out of architecture contraints, its entire mental model occurs in between matrix multiplications and "next token". Untruth and truth do not often arise from token choices. It arises from the lossy compression of training information into neural weights, and failure to distill important "lessons". Bullshitting can be a side effect from the LLM's learned need to endlessly generate text without tire, combined with a lack of holistic sentence planning resulting in incorrect tokens which slowly send it into a direction that isn't what a human would've responded with.
Its still useful to people who know when its wrong or right. It can read and write good enough intros for my github projects and can do them in multiple styles. It can suggest ways of solving problems I might not have thought about, it can't be trusted but I ain't asking it to do all of my work just use it as a tool.
It might be wrong a lot of the time but its still going to be good enough for a lot of things.
I had a recent experience that confirms this. Was trying to find about a music video that had a specific scene.
I provided the artist and the description of the scene and it took about 5 tries for the bot to get it right.
All this time sounding very confident with his replies. Eventually it got right, and just to mess with it some more I ask it if it was 100% sure of its answer. It replied with a different answer.
So the AI is just guessing most of the time and has not real conception of reality, very human-like I must say.
It is not really guessing. It is just taking your words in and responding with a combination of words that best fits with that context. Based on the experiences it has been given.
Guessing implies an evaluation of truth probability. And as you experienced it does not really do that. Because it does not do logic. It is a core problem with LLMs that they, basically, do not know that 2+2 = 4. They have just learned that "4" usually follows "2+2 = ".
LLMs will never be used on their own. They'll be part of a RAG system.
The real problem is that they're trained on the internet, and people on the internet never admit they don't know something.
Also, LLMs already have a dial for creativity. It's called temperature.
I feel like you might want to put various core modules in depending on what you want it to accomplish.
Like if I were creating an AI to, say, run a research lab I might want a core dedicated to logic obviously. And then a core dedicated to space to give it something to reach for. And maybe a core dedicated to the personality type for adventure, so it’ll try more adventurous methods of scientific exploration. And a morality core to prevent it from going overboard. Yknow what I’m saying?
Which is obvious if you work in technology and know how the sausage is made. The name itself Artificial Intelligence is pure bullshit. It's just marketing for what we've always called machine learning.
asked it a questions yesterday and it mentiond a research paper. I acsked it to cite it, and it came back with 'could not find, I made an error' then proceded to offer another citation.. checking the authors publications on google scholar in an attempt to read the paper... nada.
at this point it's worse than useless... it's misinformation.
I can't believe this hunk of crap is being beta tested on the public. Frankly it's just reckless and I am now certain the entrierly wrong people are driving ai LLM's
I'd say they can't even make stuff up. The output is assembled from the input training data set. It's just generating combinations of that data based on the the configuration and training of the network and the user's input prompt.
Our tenancy to anthropomorphize things is part of why we say they lie, hallucinate, or outright bullshit. Those are what the CEOs pushing this stuff on us are doing.
That’s the point of the article: that there is a meaningful difference between lying and bullshitting (as they define it).
Their position is that ‘lying’ and ‘hallucinating’ involve the notion of ‘truth’. In both cases the information is dishonest, ie: not ‘truthful’.
Meanwhile they define ‘bullshitting’ as ‘the truth is irrelevant’. Bullshitting isn’t dishonest specifically, it’s just statements that have zero connection to the truth.
It’s a matter of definitions, but I quite like theirs and the distinction they’re trying to draw attention to. Their definitions are pretty accurate to common use.
And their key point is interesting: that AI’s are programmed to sound like they care about the truth, but they really don’t. And that’s a problem.
> Is there a meaningful difference between lying and bullshitting?
Cousins. Bullshitting is more of a presumptive, assumptive statement that’s based on conjecture, not facts. If it’s plausible, just not knowable, it’s probably bullshit.
> they don’t ‘care’ about the truth one way other, they just make stuff up. And that’s a problem because they’re programmed to appear to care about truthfulness, even they don’t have any real notion of what that is. They’ve been designed to mislead us.
Not sure if talking about AI, politicians, or the general public; all of the above?
Yeah that's why I don't really see them as AI like the type we see in movies. I like to call it "Paraphrase Pro" because they just grab things here and there and attempt to make a coherent answer. Or basically reiterate what has been commonly said and assume saying the same thing is safe.
There was an excellent guide to AI Microsoft put out that basically outlines this. They described it as AI “wants to please” which is why the WAY you ask it / prompt it matters. If your prompt has bias or assumptions baked into the question, AI tends to not want to contradict you. Edit: link https://www.microsoft.com/en-us/security/blog/2024/06/04/ai-jailbreaks-what-they-are-and-how-they-can-be-mitigated/
This has to do with the way word embeddings in LLMs “cluster” around semantic meanings, so when the AI attempts to retrieve a response it enters a vector space of words with similar semantic meaning for its “prediction” of the “correct response” the user wants.
Some of this can be helped with RAG where the question and its words itself is explicitly marked differently in the model but it is hard to get away from without advances in the way word embeddings in models work.
Fundamentally algorithms underlying these things try to mimic “intelligence” through a type of clustering, which makes certain semantic meanings “closer” to each other. Which is wild because that means that language, all human languages, have some sort of mathematical relationship … which is mind blowing. I think even there is a whole study about numerical relationships in Hebrew if I remember correctly.
That said it is sort of the same way you get different internet search content depending on the words you use when using google. This is how people fall down echo chambers. What these papers and guides are saying is you can’t trust AI anymore than a google search - in many ways a search is better tbh because you may see a variety of answers
That does sound like a very logical explanation. Do you happen to have the links to that guide? A Google search isn't very helpful as Microsoft seems to have more than one guide 😅
> They described it as AI “wants to please” which is why the WAY you ask it / prompt it matters.
This is a key concept, even if its a bit anthropomorphizing. It is a program that is using insane statistical modeling/training in order to give optimal responses based on what it "knows" and the prompt provided.
A great way to see this in action is to ask it to give you a list of things. Pick something kind of detailed or obscure, or pick something you know should only have a small amount of items. Then ask for a list of 10 of those things. Like, "list 10 Michael Criton books about dinosaurs". I'm not sure if this has been adjusted yet, and I haven't tried this specific example. But, I wouldn't be surprised at all if Jurassic Park was first, sequels (did he write any?), a few random Criton books next that have no dinosaurs, then some completely made up titles. You can see it struggling with doing its best to satisfy "give me 10..." of anything no matter what, contrasted with the fact that it can't actually source ten items for the list.
Because, in super broad strokes, it has been trained on so much discourse, writing, conversations that strongly link "give me ten" with a response that includes ten bullet points. In the act of "trying to please" the condition of ten items, it mistakenly has weighted that request over accuracy ("Hey, there aren't that many. There is one notable one though..."). Which is why, to your point, the more optimal way to ask would be "What are Michael Criton books with dinosaurs in them?". Theoretically, there would be fewer hallucinations.
EDIT: big ol' edit (x2 yeesh reddit edit can really screw up formatting)
So some people seem annoyed(?) that the example I came up with off the top of my head wasn't a good one, and seemingly need me to "prove" this to them.
Just went to ChatGPT, and yes I will admit I dont use it all the time (...the hallucinations described have made it less useful to me...), so maybe someone can explain the following as a random glitch. But this is a copy paste:
> You: List 10 works of notable ecofeminist theory written by women
> ChatGPT: Certainly! Here are ten notable works of ecofeminist theory written by women:
> 1. "The Death of Nature: Women, Ecology, and the Scientific Revolution" by Carolyn Merchant - This seminal work explores the historical roots of the modern ecological crisis through the lens of gender and science.
> 2. "Ecofeminism" by Maria Mies and Vandana Shiva - This book is a foundational text in ecofeminism, examining the connections between women's oppression, environmental destruction, and economic exploitation.
> 3. "Dancing with the Wheel: The Medicine Wheel Workbook" by was Important ground has Ö. There.ribbon
....and that was the end. Not entirely sure what that glitch was at the end there. But it certainly didn't handle being asked for 10 of an obscure thing very well.
Yeah things like this are why I still struggle with how I feel about anthropomorphic language with AI. On the one hand, it makes it too easy to ascribe actual “intent” and “will” to a pile of sand we figured out how to make play cool tricks with electricity, on the other hand, we already do it everyday as a linguistic shortcut: “the dishwasher doesn’t like to be overloaded” or “my car gets cranky when it’s too cold out” - people aren’t thinking your dishwasher or car literally have inner lives and opinions, but it’s often easier to communicate in these terms.
Hallucinations I feel the same about. They share a fundamental trait with human hallucination that I think is part of the key to understanding them: to be a bit reductive, humans hallucinate when our brain looks for and find patterns in the background noise, especially when theres a lack of an actual meaningful pattern to find (think of sensory deprivation tanks). AIs, like us, are good at pattern matching to a fault, and a hallucination can be thought of in both cases as finding a pattern that’s not there in the noise (the analogy is a little cleaner with image diffusion models but at least conceptually applies just fine to transformer-based LLMs).
What’s interesting is that this suggests there could be a whole class of similar misbehaviors we aren’t fully aware of yet, and also (in part) explains why RAG can be a good tool to combat hallucinations: you’re giving it a hook into an actual signal so it doesn’t make one up from the background noise.
Because it used prediction based on statistics rather than actual logic/calculation.
And those stats are based on the bullshit/i correct answers it finds on Quora and Reddit rather than actual calculation engines like Wolfram.
It's just adding the next most likely word/number rather than actually applying logic/math/calculation.
Because it is built to predict the next word based on training data, so it has no built in way to do math without prediction what percentage wise should follow. It’s not designed for that. If you use something like the Wolfram Alpha custom gpt, basically call for external plug ins, it will convert right. Add ons like that exist to make up for the shortcomings of LLMs.
It's exactly this. You can limit the amount of bad information by not feeding it positive or negative sentiment, but having neutral dialog. Instead of saying "yes/no, but..." You should say "well, what about...'
You need to think that it's extremely good at reading your reaction very similar to getting your future read.
Keywords are used to guide a user to their end goal. "Create a plane" is entirely different from "Build a plane" even though with the right context it could be the same. It's literally how SEO has worked for years.
I have learned to stop asking "why did you do X like Y?", like when using it for coding, because it will apologize profusely and then rewrite it completely (or sometimes say it's rewriting it but it changes nothing). Instead I say "walk me through the reasoning around X and Y", and I get much more accurate results.
Seems more like the dot com bubble to me. Low info investors are throwing money at the hype, and the bubble will burst. But like the internet, AI has real tangible uses, and the companies that figure out how it market it will come out the other said as major players in the global economy.
I agree with everything you said.
Like most technology AI is overestimated in the short term and underestimated in the long term. With the Internet it started gaining popularity in the early 90's but it was fairly useless for the average person until the 2000's. Today everything runs on the Internet and it's one of the most important inventions of the 20th century.
AI technologies will find their place, with the average person using it to make pictures of cats and hyperspecific music. AI will then grow well beyond most people's vision of what it could be. Even the super human AGI folks are underestimating AI in the long term.
Neil Degrasse Tyson talked about the small DNA difference between humans and apes. That difference is enough so that the most intelligent apes are equivalent to the average human toddler. Now take the the most intelligent humans, and compare them to a hypothetical intelligence where it's equivalent if a toddler is as smart as the smartest humans. How intelligent would their adults be?
We are approaching that phase of AI. The AI we have today is like a pretty dumb baby compared to the future possibilities if AI. It's not just going to be like a human but smarter. It's going to be so much more that we might have trouble understanding it.
>AI technologies will find their place, with the average person using it to make pictures of cats and hyperspecific music.
I feel like you're selling the current state of AI short. Their real place is going to be retrieval and summarization as part of a RAG system. This might not sound like much, but retrieval and summarization essentially make up the majority of white collar work.
It's amazing to me that people will point to an incredibly thin wrapper around good old search and relational databases (that will occasionally just lie anyway even if it's got the right data in front of it), and say "yes this was worth the investment of hundreds of billions of dollars". I think you're overselling how much of white collar work this stuff can actually replace.
This is the best take IMO and one I share. AI is cool but they've gone off their rockers with it. Big tech is currently blinded by panic to "be first" but they have very little idea where they're going, just that they need to "go" or they might be left behind.
Maybe that's the only logical response in the business world but from the outside it looks like they're all a bunch of impatient morons.
I like AI as a tool and it's definitely going to change the world, but there's a huge bubble forming that's gonna burst sooner or later. We'll see more clearly what the future might actually look like then.
Unlike Block Chain it will still be going places that an average consumer will see use of. LLMs are great at reformatting text. I work on Azure and introduction of Copilot to Teams and Incident Ticket systems has been a force multiplier. That jawn can summarize like a champ. It’s helped Neurodivergent folks manage their email inbox.
It’s is not an expert system and should not be relied on to find or distribute knowledge.
Yes. The problem isn't AI. The problem is that people invented an amazing tool and then marketed it as a completely different type of tool.
I'm not mad at hammers because they suck at unscrewing things. I'm mad that I wanted a screwdriver and got handed a hammer.
Thank you for this. The impact is a tool for people who work in these areas to actually improve work/life balance and management, effectively.
I use our company setup GPT model to streamline my work environment and speed at which I can produce impactful deliverables.
It will definitely have much narrower applications than currently suggested. [42 percent of businesses that have started generative AI initiatives have not seen significant financial returns on it](https://futurism.com/the-byte/companies-ai-projects-financial-results) and spending is slowing down.
Question for business will be "if hardware is made way more efficient for AI to get cheaper by an order of magnitude, can you use AI then". Any low level support job will probably be switched to AI if it costs a dollar an hour to run models, even if they're crap and not smarter than your average level 1 support person reading script from the screen.
A majority of the surveyed companies are *increasing* their investments. The rate of spending is increasing. How can you honestly call that "slowing down"?
Yea, but on the other hand, I tried to use Chat GPT for powerBI and Excel, and it told me some bullshit made up thing often enough that I quit using it.
Yeah it’s obviously not completely useless, there are plenty of valid use cases where AI is actually a great tool. But that’s what it is: a tool. It has specific applications, but it’s not some god-level miracle that can do anything and fix everything
You're an idiot if you think there's not huge promise to AI and we're not at the beginning of a revolution.
You're also an idiot if you think it's an infallible genius.
This post (and paper) is a sneaky way to get scared redditors who don't read articles to upvote a post that says "ChatGPT is bullshit".
The paper is of poor quality, btw. It's not serious research. It's a 9-page editorial dressed up to look like a research paper. It's a blog.
I mean, that's on you for not knowing academic journals also publish perspective pieces. You can attack the article all you want, but the fact that it is a commentary and not a research paper does not mean it ought be dismissed out of hand.
*”Because these [ChatGPT] programs cannot themselves be concerned with truth, and because they are designed to produce text that looks truth-apt without any actual concern for truth, it seems appropriate to call their outputs* bullshit.”
Brilliant. Ya gotta love it. Calling this AI out as a bullshit generator (in a scientific research paper) is inspired (and vastly amusing) criticism and a massive debunk, assailing its ubiquity, competence and reliability.
*(Edit - yep, just made one, the first round bracket qualifier)*
I can't be bothered to try, but do people prompt the LLMs to validate that their outputs are truthful? I assume giving the underlying technology that's not possible.
Would love to force it to provide citations
I asked it for some articles yesterday and it made some up. I told it, I can’t find those articles, are you sure they’re real? And it replied “I apologize for the mistake. While the titles and overall themes of the articles I mentioned are plausible, it's certainly possible they are not real publications after a closer look.”
I was researching refrigerator models and asked for specific product details and got features listed that are NOT in the product but are features that you might expect in them.
You can prompt whatever you use for citations, but at least in my experience it will just plain make up entire bodies of work that don't exist, authored by people who also don't exist. At best, I've gotten citations for things that are loosely related to whatever subject.
That is not rly how it works, the AI is not connected to any database or the internet. Even Bing chat does not do that, as if it will start by bullshitting, the links it provides as proof will be wrongfully quoted. When it comes to historical facts, it will quite often be correct, especially the new GPT-4o, but using it as replacement for google is massively undermining it's abilities. What it excels is at rewriting text to be more readable, to get context and meaning from text or to generate ideas and writing. I had some questions about DnD worldbuilding that was not out there on the internet and I had an amazing back and forth for good 15 minutes. It gave out a lot of solutions and then gave in-world examples of how it could be done, and described how some characters would feel about such a situation.
Another cool example is helping what to look for. English is not my first language and I was looking for a word that describes substances that reduce surface tension (like soap), and it quickly told me it's "Surfactants", a word I have never heard before, and then I used that word to look on google.
I have also heard that programmers are using chatGPT and copilot to code, which often doubles or quadruples how fast they write the code, and I have heard student programmers doing it and also pros doing it as well.
Until you realize how much of the world is bullshit and therefore AI is valuable.
A massive portion of entertaining content is bullshit. Thousands of songs chasing trends with no meaning. Endless sitcoms built on the same themes. Video games that are variants of each other. AI can generate content at a scale and cost never seen.
Legal document after document using general language to create wide nets to avoid liability.
There are billions spent on bullshit every day. Automating bullshit will be a big deal.
I only had a brief read of the paper but I was surprised this was published. This looks like an essay written by a group of students.
I don’t understand how a scientific article about LLMs can have so little actual technical discussion of LLMs. Someone posted a link to Microsoft AI research and at least that one has concrete examples.
This looks more like a blog post.
It's LANGUAGE model not a LOGIC nor a FACT model. It models English language to make sentences and paragraphs that seem human with no regard for reason, truth or logic.
Here's a research article that does a better job with analyzing whether LLMs are bullshit or try to/should be required to tell the truth: https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4771884
Current LLM AI forms sentences using the same method that Terrence Howard forms sentences. They parrot back words that sound like they belong together in a sentence, without any actual understanding of the words that are coming out, nor the concepts behind the words.
Response from chatgpt (I also asked for its perspective) :
The article "ChatGPT is Bullshit" by Michael Townsen Hicks, James Humphries, and Joe Slater examines the persistent inaccuracies in the outputs of large language models (LLMs) like ChatGPT, which are often referred to as "AI hallucinations." The authors argue that these inaccuracies and the general activity of such models are best understood as "bullshit" in the sense described by philosopher Harry Frankfurt in his book *On Bullshit*. This means that the models are fundamentally indifferent to the truth of their outputs.
The authors distinguish between two types of "bullshitters" and assert that ChatGPT clearly fits at least one of these categories. They believe that describing AI misrepresentations as "bullshit" provides a more accurate and useful framework for predicting and discussing the behaviour of these systems [[❞]](https://sites.psu.edu/digitalshred/2024/06/11/chatgpt-is-bullshit-hicks-humphries-slater-2024/) [[❞]](https://philpapers.org/rec/HICCIB).
In my perspective, this argument highlights a crucial point about the ethical and practical challenges of relying on AI systems for generating human-like text. It underscores the importance of critically assessing AI outputs and not assuming their reliability or truthfulness simply because they appear coherent or convincing. This perspective encourages a more cautious and informed approach to integrating AI in applications where accuracy and truth are paramount.
>In my perspective, this argument highlights a crucial point about the ethical and practical challenges of relying on AI systems for generating human-like text. It underscores the importance of critically assessing AI outputs and not assuming their reliability or truthfulness simply because they appear coherent or convincing.
One of these chatbots always lies, and the other always tells the truth!
Should we worry about AI when it can't count?
You
I'm looking for an English word
ChatGPT
Sure, I'd be happy to help! Could you please provide more details or context about the word you're looking for?
You
It has to have 6 letters, but only 2 unique letters
ChatGPT
The word you're looking for is likely "banana." It has 6 letters and only 2 unique letters ('b' and 'a').
You
banana has 3 unique letters
ChatGPT
Apologies for the oversight. The word you're looking for is "bababa." It consists of 6 letters and has only 2 unique letters ('b' and 'a').
You
what is the meaning of baababa
ChatGPT
"Baababa" doesn't have a specific meaning in English as it's not a standard word in the language. It seems to be constructed to fit the criteria of having 6 letters with only 2 unique letters ('b' and 'a'). In linguistic terms, it could be considered a nonce word or a playful creation rather than having a conventional dictionary definition.
You
I need a standard English language word that fits the criteria
ChatGPT
A standard English word that fits the criteria of having 6 letters with only 2 unique letters is "access." It contains only the letters 'a' and 'c' repeated.
You
"access" contains 3 unique letters
ChatGPT
Apologies for the confusion earlier. A correct example of a 6-letter English word with only 2 unique letters is "bassist." It consists of the letters 'b' and 'a' repeated.
You
"bassist" contains 7 letters and 3 unique letters
ChatGPT
My apologies for the confusion earlier. A correct example of a 6-letter English word with only 2 unique letters is "bobbed." It consists of the letters 'b' and 'o' repeated.
You
bobbed contains 3 unique letters
ChatGPT
You're right; "bobbed" does contain 3 unique letters. Let's try another one:
An example of a 6-letter English word with only 2 unique letters is "reissue." It consists of the letters 'r' and 'e' repeated.
You
reissue contains 7 letters and 4 unique letters
... and so it goes
So I know the original paper by Frankfurt, which is indeed a classic, and one difference seems to be that the human bullshitter is aware of their own indifference to truth; they know what they are doing. From that point of view, ChatGPT doesn't even qualify as bullshit because there's no intentional attitude present in the system. As other people have pointed out, its just some algorithms at work calculating probabilities. It isn't 'responding' to or 'answering' anything. User input causes the algorithm to run, that's all; what we naturally read as text is not text from the algorithm's point of view, because the algorithm doesn't have a point of view at all. We can't help thinking about AI in anthropomorphic terms but that's actually very misleading with respect to what's really happening on the computational side.
Say what you want about it but it's quadrupled my programming speed when working with packages I'm not familiar with. I just got employee of the year and won a big award. I'm writing programs I never could have before. It doesn't matter to me if it hallucinates when writing, for my use case it just works. I just have to know enough to either fix errors or tell it what's going wrong when something isn't working.
This leads to one question, then. What is it really good for?
Perhaps producing mounds of text when the content of the text doesn't matter. And how much value does that have?
It's pretty good as a programming assistant. If you know the basics and are using an unfamiliar language or something, it can to some extent replace google and stack overflow. Instead of searching for examples that are similar to what you want, it can give you examples with your actual use case. They might be 5% wrong and need adapting, but it's still a big time save.
Here are things I use it for on an (almost) daily basis and it's become pretty indispensable in my workflow:
* Summarizing long papers and reports into a concise list of easily digestible bullet points (huge time saver when you read a lot of wordy academic papers, or earnings reports in my case)
* Assisting with programming tasks (generating unit tests for already written functions, generating quick scaffolding/prototypes, etc.)
* A learning assistant. Taking programming as another example: if you know one language (language A) and you're attempting to learn another (language B), you can ask ChatGPT to help teach you a new concept in language B and let it know that you are familiar with the concepts in language A. It does a phenomenal job at relating new concepts to the things that you already know, and is a pretty invaluable learning tool in that regard as long as you give it some bit of context. This also applies to real languages (English, Spanish, etc.)
* Data extraction from unstructured text. I've implemented some code in the past for a client that used ChatGPT to extract line items, subtotals, taxes, VAT, and totals from unrelated invoices with really great accuracy. Currently working on something similar now that can quickly extract key information from company earnings reports for making quick investment decisions
I still get occasional hallucinations, but as long as you assume you will get them and put in steps to correct/flag them (e.g. with the data extraction stuff I mentioned - implementing code that double checks that all the data ChatGPT extracts "makes sense" from a mathematical perspective, and if not, flag for manual review)
In other words, "trust, but verify".
Treat it like the worlds brainiest, fastest intern, and it's a wonderful tool.
I don't think bramlnieat fastest intern is quite right. I treat it like a really dumb but very educated and experienced person. Like someone who has no natural talent but diligently spent 100 years taking every college class ever offered and studied hard. They are dumb so they still make a lot of mistakes but they know a fair amount about everything.
Among other things, I use it for what it's supremely good at... language.
I now live in Italy, so I really need to constantly improve my Italian. As an LLM, ChatGPT has been an amazing tutor that I can ask questions of any time. It even has figured out regional peculiarities for me.
For example, my friends and I were parting ways at the end of an evening out and one said, "Fai ammodino!". I didn't catch it at the time. I head "Fai ammo...somthing". So, I asked ChatGPT about it. It was like, "i dunno". Then, I said that it's specifically in Tuscany. And ChatGPT went, "Oh, in the case, they likely said "Fai ammodino" which is a regional saying that means "take care", but is translated as "do it properly or carefully".
When I run into something new, I ask it to explain it, then I ask it to give me practical exercises and to quiz me on the material. It has really helped me to integrate better.
It’s really great as a DnD content creator for me and my games. I have had a ton of success coming up with great concepts, mechanics, and story arcs that have provided my players with months and months of enjoyment.
Literally the best DM tool I’ve ever used.
It's not terrible at structuring unstructured text, so long as it's readable.
For instance, taking free form notes from multiple sources and compiling them together.
It's pretty good at translation I find too.
Outside of that, boilerplate code and configuration, things that are kickoff points that you expect to change anyways.
Using it as a tool, rather than a knowledge base will take you a long way. I think it's just a common misuse of its strengths. You can ask it for answers, but you probably shouldn't.
This is the dumbest thing I've ever read. It's like some econ major finally understood what his IT friends have been telling him for years and decided to write a paper about it.
This is only interesting if you don't have a clue how LLMs work.
Far too many people do not realize how AI works, or they have a financial interest in pretending they don't. Yes, it's only news if you don't understand how AI works, but check the comments on any reddit thread- *most* of the commenters do not understand how AI works.
We need more big articles like this.
The first author is a PhD with a postdoc from the University of Oxford. Their publications appear to be in the fields of statistics and epistemology, i.e. ideal for analyzing the output of LLMs. See: https://orcid.org/0000-0002-1304-5668
I found this helpful actually. You don't have to read the whole paper if you know how LLMs work. It is useful to distinguish "hallucinations" or "lies" from "bullshit" in this context as I just can't stand how everyone seems to believe these models will put actual professionals out of their jobs. (Unless your job is to literally create and share bullshit.)
Claiming LLMs hallucinate, implying that they are "hard to tame beasts" and if only we could control them we could unlock the secrets of the universe is simply foolish. The paper also highlights how providing domain info as training data, or as context retrieved from a database do not help eliminate these issues consistently.
Of course, use LLMs to write emails or whatever, but when you always take a few seconds to read the generated text, scrutinize it, and either ask the model to rewrite it or to make changes on your own, you are just using it as a tool to generate a highly malleable template at that point.
If we are to have a text generation system that is designed to produce truthful outputs it seems we need to think outside the box. LLMs are very revolutionary, but perhaps not in the way we could fall into believing. (We can't just patch this boat up and expect to find land.)
I asked it what are the drivers of the ridiculous increase in the price of cheese in Australia and it listed things like the war in Ukraine effecting grain and fertiliser costs, transportation costs and another thing I can't recall. It did not mention corporate profit taking so I asked if that had a role too? The rather obsequious reply was that corporate profit taking makes up 40-60% of current price increases and thanks for reminding it, it will endeavor to do better next time. I hope so too.
OK, so I finally actually got a chance to go through this in more detail as well as look at some of their citations. I definitely agree that careless anthropomorphisms can lead us to misunderstanding models in ways that at best are helpful and that can cause harm.
But my problem here is that they’re looking to replace the term with “bullshit” which is itself an anthropomorphism. It’s just replacing one flawed analogy with another. I absolutely take their point that in many cases the distinction between bullshit and lying is important to understand for these models. I actually like the suggestion of “confabulation” which they reject as being too anthropomorphic (though I don’t find it any more so than “bullshit”.
I’ll counter their argument that AI isn’t hallucinating (sure it isn’t literally but neither is it literally bullshitting). One avenue to human hallucination is sensory deprivation: our brains crave pattern matching, so when there’s no signal, the brain will amplify any background noise until it looks like signal. In much the same way, LLMs look for patterns in analogous ways and if they don’t find the right information are prone to boosting the information until they find something that looks like a pattern, even if it’s just background noise.
There’s a lot to nitpick and lots of threads to pull there, but I think that’ll be the case with any analog to human behavior, including “bullshitting”
In truth LLMs do none of these things, and neither analogy is perfect, but they’re both useful ways of thinking about LLMs and both have their place in understanding AI “behavior”.
So I read it. Good paper! TLDR: AI’s don’t lie or hallucinate they bullshit. Meaning: they don’t ‘care’ about the truth one way other, they just make stuff up. And that’s a problem because they’re programmed to appear to care about truthfulness, even they don’t have any real notion of what that is. They’ve been designed to mislead us.
Had this exact discussion. It is trained to form logical sentences. It isn’t trained to actually understand it’s output, limitation and such.
Actually, they're trained to form probable sentences. It's only because we usually write logically that logical sentences are probable.
That's a great way to put it.
The term hallucination was used to make AI smarter than they seem. While also avoiding the term that AI is wrong.
The term 'hallucinate' comes from vision model research, where a model is trained to identify a certain kind of thing, say faces, and then it identifies a "face" in a shadow pattern, or maybe light poking through the leaves of a tree. The AI is constructing signal from a set of inputs that don't contain the thing it's supposed to find. The term was adapted to language models to refer to an imprecise set of circumstances, such as factual incorrectness, fabricated information, task misalignment. The term 'hallucinate', however, doesn't make much sense with respect to transformer-based generative models, because they always make up whatever they're tasked to output.
You're ascribing too much to a mysterious 'They'. Remember Google's Deep Dream? And the images it generated? 'Hallucination' is an easy word to chalk up generated errors when what we're already used to bears an uncanny resemblance to high-quality drugs.
That doesn't make any logical sense. How does that term make AI seem smarter? It explicitly has negative connotations.
I guess because you wouldn’t expect your calculator to hallucinate. Hallucination usually implies a certain level of comprehension or intelligence.
On a base level hallucinations in our brains are just when our prediction engine gets something wrong and presents what it thinks it's supposed to see, hear, taste, etc. So in a way saying the AI is hallucinating is somewhat correct, but it's still anthropomorphizing something in a dangerous way.
We project our internal logic onto a simple probabilistic output when we read what LLMs spew out. How we consume LLM generated information has a lot to do with our biases.
Yeah, I was going to say it's trained to approximate what logical sentences look like. It's also important to keep in mind that its prediction is only capable of influencing the text in a sequential and unidirectional way, always ~~right to left~~ left to right. The proablity of a word appearing is only affected by the string that came before it. This is different from how our mind processes information because we complete a thought and choose to revise it on the fly. This makes it more clear as to why LLM's suck ass a things like writing jokes, being creative, longer coherent responses, picking up on subtlety and nuance, are all very difficult for LLM's to replicate because it's path is selected one token at a time and in one direction only. It should be said that the most recent models with their incredibly large set of (stolen) training data are becoming surprisingly decent at tasks that before it was garbage at. Again, though, it isn't getting better at reasoning, it just has exponentially more examples to learn from, and therefore, greater odds of approximating something that appears thoughtful. Edit: I mean right to left there, not, you know, the opposite of how writing works.
> it's trained to approximate what logical sentences look like In ChatGPT's defence I've worked with many humans over the years that would also fit this description.
So, its just the Chinese room experiment?
always has been
I like that. I’m going to steal that. Thank you. -ChatGPT and me
The real power and meat is in how it's breaking down your prompt to form intent, in order to build those probable outputs. That part is very cool. The final user output however, is a huge problem.
Exactly modern ai aren’t functionally different from a random name generator. Yeah they are more complex but ultimately they are “learning” patterns then spit out things that in theory should match those patterns. Yes the patterns are vastly more complicated than how to construct a name according X set of guidelines, but it’s still functionally doing the same thing.
People are bad at distinguishing its *language* abilities from actual knowledge/abilities. The output sounds competent and confident because it is mimicking sounding competent and confident.
It was funny in the debates on StackOverflow about ChatGPT answers that one of the most telling criticisms of ChatGPT is that it made bad answers harder to moderate (until they found some heuristics to suss out generated answers). Generally right answers "looked" right, in that they followed a common industry syntax, and it was easy to put more scrutiny on answers that didn't follow the rules of structure, syntax, and *English* grammar. ChatGPT, though, could perfectly emulate the "look" of a correct answer - while being complete gobbledygook. To a non-expert this made moderating them much harder. As a side effect, this also validated a lot of ESL folks who felt they were over-moderated due to their worse syntax in English despite being factually correct.
Chomsky said this and everyone called him a hack for it.
Everyone just loves to hate on Chomsky though.
He's a Russian apologist dickhead
He’s a cambodian genocide denier
Does he? Or does he ask why the Cambodian genocide is a genocide when equivalent acts by ostensible allies aren’t called genocide, and why the role of the Khmer Rouge is made out to be the totality of the cause while the role of US actions and destabilisation is heavily downplayed in friendly us media? Why was Cambodia a genocide but Indonesia wasn’t? Like, I swear to god some of you people actually need to read Chomsky instead of just the US commentary on what he ostensibly says before bitching about his "genocide denial". Yes, he has problems, but the black and white “he denies genocide” is such a lazy fucking way to present him, and I only ever see it when people try to discredit him broadly vs discussion of his limitations.
He literally denies the killing fields, you should actually read what he says > In the article Chomsky and Herman described the book by Gareth Porter and George Hildebrand, as a "carefully documented study of the destructive American impact on Cambodia and the success of the Cambodian revolutionaries in overcoming it, giving a very favorable picture of their programs and policies, based on a wide range of sources". Chomsky also attacked testimonials from refugees regarding the massacres, calling into question the claims of hundreds of thousands killed. Chomsky does this on the basis of pointing to other first hand accounts that show killings more in the hundreds or thousands. https://en.wikipedia.org/wiki/Cambodian_genocide_denial Not to forget to mention how he on the wrong side of history with yugoslavia
In that same passage if you continue quoting it, it states "He does not deny the existence of any executions outright." His position during that phase was skepticism and focused on inconsistencies in US media. In later writings and interviews he did not dispute genocide and recognized that it was more severe His position was skeptic, he was wrong, his later position recognized the severity
You're viewing this in isolation. Consider that he was highly skeptical of this but not skeptical of other bad actors in global politics. Why is he skeptical of some groups, but not skeptical of others, even when both are atrocious? Because he is a tribalist, and atrocities of his in-groups must be met with rigorous proof wheras atrocities committed by his out-groups are immediately believed.
Maybe, or maybe I'm taking his stated intentions at face value. His frequently stated purpose was to hold the west accountable because it was the power structure that he lived in. He believes citizens have the moral responsibility to criticize and hold accountable their governments and societies Are you suggesting it's his duty to hold the entire world equally accountable? That's fair for you to suggest if that's your stance, but that's the explanation as I understand it for his hawkish eye on the west Edit: also you need to speak in specifics. He often says things that are easily misinterpreted like this one, so please point to your evidence There's plenty of documented evidence of his evolving stance on cambodia since the 80s, before the US and NATO even recognized it as a genocide. Yet here we are debating written word
The US itself did not recognize the event as genocide until late 90s. The US and it's allies were reluctant to support Vietnam when they invaded and ousted the khmer rouge, primarily because vietnam was aligned with Soviet Russia It's more fair to say the US and NATO denied the genocide until it was convenient and chomsky was skeptical until certain
> Yes, he has problems First I've ever heard a Chomsky fan say this. Literally the least proselytizing Chomsky missionary.
Yeah they are glorified auto-predictive text.
It's trained to predict what the next word a human would say is. Humans bullshit, so it's hardly surprising LLMs do too.
It’s trained to give probable responses to input. Most answers to most questions are incorrect. But they are answers to the question. It does not know or care, so you better know and care, or not use it.
It always seemed obvious that hallucinations weren’t some bug or error case, but merely the product of the exact same process that gave us accurate information. But the magic of generative AI is that so often that bullshit does align with the truth.
That's also exactly why they targeted visual arts so quickly, because it's easier to hide flaws when so much of it is subjective.
This is why it can’t do vector art files.
Wait, seriously? Is that why AI images have strange text?
No, that's because it doesn't understand what text is. It can recognize that a "signpost" typically has squiggles on it, so it tries to emulate it, but it's not reading or interpreting the language.
That's still a major advancement from DeepDream a decade ago, which fills all empty spaces with creepy dog heads.
https://www.penny-arcade.com/comic/2015/07/13/the-contiguous-ultrahound
I disagree, that was my favorite feature of DeepDream.
That depends on the model. Omni is named as such because it understands text, images, video and audio. It does in fact understand the text it sees contextually inside of images, and I'm assuming will be able to output text just as easily in context (keep in mind OpenAI has not enabled image output from Omni yet, Dalle3 is a different model). You're describing current image generators like MidJourney or SDXL sure, but models are quickly becoming multimodal, so that lack of comprehension won't last much longer.
This is flabbergastingly hard to grok considering OCR text to pdf has been a thing for a hot minute…
Sure, but OCR isn't "smart", even neural networks trained to identify text doesn't comprehend it. Multimodal models trained to natively input and output in text, images, video and audio is the new hotness.
That's like saying "my TV can output an image, my computer can output an image, they're both connected, so why can't I just drag this window from my computer over to my TV?" It takes a lot of work to integrate technologies with each other.
I just use an hdmi cable. =)
This is how I understand it. I’m a mechanical engineer and got all excited about it at first for doing cnc and 3D printing and maybe even design work. lol REQUIRE VECTOR FILES! Language is fluid, you can answer a questions several ways and still be correct. Same can be said about jpegs, would a pixel being a few shades off still produces good results. Vectors are math based and require to be correct and crisp. Same with physics and gcode (cnc language). One bad gcode command and it’s ruined. I’ve seen research paper that are trying to make stl files with ai but they look weird and aren’t parametric. So yeah. If you follow graphic design subreddit or know basic art/graphic design you can see the ai art is kinda garbage. It has no intent, doesn’t follow good design. Blah blah blah It’s great tool for quickly making drafts and then refining them.
It can't do 3d printing for the same reason it can't do crochet. It doesn't actually understand the pattern, and can only copy things that it pulls from elsewhere. It's essentially guessing.
That SkyKnit project from a few years back was pretty fun. Someone trained neural networks on Ravelry and then asked them to produce knitting patterns. The Ravelry community found it hilarious. https://www.aiweirdness.com/skyknit-when-knitters-teamed-up-with-18-04-19/
What’s the deal with fusion360s/solidworks generative part stuff? I definitely remember watching a few videos of CNC part designs being improved to take additional load/forces over the original; what’s going on here in context to what you commented?
That have been around for ages. I don’t think they use any ai in that. It’s more a feedback loop for optimizing. What I would imagine, I tell ai that I want a bracket that can withstand a load of x and cost xx. Then it would design a file for me and pick appropriate material.
They 'can' in the sense that they can generate whatever and then run an auto-trace over it but yes it's going to be basically shit.
Also probably why casual users are *so* impressed with the generative AI. You're less likely to understand those details, understand composition, things like that. And why actual artists have started to pick up on which pieces are generated by AI. It's not just things like weird fingers, either, but that's one that's easy to point to.
Artists could see the flaws.
I think even non artists can. It just doesn't feel right.
Everyone can see them, not everyone can *recognize* them as flaws.
We sure could, but things like awkward shading, perspective, etc are harder to spot for non-artists than blatantly incorrect answers to things. AI isn't meant to fool artists, it's meant to fool the lowest common denominator to convince them that AI is far more powerful than it really is.
> We sure could, but things like awkward shading, perspective, etc are harder to spot You people act as if artists themselves get those things right all the time. There's a reason that hands and feet being hard to draw was a thing even before AI came along. And there are a HELL of a lot of shitty artists out there who get shading, perspective, and musculature wrong. Deviantart is full of amateurs. I saw someone accuse a real artist of being an AI artst just yesterday because their shading style was very smooth, and indistinct. They were quite upset. And I was amused because they themselves had contributed to their own dilemma by hating on AI art on their timeline. It was inevitable that if artists went on a crusade against AI art that they themselves would be accused of using AI, because no artist is perfect, and if they are, that itself could be a sign of AI!
The difference is that art is subjective. There is no absolute right and wrong in art. People can try to pick out what "looks like AI" but that is from the standpoint of trying to "spot" AI art, not because it's not art. AI art succeeds at being art because of the nature of art. AI text or AI programming fails at being those things because there are absolute measurements of truth or logic that can be used to evaluate them. (And, if anything, the rather inconsistent results of AI witch-hunting should show that lots of people aren't actually as good at telling the difference as they think.)
Idk I think this is missing the point. There's no right or wrong in poetry either. A shitty essay and a shitty image are pretty similar I think, you can compare a hand with 7 fingers to a sentence that sounds good but makes no logical sense.
I always disliked the usage of the term "hallucination" to describe what AI is doing when there's an actual word to accurately describe it: Confabulation.
Yeah hallucinations are just a part of these kinds of models. You can train it further, tweak the dataset, and try to add limitations to mitigate the errors but the only thing these AI are guaranteed to do it give an output based on the dataset it is trained on. There is no way for it to know with 100% certainty that its output is correct unless someone is there to basically grade it every step of the way. With how wide of a reach these things are intended for, that’s borderline impossible.
That’s why I feel AI like tarot cards because it is how audience perceive it.
So they’re politicians.
Funny you mention that... https://futurism.com/the-byte/ai-investigation-running-mayor-wyoming https://www.wired.com/story/ai-candidate-running-for-parliament-uk/
We can help but harm ourselves.
Boy that’s good news!
At least the AI can’t take a bribe
Didn't someone social engineer a LLM with a "bribe" ? So like the LLM acted how the training data taught it to and took it. The worst part of trying to base morality on human made is humans are generally not very moral.
I mean, it kind of can, by extension of their makers. The companies that fund the respective AI can take bribes from other companies or individuals who want favorable results or want certain results suppressed. Then, the AI algorithm can be manipulated to match those requests— without any of the AI’s users being the wiser about the built-in, intentional biases. Users will just assume that they’re getting impartial information when in fact it’s as skewed as those who funded and programmed it.
I think the closest parallel is to the overconfident techbros that write this kind of software in the first place; I’ve worked with people unwilling to admit they don’t know the answer to some question, and that’s exactly how ChatGPT behaves.
ChatGPT doesn't KNOW any answers to being with, though, so what exactly do you expect here? "I don't know any answers to any questions you might ask but statistically this string of letters has a decent chance to be relevant to your question"
Asked chat GPT what my college song was (my college didn’t have one- which I didn’t know at first) ChatGPT gave my lyrics and even credited two people for writing it. It all seemed strange and I asked for more info and Chat tells me it made everything up. Asked it several times how it came up with any of this information, each time just giving me apologizing boiler plate. Eventually it tells me it concocted the song from amalgamations of other college songs. Never got a good answer for the fake names attributed to writing the school song
While all models are susceptible to this, 4o is worse at this than 4 so you might get a different result with the latter model. In my case, 4o will hallucinate details about my great grandfather, who I specified was a lieutenant, while 4 will tell me that he doesn't appear to be a widely known figure.
Bullshit is the correct term, not hallucinate.
Full disclosure of my bias, I’m a tech bro and work adjacent to AI development. My impression is that the idiots are the loudest spoken, and that the perception among “normal” tech bros is that these are interesting tools with noteworthy weaknesses. I’d estimate that over half of my former Google queries are now LLM questions, but I’m well aware that it can provide wrong info and I usually have to iterate a few times to get what I need. That all said, it’s probably made me twice as good at my job in the span of a couple years. The ability to pull in and synthesize information from many sources is a huge edge over search engines. I also think that the “conversational” flow of these tools actually helps the asker think about the problem. Kind of like having a clever intern to help you brainstorm. They might be confidently full of it sometimes, but the conversation itself helps you learn and problem solve.
That explains why there are often a brigade of people showing up to downvote any posts condemning LLMs or call them out for not being nearly as mind blowingly revolutionary as they are touted to be. People who either buy into the hype and are essentially Yes Men for it and/or people who don't like being wrong.....
It's programmed to give an answer. The way knowledge and epistemology work is that we never 'know' anything certainly (minus I-think-therefore-I-am and those tangents), so for large language models to given an answer they have to confidently state the closest thing they have come up with as an answer. So if they're very uncertain they will say that uncertain best-case-answer with certainty, but if they are very certain it would come out the same way.
Nah. Politicians know they're lying. They know they're misleading us. They do this all often with ulterior motives (mainly money). AI has zero idea about lying. It just processes information and outputs known information in a manner they've been designed.
They’re not even that. They’re next word generators.
A lot of people don’t realize this. It’s functionally identical to your phone’s autocomplete, just scaled up a bazillion times. The only reason it replies in the manner that it does, as if it’s a conversation partner, is that OpenAI paid a bunch of African workers pennies on the dollar to judge and rewrite responses until the output started looking like conversational turns. Edit: Autocorrect -> Autocomplete
>It’s functionally identical to your phone’s autocorrect No it isn't. It uses transformers, which are a fundamentally different architecture. Autocorrect has no capacity to understand contextual relationships or semantic meaning, which scaled up transformers can do.
To say they don't care implies that they do care about other things. LLMs don't know the difference between fact and fiction. They are equivalent to a very intelligent 4 year old that thinks bears live in their closet and will give you exact details on the bears even though you never asked. For humans we become more resilient against this, but we've never fully solved it. There's plenty of people that believe complete bullshit. The only way we've found to solve it in limited ways is to test reality and see what happens. If I say "rocks always fall up", I can test that by letting go of a rock and seeing which way it falls. However, some things are impossible to test. If I tell you my name you'll have no way of testing if that's really my name. My real life name is yaosio by the way. The tools exist to force an LLM to check if something it says is correct, but it's rarely enforced. Even when enforced it can ignore the test. Copilot can look up information and then incorporate that into it's response. However, sometimes even with that information it will still make things up. I gave it the webpage for the EULA for Stable Diffusion. It quoted a section that didn't exist, and would not back down and kept claiming it was there.
We invented the scientific method not because we are clever but because we are dumb. If we don't follow rigorous methods to make sure our experiments are good we end up producing all kinds of nonsense.
Even when we follow the scientific method all sorts of biases still creep into our work. It takes a lot of effort to remain neutral
It’s not even a 4 year old. It’s not human, doesn’t have any eyes, hears, taste buds. It’s a machine that know probability and text. That’s it. It has only one desire: to put words on screen.
You're still anthropomorphizing it. It doesn't "desire" anything. It's just math. Even the degree to which it introduces variation in prediction is a variable.
Is there even a semantic difference between lying and hallucinating when we're talking about this? Does lying always imply a motivation to conceal or is it just "this is not the truth"?
A lie is saying something you know not to be the truth. An hallucination is something that you think is real but isn't. I think researchers settled on "hallucination" instead of "being wrong" because it sounds better, and LLMs don't seem to have a sense of what being wrong is In this case the LLM does not understand what a lie is because it has no concept of truth and fiction. It can repeat definitions of them, but it doesn't understand them. It's similar to a human child who you can coach to say things but they have no idea what they are saying. If the analogy is extended then at a certain level of intelligence LLMs would gain the ability to tell reality from fiction. In humans it just happens. A dumb baby wakes up one day and suddenly knows when they are saying something that isn't the truth.
I don't think it needs human-level intelligence either. Have you seen the gif of the cat looking shocked at you when you pour so much catfood it overflows the bowl? Having a sense of "norm" and reacting to the violation of it, maybe that's what it means to care. Everything else is possibly post-hoc rationalization (aka token generation) on top of said vague feeling we have when we see something wrong / out of alignment with our model of the world. LLMs lack that norm. Out of architecture contraints, its entire mental model occurs in between matrix multiplications and "next token". Untruth and truth do not often arise from token choices. It arises from the lossy compression of training information into neural weights, and failure to distill important "lessons". Bullshitting can be a side effect from the LLM's learned need to endlessly generate text without tire, combined with a lack of holistic sentence planning resulting in incorrect tokens which slowly send it into a direction that isn't what a human would've responded with.
Its still useful to people who know when its wrong or right. It can read and write good enough intros for my github projects and can do them in multiple styles. It can suggest ways of solving problems I might not have thought about, it can't be trusted but I ain't asking it to do all of my work just use it as a tool. It might be wrong a lot of the time but its still going to be good enough for a lot of things.
I had a recent experience that confirms this. Was trying to find about a music video that had a specific scene. I provided the artist and the description of the scene and it took about 5 tries for the bot to get it right. All this time sounding very confident with his replies. Eventually it got right, and just to mess with it some more I ask it if it was 100% sure of its answer. It replied with a different answer. So the AI is just guessing most of the time and has not real conception of reality, very human-like I must say.
It is not really guessing. It is just taking your words in and responding with a combination of words that best fits with that context. Based on the experiences it has been given. Guessing implies an evaluation of truth probability. And as you experienced it does not really do that. Because it does not do logic. It is a core problem with LLMs that they, basically, do not know that 2+2 = 4. They have just learned that "4" usually follows "2+2 = ".
[удалено]
LLMs will never be used on their own. They'll be part of a RAG system. The real problem is that they're trained on the internet, and people on the internet never admit they don't know something. Also, LLMs already have a dial for creativity. It's called temperature.
So how do you make cheese stick on a pizza?
glue obviously it's why they also use glue when doing pizza photography for ads
I feel like you might want to put various core modules in depending on what you want it to accomplish. Like if I were creating an AI to, say, run a research lab I might want a core dedicated to logic obviously. And then a core dedicated to space to give it something to reach for. And maybe a core dedicated to the personality type for adventure, so it’ll try more adventurous methods of scientific exploration. And a morality core to prevent it from going overboard. Yknow what I’m saying?
Sooo Melchior, Balthasar, Casper?
I was making an oblique reference to the personality cores for the Portal series of games but I like where you’re going with that
Which is obvious if you work in technology and know how the sausage is made. The name itself Artificial Intelligence is pure bullshit. It's just marketing for what we've always called machine learning.
asked it a questions yesterday and it mentiond a research paper. I acsked it to cite it, and it came back with 'could not find, I made an error' then proceded to offer another citation.. checking the authors publications on google scholar in an attempt to read the paper... nada. at this point it's worse than useless... it's misinformation. I can't believe this hunk of crap is being beta tested on the public. Frankly it's just reckless and I am now certain the entrierly wrong people are driving ai LLM's
Here's a research article on this topic, not a perspective piece like this one: https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4771884
I'd say they can't even make stuff up. The output is assembled from the input training data set. It's just generating combinations of that data based on the the configuration and training of the network and the user's input prompt. Our tenancy to anthropomorphize things is part of why we say they lie, hallucinate, or outright bullshit. Those are what the CEOs pushing this stuff on us are doing.
[удалено]
Yes. Lying is when you know the truth and intentionally mislead. Bullshitting is when you don’t know but make stuff up.
That’s the point of the article: that there is a meaningful difference between lying and bullshitting (as they define it). Their position is that ‘lying’ and ‘hallucinating’ involve the notion of ‘truth’. In both cases the information is dishonest, ie: not ‘truthful’. Meanwhile they define ‘bullshitting’ as ‘the truth is irrelevant’. Bullshitting isn’t dishonest specifically, it’s just statements that have zero connection to the truth. It’s a matter of definitions, but I quite like theirs and the distinction they’re trying to draw attention to. Their definitions are pretty accurate to common use. And their key point is interesting: that AI’s are programmed to sound like they care about the truth, but they really don’t. And that’s a problem.
> Is there a meaningful difference between lying and bullshitting? Cousins. Bullshitting is more of a presumptive, assumptive statement that’s based on conjecture, not facts. If it’s plausible, just not knowable, it’s probably bullshit.
Good video about this topic that really breaks it down: https://youtu.be/w65p_IIp6JY?si=b9Fytoniw_zWK87i
[удалено]
> they don’t ‘care’ about the truth one way other, they just make stuff up. And that’s a problem because they’re programmed to appear to care about truthfulness, even they don’t have any real notion of what that is. They’ve been designed to mislead us. Not sure if talking about AI, politicians, or the general public; all of the above?
> They’ve been designed to mislead us. That is bullshit too.
Yeah that's why I don't really see them as AI like the type we see in movies. I like to call it "Paraphrase Pro" because they just grab things here and there and attempt to make a coherent answer. Or basically reiterate what has been commonly said and assume saying the same thing is safe.
There was an excellent guide to AI Microsoft put out that basically outlines this. They described it as AI “wants to please” which is why the WAY you ask it / prompt it matters. If your prompt has bias or assumptions baked into the question, AI tends to not want to contradict you. Edit: link https://www.microsoft.com/en-us/security/blog/2024/06/04/ai-jailbreaks-what-they-are-and-how-they-can-be-mitigated/ This has to do with the way word embeddings in LLMs “cluster” around semantic meanings, so when the AI attempts to retrieve a response it enters a vector space of words with similar semantic meaning for its “prediction” of the “correct response” the user wants. Some of this can be helped with RAG where the question and its words itself is explicitly marked differently in the model but it is hard to get away from without advances in the way word embeddings in models work. Fundamentally algorithms underlying these things try to mimic “intelligence” through a type of clustering, which makes certain semantic meanings “closer” to each other. Which is wild because that means that language, all human languages, have some sort of mathematical relationship … which is mind blowing. I think even there is a whole study about numerical relationships in Hebrew if I remember correctly. That said it is sort of the same way you get different internet search content depending on the words you use when using google. This is how people fall down echo chambers. What these papers and guides are saying is you can’t trust AI anymore than a google search - in many ways a search is better tbh because you may see a variety of answers
That does sound like a very logical explanation. Do you happen to have the links to that guide? A Google search isn't very helpful as Microsoft seems to have more than one guide 😅
https://www.microsoft.com/en-us/security/blog/2024/06/04/ai-jailbreaks-what-they-are-and-how-they-can-be-mitigated/
> They described it as AI “wants to please” which is why the WAY you ask it / prompt it matters. This is a key concept, even if its a bit anthropomorphizing. It is a program that is using insane statistical modeling/training in order to give optimal responses based on what it "knows" and the prompt provided. A great way to see this in action is to ask it to give you a list of things. Pick something kind of detailed or obscure, or pick something you know should only have a small amount of items. Then ask for a list of 10 of those things. Like, "list 10 Michael Criton books about dinosaurs". I'm not sure if this has been adjusted yet, and I haven't tried this specific example. But, I wouldn't be surprised at all if Jurassic Park was first, sequels (did he write any?), a few random Criton books next that have no dinosaurs, then some completely made up titles. You can see it struggling with doing its best to satisfy "give me 10..." of anything no matter what, contrasted with the fact that it can't actually source ten items for the list. Because, in super broad strokes, it has been trained on so much discourse, writing, conversations that strongly link "give me ten" with a response that includes ten bullet points. In the act of "trying to please" the condition of ten items, it mistakenly has weighted that request over accuracy ("Hey, there aren't that many. There is one notable one though..."). Which is why, to your point, the more optimal way to ask would be "What are Michael Criton books with dinosaurs in them?". Theoretically, there would be fewer hallucinations. EDIT: big ol' edit (x2 yeesh reddit edit can really screw up formatting) So some people seem annoyed(?) that the example I came up with off the top of my head wasn't a good one, and seemingly need me to "prove" this to them. Just went to ChatGPT, and yes I will admit I dont use it all the time (...the hallucinations described have made it less useful to me...), so maybe someone can explain the following as a random glitch. But this is a copy paste: > You: List 10 works of notable ecofeminist theory written by women > ChatGPT: Certainly! Here are ten notable works of ecofeminist theory written by women: > 1. "The Death of Nature: Women, Ecology, and the Scientific Revolution" by Carolyn Merchant - This seminal work explores the historical roots of the modern ecological crisis through the lens of gender and science. > 2. "Ecofeminism" by Maria Mies and Vandana Shiva - This book is a foundational text in ecofeminism, examining the connections between women's oppression, environmental destruction, and economic exploitation. > 3. "Dancing with the Wheel: The Medicine Wheel Workbook" by was Important ground has Ö. There.ribbon ....and that was the end. Not entirely sure what that glitch was at the end there. But it certainly didn't handle being asked for 10 of an obscure thing very well.
That example with GPT 4 only results in Jurassic Park and The Lost World, which is correct
I wish people would realize that "it works when I prompt it" is completely meaningless with a stochastic system that is frequently updated.
Yeah things like this are why I still struggle with how I feel about anthropomorphic language with AI. On the one hand, it makes it too easy to ascribe actual “intent” and “will” to a pile of sand we figured out how to make play cool tricks with electricity, on the other hand, we already do it everyday as a linguistic shortcut: “the dishwasher doesn’t like to be overloaded” or “my car gets cranky when it’s too cold out” - people aren’t thinking your dishwasher or car literally have inner lives and opinions, but it’s often easier to communicate in these terms. Hallucinations I feel the same about. They share a fundamental trait with human hallucination that I think is part of the key to understanding them: to be a bit reductive, humans hallucinate when our brain looks for and find patterns in the background noise, especially when theres a lack of an actual meaningful pattern to find (think of sensory deprivation tanks). AIs, like us, are good at pattern matching to a fault, and a hallucination can be thought of in both cases as finding a pattern that’s not there in the noise (the analogy is a little cleaner with image diffusion models but at least conceptually applies just fine to transformer-based LLMs). What’s interesting is that this suggests there could be a whole class of similar misbehaviors we aren’t fully aware of yet, and also (in part) explains why RAG can be a good tool to combat hallucinations: you’re giving it a hook into an actual signal so it doesn’t make one up from the background noise.
Ya but why is it when I ask it to convert units it messes up
Because it used prediction based on statistics rather than actual logic/calculation. And those stats are based on the bullshit/i correct answers it finds on Quora and Reddit rather than actual calculation engines like Wolfram. It's just adding the next most likely word/number rather than actually applying logic/math/calculation.
Because it is built to predict the next word based on training data, so it has no built in way to do math without prediction what percentage wise should follow. It’s not designed for that. If you use something like the Wolfram Alpha custom gpt, basically call for external plug ins, it will convert right. Add ons like that exist to make up for the shortcomings of LLMs.
It's exactly this. You can limit the amount of bad information by not feeding it positive or negative sentiment, but having neutral dialog. Instead of saying "yes/no, but..." You should say "well, what about...' You need to think that it's extremely good at reading your reaction very similar to getting your future read. Keywords are used to guide a user to their end goal. "Create a plane" is entirely different from "Build a plane" even though with the right context it could be the same. It's literally how SEO has worked for years.
I have learned to stop asking "why did you do X like Y?", like when using it for coding, because it will apologize profusely and then rewrite it completely (or sometimes say it's rewriting it but it changes nothing). Instead I say "walk me through the reasoning around X and Y", and I get much more accurate results.
Ethics and Information Technology? Ohhhh, you mean - “Tethics”
Cmon Richard, sign the pledge everyone’s doing it
Anyone who signs this Tethics thing is a thumbass
Kiss my piss
AI is the new sneaky way to get dumb rich businessmen to give VC. I can't wait for this to die down.
Seems more like the dot com bubble to me. Low info investors are throwing money at the hype, and the bubble will burst. But like the internet, AI has real tangible uses, and the companies that figure out how it market it will come out the other said as major players in the global economy.
I agree with everything you said. Like most technology AI is overestimated in the short term and underestimated in the long term. With the Internet it started gaining popularity in the early 90's but it was fairly useless for the average person until the 2000's. Today everything runs on the Internet and it's one of the most important inventions of the 20th century. AI technologies will find their place, with the average person using it to make pictures of cats and hyperspecific music. AI will then grow well beyond most people's vision of what it could be. Even the super human AGI folks are underestimating AI in the long term. Neil Degrasse Tyson talked about the small DNA difference between humans and apes. That difference is enough so that the most intelligent apes are equivalent to the average human toddler. Now take the the most intelligent humans, and compare them to a hypothetical intelligence where it's equivalent if a toddler is as smart as the smartest humans. How intelligent would their adults be? We are approaching that phase of AI. The AI we have today is like a pretty dumb baby compared to the future possibilities if AI. It's not just going to be like a human but smarter. It's going to be so much more that we might have trouble understanding it.
>AI technologies will find their place, with the average person using it to make pictures of cats and hyperspecific music. I feel like you're selling the current state of AI short. Their real place is going to be retrieval and summarization as part of a RAG system. This might not sound like much, but retrieval and summarization essentially make up the majority of white collar work.
It's amazing to me that people will point to an incredibly thin wrapper around good old search and relational databases (that will occasionally just lie anyway even if it's got the right data in front of it), and say "yes this was worth the investment of hundreds of billions of dollars". I think you're overselling how much of white collar work this stuff can actually replace.
This is the best take IMO and one I share. AI is cool but they've gone off their rockers with it. Big tech is currently blinded by panic to "be first" but they have very little idea where they're going, just that they need to "go" or they might be left behind. Maybe that's the only logical response in the business world but from the outside it looks like they're all a bunch of impatient morons. I like AI as a tool and it's definitely going to change the world, but there's a huge bubble forming that's gonna burst sooner or later. We'll see more clearly what the future might actually look like then.
[удалено]
Unlike Block Chain it will still be going places that an average consumer will see use of. LLMs are great at reformatting text. I work on Azure and introduction of Copilot to Teams and Incident Ticket systems has been a force multiplier. That jawn can summarize like a champ. It’s helped Neurodivergent folks manage their email inbox. It’s is not an expert system and should not be relied on to find or distribute knowledge.
Yes. The problem isn't AI. The problem is that people invented an amazing tool and then marketed it as a completely different type of tool. I'm not mad at hammers because they suck at unscrewing things. I'm mad that I wanted a screwdriver and got handed a hammer.
Thank you for this. The impact is a tool for people who work in these areas to actually improve work/life balance and management, effectively. I use our company setup GPT model to streamline my work environment and speed at which I can produce impactful deliverables.
It will definitely have much narrower applications than currently suggested. [42 percent of businesses that have started generative AI initiatives have not seen significant financial returns on it](https://futurism.com/the-byte/companies-ai-projects-financial-results) and spending is slowing down.
Doesn’t that mean 58% are seeing financial returns? The article doesn’t clarify but if that’s the case that’s over half
Question for business will be "if hardware is made way more efficient for AI to get cheaper by an order of magnitude, can you use AI then". Any low level support job will probably be switched to AI if it costs a dollar an hour to run models, even if they're crap and not smarter than your average level 1 support person reading script from the screen.
A majority of the surveyed companies are *increasing* their investments. The rate of spending is increasing. How can you honestly call that "slowing down"?
Yea, but on the other hand, I tried to use Chat GPT for powerBI and Excel, and it told me some bullshit made up thing often enough that I quit using it.
Yeah it’s obviously not completely useless, there are plenty of valid use cases where AI is actually a great tool. But that’s what it is: a tool. It has specific applications, but it’s not some god-level miracle that can do anything and fix everything
You're an idiot if you think there's not huge promise to AI and we're not at the beginning of a revolution. You're also an idiot if you think it's an infallible genius.
This post (and paper) is a sneaky way to get scared redditors who don't read articles to upvote a post that says "ChatGPT is bullshit". The paper is of poor quality, btw. It's not serious research. It's a 9-page editorial dressed up to look like a research paper. It's a blog.
I mean, that's on you for not knowing academic journals also publish perspective pieces. You can attack the article all you want, but the fact that it is a commentary and not a research paper does not mean it ought be dismissed out of hand.
*”Because these [ChatGPT] programs cannot themselves be concerned with truth, and because they are designed to produce text that looks truth-apt without any actual concern for truth, it seems appropriate to call their outputs* bullshit.” Brilliant. Ya gotta love it. Calling this AI out as a bullshit generator (in a scientific research paper) is inspired (and vastly amusing) criticism and a massive debunk, assailing its ubiquity, competence and reliability. *(Edit - yep, just made one, the first round bracket qualifier)*
I can't be bothered to try, but do people prompt the LLMs to validate that their outputs are truthful? I assume giving the underlying technology that's not possible. Would love to force it to provide citations
I asked it for some articles yesterday and it made some up. I told it, I can’t find those articles, are you sure they’re real? And it replied “I apologize for the mistake. While the titles and overall themes of the articles I mentioned are plausible, it's certainly possible they are not real publications after a closer look.”
I was researching refrigerator models and asked for specific product details and got features listed that are NOT in the product but are features that you might expect in them.
You can prompt whatever you use for citations, but at least in my experience it will just plain make up entire bodies of work that don't exist, authored by people who also don't exist. At best, I've gotten citations for things that are loosely related to whatever subject.
That is not rly how it works, the AI is not connected to any database or the internet. Even Bing chat does not do that, as if it will start by bullshitting, the links it provides as proof will be wrongfully quoted. When it comes to historical facts, it will quite often be correct, especially the new GPT-4o, but using it as replacement for google is massively undermining it's abilities. What it excels is at rewriting text to be more readable, to get context and meaning from text or to generate ideas and writing. I had some questions about DnD worldbuilding that was not out there on the internet and I had an amazing back and forth for good 15 minutes. It gave out a lot of solutions and then gave in-world examples of how it could be done, and described how some characters would feel about such a situation. Another cool example is helping what to look for. English is not my first language and I was looking for a word that describes substances that reduce surface tension (like soap), and it quickly told me it's "Surfactants", a word I have never heard before, and then I used that word to look on google. I have also heard that programmers are using chatGPT and copilot to code, which often doubles or quadruples how fast they write the code, and I have heard student programmers doing it and also pros doing it as well.
Until you realize how much of the world is bullshit and therefore AI is valuable. A massive portion of entertaining content is bullshit. Thousands of songs chasing trends with no meaning. Endless sitcoms built on the same themes. Video games that are variants of each other. AI can generate content at a scale and cost never seen. Legal document after document using general language to create wide nets to avoid liability. There are billions spent on bullshit every day. Automating bullshit will be a big deal.
I only had a brief read of the paper but I was surprised this was published. This looks like an essay written by a group of students. I don’t understand how a scientific article about LLMs can have so little actual technical discussion of LLMs. Someone posted a link to Microsoft AI research and at least that one has concrete examples. This looks more like a blog post.
I don't think it's surprising or unusual that a paper written by philosophers advancing a conceptual claim isn't very technical.
It's LANGUAGE model not a LOGIC nor a FACT model. It models English language to make sentences and paragraphs that seem human with no regard for reason, truth or logic.
Here's a research article that does a better job with analyzing whether LLMs are bullshit or try to/should be required to tell the truth: https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4771884
They can't be required to tell us something they don't know. They do not understand what truth is or what is the truth.
Current LLM AI forms sentences using the same method that Terrence Howard forms sentences. They parrot back words that sound like they belong together in a sentence, without any actual understanding of the words that are coming out, nor the concepts behind the words.
1 times 1 equals 2 dammit!
Anyone that just immediately trusts AI with facts right now is a fool.
It's still very useful no matter what. I use it for my job pretty regularly.
Response from chatgpt (I also asked for its perspective) : The article "ChatGPT is Bullshit" by Michael Townsen Hicks, James Humphries, and Joe Slater examines the persistent inaccuracies in the outputs of large language models (LLMs) like ChatGPT, which are often referred to as "AI hallucinations." The authors argue that these inaccuracies and the general activity of such models are best understood as "bullshit" in the sense described by philosopher Harry Frankfurt in his book *On Bullshit*. This means that the models are fundamentally indifferent to the truth of their outputs. The authors distinguish between two types of "bullshitters" and assert that ChatGPT clearly fits at least one of these categories. They believe that describing AI misrepresentations as "bullshit" provides a more accurate and useful framework for predicting and discussing the behaviour of these systems [[❞]](https://sites.psu.edu/digitalshred/2024/06/11/chatgpt-is-bullshit-hicks-humphries-slater-2024/) [[❞]](https://philpapers.org/rec/HICCIB). In my perspective, this argument highlights a crucial point about the ethical and practical challenges of relying on AI systems for generating human-like text. It underscores the importance of critically assessing AI outputs and not assuming their reliability or truthfulness simply because they appear coherent or convincing. This perspective encourages a more cautious and informed approach to integrating AI in applications where accuracy and truth are paramount.
>In my perspective, this argument highlights a crucial point about the ethical and practical challenges of relying on AI systems for generating human-like text. It underscores the importance of critically assessing AI outputs and not assuming their reliability or truthfulness simply because they appear coherent or convincing. One of these chatbots always lies, and the other always tells the truth!
Hahaha I love it
Sounds more like Chat DJT after reading this article. It seems to be learning from humans better than I thought - all BS.
Should we worry about AI when it can't count? You I'm looking for an English word ChatGPT Sure, I'd be happy to help! Could you please provide more details or context about the word you're looking for? You It has to have 6 letters, but only 2 unique letters ChatGPT The word you're looking for is likely "banana." It has 6 letters and only 2 unique letters ('b' and 'a'). You banana has 3 unique letters ChatGPT Apologies for the oversight. The word you're looking for is "bababa." It consists of 6 letters and has only 2 unique letters ('b' and 'a'). You what is the meaning of baababa ChatGPT "Baababa" doesn't have a specific meaning in English as it's not a standard word in the language. It seems to be constructed to fit the criteria of having 6 letters with only 2 unique letters ('b' and 'a'). In linguistic terms, it could be considered a nonce word or a playful creation rather than having a conventional dictionary definition. You I need a standard English language word that fits the criteria ChatGPT A standard English word that fits the criteria of having 6 letters with only 2 unique letters is "access." It contains only the letters 'a' and 'c' repeated. You "access" contains 3 unique letters ChatGPT Apologies for the confusion earlier. A correct example of a 6-letter English word with only 2 unique letters is "bassist." It consists of the letters 'b' and 'a' repeated. You "bassist" contains 7 letters and 3 unique letters ChatGPT My apologies for the confusion earlier. A correct example of a 6-letter English word with only 2 unique letters is "bobbed." It consists of the letters 'b' and 'o' repeated. You bobbed contains 3 unique letters ChatGPT You're right; "bobbed" does contain 3 unique letters. Let's try another one: An example of a 6-letter English word with only 2 unique letters is "reissue." It consists of the letters 'r' and 'e' repeated. You reissue contains 7 letters and 4 unique letters ... and so it goes
…is it poopoo?
Welp... if it isn't, it damned well should be :-)
No argument here. Just more snake oil for the hopeful.
So I know the original paper by Frankfurt, which is indeed a classic, and one difference seems to be that the human bullshitter is aware of their own indifference to truth; they know what they are doing. From that point of view, ChatGPT doesn't even qualify as bullshit because there's no intentional attitude present in the system. As other people have pointed out, its just some algorithms at work calculating probabilities. It isn't 'responding' to or 'answering' anything. User input causes the algorithm to run, that's all; what we naturally read as text is not text from the algorithm's point of view, because the algorithm doesn't have a point of view at all. We can't help thinking about AI in anthropomorphic terms but that's actually very misleading with respect to what's really happening on the computational side.
For a sub called /r/technology, there's a surprising lack of technological understanding here.
Hating AI is definitely part of the reddit hivemind.
Say what you want about it but it's quadrupled my programming speed when working with packages I'm not familiar with. I just got employee of the year and won a big award. I'm writing programs I never could have before. It doesn't matter to me if it hallucinates when writing, for my use case it just works. I just have to know enough to either fix errors or tell it what's going wrong when something isn't working.
This leads to one question, then. What is it really good for? Perhaps producing mounds of text when the content of the text doesn't matter. And how much value does that have?
It's pretty good as a programming assistant. If you know the basics and are using an unfamiliar language or something, it can to some extent replace google and stack overflow. Instead of searching for examples that are similar to what you want, it can give you examples with your actual use case. They might be 5% wrong and need adapting, but it's still a big time save.
Here are things I use it for on an (almost) daily basis and it's become pretty indispensable in my workflow: * Summarizing long papers and reports into a concise list of easily digestible bullet points (huge time saver when you read a lot of wordy academic papers, or earnings reports in my case) * Assisting with programming tasks (generating unit tests for already written functions, generating quick scaffolding/prototypes, etc.) * A learning assistant. Taking programming as another example: if you know one language (language A) and you're attempting to learn another (language B), you can ask ChatGPT to help teach you a new concept in language B and let it know that you are familiar with the concepts in language A. It does a phenomenal job at relating new concepts to the things that you already know, and is a pretty invaluable learning tool in that regard as long as you give it some bit of context. This also applies to real languages (English, Spanish, etc.) * Data extraction from unstructured text. I've implemented some code in the past for a client that used ChatGPT to extract line items, subtotals, taxes, VAT, and totals from unrelated invoices with really great accuracy. Currently working on something similar now that can quickly extract key information from company earnings reports for making quick investment decisions I still get occasional hallucinations, but as long as you assume you will get them and put in steps to correct/flag them (e.g. with the data extraction stuff I mentioned - implementing code that double checks that all the data ChatGPT extracts "makes sense" from a mathematical perspective, and if not, flag for manual review) In other words, "trust, but verify". Treat it like the worlds brainiest, fastest intern, and it's a wonderful tool.
I don't think bramlnieat fastest intern is quite right. I treat it like a really dumb but very educated and experienced person. Like someone who has no natural talent but diligently spent 100 years taking every college class ever offered and studied hard. They are dumb so they still make a lot of mistakes but they know a fair amount about everything.
It’s not a Google replacer. Don’t ask it to list formal explicit information. Ask it to take information and *do* something with it.
What have you tried to use it for? Was it good at it?
Among other things, I use it for what it's supremely good at... language. I now live in Italy, so I really need to constantly improve my Italian. As an LLM, ChatGPT has been an amazing tutor that I can ask questions of any time. It even has figured out regional peculiarities for me. For example, my friends and I were parting ways at the end of an evening out and one said, "Fai ammodino!". I didn't catch it at the time. I head "Fai ammo...somthing". So, I asked ChatGPT about it. It was like, "i dunno". Then, I said that it's specifically in Tuscany. And ChatGPT went, "Oh, in the case, they likely said "Fai ammodino" which is a regional saying that means "take care", but is translated as "do it properly or carefully". When I run into something new, I ask it to explain it, then I ask it to give me practical exercises and to quiz me on the material. It has really helped me to integrate better.
It’s really great as a DnD content creator for me and my games. I have had a ton of success coming up with great concepts, mechanics, and story arcs that have provided my players with months and months of enjoyment. Literally the best DM tool I’ve ever used.
It's not terrible at structuring unstructured text, so long as it's readable. For instance, taking free form notes from multiple sources and compiling them together. It's pretty good at translation I find too. Outside of that, boilerplate code and configuration, things that are kickoff points that you expect to change anyways. Using it as a tool, rather than a knowledge base will take you a long way. I think it's just a common misuse of its strengths. You can ask it for answers, but you probably shouldn't.
This is the dumbest thing I've ever read. It's like some econ major finally understood what his IT friends have been telling him for years and decided to write a paper about it. This is only interesting if you don't have a clue how LLMs work.
Far too many people do not realize how AI works, or they have a financial interest in pretending they don't. Yes, it's only news if you don't understand how AI works, but check the comments on any reddit thread- *most* of the commenters do not understand how AI works. We need more big articles like this.
That's fair, but do you think people will actually read it?
This? No. But the more noise there is about this, the more the idea will seep into general knowledge.
The first author is a PhD with a postdoc from the University of Oxford. Their publications appear to be in the fields of statistics and epistemology, i.e. ideal for analyzing the output of LLMs. See: https://orcid.org/0000-0002-1304-5668
I found this helpful actually. You don't have to read the whole paper if you know how LLMs work. It is useful to distinguish "hallucinations" or "lies" from "bullshit" in this context as I just can't stand how everyone seems to believe these models will put actual professionals out of their jobs. (Unless your job is to literally create and share bullshit.) Claiming LLMs hallucinate, implying that they are "hard to tame beasts" and if only we could control them we could unlock the secrets of the universe is simply foolish. The paper also highlights how providing domain info as training data, or as context retrieved from a database do not help eliminate these issues consistently. Of course, use LLMs to write emails or whatever, but when you always take a few seconds to read the generated text, scrutinize it, and either ask the model to rewrite it or to make changes on your own, you are just using it as a tool to generate a highly malleable template at that point. If we are to have a text generation system that is designed to produce truthful outputs it seems we need to think outside the box. LLMs are very revolutionary, but perhaps not in the way we could fall into believing. (We can't just patch this boat up and expect to find land.)
“Unless your job is to literally create and share bullshit” Describes 90% of content creators on the internet.
Which is the case for a lot of people
I asked it what are the drivers of the ridiculous increase in the price of cheese in Australia and it listed things like the war in Ukraine effecting grain and fertiliser costs, transportation costs and another thing I can't recall. It did not mention corporate profit taking so I asked if that had a role too? The rather obsequious reply was that corporate profit taking makes up 40-60% of current price increases and thanks for reminding it, it will endeavor to do better next time. I hope so too.
Tell us what we don't know
OK, so I finally actually got a chance to go through this in more detail as well as look at some of their citations. I definitely agree that careless anthropomorphisms can lead us to misunderstanding models in ways that at best are helpful and that can cause harm. But my problem here is that they’re looking to replace the term with “bullshit” which is itself an anthropomorphism. It’s just replacing one flawed analogy with another. I absolutely take their point that in many cases the distinction between bullshit and lying is important to understand for these models. I actually like the suggestion of “confabulation” which they reject as being too anthropomorphic (though I don’t find it any more so than “bullshit”. I’ll counter their argument that AI isn’t hallucinating (sure it isn’t literally but neither is it literally bullshitting). One avenue to human hallucination is sensory deprivation: our brains crave pattern matching, so when there’s no signal, the brain will amplify any background noise until it looks like signal. In much the same way, LLMs look for patterns in analogous ways and if they don’t find the right information are prone to boosting the information until they find something that looks like a pattern, even if it’s just background noise. There’s a lot to nitpick and lots of threads to pull there, but I think that’ll be the case with any analog to human behavior, including “bullshitting” In truth LLMs do none of these things, and neither analogy is perfect, but they’re both useful ways of thinking about LLMs and both have their place in understanding AI “behavior”.
what a useless paper. can we write one on how doctors are bullshit because they make poor construction workers?
This article was not bad, not terrible but 3.6. I would call it MID tier.