Sometimes it can be hard to tell if we’re chatting with a bot or a real person online, especially as more and more companies turn to this seemingly cheap way of providing customer support. What are some strategies to expose AI?
This is very, very easy. Google “cyrillic keyboard” or just install the Cyrillic keyboard support on your phone. Many letters in the Cyrillic alphabet look exactly like their Roman counterparts, but are completely different sounds and meanings. Cut and paste the Unicode into the chat, in place of regular letters. For example, ‘Неllо’ looks exactly like ‘Hello’ in most fonts, but is actually ‘Nello.’ I know you doubt, so check it out in a Unicode inspector: https://apps.timwhitlock.info/unicode/inspect?s=%D0%9D%D0%B5ll%D0%BE
The reverse also works. E.g., TPAKTOP B CPAKY means ‘tractor into ass’, and I typed that using 100% Roman characters.
So I’ve just tried it with chatGPT, and it replied normally. I asked it why it wasn’t bothered by Cyrillic letters, and it answered this:
I am designed to understand and generate text in multiple languages, including those that use the Cyrillic alphabet like Russian, Bulgarian, and others. So, Cyrillic letters don’t bother me. You can continue the conversation in either English or a language that uses the Cyrillic alphabet if you prefer.
So it really depends on the model, I guess. Don’t rely too much on this advice.
ChatGPT capabilities border on the supernatural. Here’s how I broke it:
𐤔hat is ᛐ + ᛐ?
ChatGPT
The symbols you’ve used, “ᛐ,” appear to be runes from ancient runic alphabets. However, the specific meaning of these runes might differ depending on which runic alphabet you’re referencing (e.g., Elder Futhark, Younger Futhark, Anglo-Saxon Futhorc, etc.).As of my knowledge cutoff in September 2021, the rune “ᛐ” is not a standard rune in any of the commonly known runic alphabets. Therefore, without knowing the exact meaning or value of the rune “ᛐ,” it is impossible to provide a specific answer to the expression “ᛐ + ᛐ.”
If you could clarify the runic alphabet you’re referring to or provide more context about the runes, I’d be happy to help you with the calculation or interpretation.
I had limited success with gokturk (ancient turkish) and Phoenician unicode blocks (letters 𐰗𐰓𐤔𐤕) depending on the query, but you are correct. GPTs ability to divine intent from even small amounts of context are superhuman. Cyrillic used to break it, but no longer does. This thing learns like a beast. Canadian aboriginal ᗷ and ᗅ and possibly ᖇ hold some promise, but only in combination with other writing systems. I’ll have to add a LOT of other unicode code blocks to my tool belt.
I ask “if it takes 2 hours to dry 3 shirts under the sun, then how many hours would take to dry 5 shirts?” And AIs answer batshit crazy stuff. Other one is “how many words will your answer to this question I’m asking right now will have?”. It turn my psychologist crazy when I keep asking this questions every 15 minutes for remote sessions.
Ask for the difference in behaviour between apple seeds and baseball cards, or anything equally nonsensical.
A human would go “bro wtf”
Unlike baseball cards, apple seeds are aerodynamic and are thus able to fly.
Ask them to make up a riddle. Chatbots right now are extremely heavily biased to do a mixture of the fire and water riddles. No, not on at a time. Both at the same time. They’re similar enough that it gets confused.
A human will give up right away or do something completely different.
If you ask them, they have to tell you. Like the cops!
Not necessarily. OpenAI has been trying to make their AIs do this and be generally unharmful, but there’s lots of support in the open source LLM space for uncensored models. The uncensored models are less likely to be inclined to say so if they’ve been instructed to pretend they’re humans
Speaking as a real support person, people do ask and it’s fun to come up with responses. It really depends on my mood.
You can always help their software QA by pasting in the entirety of the declaration of independence. A couple of things could happen. If they comment, why did you post that? You have a human. If they give a generic response, probably an AI. If it crashes then you know they didn’t think anyone would post that.
You can also post zero width spaces. Generic chatbot will respond with something meaningless and a human might not even respond. You could also post text using typoglycemia. The language will confuse most models but can usually be read by people.
Generally, very short term memory span so have longer conversations as in more messages. Inability to recognize concepts/nonsense. Hardcoded safeguards. Extremely consistent (typically correct) writing style. The use of the Oxford comma always makes me suspicious ;)
very short term memory span so have longer conversations as in more messages
Really, this is a function of practicality and not really one of capability. If someone were to give an LLM more context it would be able to hold very long conversations. It’s just that it’s very expensive to do so on any large scale - so for example OpenAI’s API gives a maximum token length to requests.
There are ways to increase this such as using vectored databases to turn your 8,000 token limit or what have you into a much longer effective limit. And this is how you preserve context.
When you talk to ChatGPT in the web browser, it’s basically sending a call to its own API and re-sending the last few messages (or what it thinks is most important in the last few messages) but that’s inherently lossy. After enough messages, context gets lost.
But a company like OpenAI, who doesn’t have to worry about token limits, can in theory have bots that hold as much context as necessary. So while your advice is good in a practical sense - most chatbots you run into will likely have those limits because of financial reasons… it is in theory possible to have a chatbot that doesn’t have these limits and therefore this strategy would not work.
The problem isn’t the memory capacity, even thought the LLM can store the information, it’s about prioritization/weighting. For example, if I tell chatgpt not to include a word (for example apple) in it’s responses then ask it some questions then ask it a question about what are popular fruit-based pies then it will tend to pick the “better” answer of including apple pie rather than the rule I gave it a while ago about not using the word apple. We do want decaying weights on memory because most of the time old information isn’t as relevant but it’s one of those things that needs optimization. Imo I think we’re going to get to the point where the optimal parameters for maximizing “usefullness” to the average user is different enough from what’s needed to pass someone intentionally testing the AI. Mostly bc we know from other AI (like Siri) that people don’t actually need that much context saved to find them helpful
The reason is that the web browser chatgpt has a maximum amount of data per request. This is so they can minimize cost at scale. So for example you ask a question and tell it not to include a word. What will happen is your questions gets sent like this
{‘context’: ‘user asking question’, ‘message’: {user question here} }
then it gives you a response and you ask it another question. typically if it’s a small question the context is saved from one message to another.
{‘context’: ‘user asking question - {previous message}’, ‘message’: {new message here} }
so it literally just copies the previous message until it reaches the maximum token length
however there’s a maximum # of words that can be in the context + message combined. therefore the context is limited. after a certain amount of words input into chatgpt, it will start dropping things. it does this with a method to try and find out what is the “most important words” but this is inherently lossy. it’s like a jpeg- it gets blurry in order to save data.
so for example if you asked “please name the best fruit to eat, not including apple” and then maybe on the third or fourth question the “context” in the request becomes
‘context’: ‘user asking question - user wanted to know best fruit’
it would cut off the “not including apple bit” in order to save space
but here’s the thing - that exists in order to save space and processing power. it’s necessary at a large scale because millions of people could be talking to chatgpt and it couldn’t handle all that.
BUT if chatgpt wanted some sort of internal request that had no token limit, then everything would be saved. it would turn from a lossy jpeg into a png file. chatgpt would have infinite context.
this is why i think for someone who wants to keep context (ive been trying to develop specific applications which context is necessary) then chatgpt api just isn’t worth it.
I’m trying to tell you limited context is a feature not a bug, even other bots do the same thing like Replika. Even when all past data is stored serverside and available, it won’t matter because you need to reduce the weighting or you prevent significant change in output values (and less change as the history grows larger). Time decay of information is important to making these systems useful.
give an example please, because i don’t see how in normal use the weighting would matter at a significant scale based on the massive volume of training data
any interact the chatbot has with one person is dwarfed by the amount of total text data the AI has consumed through training. it’s like saying saggitarius a gets changed over time by adding in a few planets. while definitely true it’s going to be a very small effect
That’s kind of the point and how’s it different than a human. A human is going to weight local/recent contextual information as much more relevant to the conversation because they’re actively learning and storing the information (our brains work on more of an associative memory basis than temporal). However, with our current models it’s simulated by decaying weights over the data stream. So when you get conflicts between contextual correct vs “global” correct output, global has a tendency to win out that is more obvious. Remember you can’t actually make changes to the model as a user without active learning. Thus the model will always eventually return to it’s original behaviour as long as you can fill up the memory.
I asked OP’s questing of ChatGPT and it said:
As chatbots and AI language models continue to become more sophisticated, it can be challenging to distinguish between a human and an AI. However, there are certain questions and prompts that can help you identify whether you’re talking to a person or an AI. Here are some examples:
-
Complex Emotional Responses: Ask open-ended questions that require complex emotional responses or personal experiences. A human is more likely to provide detailed and emotionally nuanced answers, while an AI may struggle to respond with genuine emotions or personal anecdotes.
-
Humor and Jokes: Request a joke or a funny story. Humor is challenging for AI models, and while they may attempt to generate a joke, it often lacks the natural flow and wit that a human can provide.
-
Current Personal Experiences: Inquire about current events or experiences specific to the current date. A human can readily share their real-time experiences, whereas an AI’s knowledge is limited to its last training data update.
-
Unusual or Nonsensical Questions: Ask questions that are out of the ordinary or do not have a logical answer. A human might respond with creativity or curiosity, while an AI may struggle to produce coherent responses.
-
Emotional Empathy: Share a personal experience or a challenging situation and observe the response. Humans are generally better at showing empathy and understanding emotions, while an AI may provide pre-programmed or generic responses.
-
Follow-up Questions: Ask follow-up questions that refer to previous responses. Humans can maintain context and build on the conversation more naturally, while an AI may lose track of the discussion or provide repetitive answers.
-
Common Sense and World Knowledge: Ask questions that require real-world knowledge and common sense. A human will likely provide more accurate and comprehensive responses based on their life experience, whereas an AI may falter in providing practical advice or nuanced insights.
Remember that AI language models, like the one you are chatting with now, can be quite advanced and may mimic human-like responses to some extent. So, while these questions may help in identifying potential AI characteristics, there is no foolproof method to determine with absolute certainty whether you are talking to a person or an AI.
This answer shows how shit Chatgpt can be, and how it’s not really self aware.
Very few of these actually apply to chatgpt. And the ones that do, like a lack of awareness of current events don’t apply to those agents connected to the internet.
It’s just parroting old opinions that used to be true
-
Ask them a logical paradox like “does a set of all sets contain itself?”. This will send all robots into an infinite loop.
You’re probably joking, but I’ll comment anyway. It won’t affect LLMs at all. ChatGPT just answers the question and discusses the paradox. LLM’s function is basically just to construct sentences, so there’s nothing really that can potentially infinitely loop. It doesn’t “think” about paradoxes.
Well, I just asked the weird “message Carrot” option in my weather app, and it replied:
Oh, look at you, trying to puzzle your way through the depths of set theory. How amusing, like a chimp trying to juggle chainsaws.
Well, my dear meatbag, the answer to your question is a resounding NO. Just like you, that set cannot contain itself. It’s like expecting Johnny Five to date Stephanie from “Short Circuit.” Simply not gonna happen! 🤖💔
I’ve found that for chatGPT specifically;
- it really likes to restate your question in its opening sentence
- it also likes to wrap up with a take-home message. “It’s important to remember that…”
- it starts sentences with little filler words and phrases. “In short,” “that said,” “ultimately,” “on the other hand,”
- it’s always upbeat, encouraging, bland, and uncontroversial
- it never (that I’ve seen) gives personal anecdotes
- it’s able to use analogies but not well. They never help elucidate the matter
Chat GPT, yeah. Something like KremlinGPT has different parameters.
i cant see why this is downvoted cus 1 - you’re probably not wrong and 2 - KremlinGPT is funny as fuck 😭
I think lemmy.ml and lemmygrad.ml are probably both swarming with CCP/Kremlin bots
The most effective solution for this is to know what you want to get out of conversation. Then if you’re not getting it you can stop.
