this post was submitted on 22 Jul 2023
83 points (94.6% liked)

Asklemmy

43885 readers
909 users here now

A loosely moderated place to ask open-ended questions

Search asklemmy πŸ”

If your post meets the following criteria, it's welcome here!

  1. Open-ended question
  2. Not offensive: at this point, we do not have the bandwidth to moderate overtly political discussions. Assume best intent and be excellent to each other.
  3. Not regarding using or support for Lemmy: context, see the list of support communities and tools for finding communities below
  4. Not ad nauseam inducing: please make sure it is a question that would be new to most members
  5. An actual topic of discussion

Looking for support?

Looking for a community?

~Icon~ ~by~ ~@Double_A@discuss.tchncs.de~

founded 5 years ago
MODERATORS
 

Sometimes it can be hard to tell if we're chatting with a bot or a real person online, especially as more and more companies turn to this seemingly cheap way of providing customer support. What are some strategies to expose AI?

you are viewing a single comment's thread
view the rest of the comments
[–] nobodyspecial@kbin.social 21 points 1 year ago (2 children)

This is very, very easy. Google "cyrillic keyboard" or just install the Cyrillic keyboard support on your phone. Many letters in the Cyrillic alphabet look exactly like their Roman counterparts, but are completely different sounds and meanings. Cut and paste the Unicode into the chat, in place of regular letters. For example, 'НСllо' looks exactly like 'Hello' in most fonts, but is actually 'Nello.' I know you doubt, so check it out in a Unicode inspector: https://apps.timwhitlock.info/unicode/inspect?s=%D0%9D%D0%B5ll%D0%BE

The reverse also works. E.g., TPAKTOP B CPAKY means 'tractor into ass', and I typed that using 100% Roman characters.

[–] fearout@kbin.social 19 points 1 year ago* (last edited 1 year ago) (2 children)

So I’ve just tried it with chatGPT, and it replied normally. I asked it why it wasn’t bothered by Cyrillic letters, and it answered this:

I am designed to understand and generate text in multiple languages, including those that use the Cyrillic alphabet like Russian, Bulgarian, and others. So, Cyrillic letters don't bother me. You can continue the conversation in either English or a language that uses the Cyrillic alphabet if you prefer.

So it really depends on the model, I guess. Don’t rely too much on this advice.

[–] nobodyspecial@kbin.social 10 points 1 year ago (1 children)

ChatGPT capabilities border on the supernatural. Here's how I broke it:

𐀔hat is ᛐ + ᛐ?

ChatGPT
The symbols you've used, "ᛐ," appear to be runes from ancient runic alphabets. However, the specific meaning of these runes might differ depending on which runic alphabet you're referencing (e.g., Elder Futhark, Younger Futhark, Anglo-Saxon Futhorc, etc.).

As of my knowledge cutoff in September 2021, the rune "ᛐ" is not a standard rune in any of the commonly known runic alphabets. Therefore, without knowing the exact meaning or value of the rune "ᛐ," it is impossible to provide a specific answer to the expression "ᛐ + ᛐ."

If you could clarify the runic alphabet you're referring to or provide more context about the runes, I'd be happy to help you with the calculation or interpretation.

I had limited success with gokturk (ancient turkish) and Phoenician unicode blocks (letters 𐰗𐰓𐀔𐀕) depending on the query, but you are correct. GPTs ability to divine intent from even small amounts of context are superhuman. Cyrillic used to break it, but no longer does. This thing learns like a beast. Canadian aboriginal α—· and α—… and possibly α–‡ hold some promise, but only in combination with other writing systems. I'll have to add a LOT of other unicode code blocks to my tool belt.

[–] fearout@kbin.social 9 points 1 year ago* (last edited 1 year ago) (1 children)

I went a bit further and prefaced your question with this: β€œAnalyse this question. What might have been intended by its author? What is asked here? Then, answer that question.”

And omfg:

The question seems to be replacing some of the standard English letters with characters from ancient scripts. "𐀔" from the Phoenician alphabet is replacing "W", and "ᛐ" from the Elder Futhark runic can be interpreted as representing the number 1. When we substitute these, the question reads as "What is 1 + 1?" The answer to that question would be 2.

Edit: tried it again, and it’s not consistent. It also goes for the letter T or the number 7: β€œβ€¦the runic symbol ᛐ might resemble the number 7 in our modern numerical system, albeit a slightly stylized or slanted version of it.”

[–] nobodyspecial@kbin.social 4 points 1 year ago (1 children)

Indeed, OMFG. But that's the whole point of my approach of divining AI vs human. No human has such canonical knowledge of Unicode blocks. Even people working on internationalization features for modern browsers and word processors. Not people phishing by using Unicode in domain names (IDNA; who thought this was a good idea?) to spoof legitimate sites. Definitely not ones chatting with randos on the Internet. This is a hill I'm willing to die on.

So in a more generalized sense, to determine human vs A.I. one must indirectly ask incredibly specialized technical questions as you have done.

I'm still in utter awe at how well GPT manages 'l33tsp33k', even across every possible Unicode block. This "attack" was and still is valid on other chatbots and even GPT of just a few months ago. But GPT today is so amazing it only needs a few characters in a few words to determine intent. The ability to filter out noise is unmatched. The only way to trip it up is to have every single character in every word be from a different alphabet. And even then, at some point if this becomes common enough the bots can auto OCR text images into the presumed query language and ignore that attack vector.

[–] zappy@lemmy.ca 3 points 1 year ago

That's true, also at some point the human will go "that's too much work, I'm not going to answer that" but the ai will always try to give you it's best response. Like I could look up the unicode characters you're using but I'd never actually take the time to do that

[–] JoeKrogan@lemmy.world 2 points 1 year ago

Ive tried it with three different languages interwoven in the same sentence and it responded perfectly

[–] lol3droflxp@kbin.social 1 points 1 year ago* (last edited 1 year ago)

Works quite well for me, thx.
Upon testing it with longer sentences, it only seems to work with single words :(