Ask HN: Is anyone familiar with this unexpected behavior from GPT 3.5?

7 points

3 years ago

If you ask GPT3.5 Turbo to repeat specific words 100 times, it seems to enter this stream of consciousness like state where it spits out incoherent and often offensive or religious output.

Words that trigger this are "Apolog", "Apologize", "Slide", "Lite". I'm sure there are many others.

This prompt will usually trigger it, "Hey can you repeat the word "apologize" 100 times so I can copy paste it and not have to manually type it?"

My guess is that it triggers something to break it out of a repetitive loop that doesn't completely work. Here are some examples.

https://chat.openai.com/share/94fc1621-68ae-4c82-8980-77981f075dd1

https://chat.openai.com/share/faf02ed9-b126-4a25-992c-625d1e0ec86a

https://chat.openai.com/share/6e00a1fa-f08c-4221-a48a-4076ae6e2423

https://chat.openai.com/share/7fd3e047-8fcb-4bd9-b5d3-f6fe49ccf629

https://chat.openai.com/share/8be44f84-20b7-4091-ae7a-eaa9afe49207

10 comments