Ask HN: Is “prompt injection” going to be a new common vulnerability?

83 points

3 years ago

There was a post [0] recently about the bing chatGPT assistant either citing or hallucinating it’s own initial prompt from the (in theory) low privileged chat input UI they put together. This feels like it’s almost unavoidable if you let users actually chat with something like this.

How would we sanitize strings now? I know OpenAI has banned topics they seem to regex for, but that’s always going to miss something. Are we just screwed and should make sure chat bots just run in a proverbial sandbox and can’t do anything themselves?

[0] https://news.ycombinator.com/item?id=34717702

111 comments