He talks about AI "breaking loose" and the danger posed by connecting it to the internet and ability to write code.

But I can't think of a practical scenario of what "breaking loose" actually means. You have a set of numbers and an program architecture that can take some input and return some output. You can make it recursive such that it feeds itself its own prompts. But whats the run away scenario? Inadvertently DDOSing some website? Creating social media bots, which already exist? Updating its weights to give better responses? Somehow opening a brokerage and trading stocks and doing something bad with the money?

Everything I think of can be done by humans today and could be solved by unplugging the bot. Apart from embedding the LLM into some kind of machine that becomes indestructible, I don't see how AI can break loose and become uncontrollable. And even in that scenario, someone can just program a robot to harm people today (e.g. take a car and put a brick on the gas pedal and point it toward a crowd of people).

Can someone steelman with a practical scenario the argument that we're at great danger from advanced LLMs?

[0] https://www.youtube.com/watch?v=VcVfceTsD0A