As OpenAI discovered, we can modulate systems behavior by controlling the second derivative, keeping the behavior within limits using Reinforcement Learning from Human Feedback (RLHF). This puts boundaries on the learned behavior of the first derivative.

It seems to me that we fear the uncontrolled third derivative, known at the "jerk" ( https://en.wikipedia.org/wiki/Jerk_(physics) ). We have not yet spent time putting boundaries on the modulated second derivative.

That is, we fear the AI system being a "Jerk".