The whole point of LLMs is to fake being human through following the likely paths text will follow. Is that not what a philosophical zombie[0] is: appearing to be human while having no actual intelligence?[0] https://en.wikipedia.org/wiki/Philosophical_zombie