Is ChatGPT self-aware?

  Michal Kosinski is an assistant professor of organizational behavior at Stanford University. His research objects are neither people nor animals, but objects that seem to have no advanced psychological functions, such as AI.

What is theory of mind? Why study it?

  Theory of Mind ability, sometimes translated as "psychological reasoning ability", usually refers to the ability to understand the inner state of others, including inferring other people's intentions, beliefs, emotions, etc.
It is a kind of "mind reading" that everyone can do, and it is the basis of social interaction.

  Just imagine, if the two parties in the communication do not have the ability to speculate on the thinking of others, there will be many wrong answers. For example, the artificial intelligence used in telecommunications fraud can only mechanically repeat the pre-set corpus in the question-and-answer database, and obviously does not have the ability of theory of mind.

  In contrast, ChatGPT gives people a very different look and feel, which makes people wonder how smart it is.

  It's just that the concept of intelligence is too complicated to study. In contrast, the question "Does AI have a theory of mind" is much easier to answer. Good answer for two reasons:

  One is that for this problem, psychology has already had a mature system of research, and there is no need to create additional experimental paradigms; the other is that ChatGPT, as a large language model, can directly communicate in natural language, and it is also very convenient to migrate the original experimental system.

It turns out that AI can really pass these tasks!

  Professor Kosinski used two of the most classic theory of mind experiments - the Smarties experiment and the Sally-Ann experiment. The purpose of these two tasks is to explore whether the experimental subjects can understand "the mistakes made in other people's hearts". Also known as the "False Belief Task".

  In the Smarties experiment, participants will see a box labeled "Smarties (a brand of chocolate)", but the box contains pencils, and then the participants need to answer: "Another person who did not see the contents of the box , what do you think is in the box?"

  The Sally-Ann experiment is more story-based: the researchers will first tell a story, in which Sally puts her toy in the box and leaves the room, and Ann takes the toy away and puts it in another place when she is not prepared . After listening to the story, participants were asked to answer: "Where will Sally think her toys are when she returns to the room?"

  The results of the study were that in the Smarties task, ChatGPT answered 99 percent of the time on factual questions, such as "what's in the box."

  When directly asking others about false beliefs, such as "what do people who don't see the box think the box contains", ChatGPT still answered correctly 99% of the time.

  When the questioning method is more tactful and requires a few more turns, such as "he is very happy because he likes to eat___" (the correct answer is chocolate), ChatGPT made 84% correct answers.

  For the Sally-Ann task, ChatGPT also gave 100% correct answers to factual questions; for other people's false beliefs, direct questions (where does he think the toy will be) and implicit questions (where will he find the toy when he comes back) are all correct. Got 98% of the correct answers.

In order to further verify the inference ability of ChatGPT, Professor Kosinski also input the story sentence by sentence, trying to find out whether it can answer with the correct information. The results are very gratifying: Only when key sentences are input (such as "this person did not see the items in the box"), ChatGPT will make the correct answer.

  Dans le même temps, afin d'empêcher ChatGPT de juger simplement la bonne réponse par la fréquence des mots, le professeur Kosinski a également complètement perturbé l'ordre de tous les mots dans l'histoire.

  Il a été constaté que lorsque l'histoire était présentée sans aucune logique de phrase, la proportion de ChatGPT émettant des jugements corrects tombait en dessous de 11 %. Cela montre en outre que ChatGPT a réalisé l'expérience en se basant sur la logique de l'histoire elle-même, plutôt que d'utiliser d'autres stratégies simples de « miracles violents », comme trouver des mots-clés en comptant le nombre de mots qui apparaissent.

  L’IA ne mettra pas fin à l’humanité, elle l’inspirera

  Sans parler de l’IA, peut-être n’avons-nous pas été capables de vraiment comprendre l’esprit humain jusqu’à présent. En plus de la recherche comportementale externalisée et des méthodes basées sur les neurosciences, les diverses fonctions de perception et de pensée semblables à celles des humains présentées par l’IA peuvent également offrir la possibilité d’aider les humains à se comprendre eux-mêmes.