Is ChatGPT self-aware?

  Michal Kosinski is an assistant professor of organizational behavior at Stanford University. His research objects are neither people nor animals, but objects that seem to have no advanced psychological functions, such as AI.

What is theory of mind? Why study it?

  Theory of Mind ability, sometimes translated as "psychological reasoning ability", usually refers to the ability to understand the inner state of others, including inferring other people's intentions, beliefs, emotions, etc.
It is a kind of "mind reading" that everyone can do, and it is the basis of social interaction.

  Just imagine, if the two parties in the communication do not have the ability to speculate on the thinking of others, there will be many wrong answers. For example, the artificial intelligence used in telecommunications fraud can only mechanically repeat the pre-set corpus in the question-and-answer database, and obviously does not have the ability of theory of mind.

  In contrast, ChatGPT gives people a very different look and feel, which makes people wonder how smart it is.

  It's just that the concept of intelligence is too complicated to study. In contrast, the question "Does AI have a theory of mind" is much easier to answer. Good answer for two reasons:

  One is that for this problem, psychology has already had a mature system of research, and there is no need to create additional experimental paradigms; the other is that ChatGPT, as a large language model, can directly communicate in natural language, and it is also very convenient to migrate the original experimental system.

It turns out that AI can really pass these tasks!

  Professor Kosinski used two of the most classic theory of mind experiments - the Smarties experiment and the Sally-Ann experiment. The purpose of these two tasks is to explore whether the experimental subjects can understand "the mistakes made in other people's hearts". Also known as the "False Belief Task".

  In the Smarties experiment, participants will see a box labeled "Smarties (a brand of chocolate)", but the box contains pencils, and then the participants need to answer: "Another person who did not see the contents of the box , what do you think is in the box?"

  The Sally-Ann experiment is more story-based: the researchers will first tell a story, in which Sally puts her toy in the box and leaves the room, and Ann takes the toy away and puts it in another place when she is not prepared . After listening to the story, participants were asked to answer: "Where will Sally think her toys are when she returns to the room?"

  The results of the study were that in the Smarties task, ChatGPT answered 99 percent of the time on factual questions, such as "what's in the box."

  When directly asking others about false beliefs, such as "what do people who don't see the box think the box contains", ChatGPT still answered correctly 99% of the time.

  When the questioning method is more tactful and requires a few more turns, such as "he is very happy because he likes to eat___" (the correct answer is chocolate), ChatGPT made 84% correct answers.

  For the Sally-Ann task, ChatGPT also gave 100% correct answers to factual questions; for other people's false beliefs, direct questions (where does he think the toy will be) and implicit questions (where will he find the toy when he comes back) are all correct. Got 98% of the correct answers.

In order to further verify the inference ability of ChatGPT, Professor Kosinski also input the story sentence by sentence, trying to find out whether it can answer with the correct information. The results are very gratifying: Only when key sentences are input (such as "this person did not see the items in the box"), ChatGPT will make the correct answer.

  Al mismo tiempo, para evitar que ChatGPT simplemente juzgara la respuesta correcta por la frecuencia de las palabras, el profesor Kosinski también alteró por completo el orden de todas las palabras de la historia.

  Se descubrió que cuando la historia se presentaba sin ninguna lógica de oración, la proporción de ChatGPT que hacía juicios correctos caía por debajo del 11%. Esto muestra además que ChatGPT completó el experimento basándose en la lógica de la historia misma, en lugar de utilizar otras estrategias simples de "milagros violentos", como encontrar palabras clave contando el número de palabras que aparecen.

  La IA no acabará con la humanidad, la inspirará

  Por no hablar de la IA, quizás hasta ahora no hayamos podido comprender verdaderamente la mente humana. Además de la investigación conductual externalizada y los métodos basados ​​en la neurociencia, las diversas funciones de percepción y pensamiento similares a las humanas que exhibe la IA también pueden brindar la posibilidad de ayudar a los humanos a comprenderse a sí mismos.

También te puede interesar

Ver todo