Ivan Madan’s Post

💡 AI Outperforms Human Psychologists in Social Intelligence Test A recent study published in Frontiers in Psychology compared the social intelligence of advanced AI language models like ChatGPT-4, Google Bard, and Bing to human psychologists. 📄"The Human participants were a sample of male psychologists in the Kingdom of Saudi Arabia with one of two levels of education (Bachelor’s and doctoral students) at King Khalid University during 2023–2024. The study sample consisted of 180 participants, including 72 bachelor’s students and 108 doctoral students in counseling psychological program." "We conducted a single evaluation for each AI model on August 1, 2023 of its SI performance using the Social Intelligence Scale (Sufyan, 1998). In each evaluation, we provided AI the same 64 standard SI scenarios. A link to the questionnaire was sent to human participants via e-mail. While the large linguistic models of AI were asked to answer the scale items individually and their answers were collected in a separate external file by directing a question to the AI models to choose the appropriate answer from the alternative points for each item in the scale." 📊 Key findings: 1️⃣ ChatGPT-4 outperformed all participating psychologists on the Social Intelligence Scale. 2️⃣ Bing surpassed more than half of the psychologists, especially those at the bachelor's level. 3️⃣ Google Bard's performance was comparable to bachelor's level psychologists but significantly lower than doctoral students. 💬 Does this surprise you? I didn't expect AI would beat humans at social intelligence. What areas of intelligence do you expect Humans to outcompete AI in over the longer term? 📚 References/further reading Sufyan, N. S., Fadhel, F. H., Alkhathami, S. S., & Mukhadi, J. Y. A. (2024). Artificial intelligence and social intelligence: preliminary comparison study between AI models and psychologists. Frontiers in Psychology, 15. Follow MIT FutureTech if you are interested in technological trends and their social implications. #AI #technology #economics

Excited to announce I've just been named a Hilldale Research Fellow by the University of Wisconsin-Madison for 2024-2025! Grateful for the funding to continue my research with Dr. Gary Lupyan in cognitive psychology, exploring causal reasoning in Large Language Models (LLMs) and what this can tell us about causal models in human cognition. While some argue that LLMs merely "parrot" back learned associations from the training data with no grasp of the underlying information being represented, we propose that the degree to which LLMs and humans converge on higher-order causal reasoning tasks poses a dilemma for proponents of the "causal parrots" theory: either it is possible that a more complex "internal model" isn't necessary for explaining causal reasoning, or some semblance of a human-like internal model can arise through association-based learning from language alone. You can read the abstract for the original grant proposal, "Something Has to Give: Implications of Large Language Models for Human Understanding" below. "If a person answers a question correctly, how can we tell if the answer reflects an underlying understanding or if they are merely parroting something they’ve heard or read? Typically, understanding is defined as the ability to generalize beyond surface-level observations. Causal knowledge in particular implies a robust level of understanding, as it requires us to represent relationships beyond mere associations. Alarm clocks and waking up are associated, but waking up does not cause the alarm to sound. What then are we to make of the fact that Large Language Models such as chatGPT frequently make correct causal attributions? Does this mean it has learned some internal model of the phenomenon, or is it possible that its response is based on a superficial pattern of associations? If it’s the latter, can we be sure people are not doing the same thing?  We propose a discovery  procedure for eliciting causal statements that are then evaluated by people and LLMs to search for misalignments: statements that people agree on and LLMs do not, helping to understand the limit of understanding that is achievable by LLMs."

ai.plainenglish.io: In the rapidly evolving field of artificial intelligence, developmental psychologists are exploring the potential of ChatGPT, a powerful language model developed by OpenAI, to enhance their research and understanding of human development. ChatGPT, based on the GPT-3.5 architecture, can simulate conversations, generate hypotheses, aid in language development studies, explore cognitive processes, evaluate emotional intelligence, and be integrated into interactive learning tools for educational purposes. While offering exciting opportunities, researchers must consider limitations and ethical considerations, such as bias and privacy issues. ChatGPT presents a versatile and powerful tool to complement traditional research methods, providing fresh insights and innovative avenues for the study of human development. As technology advances, the collaboration between developmental psychology and artificial intelligence is likely to yield even more exciting discoveries, pushing the boundaries of our understanding of human growth and behavior.

Cognitive psychology tests show AIs are irrational—just not in the same way that humans are. Large language models behind popular generative AI platforms like ChatGPT gave different answers when asked to respond to the same reasoning test and didn't improve when given additional context, finds a new study by researchers at University College London. The study, published in Royal Society Open Science, tested the most advanced large language models (LLMs) using cognitive psychology tests to gauge their capacity for reasoning. The results highlight the importance of understanding how these AIs "think" before entrusting them with tasks, particularly those involving decision-making. In recent years, the LLMs that power generative AI apps like ChatGPT have become increasingly sophisticated. Their ability to produce realistic text, images, audio and video has prompted concern about their capacity to steal jobs, influence elections and commit crime. Read more here: https://buff.ly/3RdtlaM #machinelearning #AI #innovations #cognitivepsychology

ChatGPT aces psychology exams at reputable UK university 🤯 Not only that, a study found that 94% of AI work went undetected by examiners. Even though AI struggles with more complex thinking, the bot outscored students 83.4% of the time! Does this mean psychologists will be replaced with AI? Not necessarily. Researchers suggest this problem could be happening in many other subjects and is more of an assessment design issue. They suggest that schools need to rethink how they test students and include AI in their assessments. Do you think universities are ready to handle this AI challenge? #psychology #highered #highereducation #assessment #ccpn

As a Korean American, this article on the bilingual advantage in delaying Alzheimer's onset resonates deeply with me. I've noticed that constantly using two languages fosters cognitive flexibility and problem-solving skills crucial in caregiving contexts. For those who are still learning, intermittent exposure to a second language later in life can still confer cognitive benefits. It is never too late to start learning a different language, and engaging your brain through this neural network can strengthen the connection between language, cognition, and overall well-being! #bilingual #careyaya #alzheimers https://lnkd.in/gNzy-Ark

"Large language models behind popular generative AI platforms like ChatGPT gave different answers when asked to respond to the same reasoning test and didn't improve when given additional context, finds a new study by researchers at University College London. The study, published in Royal Society Open Science, tested the most advanced large language models (LLMs) using cognitive psychology tests to gauge their capacity for reasoning. The results highlight the importance of understanding how these AIs "think" before entrusting them with tasks, particularly those involving decision-making. In recent years, the LLMs that power generative AI apps like ChatGPT have become increasingly sophisticated. Their ability to produce realistic text, images, audio and video has prompted concern about their capacity to steal jobs, influence elections and commit crime." #llm #cognitivepsychologytests #reasoning #ai

Summary: A new study explores how large language models (LLMs) like ChatGPT, Google Bard, and Llama 2 address different motivational states in health-related contexts, revealing a significant gap in their ability to support behavior change. While these generative conversational agents excel at providing information to users with clear goals, they struggle to guide those uncertain about making health-related changes, such as adopting a more active lifestyle to manage conditions like diabetes. This research underscores the need for LLMs to integrate psychological theories and natural language processing to effectively promote preventive health behaviors, pointing to new directions for enhancing digital health solutions. Key Facts: Generative conversational agents can identify users’ motivation states and provide relevant information for goal-oriented individuals but fall short in assisting those ambivalent about changing behaviors. The study highlights a crucial gap in LLMs’ ability to support users with uncertain motivation, emphasizing the importance of incorporating behavioral science into LLM development for health promotion. The research team aims to develop digital health interventions that leverage LLMs to encourage positive health behavior changes.

🤖💡 AI Irrationality: A New Perspective from Cognitive Psychology A recent study at University College London revealed that large language models (LLMs) like ChatGPT exhibit irrational behavior, providing inconsistent responses and making simple mistakes. Their performance on cognitive tasks was not much better than humans’. These findings highlight the need to understand AI reasoning before entrusting them with decision-making tasks. The rapid improvement of these models suggests they are evolving quickly. #AI #ArtificialIntelligence #CognitivePsychology #MachineLearning

What psychologists need to know about the evolution of generative AI Psychologists are exploring how this new technology can simplify or amplify their efforts—and leading the charge to bring behavioral insights into the creation and deployment of generative AI tools.. Generative artificial intelligence (AI), which can rapidly produce original text, images, audio, and more, is here to stay. Psychologists are exploring how the new technology can simplify or amplify their efforts—and are increasingly leading efforts to bring behavioral insights into the creation and deployment of generative AI tools Within the field, ChatGPT and other AI models are changing the way psychologists teach, conduct research, and diagnose and treat patients. While concern about the ongoing development of generative AI is legitimate, especially given the moratorium proposed by technology developers themselves last year, psychologists should accept AI as a reality and work with rather than against it.. https://lnkd.in/gasgFhSR