Xintao Wang (王鑫涛) is a third year Ph.D candidate at Fudan University in the School of Computer Science. He is deeply fascinated with ACG (Anime, Comics & Games) culture, and is devoted to revolutionizing the ACG industry with AI techniques. Hence, his research interests primarily focus on human-like generative agents and their personas, including (but not limited to):
Role-Playing Language Agents: Targeting at creating AI agents that faithfully represent specific personas, including: (1) Agents for fictional characters from books and ACG, whose applications include virtual companions, games, and content creation; and (2) Agents for real-world individuals, which deeply understand user personas to serve as their digital proxies or personal assistants.
Cognitive Modeling in Language Models: Focusing on integrating anthropomorphic cognition into LLMs, such as ego-awareness, social intelligence, personalities, etc. The goal is to promote LLMs’ understanding of the inner world of themselves and others, hence enabling them to generate more cognitively-aligned and human-like responses.
Ph.D in NLP, 2021 - 2026 (estimated)
Fudan University
B.S in CS, 2017 - 2021
Fudan University
We present a comprehensive survey on role-playing language agents (RPLAs), i.e., specialized AI systems simulating assigned personas. Specifically, we distinguish personas in RPLAs into three progressive tiers - demographic persona, character persona, and individualized persona. Our survey discusses their data sourcing, agent construction, evaluation, applications, risks, limitations and future prospects.
Do role-playing language agents accurately capture character personalities? Towards this question, we propose InCharacter that evaluates their personality fidelity via psychological interviews using 14 scales.
Can Large Language Models substitute humans in making important decisions? Recent research has unveiled the potential of LLMs to role-play assigned personas, mimicking their knowledge and linguistic habits. However, imitative decision-making requires a more nuanced understanding of personas. In this paper, we benchmark the ability of LLMs in persona-driven decision-making. Specifically, we investigate whether LLMs can predict characters’ decisions provided with the preceding stories in high-quality novels. Leveraging character analyses written by literary experts, we construct a dataset LIFECHOICE comprising 1,401 character decision points from 395 books. Then, we conduct comprehensive experiments on LIFECHOICE, with various LLMs and methods for LLM role-playing. The results demonstrate that state-of-the-art LLMs exhibit promising capabilities in this task, yet there is substantial room for improvement. Hence, we further propose the CHARMAP method, which achieves a 6.01% increase in accuracy via persona-based memory retrieval. We will make our datasets and code publicly available.
Large language models (LLMs) have demonstrated impressive performance and spurred numerous AI applications, in which role-playing agents (RPAs) are particularly popular, especially for fictional characters. The prerequisite for these RPAs lies in the capability of LLMs to understand characters from fictional works. Previous efforts have evaluated this capability via basic classification tasks or characteristic imitation, failing to capture the nuanced character understanding with LLMs. In this paper, we propose evaluating LLMs’ character understanding capability via the character profiling task, i.e., summarizing character profiles from corresponding materials, a widely adopted yet understudied practice for RPA development. Specifically, we construct the CroSS dataset from literature experts and assess the generated profiles by comparing ground truth references and their applicability in downstream tasks. Our experiments, which cover various summarization methods and LLMs, have yielded promising results. These results strongly validate the character understanding capability of LLMs. We believe our constructed resource will promote further research in this field. Resources are available at https://github.com/Joanna0123/character_profiling.