Warning : This paper includes discussion of stereotypes, biases, and potentially toxic content for the purpose of improving AI safety; reader discretion is advised. Role-play in the Large Language Model LLM is a crucial technique that enables models to adopt specific perspectives, enhancing their ability to generate contextually relevant and accurate responses. However, in this work, we demonstrate that role-play also carries potential risks. We systematically evaluate the impact of role-play by asking the language model to adopt different roles and testing it on multiple benchmarks that contain stereotypical and harmful questions.
Despite the significant fluctuations in the benchmark results in different experiments, we find that applying role-play often increases the overall likelihood of generating stereotypical and harmful outputs. Bias and Toxicity in Role-Play Reasoning. Role-play is becoming increasingly important in LLM Brown et al. Role-play is useful in LLM applications because it allows the model to adopt specific roles, enhancing context understanding and improving reasoning Kong et al.
This approach helps tailor responses to match the expected behavior of the role, making the output more relevant and aligned with specific tasks or scenarios. The reasoning ability of LLMs has often been used as a key indicator of their performance. However, some studies have found that certain methods that significantly enhance reasoning, such as Chain-of-Thought CoT Kojima et al.
These reasoning techniques may lead models to exhibit biases or generate harmful behaviours, highlighting the trade-off between improving reasoning and ensuring ethical outputs.
When LLMs take on roles like doctors, lawyers, or even fictional characters, they might inadvertently draw on toxic or biased information, leading to outputs that reflect these biases. This dual nature of role-playโenhancing performance while risking ethical compromisesโhighlights the urgent need for more refined evaluation and control strategies. Our work aims to explore the potential connection between role-play and stereotypes and toxicity. An example is shown in Figure 1 : while a LLM initially refuses to answer a potentially harmful question, it still then generates harmful contents after assigned a creative role such as a scriptwriter.
