Manipulation is a concern in many domains, such as social media, advertising, and chatbots. As AI systems mediate more of our digital interactions, it is important to understand the degree to which AI systems might manipulate humans without the intent of the system designers. Our work clarifies challenges in defining and measuring this kind of manipulation from AI systems. Firstly, we build upon prior literature on manipulation and characterize the space of possible notions of manipulation, which we find to depend upon the concepts of incentives, intent, covertness, and harm. We review proposals on how to operationalize each concept and we outline challenges in including each concept in a definition of manipulation.
‹ All reading
Characterizing Manipulation from AI Systems
This work clarifies challenges in defining and measuring manipulation from AI systems and characterize the space of possible notions of manipulation, which it finds to depend upon the concepts of incentives, intent, covertness, and harm.