While AI ethics ensures fairness, accountability, and protection of user rights, dark patterns manipulate users to take unintended actions on digital interfaces. Related studies uncover limited insights into how reliably; human experts and AI models can detect dark patterns within a specific taxonomy. Our research fills this gap by asymmetrically examining cross-origin detection performance of human and AI/LLM evaluators (each evaluator’s ability to detect dark patterns generated by the opposite source) to understand their limitations and future potentials. Using GPT-4.1, we generated 200 UI images (with matched dark and non-dark pattern pairs) and selected 200 UI images collected 200 human-created UI screenshots from the ContextDP/AidUI dataset, based on computational, methodological, and statistical considerations.
‹ All reading
UX experts vs. AI: exploring the performance of large language models and humans on detecting dark patterns
A novel study which explores the performance of AI/LLMs and UX experts in detecting dark patterns in UI images, and provides a benchmark dataset that could be useful to future research, while discussing empirical insights into the role, limitations, and promise of AI/LLMs in UI/UX design ethics and auditing, in realistic deployment scenarios.