Deep Learning-Based Detection of Deceptive UI Patterns in Web Applications Using Convolutional Neural Networks

The growth of web applications has been accompanied by a parallel growth in the variety of deceptive user interface (UI) patterns or, alternatively referred to as, dark patterns that manipulate their users by taking advantage of human behavior by manipulating their visual presentation. The privacy, freedom, and economic wellbeing of consumers are at a high risk due to these trends, e.g., disguised advertisements, unnoticed subscription traps, misleading sense of urgency, and confirm shaming elements. Although increased regulatory interest has led to an increase in regulatory focus, scale-based automated detection of such patterns is an ongoing research problem. Manual auditing is labor intensive, not consistent and cannot be used to monitor the web continuously. The paper suggests a Convolutional Neural Network (CNN)-based system to identify and classify fake UI patterns in web application screenshots. The systematic data collection and strict inter-annotator annotation created a curated, annotated set of 4,800 web screenshots in six categories of deception and a non-deceptive baseline. It has been compared to five state-of-the-art baseline models, such as VGG16, ResNet50, EfficientNetB3, a Support Vector Machine (SVM) and a Random Forest classifier. Gradient-weighted Class Activation Mapping (Grad-CAM) was used to explain the activation of salient regions of the UI that used the activation to make classification decisions. The proposed CNN obtained a macro-averaged F1-score of 0.914 and AUC of 0.971, which is higher than any baselines. Grad-CAM analyses provided support that the model pays attention to semantically meaningful areas including call-to-action buttons, pre-ticked checkboxes, and obfuscated pricing items.