Yu Ran, Wentao Zhao, Xin Zhang, Yi Pan
The paper introduces MissClick, a white-box adversarial attack that exploits digit-serialized coordinates in GUI grounding models. MissClick achieves high success rates for untargeted and targeted attacks. It outperforms existing attacks on various platforms.
The research highlights the security implications of GUI grounding models, which can be exploited to induce large displacements of executable clicks. This has significant practical implications for the security of graphical user interfaces. The findings can inform the development of more secure GUI grounding models.
Thank you to arXiv for use of its open access interoperability. arxiv.org. aibrief.ai is not affiliated with or endorsed by arXiv.