Yiming Zhang, Zhonghan Zhao, Wenwei Zhang, Haiteng Zhao, Tianyang Lin, Yunhua Zhou, Demin Song, Kuikun Liu, Haochen Ye, Haian Huang, Yuzhe Gu, Haijun Lv, Qipeng Guo, Bin Liu, Gaoang Wang, Kai Chen
This paper introduces Visual Pretraining, a scalable learner for foundation model intelligence that leverages visual documents without text extraction. It outperforms text-only pretraining across multiple backbones and benchmarks. Visual Pretraining offers an efficient pathway to scalable language intelligence.
The approach has practical significance as it can capture rich information from visual representations, enabling more accurate language understanding. This can improve various applications such as document analysis and web page processing. It also opens up new possibilities for multimodal learning.
Thank you to arXiv for use of its open access interoperability. arxiv.org. aibrief.ai is not affiliated with or endorsed by arXiv.