{"technology":{"slug":"computer-vision","name":"Computer Vision","description":"Visual AI and image understanding. Object detection, segmentation, 3D vision, video understanding, visual transformers, and multimodal vision-language models.","discipline":"Computer Science / AI","icon":"👁️"},"lastUpdated":"2026-07-21T06:28:13.087Z","articleCount":29,"articles":[{"id":"oa-W2112796928","title":"Gradient-based learning applied to document recognition","authors":"Yann LeCun, Léon Bottou, Yoshua Bengio, Patrick Haffner","journal":"Proceedings of the IEEE","pubDate":"1998-01-01","doi":"10.1109/5.726791","abstract":"Multilayer neural networks trained with the back-propagation algorithm constitute the best example of a successful gradient based learning technique. Given an appropriate network architecture, gradient-based learning algorithms can be used to synthesize a complex decision surface that can classify high-dimensional patterns, such as handwritten characters, with minimal preprocessing. This paper reviews various methods applied to handwritten character recognition and compares them on a standard handwritten digit recognition task. Convolutional neural networks, which are specifically designed to deal with the variability of 2D shapes, are shown to outperform all other techniques. Real-life document recognition systems are composed of multiple modules including field extraction, segmentation recognition, and language modeling. A new learning paradigm, called graph transformer networks (GTN), allows such multimodule systems to be trained globally using gradient-based methods so as to minimize an overall performance measure. Two systems for online handwriting recognition are described. Experiments demonstrate the advantage of global training, and the flexibility of graph transformer networks. A graph transformer network for reading a bank cheque is also described. It uses convolutional neural network character recognizers combined with global training techniques to provide record accuracy on business and personal cheques. It is deployed commercially and reads several million cheques per day.","tldr":"","source":"OpenAlex","sourceUrl":"https://openalex.org/W2112796928","citationCount":58620,"isOpenAccess":true,"pdfUrl":"https://hal.science/hal-03926082/document"},{"id":"s2-c8b25fab5608c3e033d34b4483ec47e68ba109b7","title":"Swin Transformer: Hierarchical Vision Transformer using Shifted Windows","authors":"Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, B. Guo","journal":"2021 IEEE/CVF International Conference on Computer Vision (ICCV)","pubDate":"2021","doi":"10.1109/ICCV48922.2021.00986","abstract":"This paper presents a new vision Transformer, called Swin Transformer, that capably serves as a general-purpose backbone for computer vision. Challenges in adapting Transformer from language to vision arise from differences between the two domains, such as large variations in the scale of visual entities and the high resolution of pixels in images compared to words in text. To address these differences, we propose a hierarchical Transformer whose representation is computed with Shifted windows. The shifted windowing scheme brings greater efficiency by limiting self-attention computation to non-overlapping local windows while also allowing for cross-window connection. This hierarchical architecture has the flexibility to model at various scales and has linear computational complexity with respect to image size. These qualities of Swin Transformer make it compatible with a broad range of vision tasks, including image classification (87.3 top-1 accuracy on ImageNet-1K) and dense prediction tasks such as object detection (58.7 box AP and 51.1 mask AP on COCO test-dev) and semantic segmentation (53.5 mIoU on ADE20K val). Its performance surpasses the previous state-of-the-art by a large margin of +2.7 box AP and +2.6 mask AP on COCO, and +3.2 mIoU on ADE20K, demonstrating the potential of Transformer-based models as vision backbones. The hierarchical design and the shifted window approach also prove beneficial for all-MLP architectures. The code and models are publicly available at https://github.com/microsoft/Swin-Transformer.","tldr":"A hierarchical Transformer whose representation is computed with Shifted windows, which has the flexibility to model at various scales and has linear computational complexity with respect to image size and will prove beneficial for all-MLP architectures.","source":"Semantic Scholar","sourceUrl":"https://www.semanticscholar.org/paper/c8b25fab5608c3e033d34b4483ec47e68ba109b7","citationCount":33965,"isOpenAccess":true,"pdfUrl":"http://arxiv.org/pdf/2103.14030"},{"id":"oa-W3138516171","title":"Swin Transformer: Hierarchical Vision Transformer using Shifted Windows","authors":"Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, Baining Guo","journal":"2021 IEEE/CVF International Conference on Computer Vision (ICCV)","pubDate":"2021-10-01","doi":"10.1109/iccv48922.2021.00986","abstract":"This paper presents a new vision Transformer, called Swin Transformer, that capably serves as a general-purpose backbone for computer vision. Challenges in adapting Transformer from language to vision arise from differences between the two domains, such as large variations in the scale of visual entities and the high resolution of pixels in images compared to words in text. To address these differences, we propose a hierarchical Transformer whose representation is computed with Shifted windows. The shifted windowing scheme brings greater efficiency by limiting self-attention computation to non-overlapping local windows while also allowing for cross-window connection. This hierarchical architecture has the flexibility to model at various scales and has linear computational complexity with respect to image size. These qualities of Swin Transformer make it compatible with a broad range of vision tasks, including image classification (87.3 top-1 accuracy on ImageNet-1K) and dense prediction tasks such as object detection (58.7 box AP and 51.1 mask AP on COCO test-dev) and semantic segmentation (53.5 mIoU on ADE20K val). Its performance surpasses the previous state-of-the-art by a large margin of +2.7 box AP and +2.6 mask AP on COCO, and +3.2 mIoU on ADE20K, demonstrating the potential of Transformer-based models as vision backbones. The hierarchical design and the shifted window approach also prove beneficial for all-MLP architectures. The code and models are publicly available at https://github.com/microsoft/Swin-Transformer.","tldr":"","source":"OpenAlex","sourceUrl":"https://openalex.org/W3138516171","citationCount":30647,"isOpenAccess":false,"pdfUrl":""},{"id":"oa-W4312443924","title":"A ConvNet for the 2020s","authors":"Zhuang Liu, Hanzi Mao, Chao-Yuan Wu, Christoph Feichtenhofer, Trevor Darrell, Saining Xie","journal":"2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)","pubDate":"2022-06-01","doi":"10.1109/cvpr52688.2022.01167","abstract":"The “Roaring 20s” of visual recognition began with the introduction of Vision Transformers (ViTs), which quickly superseded ConvNets as the state-of-the-art image classification model. A vanilla ViT, on the other hand, faces difficulties when applied to general computer vision tasks such as object detection and semantic segmentation. It is the hierarchical Transformers (e.g., Swin Transformers) that reintroduced several ConvNet priors, making Transformers practically viable as a generic vision backbone and demonstrating remarkable performance on a wide variety of vision tasks. However, the effectiveness of such hybrid approaches is still largely credited to the intrinsic superiority of Transformers, rather than the inherent inductive biases of convolutions. In this work, we reexamine the design spaces and test the limits of what a pure ConvNet can achieve. We gradually “modernize” a standard ResNet toward the design of a vision Transformer, and discover several key components that contribute to the performance difference along the way. The outcome of this exploration is a family of pure ConvNet models dubbed ConvNeXt. Constructed entirely from standard ConvNet modules, ConvNeXts compete favorably with Transformers in terms of accuracy and scalability, achieving 87.8% ImageNet top-1 accuracy and outperforming Swin Transformers on COCO detection and ADE20K segmentation, while maintaining the simplicity and efficiency of standard ConvNets.","tldr":"","source":"OpenAlex","sourceUrl":"https://openalex.org/W4312443924","citationCount":7261,"isOpenAccess":false,"pdfUrl":""},{"id":"oa-W3131500599","title":"Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutions","authors":"Wenhai Wang, Enze Xie, Xiang Li, Deng-Ping Fan, Kaitao Song, Ding Liang, Tong Lü, Ping Luo, Ling Shao","journal":"2021 IEEE/CVF International Conference on Computer Vision (ICCV)","pubDate":"2021-10-01","doi":"10.1109/iccv48922.2021.00061","abstract":"Although convolutional neural networks (CNNs) have achieved great success in computer vision, this work investigates a simpler, convolution-free backbone network use-fid for many dense prediction tasks. Unlike the recently-proposed Vision Transformer (ViT) that was designed for image classification specifically, we introduce the Pyramid Vision Transformer (PVT), which overcomes the difficulties of porting Transformer to various dense prediction tasks. PVT has several merits compared to current state of the arts. (1) Different from ViT that typically yields low-resolution outputs and incurs high computational and memory costs, PVT not only can be trained on dense partitions of an image to achieve high output resolution, which is important for dense prediction, but also uses a progressive shrinking pyramid to reduce the computations of large feature maps. (2) PVT inherits the advantages of both CNN and Transformer, making it a unified backbone for various vision tasks without convolutions, where it can be used as a direct replacement for CNN backbones. (3) We validate PVT through extensive experiments, showing that it boosts the performance of many downstream tasks, including object detection, instance and semantic segmentation. For example, with a comparable number of parameters, PVT+RetinaNet achieves 40.4 AP on the COCO dataset, surpassing ResNet50+RetinNet (36.3 AP) by 4.1 absolute AP (see Figure 2). We hope that PVT could, serre as an alternative and useful backbone for pixel-level predictions and facilitate future research.","tldr":"","source":"OpenAlex","sourceUrl":"https://openalex.org/W3131500599","citationCount":4737,"isOpenAccess":false,"pdfUrl":""},{"id":"oa-W4382203079","title":"Are Transformers Effective for Time Series Forecasting?","authors":"Ailing Zeng, Muxi Chen, Lei Zhang, Qiang Xu","journal":"Proceedings of the AAAI Conference on Artificial Intelligence","pubDate":"2023-06-26","doi":"10.1609/aaai.v37i9.26317","abstract":"Recently, there has been a surge of Transformer-based solutions for the long-term time series forecasting (LTSF) task. Despite the growing performance over the past few years, we question the validity of this line of research in this work. Specifically, Transformers is arguably the most successful solution to extract the semantic correlations among the elements in a long sequence. However, in time series modeling, we are to extract the temporal relations in an ordered set of continuous points. While employing positional encoding and using tokens to embed sub-series in Transformers facilitate preserving some ordering information, the nature of the permutation-invariant self-attention mechanism inevitably results in temporal information loss. To validate our claim, we introduce a set of embarrassingly simple one-layer linear models named LTSF-Linear for comparison. Experimental results on nine real-life datasets show that LTSF-Linear surprisingly outperforms existing sophisticated Transformer-based LTSF models in all cases, and often by a large margin. Moreover, we conduct comprehensive empirical studies to explore the impacts of various design elements of LTSF models on their temporal relation extraction capability. We hope this surprising finding opens up new research directions for the LTSF task. We also advocate revisiting the validity of Transformer-based solutions for other time series analysis tasks (e.g., anomaly detection) in the future.","tldr":"","source":"OpenAlex","sourceUrl":"https://openalex.org/W4382203079","citationCount":2720,"isOpenAccess":true,"pdfUrl":"https://ojs.aaai.org/index.php/AAAI/article/download/26317/26089"},{"id":"oa-W4388823657","title":"A Comprehensive Review of YOLO Architectures in Computer Vision: From YOLOv1 to YOLOv8 and YOLO-NAS","authors":"Juan Terven, Diana‐Margarita Córdova‐Esparza, Julio-Alejandro Romero-González","journal":"Machine Learning and Knowledge Extraction","pubDate":"2023-11-20","doi":"10.3390/make5040083","abstract":"YOLO has become a central real-time object detection system for robotics, driverless cars, and video monitoring applications. We present a comprehensive analysis of YOLO’s evolution, examining the innovations and contributions in each iteration from the original YOLO up to YOLOv8, YOLO-NAS, and YOLO with transformers. We start by describing the standard metrics and postprocessing; then, we discuss the major changes in network architecture and training tricks for each model. Finally, we summarize the essential lessons from YOLO’s development and provide a perspective on its future, highlighting potential research directions to enhance real-time object detection systems.","tldr":"","source":"OpenAlex","sourceUrl":"https://openalex.org/W4388823657","citationCount":2645,"isOpenAccess":true,"pdfUrl":"https://www.mdpi.com/2504-4990/5/4/83/pdf?version=1700497489"},{"id":"s2-67040b931c1a384426c44ae73f9553e97f08cf6a","title":"PVT v2: Improved baselines with Pyramid Vision Transformer","authors":"Wenhai Wang, Enze Xie, Xiang Li, Deng-Ping Fan, Kaitao Song, Ding Liang, Tong Lu, P. Luo, Ling Shao","journal":"Computational Visual Media","pubDate":"2021","doi":"10.1007/s41095-022-0274-8","abstract":"Transformers have recently lead to encouraging progress in computer vision. In this work, we present new baselines by improving the original Pyramid Vision Transformer (PVT v1) by adding three designs: (i) a linear complexity attention layer, (ii) an overlapping patch embedding, and (iii) a convolutional feed-forward network. With these modifications, PVT v2 reduces the computational complexity of PVT v1 to linearity and provides significant improvements on fundamental vision tasks such as classification, detection, and segmentation. In particular, PVT v2 achieves comparable or better performance than recent work such as the Swin transformer. We hope this work will facilitate state-of-the-art transformer research in computer vision. Code is available at https://github.com/whai362/PVT.","tldr":"This work improves the original Pyramid Vision Transformer (PVT v1) by adding three designs: a linear complexity attention layer, an overlapping patch embedding, and a convolutional feed-forward network to reduce the computational complexity of PVT v1 to linearity and provide significant improvements on fundamental vision tasks.","source":"Semantic Scholar","sourceUrl":"https://www.semanticscholar.org/paper/67040b931c1a384426c44ae73f9553e97f08cf6a","citationCount":2512,"isOpenAccess":true,"pdfUrl":"https://link.springer.com/content/pdf/10.1007/s41095-022-0274-8.pdf"},{"id":"oa-W3212386989","title":"Attention mechanisms in computer vision: A survey","authors":"Meng-Hao Guo, Tian-Xing Xu, Jiangjiang Liu, Zheng-Ning Liu, Peng-Tao Jiang, Tai‐Jiang Mu, Song–Hai Zhang, Ralph R. Martin, Ming‐Ming Cheng, Shi‐Min Hu","journal":"Computational Visual Media","pubDate":"2022-03-15","doi":"10.1007/s41095-022-0271-y","abstract":"Humans can naturally and effectively find salient regions in complex scenes. Motivated by this observation, attention mechanisms were introduced into computer vision with the aim of imitating this aspect of the human visual system. Such an attention mechanism can be regarded as a dynamic weight adjustment process based on features of the input image. Attention mechanisms have achieved great success in many visual tasks, including image classification, object detection, semantic segmentation, video understanding, image generation, 3D vision, multimodal tasks, and self-supervised learning. In this survey, we provide a comprehensive review of various attention mechanisms in computer vision and categorize them according to approach, such as channel attention, spatial attention, temporal attention, and branch attention; a related repository https://github.com/MenghaoGuo/Awesome-Vision-Attentions is dedicated to collecting related work. We also suggest future directions for attention mechanism research.","tldr":"","source":"OpenAlex","sourceUrl":"https://openalex.org/W3212386989","citationCount":2403,"isOpenAccess":true,"pdfUrl":"https://link.springer.com/content/pdf/10.1007/s41095-022-0271-y.pdf"},{"id":"oa-W2964979676","title":"Learning RoI Transformer for Oriented Object Detection in Aerial Images","authors":"Jian Ding, Nan Xue, Yang Long, Gui-Song Xia, Qikai Lu","journal":"","pubDate":"2019-06-01","doi":"10.1109/cvpr.2019.00296","abstract":"Object detection in aerial images is an active yet challenging task in computer vision because of the bird’s-eye view perspective, the highly complex backgrounds, and the variant appearances of objects. Especially when detecting densely packed objects in aerial images, methods relying on horizontal proposals for common object detection often introduce mismatches between the Region of Interests (RoIs) and objects. This leads to the common misalignment between the final object classification confidence and localization accuracy. In this paper, we propose a RoI Transformer to address these problems. The core idea of RoI Transformer is to apply spatial transformations on RoIs and learn the transformation parameters under the supervision of oriented bounding box (OBB) annotations. RoI Transformer is with lightweight and can be easily embedded into detectors for oriented object detection. Simply apply the RoI Transformer to light head RCNN has achieved state-of-the-art performances on two common and challenging aerial datasets, i.e., DOTA and HRSC2016, with a neglectable reduction to detection speed. Our RoI Transformer exceeds the deformable Position Sensitive RoI pooling when oriented bounding-box annotations are available. Extensive experiments have also validated the flexibility and effectiveness of our RoI Transformer.","tldr":"","source":"OpenAlex","sourceUrl":"https://openalex.org/W2964979676","citationCount":1513,"isOpenAccess":false,"pdfUrl":""},{"id":"s2-a09cbcaac305884f043810afc4fa4053099b5970","title":"Exploring Plain Vision Transformer Backbones for Object Detection","authors":"Yanghao Li, Hanzi Mao, Ross B. Girshick, Kaiming He","journal":"ArXiv","pubDate":"2022","doi":"10.48550/arXiv.2203.16527","abstract":"We explore the plain, non-hierarchical Vision Transformer (ViT) as a backbone network for object detection. This design enables the original ViT architecture to be fine-tuned for object detection without needing to redesign a hierarchical backbone for pre-training. With minimal adaptations for fine-tuning, our plain-backbone detector can achieve competitive results. Surprisingly, we observe: (i) it is sufficient to build a simple feature pyramid from a single-scale feature map (without the common FPN design) and (ii) it is sufficient to use window attention (without shifting) aided with very few cross-window propagation blocks. With plain ViT backbones pre-trained as Masked Autoencoders (MAE), our detector, named ViTDet, can compete with the previous leading methods that were all based on hierarchical backbones, reaching up to 61.3 AP_box on the COCO dataset using only ImageNet-1K pre-training. We hope our study will draw attention to research on plain-backbone detectors. Code for ViTDet is available in Detectron2.","tldr":"This design enables the original ViT architecture to be fine-tuned for object detection without needing to redesign a hierarchical backbone for pre-training, and can compete with the previous leading methods that were all based on hierarchical backbones.","source":"Semantic Scholar","sourceUrl":"https://www.semanticscholar.org/paper/a09cbcaac305884f043810afc4fa4053099b5970","citationCount":1190,"isOpenAccess":true,"pdfUrl":"https://arxiv.org/pdf/2203.16527"},{"id":"oa-W4283450732","title":"UNetFormer: A UNet-like transformer for efficient semantic segmentation of remote sensing urban scene imagery","authors":"Libo Wang, Rui Li, Ce Zhang, Shenghui Fang, Chenxi Duan, Xiaoliang Meng, Peter M. Atkinson","journal":"ISPRS Journal of Photogrammetry and Remote Sensing","pubDate":"2022-06-24","doi":"10.1016/j.isprsjprs.2022.06.008","abstract":"","tldr":"","source":"OpenAlex","sourceUrl":"https://openalex.org/W4283450732","citationCount":1172,"isOpenAccess":true,"pdfUrl":"https://arxiv.org/pdf/2109.08937"},{"id":"oa-W4362603432","title":"Transformers in medical imaging: A survey","authors":"Fahad Shamshad, Salman Khan, Syed Waqas Zamir, Muhammad Haris Khan, Munawar Hayat, Fahad Shahbaz Khan, Huazhu Fu","journal":"Medical Image Analysis","pubDate":"2023-04-05","doi":"10.1016/j.media.2023.102802","abstract":"","tldr":"","source":"OpenAlex","sourceUrl":"https://openalex.org/W4362603432","citationCount":1163,"isOpenAccess":false,"pdfUrl":""},{"id":"oa-W4386075524","title":"BiFormer: Vision Transformer with Bi-Level Routing Attention","authors":"Lei Zhu, Xinjiang Wang, Zhanghan Ke, Wayne Zhang, Rynson W. H. Lau","journal":"","pubDate":"2023-06-01","doi":"10.1109/cvpr52729.2023.00995","abstract":"As the core building block of vision transformers, attention is a powerful tool to capture long-range dependency. However, such power comes at a cost: it incurs a huge computation burden and heavy memory footprint as pairwise token interaction across all spatial locations is computed. A series of works attempt to alleviate this problem by introducing handcrafted and content-agnostic sparsity into attention, such as restricting the attention operation to be inside local windows, axial stripes, or dilated windows. In contrast to these approaches, we propose a novel dynamic sparse attention via bi-level routing to enable a more flexible allocation of computations with content awareness. Specifically, for a query, irrelevant key-value pairs are first filtered out at a coarse region level, and then fine-grained token-to-token attention is applied in the union of remaining candidate regions (i.e., routed regions). We provide a simple yet effective implementation of the proposed bilevel routing attention, which utilizes the sparsity to save both computation and memory while involving only GPU-friendly dense matrix multiplications. Built with the proposed bi-level routing attention, a new general vision transformer, named BiFormer, is then presented. As BiFormer attends to a small subset of relevant tokens in a query adaptive manner without distraction from other irrelevant ones, it enjoys both good performance and high computational efficiency, especially in dense prediction tasks. Empirical results across several computer vision tasks such as image classification, object detection, and semantic segmentation verify the effectiveness of our design. Code is available at https://github.com/rayleizhu/BiFormer.","tldr":"","source":"OpenAlex","sourceUrl":"https://openalex.org/W4386075524","citationCount":1060,"isOpenAccess":false,"pdfUrl":""},{"id":"oa-W4385763767","title":"Transformers in Time Series: A Survey","authors":"Qingsong Wen, Tian Zhou, Chaoli Zhang, Weiqi Chen, Ziqing Ma, Junchi Yan, Liang Sun","journal":"","pubDate":"2023-08-01","doi":"10.24963/ijcai.2023/759","abstract":"Transformers have achieved superior performances in many tasks in natural language processing and computer vision, which also triggered great interest in the time series community. Among multiple advantages of Transformers, the ability to capture long-range dependencies and interactions is especially attractive for time series modeling, leading to exciting progress in various time series applications. In this paper, we systematically review Transformer schemes for time series modeling by highlighting their strengths as well as limitations. In particular, we examine the development of time series Transformers in two perspectives. From the perspective of network structure, we summarize the adaptations and modifications that have been made to Transformers in order to accommodate the challenges in time series analysis. From the perspective of applications, we categorize time series Transformers based on common tasks including forecasting, anomaly detection, and classification. Empirically, we perform robust analysis, model size analysis, and seasonal-trend decomposition analysis to study how Transformers perform in time series. Finally, we discuss and suggest future directions to provide useful research guidance.","tldr":"","source":"OpenAlex","sourceUrl":"https://openalex.org/W4385763767","citationCount":1023,"isOpenAccess":true,"pdfUrl":"https://www.ijcai.org/proceedings/2023/0759.pdf"},{"id":"oa-W4312847199","title":"MaxViT: Multi-axis Vision Transformer","authors":"Zhengzhong Tu, Hossein Talebi, Han Zhang, Feng Yang, Peyman Milanfar, Alan C. Bovik, Yinxiao Li","journal":"Lecture notes in computer science","pubDate":"2022-01-01","doi":"10.1007/978-3-031-20053-3_27","abstract":"","tldr":"","source":"OpenAlex","sourceUrl":"https://openalex.org/W4312847199","citationCount":821,"isOpenAccess":false,"pdfUrl":""},{"id":"oa-W3202406646","title":"Swin Transformer: Hierarchical Vision Transformer using Shifted Windows","authors":"Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, Baining Guo","journal":"arXiv (Cornell University)","pubDate":"2021-03-25","doi":"10.48550/arxiv.2103.14030","abstract":"This paper presents a new vision Transformer, called Swin Transformer, that capably serves as a general-purpose backbone for computer vision. Challenges in adapting Transformer from language to vision arise from differences between the two domains, such as large variations in the scale of visual entities and the high resolution of pixels in images compared to words in text. To address these differences, we propose a hierarchical Transformer whose representation is computed with \\textbf{S}hifted \\textbf{win}dows. The shifted windowing scheme brings greater efficiency by limiting self-attention computation to non-overlapping local windows while also allowing for cross-window connection. This hierarchical architecture has the flexibility to model at various scales and has linear computational complexity with respect to image size. These qualities of Swin Transformer make it compatible with a broad range of vision tasks, including image classification (87.3 top-1 accuracy on ImageNet-1K) and dense prediction tasks such as object detection (58.7 box AP and 51.1 mask AP on COCO test-dev) and semantic segmentation (53.5 mIoU on ADE20K val). Its performance surpasses the previous state-of-the-art by a large margin of +2.7 box AP and +2.6 mask AP on COCO, and +3.2 mIoU on ADE20K, demonstrating the potential of Transformer-based models as vision backbones. The hierarchical design and the shifted window approach also prove beneficial for all-MLP architectures. The code and models are publicly available at~\\url{https://github.com/microsoft/Swin-Transformer}.","tldr":"","source":"OpenAlex","sourceUrl":"https://openalex.org/W3202406646","citationCount":400,"isOpenAccess":true,"pdfUrl":"https://arxiv.org/pdf/2103.14030"},{"id":"s2-4702a22a3c2da1284a88d5e608d38cd106d66736","title":"A Simple Yet Effective Network Based on Vision Transformer for Camouflaged Object and Salient Object Detection","authors":"C. Hao, Zitong Yu, Xin Liu, Jun Xu, Huanjing Yue, Jingyu Yang","journal":"IEEE Transactions on Image Processing","pubDate":"2024","doi":"10.1109/TIP.2025.3528347","abstract":"Camouflaged object detection (COD) and salient object detection (SOD) are two distinct yet closely-related computer vision tasks widely studied during the past decades. Though sharing the same purpose of segmenting an image into binary foreground and background regions, their distinction lies in the fact that COD focuses on concealed objects hidden in the image, while SOD concentrates on the most prominent objects in the image. Building universal segmentation models is currently a hot topic in the community. Previous works achieved good performance on certain task by stacking various hand-designed modules and multi-scale features. However, these careful task-specific designs also make them lose their potential as general-purpose architectures. Therefore, we hope to build general architectures that can be applied to both tasks. In this work, we propose a simple yet effective network (SENet) based on vision Transformer (ViT), by employing a simple design of an asymmetric ViT-based encoder-decoder structure, we yield competitive results on both tasks, exhibiting greater versatility than meticulously crafted ones. To enhance the performance of universal architectures on both tasks, we propose some general methods targeting some common difficulties of the two tasks. First, we use image reconstruction as an auxiliary task during training to increase the difficulty of training, forcing the network to have a better perception of the image as a whole to help with segmentation tasks. In addition, we propose a local information capture module (LICM) to make up for the limitations of the patch-level attention mechanism in pixel-level COD and SOD tasks and a dynamic weighted loss (DW loss) to solve the problem that small target samples are more difficult to locate and segment in both tasks. Finally, we also conduct a preliminary exploration of joint training, trying to use one model to complete two tasks simultaneously. Extensive experiments on multiple benchmark datasets demonstrate the effectiveness of our method. The code is available at https://github.com/linuxsino/SENet.","tldr":"A simple yet effective network (SENet) based on vision Transformer, employing a simple design of an asymmetric ViT-based encoder-decoder structure that yield competitive results on both tasks, exhibiting greater versatility than meticulously crafted ones is proposed.","source":"Semantic Scholar","sourceUrl":"https://www.semanticscholar.org/paper/4702a22a3c2da1284a88d5e608d38cd106d66736","citationCount":50,"isOpenAccess":false,"pdfUrl":""},{"id":"s2-62ea0abd2a4c8b734e564542fe35ebb971cdae20","title":"Vision Transformer Based Effective Model for Early Detection and Classification of Lung Cancer","authors":"Arvind Kumar, Ravishankar Mehta, B. Reddy, K. Singh","journal":"SN Computer Science","pubDate":"2024","doi":"10.1007/s42979-024-03120-9","abstract":"","tldr":"The present work encompasses the Vision Transformer (ViT) based model to classify lung cancer based on histopathological images, which has been shown to have a promising impact in deciphering complex spatial relationships within image data.","source":"Semantic Scholar","sourceUrl":"https://www.semanticscholar.org/paper/62ea0abd2a4c8b734e564542fe35ebb971cdae20","citationCount":38,"isOpenAccess":false,"pdfUrl":""},{"id":"s2-7ca23adea429d2fbd65a916e526d52b40ab8a085","title":"Ensemble Architecture of Vision Transformer and CNNs for Breast Cancer Tumor Detection From Mammograms","authors":"S. Khan, Sohaib Asif, Omair Bilal","journal":"International Journal of Imaging Systems and Technology","pubDate":"2025","doi":"10.1002/ima.70090","abstract":"Addressing the complexities of classifying distinct object classes in computer vision presents several challenges, including effectively capturing features such as color, form, and tissue size for each class, correlating class vulnerabilities, singly capturing features, and predicting class labels accurately. To tackle these issues, we introduce a novel hybrid deep dense learning technique that combines deep transfer learning with a transformer architecture. Our approach utilizes ResNet50, EfficientNetB1, and our proposed ProDense block as the backbone models. By integrating the Vit‐L16 transformer, we can focus on relevant features in mammography and extract high‐value pair features, offering two alternative methods for feature extraction. This allows our model to adaptively shift the region of interest towards the class type in slides. The transformer architecture, particularly Vit‐L16, enhances feature extraction by efficiently capturing long‐range dependencies in the data, enabling the model to better understand the context and relationships between features. This aids in more accurate classification, especially when fine‐tuning pretrained models, as it helps the model adapt to specific characteristics of the target dataset while retaining valuable information learned from the pretraining phase. Furthermore, we employ a stack ensemble technique to leverage both the deep transfer learning model and the ProDense block extension for training extensive features for breast cancer classification. The fine‐tuning process employed by our hybrid model helps refine the dense layers, enhancing classification accuracy. Evaluating our method on the INbreast dataset, we observe a significant improvement in predicting the binary cancer category, outperforming the current state‐of‐the‐art classifier by 98.08% in terms of accuracy.","tldr":"A novel hybrid deep dense learning technique that combines deep transfer learning with a transformer architecture, and employs a stack ensemble technique to leverage both the deep transfer learning model and the ProDense block extension for training extensive features for breast cancer classification.","source":"Semantic Scholar","sourceUrl":"https://www.semanticscholar.org/paper/7ca23adea429d2fbd65a916e526d52b40ab8a085","citationCount":37,"isOpenAccess":false,"pdfUrl":""},{"id":"s2-f574d841ae3d6fea7964215325921b69866d75ac","title":"Object Detection Based on CNN and Vision-Transformer: A Survey","authors":"Jinfeng Cao, Bo Peng, Mingzhong Gao, Haichun Hao, Xinfang Li, Hongwei Mou","journal":"IET Comput. Vis.","pubDate":"2025","doi":"10.1049/cvi2.70028","abstract":"Object detection is the most crucial and challenging task of computer vision and has been used in various fields in recent years, such as autonomous driving and industrial inspection. Traditional object detection methods are mainly based on the sliding windows and the handcrafted features, which have problems such as insufficient understanding of image features and low accuracy of detection. With the rapid advancements in deep learning, convolutional neural networks (CNNs) and vision transformers have become fundamental components in object detection models. These components are capable of learning more advanced and deeper image properties, leading to a transformational breakthrough in the performance of object detection. In this review, we comprehensively review the representative object detection models from deep learning periods, tracing their architectural shifts and technological breakthroughs. Furthermore, we discuss key challenges and promising research directions in the object detection. This review aims to provide a comprehensive foundation for practitioners to enhance their understanding of object detection technologies.","tldr":"This review comprehensively review the representative object detection models from deep learning periods, tracing their architectural shifts and technological breakthroughs to provide a comprehensive foundation for practitioners to enhance their understanding of object detection technologies.","source":"Semantic Scholar","sourceUrl":"https://www.semanticscholar.org/paper/f574d841ae3d6fea7964215325921b69866d75ac","citationCount":25,"isOpenAccess":true,"pdfUrl":"https://onlinelibrary.wiley.com/doi/pdfdirect/10.1049/cvi2.70028"},{"id":"s2-1e95bb5827dc784547a46058793c15effd74dccc","title":"Liveness Detection in Computer Vision: Transformer-Based Self-Supervised Learning for Face Anti-Spoofing","authors":"Arman Keresh, Pakizar Shamoi","journal":"IEEE Access","pubDate":"2024","doi":"10.1109/ACCESS.2024.3513795","abstract":"Face recognition systems are increasingly used in biometric security for convenience and effectiveness. However, they remain vulnerable to spoofing attacks, where attackers use photos, videos, or masks to impersonate legitimate users. This research addresses these vulnerabilities by exploring the Vision Transformer (ViT) architecture, fine-tuned with the DINO framework utilizing CelebA-Spoof, CASIA SURF, and a proprietary dataset. The DINO framework facilitates self-supervised learning, enabling the model to learn distinguishing features from unlabeled data. We compared the performance of the proposed fine-tuned ViT model using the DINO framework against traditional models, including CNN Model EfficientNet b2, EfficientNet b2 (Noisy Student), and Mobile ViT on the face anti-spoofing task. Numerous tests on standard datasets show that the ViT model performs better than other models in terms of accuracy and resistance to different spoofing methods. Our model’s superior performance, particularly in APCER (1.6%), the most critical metric in this domain, underscores its improved ability to detect spoofing relative to other models. Additionally, we collected our own dataset from a biometric application to validate our findings further. This study highlights the superior performance of transformer-based architecture in identifying complex spoofing cues, leading to significant advancements in biometric security.","tldr":"This study highlights the superior performance of transformer-based architecture in identifying complex spoofing cues, leading to significant advancements in biometric security.","source":"Semantic Scholar","sourceUrl":"https://www.semanticscholar.org/paper/1e95bb5827dc784547a46058793c15effd74dccc","citationCount":19,"isOpenAccess":true,"pdfUrl":"https://doi.org/10.1109/access.2024.3513795"},{"id":"s2-71ee04d684aba3d6ad426c553aa26c428d2ec578","title":"Deepfake face detection via multi-level discrete wavelet transform and vision transformer","authors":"Main Uddin, Zhangjie Fu, Xiang Zhang","journal":"The Visual Computer","pubDate":"2025","doi":"10.1007/s00371-024-03791-8","abstract":"","tldr":"A novel face tampering detection framework that combines multi-level discrete wavelet transform (DWT) and vision transformer and reveals discriminative details and frequency information, paving the way for more robust facial manipulation detection systems.","source":"Semantic Scholar","sourceUrl":"https://www.semanticscholar.org/paper/71ee04d684aba3d6ad426c553aa26c428d2ec578","citationCount":19,"isOpenAccess":false,"pdfUrl":""},{"id":"s2-5b94b3b121105d9f2ba8aa029a58ff95cc2a1341","title":"Vision Transformer-Based Anomaly Detection in Smart Grid Phasor Measurement Units Using Deep Learning Models","authors":"Zhibin Liu, Yibo Wang, Qingwei Wang, Man Hu","journal":"IEEE Access","pubDate":"2025","doi":"10.1109/ACCESS.2025.3549679","abstract":"Modern smart grids become increasingly complex and thus require advanced anomaly detection mechanisms to guarantee system stability and reliability. New real-time measurement devices (Phasor Measurement Units (PMUs) provide high-resolution, time-synchronized voltage, frequency, and phase angle measurements. However, this is insufficient in complex and subtle anomalies; traditional threshold-based and statistical approaches fail to detect such anomalies and instead require deep learning techniques for better effectiveness. On the other hand, Convolutional Neural Networks (CNNs) and Vision Transformers (ViTs) have proven to be deep learning models that can be used for PMU anomaly detection. Through hierarchical convolutional filters and self-attention mechanisms, CNNs exploit localized feature extraction counterparts to ViTs, which employ self-attention mechanisms to capture long-range dependencies of PMU data and a global contextual relationship. Although ViTs have shown capabilities for computer vision tasks, their efficacy in PMU-based anomaly detection remains under-researched, which motivates this comparative study. Based on such capability, we hypothesize that CNNs will be outperformed by ViTs in their capability to differentiate similar anomalies, especially since they can better model global spatial-temporal dependencies over CNNs’ localized feature extraction. Specifically, in this study, four CNN architectures (DenseNet201, InceptionV3, ResNet152, MobileNetV2) and four ViT models (ViT-Base-16, ViT-Base-32, ViT-Large-16, ViT-Large-32) are evaluated on a PMU anomaly dataset. Accuracy, precision, recall, F1-score, and computational efficiency are evaluated to assess these models as to whether they can perform real-time smart grid monitoring. Compared to CNN models, ViT-Large-32 achieved the highest accuracy of 98.91%, far exceeding them. ViT were more precise and recalled generation loss and motor switching anomaly numbers, while CNNs were faster to infer but more inaccurate in classification. The results confirm that ViTs achieve state-of-the-art performance in detecting PMU anomalies and are a promising grid monitoring and predictive maintenance solution. Real-time deployment of such systems remains a challenge that should be explored using computational optimizations, and hybrid CNN-ViT models for balancing sufficient performance with acceptable efficiency should be studied in large-scale innovative grid applications.","tldr":"CNNs will be outperformed by ViTs in their capability to differentiate similar anomalies, especially since they can better model global spatial-temporal dependencies over CNNs’ localized feature extraction.","source":"Semantic Scholar","sourceUrl":"https://www.semanticscholar.org/paper/5b94b3b121105d9f2ba8aa029a58ff95cc2a1341","citationCount":18,"isOpenAccess":true,"pdfUrl":"https://doi.org/10.1109/access.2025.3549679"},{"id":"s2-d01e791b4942aae325849666a9e02ecb4ad5a3c9","title":"WeedSwin hierarchical vision transformer with SAM-2 for multi-stage weed detection and classification","authors":"Taminul Islam, Toqi Tahamid Sarker, Khaled R. Ahmed, Cristiana Bernardi Rankrape, Karla Gage","journal":"Scientific Reports","pubDate":"2025","doi":"10.1038/s41598-025-05092-z","abstract":"Weed detection and classification using computer vision and deep learning techniques have emerged as crucial tools for precision agriculture, offering automated solutions for sustainable farming practices. This study presents a comprehensive approach to weed identification across multiple growth stages, addressing the challenges of detecting and classifying diverse weed species throughout their developmental cycles. We introduce two extensive datasets: the Alpha Weed Dataset (AWD) with 203,567 images and the Beta Weed Dataset (BWD) with 120,341 images, collectively documenting 16 prevalent weed species across 11 growth stages. The datasets were preprocessed using both traditional computer vision techniques and the advanced SAM-2 model, ensuring high-quality annotations with segmentation masks and precise bounding boxes. Our research evaluates several state-of-the-art object detection architectures, including DINO Transformer (with ResNet-101 and Swin backbones), Detection Transformer (DETR), EfficientNet B4, YOLO v8, and RetinaNet. Additionally, we propose a novel WeedSwin Transformer architecture specifically designed to address the unique challenges of weed detection, such as complex morphological variations and overlapping vegetation patterns. Through rigorous experimentation, WeedSwin demonstrated superior performance, achieving 0.993 ± 0.004 mAP and 0.985 mAR while maintaining practical processing speeds of 218.27 FPS, outperforming existing architectures across various metrics. The comprehensive evaluation across different growth stages reveals the robustness of our approach, particularly in detecting challenging “driver weeds” that significantly impact agricultural productivity. By providing accurate, automated weed identification capabilities, this research establishes a foundation for more efficient and environmentally sustainable weed management practices. The demonstrated success of the WeedSwin architecture, combined with our extensive temporal datasets, represents a significant advancement in agricultural computer vision, supporting the evolution of precision farming techniques while promoting reduced herbicide usage and improved crop management efficiency.","tldr":"The demonstrated success of the WeedSwin architecture, combined with the extensive temporal datasets, represents a significant advancement in agricultural computer vision, supporting the evolution of precision farming techniques while promoting reduced herbicide usage and improved crop management efficiency.","source":"Semantic Scholar","sourceUrl":"https://www.semanticscholar.org/paper/d01e791b4942aae325849666a9e02ecb4ad5a3c9","citationCount":14,"isOpenAccess":false,"pdfUrl":""},{"id":"s2-09a101e9a16d22cfa574bac42fb8400449294f1a","title":"MoSViT: a lightweight vision transformer framework for efficient disease detection via precision attention mechanism","authors":"Yuanqi Chen, Aiping Wang, Ziyang Liu, Jie Yue, Enxu Zhang, Fei Li, Ning Zhang","journal":"Frontiers in Artificial Intelligence","pubDate":"2025","doi":"10.3389/frai.2025.1498025","abstract":"Maize, a globally essential staple crop, suffers significant yield losses due to diseases. Traditional diagnostic methods are often inefficient and subjective, posing challenges for timely and accurate pest management. This study introduces MoSViT, an innovative classification model leveraging advanced machine learning and computer vision technologies. Built on the MobileViT V2 framework, MoSViT integrates the CLA focus mechanism, DRB module, MoSViT Block, and the LeakyRelu6 activation function to enhance feature extraction accuracy while reducing computational complexity. Trained on a dataset of 3,850 images encompassing Blight, Common Rust, Gray Leaf Spot, and Healthy conditions, MoSViT achieves exceptional performance, with classification accuracy, Precision, Recall, and F1 Score of 98.75%, 98.73%, 98.72%, and 98.72%, respectively. These results surpass leading models such as Swin Transformer V2, DenseNet121, and EfficientNet V2 in both accuracy and parameter efficiency. Additionally, the model's interpretability is enhanced through heatmap analysis, providing insights into its decision-making process. Testing on small sample datasets further demonstrates MoSViT's generalization capability and potential for small-sample detection scenarios.","tldr":"MoSViT, an innovative classification model leveraging advanced machine learning and computer vision technologies, achieves exceptional performance, surpassing leading models such as Swin Transformer V2, DenseNet121, and EfficientNet V2 in both accuracy and parameter efficiency.","source":"Semantic Scholar","sourceUrl":"https://www.semanticscholar.org/paper/09a101e9a16d22cfa574bac42fb8400449294f1a","citationCount":9,"isOpenAccess":true,"pdfUrl":"https://doi.org/10.3389/frai.2025.1498025"},{"id":"s2-6f63b4616cb3bcd56c659058a11c5aa4e7a959dd","title":"Object Detection using Vision Transformer and Deep Learning for Computer Vision Applications","authors":"S. S, Mohana, Ambika G, K. A, Nataraj K, Sudhangowda B S","journal":"2025 7th International Conference on Intelligent Sustainable Systems (ICISS)","pubDate":"2025","doi":"10.1109/ICISS63372.2025.11076364","abstract":"Vision Transformer (ViT) is an image recognition model that uses transformer architecture, which has a numerous advantage over Convolution Neural Networks (CNN). It offers improved accuracy, scalability, flexibility, global context, and transferability. ViT can handle images of different sizes and aspect ratios, making it more versatile than CNN. It can process an entire image at once, allowing it to capture global context information and long-range dependencies. Additionally, ViTs pre-training on huge amounts of image data can be transferred to other image recognition tasks, making it a useful tool for transfer learning. This paper describes the differences between ViT and CNN and how ViT splits images into patches for classification. The positional encoding of different features is done in ViT to avoid the requirement of filters. Proposed implementation obtained final accuracy of prediction 93% for top-1 accuracy.","tldr":"","source":"Semantic Scholar","sourceUrl":"https://www.semanticscholar.org/paper/6f63b4616cb3bcd56c659058a11c5aa4e7a959dd","citationCount":6,"isOpenAccess":false,"pdfUrl":""},{"id":"s2-e7a1db99bbf6dfdebaa5e947a20853eda7459f60","title":"Lychee Leaf Disease Detection by Vision Transformer and Computer Vision","authors":"Nayem Uddin Prince, Md. Abdullah Al Mamun, Md. Tanvir Miah Shagar, Md. Rezaul Karim Emon, Md. Sahadat Hossen Sajib","journal":"2024 IEEE International Conference on Blockchain and Distributed Systems Security (ICBDS)","pubDate":"2024","doi":"10.1109/ICBDS61829.2024.10837439","abstract":"Northern Bangladesh is home to the most lychee growing, which has a major economic impact. Lychee output and quality are reduced by various leaf and fruit diseases. Deep learning is used to construct a disease detection system for lychee diseases to detect them early and accurately. We correctly identified 6,000 photos of healthy and unhealthy lychee foliage and fruits. Six deep learning algorithms—VGG16, CustomCNN, MobileNet, InceptionV3, ResNet50, and Vision Transformer— classified the photos. Normalization and augmentation improved model resilience during data preparation. F1-score, recall, accuracy, and precision were utilized to train and evaluate the models. Vision Transformer (ViT) scores 99.91% in accuracy, recall, and F1-score, outperforming the other models. This achievement shows that ViT can better recognize complex lychee disease traits. The ViT algorithm for disease detection can help Bangladeshi farmers identify illnesses quickly and accurately. Reduced lychee loss and improved fruit quality and attractiveness should boost lychee sales in domestic and international markets.","tldr":"","source":"Semantic Scholar","sourceUrl":"https://www.semanticscholar.org/paper/e7a1db99bbf6dfdebaa5e947a20853eda7459f60","citationCount":6,"isOpenAccess":false,"pdfUrl":""},{"id":"s2-a82611b2dfefd577d19e96b2537513c8076cd787","title":"FFRViT: Frequency Feature Refinement Vision Transformer for Remote Sensing Object Detection","authors":"Lihua Fu, Wenwen Liu, Guofang Li, Weijin Huang","journal":"IEEE Transactions on Geoscience and Remote Sensing","pubDate":"2025","doi":"10.1109/TGRS.2025.3602860","abstract":"Rotated object detection in remote sensing images is a research hot spot in the field of computer vision. The following problems exist in the detection of rotating object in remote sensing images: insufficient feature extraction ability, complex scenes lead to interference and occlusion between objects, and object characteristics are sensitive to positioning. To solve the above problems, we propose the frequency feature refinement vision transformer (FFRViT). First, the initial feature extraction is performed on the input image. Then, the frequency division feature refinement of local, global, and contextual features is performed in four stages. This process not only emphasizes the key positional information of the image but also models global relationships and extracts surrounding environmental information as a supplement, thereby enhancing the model’s ability to capture comprehensive features. Finally, the detection head is used for object classification and localization. A sample collaborative optimization loss is designed to adaptively adjust the model’s emphasis on positive and negative samples, so as to further improve the detection accuracy of the model. Experiments on the HRSC2016 and DOTA-v1.0 public datasets show that the proposed method achieves better performance in rotated object detection in remote sensing images, and its detection accuracy is better than some mainstream methods.","tldr":"","source":"Semantic Scholar","sourceUrl":"https://www.semanticscholar.org/paper/a82611b2dfefd577d19e96b2537513c8076cd787","citationCount":4,"isOpenAccess":false,"pdfUrl":""}],"links":{"web":"https://science-database.com/technology/computer-vision","llms_txt":"https://science-database.com/technology/computer-vision/llms.txt","api":"https://science-database.com/api/v1/technology/computer-vision"}}