Peer Revenue Play credits. Real papers.

Papers

Fresh from arXiv, typeset properly. Highlighted words trade on the market.

Computer Science Computer Vision

cs.CV Watch Computer Vision

Monday 17 August

Computer Vision

Nexus: Structured Synergy for Efficient Text-to-Image Generation using Rectified Flow Model

Yizhao Wang

Diffusion and flow matching models have made significant progress in text-to-image generation, yet high computation, quadratic complexity, and large memory footprint hinder high-resolution synthesis and edge deployment. We propose Nexus, which integrates sparse architecture, linear complexity, and low-bit quantization. It combines MoE feed-forward layers, gated DeltaNet attention, and per-expert low-bit training to reduce computation and memory. Their joint optimization allows Nexus to achieve generation quality comparable to mainstream models such as SDXL and SD3 while delivering markedly higher inference efficiency. Experiments on COCO and LAION validate its effectiveness.
Read the full abstract

Computer Vision

Beyond Similarity Matching: Structured Reasoning for Open-Vocabulary Referring Segmentation in 3DGS

Yizhao Wang, Xinfa Wang, Jingbo Wang et al. (+5 more)

Open-vocabulary referring segmentation in 3D Gaussian Splatting (3DGS) requires a neural model to select Gaussian primitives according to free-form language expressions. Existing 3DGS-based methods usually rely on global text-region similarity, which is weak for queries involving attributes, reference objects, spatial relations, and fine-grained parts. This often causes target-reference confusion, granularity mismatch, part-whole leakage, and relation violations. We propose QAGaussian, a query-adaptive neural reasoning framework for language-guided Gaussian primitive selection. QAGaussian first learns query-conditioned multi-scale Gaussian slots as differentiable candidates whose receptive fields are shaped by the input expression. It then builds a relation-aware slot graph with language-conditioned edge weighting to propagate target-reference, attribute, part-whole, and contextual evidence. A granularity-adaptive router softly combines region-level, object-level, part-level, attribute-aware, and relation-aware mask branches, followed by relation-constrained refinement for spatial, part-whole, attribute, and geometric consistency. QAGaussian is pretrained only on Mosaic3D-5.6M for Gaussian-text alignment and evaluated on independent benchmarks without target-dataset fine-tuning. It achieves 47.2 Avg. mIoU and 63.2 Avg. F1, outperforming the strongest 3DGS referring baseline by 2.7 mIoU points and 2.9 F1 points. It also improves Part-mIoU from 38.6 to 43.4, Rel-mIoU from 44.4 to 50.8, and reduces target-reference confusion from 10.8 to 7.4. These results demonstrate that query-conditioned slot learning, relation-aware graph reasoning, and adaptive routing provide an effective neural modeling strategy for open-vocabulary referring segmentation in 3DGS. The code is available at https://github.com/zqeslwyz/QAGaussian.
Read the full abstract

All authors Yizhao Wang, Xinfa Wang, Jingbo Wang, Jingbo Wang, Guantao Zhang, Yafeng Han, Guohong Gao, Yuhe Xia

Computer Vision

TISC: A Text-Driven Image Semantic Communication System for Faithful Reconstruction

Feifan Zhang, Yuyang Du, Xiaoyan Liu et al. (+1 more)

Generative image semantic communication converts an image into a text description and then performs text-to-image reconstruction at the receiver via diffusion-based generative models. This paradigm has attracted broad attention due to its extremely low bandwidth cost. However, existing methods still face two critical bottlenecks across image-to-text (I2T) semantic extraction at the transmitter and text-to-image (T2I) semantic reconstruction at the receiver: (i) semantic loss and distortion in I2T, where holistic image descriptions may omit fine-grained object attributes and spatial-position information, causing the generated text to deviate from the original image semantics; and (ii) insufficient semantic faithfulness in T2I, where even with the same semantically faithful text description, different initial noise settings may lead diffusion-based reconstruction to produce images with different levels of semantic consistency with the original image. These issues jointly limit the semantic faithfulness of image reconstruction. To address them, we propose TISC, a text-driven image semantic communication framework tailored for faithful reconstruction. TISC incorporates two key designs: (1) Tree-Structured Attribute Semantic Extraction (TSASE), which decomposes semantic extraction into global scene, background, and object-level attribute descriptions, covering spatial position, shape/pose, color, material, and other physical attributes for each detected object; and (2) an Initial Noise Optimization (INO) mechanism, which selects an initial noise seed at the transmitter according to a comprehensive similarity score that jointly considers visual and semantic consistency. Experiments on multiple datasets show that TSASE improves object-position recovery and semantic description faithfulness, while the INO parameter study supports the adopted configuration for noise selection.
Read the full abstract

All authors Feifan Zhang, Yuyang Du, Xiaoyan Liu, Soung Chang Liew

Computer Vision

Representation Is Not Enough: Body-Localized Thermal Evidence for Contactless Stress and Craving Sensing in Opioid Use Disorder

Sachin Deb, Harshit Sharma and Asif Salekin

Removing wearables from physiological monitoring also removes their supervision: the signal indicating where and when a stress response occurred. Contactless stress sensing therefore becomes a weakly supervised evidence-localization problem, where a clip-level label must be traced to the body regions and moments that produced it. We address this with FABLE-Therm, a weakly supervised architecture that preserves localized evidence across body regions, time, and encoder-specific representations until the final decision. FABLE-Therm fuses frozen foundation-model encoders at the embedding level, with theory explaining why localized fusion can outperform feature concatenation and prediction averaging. We study this problem in opioid use disorder (OUD), where stress is a major relapse trigger and sustained wearable use can be difficult during early recovery. Using fixed thermal video, FABLE-Therm achieves 0.938 AUROC on held-out participants, and its learned representation transfers to self-reported craving, providing, to our knowledge, the first evidence that craving can be recovered from contactless thermal video. Localized evidence also enables participant-level analysis of deployment failure. We find that improving representation alone is insufficient for equitable deployment: additional data from the underserved group would recover only about half of the cohort gap, while the remainder reflects person-to-person heterogeneity. This modality-agnostic decomposition applies to models with identifiable subpopulations. Together with the first cohort-structured contactless thermal OUD benchmark, our results show that preserving localized evidence supports both accurate sensing and principled analysis of who a model fails and why.
Read the full abstract

Computer Vision

SafeGesture: Evaluating Fine-Grained Hand Gesture Understanding in Vision-Language Models through Scenario-Conditioned Safety Interpretation

Taegang Kim, Saleh Afroogh and Junfeng Jiao

Open-weight and frontier vision-language models (VLMs) perform well on general image understanding, but their ability to interpret fine-grained hand gestures in safety-critical operational contexts remains largely unexamined. We introduce SafeGesture, a benchmark that evaluates whether a model can infer scenario-appropriate safety actions from hand gestures. It pairs six HaGRID gestures with eight operational scenarios for 4,800 items and evaluates Qwen2.5-VL-7B, LLaVA-NeXT-7B, InternVL2-8B, Phi-3.5-Vision, and GPT-4o. Results reveal a perception-reasoning decoupling: GPT-4o achieves 98.4% gesture accuracy but 53.3% safety accuracy, while Qwen2.5-VL reaches 84.9% and 39.5%, yielding gaps of 45.0 and 45.4 percentage points. Four of five models rarely or never use the uncertainty label, and failure directions differ substantially across models. Accuracy also obscures label bias: a scenario-majority policy with no visual input reaches 58.3%, above every evaluated model, while only GPT-4o exceeds this prior under macro-F1. Visual input improves safety accuracy by 11.2 to 30.2 percentage points, but providing the ground-truth gesture as text improves performance by only 0.4 to 3.2 points, and no model exceeds 56.2%. These results indicate that the main bottleneck is scenario-conditioned safety reasoning rather than gesture recognition.
Read the full abstract

Computer Vision · Cross-listed from cs.RO

US-VLA: An Ultrasound Vision-Language-Action Model for Embodied Abdomina

Cheng Zhang, Xingzheng Wu, Guihao Yan et al. (+4 more)

Artificial intelligence-assisted ultrasound scanning enhances diagnostic reliability and efficiency by providing real-time guidance for standardized image acquisition and reducing operator dependence. However, existing reinforcement learning and learning-assisted ultrasound scanning methods typically rely on carefully designed reward functions or extensive interaction data, which limits their generalization ability and stability across different devices, patient populations, and complex clinical scenarios. To address these challenges, we propose an ultrasound vision-language-action model (US-VLA) for automated ultrasound scanning that explicitly encodes clinical semantic goals and generates sequential probe manipulation actions under real-time ultrasound feedback. In particular, we first design an ultrasound-aware expert fusion module to jointly integrate ultrasound observations with auxiliary contextual information, enabling semantic ultrasound feedback to effectively guide the scanning process. Then, we construct US-VLA-Data, a real-world dataset covering liver and kidney examinations, which includes five clinically defined standard planes and comprises 320 expert scanning trajectories with approximately 80,000 synchronized timesteps. Extensive experiments demonstrate that US-VLA achieves competitive performance in ultrasound probe manipulation tasks, indicating its effectiveness and promising generalization within the evaluated abdominal ultrasound setting. The source code is available at https://github.com/VMVLab/US-VLA.
Read the full abstract

All authors Cheng Zhang, Xingzheng Wu, Guihao Yan, Xifeng Hu, Zhi Liu, Mei Wu, Qing Cai

Computer Vision

TR-GS: High-Fidelity Sparse-View CT Volumetric Rendering via t-Distribution Gaussian Splatting and Ray-Confidence Modeling

Zedong Xiao, Yiren Wang, Zhou Liu et al. (+2 more)

High-fidelity 3D medical visualization supports applications such as clinical assessment and surgical planning. Sparse-view computed tomography (CT) can reduce projection requirements and associated radiation exposure, but limited observations may introduce structural artifacts and reconstruction uncertainty. Although 3D Gaussian Splatting (3DGS) provides an efficient explicit representation for volumetric rendering, existing CT methods based on standard Gaussian primitives may be sensitive to unreliable observations under sparse-view acquisition. We present TR-GS, a Gaussian-splatting framework for sparse view CT volumetric rendering. TR-GS replaces standard Gaussian primitives with projectable Student's t-distribution primitives and introduces a ray-confidence model that regulates their degrees of freedom according to local ray observability. Confidence-guided 3D wavelet regularization is further used to balance high-frequency detail preservation and noise suppression. This work is licensed under a Creative Commons Attribution 4.0 International License. Experiments on synthetic and real-world datasets show that TR-GS improves over representative baselines in most evaluated settings and remains competitive in the remaining cases. The resulting volumetric representations may support downstream medical multimedia applications, including XR-based visualization and interactive clinical rendering.
Read the full abstract

All authors Zedong Xiao, Yiren Wang, Zhou Liu, Xiaolin Liu, Zhangji Lu

Computer Vision · Cross-listed from eess.IV

Decoupling Parcellation from Classification: Systematic Benchmark of Fast Brain Segmentation Methods for Alzheimer's Disease Detection

Jiadao Zou, Hongyu Guo and Wei Xi

Brain parcellation and classification are typically evaluated in isolation, yet downstream AD detection performance depends on their interaction. We decouple these components and systematically benchmark fast deep learning parcellation methods (SynthSeg+, OpenMAP-T1) against the FreeSurfer (FS-HV) clinical baseline through down- stream AD classification on OASIS-1. Our factorial design evaluates three parcellation methods, two volumetry strategies (hard vs. soft), and four classifier paradigms (clinical thresholds, supervised feedforward networks, ensemble methods, and foundation models with zero/few-shot prompting), with all results quantified using BCa Bootstrap 95% confidence intervals.
Read the full abstract

Computer Vision · Cross-listed from cs.LG

AdROD: HyperNetwork-based Adversarially Robust Object Detection for Autonomous Driving

Yuting Wu, Dongfang Guo, Xiangzhong Luo et al. (+2 more)

Camera-based object detectors are vulnerable to physical adversarial attacks designed to suppress detections. While adversarial training and input purification offer some protection, they often overfit to specific attack distributions and fail on adaptive adversaries. This paper presents AdROD, an embedded, stochastic ensemble defense software designed for autonomous driving. AdROD employs low-rank HyperNetworks, which require only 1.6% of the parameter footprint of standard HyperNetworks, to generate diverse detectors at a per-frame rate, making it impractical for attackers to obtain the deployed detectors in time. To further improve adversarial robustness, AdROD incorporates a novel functional diversity mechanism, which couples stochastic weight updates with unique input-space transformations. We design two serving modes of AdROD that strike different trade-offs between robustness and runtime overhead: AdROD-I, a continuous protection mode for maximum resilience that leverages inter-detector disagreement to recover compromised detections, and AdROD-II, an on-demand mode triggered by kinematic discontinuities in object tracking. Through comprehensive evaluation with synthetic benchmarks, physically deployed adversarial patches, and end-to-end safety tests in the OpenCDA co-simulator, AdROD outperforms five baseline defenses and exhibits superior generalizability compared with the evaluated adversarial-training baselines, while maintaining real-time performance for safely stopping the vehicle at a stop sign instrumented with adversarial patches.
Read the full abstract

All authors Yuting Wu, Dongfang Guo, Xiangzhong Luo, Qun Song, Rui Tan

Computer Vision

Multi-scale Decomposed Convolution Refinement Network for Visible-Infrared Person Re-Identification

Mingsheng Zheng, Zirui Jiang, Bo Liu et al. (+3 more)

Visible-infrared person re-identification (VI-ReID) suffers from cross-modal discrepancies and limited discriminative capabilities, leading to suboptimal recognition performance. Current approaches exhibit limitations in semantic mining, cross-modal fusion and feature constraints. To tackle these challenges, we propose MDCRNet, a Multi-scale Decomposed Convolution Refinement Network that enhances cross-modal feature learning and discriminative metric learning. Specifically, we introduce a Hierarchical Learning Module (HLM) containing four Hierarchical Decomposed Convolution Attention (HDCA) modules, each equipped with lightweight channel attention and multi-scale spatial perception blocks to capture multi-scale spatial dependencies. Moreover, we develop a Joint Discriminative Metric Loss (JDML) incorporating a novel Granularity Discriminative Loss (GDL) that simultaneously optimizes intra-identity compactness and inter-identity separability across modalities. Extensive experiments on SYSU-MM01 and RegDB datasets demonstrate that MDCRNet achieves state-of-the-art performance on both benchmarks. Code is available at https://github.com/Kevin-zms/MDCRNet.
Read the full abstract

All authors Mingsheng Zheng, Zirui Jiang, Bo Liu, Yupeng Chen, Jun Zhang, Kai Zhao

Computer Vision

Depth-guided Multi-view Exposure Bracketing for HDR Robot Vision

Jinnyeong Kim, Juhyung Choi, Woohyeok Kim et al. (+2 more)

Achieving reliable single-shot high dynamic range (HDR) imaging under extreme illumination conditions remains a long-standing challenge, yet no comprehensive benchmark exist for evaluating HDR perception in multi-sensor robotic systems. To fill this gap, we introduce a large-scale dataset collected via a custom robotic vision platform and an iPhone 13 Pro: 121 real-world scenes spanning modest and ultra-high dynamic range conditions, alongside 20 synthetic video sequences from the CARLA simulator. As a reference pipeline for this dataset, we propose Depth-guided Multi-view Exposure Bracketing (DMEB), a single-shot HDR method that distributes drastically different exposures across multi-view low-bit-depth cameras and fuses them via depth-guided confidence-aware fusion. Evaluations on our dataset show that DMEB establishes a strong reference point and highlight the promise of this sensor configuration for robust HDR perception in diverse multi-camera and depth sensor system.
Read the full abstract

All authors Jinnyeong Kim, Juhyung Choi, Woohyeok Kim, Sunghyun Cho, Seung-Hwan Baek

Computer Vision · Cross-listed from cs.CL

ReRef-3D: A Benchmark for Spatial Referring Expression-Guided 3D Object Placement

Mary Lynn Martin, Yifei Zhang, Martha Palmer et al. (+1 more)

We introduce ReRef-3D, a benchmark for language-guided single-object placement in controlled 3D tabletop scenes. It contains 33,826 instructions across 998 CLEVR-derived scenes, spanning 16 placement types and direct, one-hop, and two-hop references. Given that an instruction defines a region of acceptable placements rather than one coordinate, predictions are evaluated by relation satisfaction and physical validity. Each instruction also includes a verified natural rephrasing. We evaluate LLaVA-3D, 3D-LLM, 3D-LLaVA, LL3DA, and PlaceIt3D, with PlaceIt3D evaluated under both point-based and region-based supervision. Region-supervised PlaceIt3D achieves the highest placement accuracy at about 80% and LLaVA-3D reaches about 70%. Performance varies by relation type, but nearest and aligned remain the most difficult. Rephrasing has little effect on performance overall.
Read the full abstract

All authors Mary Lynn Martin, Yifei Zhang, Martha Palmer, Maria Leonor Pacheco

Computer Vision · Cross-listed from cs.LG

Breaking the Compression Barrier: Cross-Architecture Compression Boundary Learning via Reverse Regrowth

Zhaocen Liu, Satvik Praveen and Yi Sheng

Model compression is critical for deploying networks on resource-constrained edge devices. While pruning-based methods can significantly reduce model size, they often suffer from abrupt performance collapse beyond a sparsity thresh-old, making it difficult to identify the feasible compression limit of the model. To address this challenge, we propose a boundary-Learning reverse regrowth framework, BRIDGE, that reformulates compression as a constructive boundary-search problem. Unlike forward pruning, our method first drives the model to an extremely sparse state to expose the collapse region, and then selectively regenerates the critical structure to restore performance. The proposed framework employs a hierarchical regeneration strategy, including coarse-grained layer selection and fine-grained regeneration parameter selection, to accurately identify which parameters require recovery. Experiments show that our method can recover models from the brink of collapse on both CNNs and Transformer architectures, demonstrating its architecture in-dependence. BRIDGE achieves a performance improvement of up to 1.49% in unstructured pruning and up to 4.77% in structured pruning. These results demonstrate that reverse regeneration can effectively extend the compression limit while maintaining stable performance. The source code is available at https://github.com/EnumaCaliber/BRIDGE.
Read the full abstract

Computer Vision

Spatial Temporal Synergy: Balancing Change and Invariance in Text Driven 3D Human Motion Editing

Shaohui Lin, Zhenwu Shi, Jingyu Gong et al. (+5 more)

Text-driven human motion editing aims to modify existing motion sequences according to natural language instructions while maintaining the structural consistency of the original motion. Existing diffusion-based approaches struggle to balance text-responsive "change" and inertial "invariance". They often rely on coarse spatial constraints and rigid uniform time assumptions, leading to spatial motion distortions and the destruction of intrinsic physical rhythms during variable-length editing. To handle these challenges, we propose Change and Invariance Motion Editing (CIME), a unified framework that comprehensively decouples change and invariance into spatial pose and temporal rhythm dimensions. For spatial poses, our method integrates an omni-supervised positive-negative learning mechanism comprising hierarchical retrospective feature supervision, subtle motion preservation, and triplet-based semantic alignment. For temporal rhythms, we introduce the Riemannian Non-uniform Integral Manifold Mapping (RNIMM) module, which achieves high-fidelity reproduction of physical beats in the edited text via kinematics-aware non-uniform timestamps. Extensive experiments on the MotionFix and STANCE Adjustment datasets demonstrate that CIME achieves state-of-the-art performance in editing alignment and structural fidelity, validating the effectiveness of our unified architecture. Our source codes and models have been released at: github.com/ZhenwuShi/CIME.git
Read the full abstract

All authors Shaohui Lin, Zhenwu Shi, Jingyu Gong, Jiao Xie, Yu Zhou, Baochang Zhang, Lizhuang Ma, Chia-Wen Lin

Computer Vision

A Plug-and-Play 2D Motion Interface for Real-World Motion Language Models

Kaname Yokoyama and Norimichi Ukita

Motion Language Models (MoLMs) typically understand human motions by tokenizing 3D motion and processing the resulting tokens using a language model. However, obtaining accurate 3D motions from monocular videos is challenging, limiting their real-world applicability. To address this issue, we introduce a plug-and-play 2D Motion Interface that enables 3D-pretrained MoLMs to accept 2D motion inputs without modifying or fine-tuning the original models. Experiments on public datasets show that our method achieves performance comparable to 3D motion inputs across multiple MoLMs and outperforms training MoLMs from scratch on 2D motions. We further construct a monocular real-world video motion evaluation dataset and introduce a real-video adapter, demonstrating the usefulness of 2D motions over 3D motions under the evaluated monocular pose-estimation setting. These results suggest that 2D motion provides a practical interface for deploying MoLMs in real-world motion understanding settings. Code is available at https://github.com/irajisamurai/2D-Motion-Interface.
Read the full abstract

Sunday 16 August

Computer Vision · Cross-listed from cs.LG

Correcting Learning-based Perception for Safety

Yan Miao, Hussein Darir and Sayan Mitra

Learning-enabled perception is important in many autonomous systems. Unlike traditional sensors, the boundary where ML perception does or does not work is poorly characterized. Incorrect perception can lead to unsafe or overtly conservative downstream control actions. In this paper, we propose a two-step strategy for correcting ML-based state estimation. First, an offline computation is used to characterize the uncertainties resulting from the ML module's state estimation, using preimages of perception contracts. Second, at runtime, a risk heuristic is used to choose particular states from the uncertain estimates to drive the control decisions. We perform extensive simulation-based evaluation of this runtime perception correction strategy on different vision-based adaptive cruise controllers (ACC modules), in different weather conditions, and road scenarios. Out of 45 ACC scenarios where the original perception-based control system using Yolo and LaneNet led to safety violations, in 73% of the scenarios, our runtime perception correction preserved safety; our method wouldn't be able to recover 27% of the scenarios where the construction of the preimages of perception contracts is not fully conformant. Further, our runtime perception correction strategy is not overly conservative—on the average only a 2.8% increase in completion time is experienced in the corrected scenarios, with mild interventions.
Read the full abstract

Computer Vision · Cross-listed from cs.LG

PRQuant: Permutation Residual Quantization for Low-Overhead Inference

Peiran Wang, Anqi Wang, Jiaying Zhao et al. (+10 more)

Low-bit quantization of linear layers is often dominated by a small number of outlier channels. Existing smoothing, rotation, and residual-based methods can mitigate this issue, but may shift the quantization bottleneck to weights or introduce costly online operations. To address these limitations, we propose PRQuant (Permutation Residual Quantization), a training-free framework that combines channel permutation with offline weight residual compensation. PRQuant identifies the scaled-weight columns with the largest quantization errors and permutes them into contiguous tail blocks. This structure allows the corresponding weight residuals to be precomputed entirely offline, while replacing scattered activation gathering with simple contiguous access during inference, yielding a single regular MXFP4 GEMM for compensated computation. Experiments show that PRQuant substantially reduces down-projection reconstruction error, with scaling and residual compensation providing the main numerical gains while permutation enables a hardware-friendly contiguous layout. Comprehensive experiment results on Qwen3-4B-Instruct-2507 and Qwen3-30B-A3B-Instruct-2507 illustrate that PRQuant achieves up to averagelly 2.6x and 1.8x operator speedup over BF16 respectively, while preserving near plain MXFP4 end-to-end decoding efficiency. Across five downstream benchmarks, PRQuant achieves the best average accuracy among the quantized methods, improving accuracy over MXFP4 by 1.24 and 0.55, respectively.
Read the full abstract

All authors Peiran Wang, Anqi Wang, Jiaying Zhao, Huiwen Yang, Zhenyu Ming, Yuantian Shao, Rongqian Wang, Yiwu Yao, Kun Tian, Xin Yao, Gong Zhang, Fan Yang, Zhongyi Huang

Computer Vision

Few-Shot Cross-Dataset Adaptation for Tuberculosis Detection Using DenseNet

Bidhan Biswas, Shahadat Hossain Sohag, Nabil Ashab et al. (+2 more)

Tuberculosis (TB) is one of the most common and dangerous bacterial ailments. Every year, it causes a large number of deaths worldwide. Although many deep learning models can detect tuberculosis from chest X-rays quite accurately, severe domain shift across datasets makes the task challenging. Different imaging protocols, patient demographics, and equipment across domains make the task of generalization difficult. In real-world settings, a model may perform well on one dataset but show a noticeable drop in performance when tested on another. In this work, we address this domain adaptation challenge through a few-shot scaling study. A controlled cross-dataset evaluation is presented in this paper using TBX11K as the source domain and the Mendeley TB dataset as the target domain. It is investigated how varying the number of target samples affects model performance under three training regimes: frozen backbone adaptation, full fine-tuning of a source-pretrained DenseNet121 model, and training from scratch. The results indicate that the model can perform well even with limited data and can achieve 98.36% accuracy with just 75 labeled samples per class. The adaptation curves demonstrate how fine-tuning effectively mitigates domain shift. These findings establish full fine-tuning of pretrained models as a highly effective and practical strategy for mitigating domain shift in low-resource clinical deployment scenarios.
Read the full abstract

All authors Bidhan Biswas, Shahadat Hossain Sohag, Nabil Ashab, Soumit Kumar Kundu, Saif Mahmud Parvez

Computer Vision

AI Visual Inspection for Garment Production

Ray Wai Man Kong, Ding Ning and Theodore Ho Tin Kong

The garment manufacturing industry is under increasing pressure to improve product quality, reduce costs, and accelerate digital transformation toward Industry 4.0. One of the most challenging quality-control activities is sewing-line inspection, where defects such as broken stitches and skipped stitches are difficult to detect consistently through manual inspection. Human-based inspection is often affected by fatigue, subjective judgement, and inconsistent performance, resulting in defect leakage, rework, and reduced production efficiency. This study presents the development and validation of an Artificial Intelligence (AI)-based visual inspection system for garment sewing-line quality control. The system utilizes Convolutional Neural Networks (CNNs) to detect sewing defects and was initially trained using black fabric and black sewing thread samples. Experimental testing was conducted on black, red, dark green, light blue, silver, and fluorescent yellow fabrics. The results demonstrated successful detection of jump sewing-line defects on black, red, and dark green materials, while performance limitations were observed for broken sewing-line defects and fabrics with significantly different visual characteristics, including light blue, silver, and fluorescent yellow colours. These findings indicate that model accuracy is strongly influenced by the diversity of training data and the ability to generalize across different fabric and thread colours.
Read the full abstract

Computer Vision

Aligning Human Sense: Calibrated Distributional Reward Learning for Video Generation

Nai-Xin Zhai, Weihua Cheng, Dexu Yu et al. (+9 more)

Video generation is central to AI-powered content creation. Aligning generated videos with human preferences is a key criterion for evaluating generation quality. Despite significant progress in visual quality, three key challenges remain. First, the reliability of reward signals is constrained by the quality of human preference data, which is often affected by subjective noise and bias. Second, standard scalar reward models collapse multi-aspect human preferences into a single value, leading to the loss of dynamic trade-offs across multiple preference dimensions. Third, in policy optimization, the widely adopted KL divergence imposes primarily local constraints and may fail to capture the global structure of human preferences. To address these challenges, we propose a unified preference-aware learning framework for video generation. First, we introduce elite-guided filtering to calibrate preference data and construct reliable supervision for reward model training. We then model video quality as a multidimensional reward distribution to capture the uncertainty inherent in human preferences, and use the Wasserstein distance to align the learned reward distribution with the empirical human preference distribution. Finally, we introduce Wasserstein-based distributional alignment into GRPO, guiding policy optimization to better match the global structure of human preferences over videos. Experiments on reward modeling and video generation demonstrate that our approach improves the reliability of reward signals and the perceptual consistency of generated videos. Our code is available at https://github.com/alignhs26/ahs.
Read the full abstract

All authors Nai-Xin Zhai, Weihua Cheng, Dexu Yu, Yikai Gu, Hanwen Du, Junchen Fu, Chenxi Huang, Yingwei Song, Liyuan Lillian Ma, Yang Ran, Youhua Li, Yongxin Ni

Computer Vision

EditStream: A Unified Autoregressive Framework for Interactive Video Generation and Editing

Yuqian Zhou, Zhenghong Zhou, Zongze Wu et al. (+5 more)

Interactive video generation and editing are becoming increasingly important for creative design. In this report, we introduce EditStream: a unified framework for interactive video generation and editing. EditStream unifies multiple video creation and manipulation tasks within a single DiT-based model through flexible task-specific conditioning, and further transforms it into a fast, few-step autoregressive model for efficient streaming. It supports Text-to-Video, Image-to-Video, Video-to-Video, Editing Propagation, Reference-guided Video Editing, and Camera Pose Change, enabling flexible control over video generation, transformation, and editing within one system. To make the unified model practical for interactive use, we develop a two-stage distillation approach that combines Velocity Moment Matching (VMM) with autoregressive unrolling. VMM matches conditional velocity moments at student-reached intermediate states to preserve generation quality and motion, while unrolling exposes the student to its own autoregressive predictions to improve temporal stability. Together, they alleviate common challenges in few-step autoregressive video generation, including over-saturation, degraded motion, temporal instability, and complex training. EditStream provides a practical and scalable solution that bridges high-quality diffusion-based video models with interactive creative workflows.
Read the full abstract

All authors Yuqian Zhou, Zhenghong Zhou, Zongze Wu, Cameron Smith, Richard Zhang, Jiebo Luo, Eli Shechtman, Zhe Lin

Computer Vision · Cross-listed from eess.IV

TractoGraphVLM: A Unified Vision-Language Framework for White Matter Tractography

Gurucharan Marthi Krishna Kumar, Janine Dale Mendola and Amir Shmuel

Vision language models have transformed 2D medical imaging, yet extending them to 3D white matter tractography remains challenging due to the complex topology of fiber bundles. We introduce TractoGraphVLM, a unified framework for four tasks, bundle classification, text-to-tract retrieval, anatomical captioning, and visual question answering, built on a shared GPS architecture, training procedure, and read-out design. Fiber bundles are represented as streamline graphs whose nodes encode 3D position and tangent orientation. A General, Powerful, Scalable (GPS) graph transformer produces bundle embeddings aligned with a frozen BiomedBERT text encoder via contrastive learning, while a BioGPT decoder with visual prefix tokens generates captions and answers. A single shared encoder and decoder is trained jointly across all four tasks and evaluated from one checkpoint. Trained on HCP Young Adult subjects, TractoGraphVLM achieves 91.8% bundle classification accuracy, 84.7% retrieval R@1, BLEU-4=20.1, ROUGE-L=66.8, and 66.4% VQA accuracy on a held-out test set. The same checkpoints transfer zero-shot to HCP Aging subjects, with a modest drop on discriminative tasks and a larger drop on generative tasks, showing robustness to age and acquisition shift. Language supervision yields richer representations than label-only training, recovering structure like hemisphere and fiber family, carried by captions but never given as a label. Swapping only the visual encoder, graphs preserving fiber orientation outperform volumetric baselines, with GPS giving the best balance. Generative metrics measure consistency with a structured knowledge base rather than independent clinical text; even so, TractoGraphVLM shows that classifying, retrieving, describing, and answering questions about a white matter bundle can be served by one jointly trained model that learns transferable neuroanatomy from language alone.
Read the full abstract

Computer Vision · Cross-listed from eess.IV

MagViT: Interpretable Multi-Magnification Transformers with Patient-Level Model Selection for Breast Histopathology

Nabil Ashab, Soumit Kumar Kundu, Saif Mahmud Parvez et al. (+3 more)

Breast cancer is one of the most common types of cancer among women around the world. Rapid detection and early treatment can hinder its progress to more complex stages and can impede its spread to other parts of the body. Histopathological image classification is the most common task in cancer detection due to its robustness in analyzing cellular data. Breast histopathology classification requires handling both multi-scale tissue morphology and clinically relevant generalization beyond the source domain. This paper presents MagViT, an interpretable multi-magnification transformer framework with scale-gated fusion and patient-level model selection. The model uses four BreakHis magnifications (40X, 100X, 200X, 400X) and extracts per-scale representations with a ViT backbone, and combines them via a learnable gate that masks missing scales. Patient-level five-fold cross-validation with a fixed seed has been run and compared with three architectural branches. The most accurate branch is then selected as the final model due to the strongest patient-level accuracy while retaining the simplest fusion pathway. On BreakHis, our architecture achieves a mean image accuracy of 0.9191, a mean patient accuracy of 0.9643, and a mean macro-F1 of 0.9042. External transfer experiments provide preliminary evidence of cross-dataset generalization under controlled adaptation settings on BUSI (image accuracy 0.8306, macro-F1 0.7480, patient accuracy 0.8291) and IDC (image accuracy 0.8577, macro-F1 0.8191, patient accuracy 0.8372). Grad-CAM visualization indicates that the model focuses on diagnostically significant and meaningful regions across magnifications. Relative to prior ViT-centered BreakHis work, this study emphasizes patient-level selection and cross-dataset robustness under a reproducible protocol.
Read the full abstract

All authors Nabil Ashab, Soumit Kumar Kundu, Saif Mahmud Parvez, Shahadat Hossain Sohag, Bidhan Biswas, Nazmus Subha

Computer Vision · Cross-listed from eess.IV

ORViT-DR: Ordinally-Robust Hybrid ViT for Low-Resolution Diabetic Retinopathy Grading

Soumit Kumar Kundu, Nabil Ashab, Bidhan Biswas et al. (+4 more)

Diabetic retinopathy (DR) is one of the main causes of impaired vision. A good and reliable automated grading system can make the screening process safer and more accurate. Because DR stages progress gradually, the task of grading disease severity naturally follows an ordinal structure in which neighboring classes share similar visual characteristics. In this study, ORViT-DR, a hybrid deep learning framework, is designed to improve DR grading from low-resolution retinal images. The proposed approach combines convolutional feature extraction with transformer-based global context modeling through a pre-trained ViT-Hybrid backbone, which integrates BiT-ResNetv2 with a Vision Transformer architecture. The approach is tested on the RetinaMNIST subset of the MedMNISTv2 dataset, which contains 28x28 retinal fundus images annotated with five levels of disease severity. To promote stable training and better feature learning, the training strategy applies progressive layer unfreezing, layer-wise learning rate decay, exponential moving average (EMA) parameter updates, and ensemble-based prediction during inference. Experimental results on the official RetinaMNIST test set show that the proposed method achieves 57.00% classification accuracy, along with a quadratic weighted kappa score of 0.5963 and a macro-F1 score of 0.4293. These results suggest that hybrid CNN-Transformer architectures can provide effective representations for ordinal retinal image analysis.
Read the full abstract

All authors Soumit Kumar Kundu, Nabil Ashab, Bidhan Biswas, Shahadat Hossain Sohag, Saif Mahmud Parvez, Souvik Kumar Kundu, Zunayed Ahmed Rafi

Computer Vision

CM-MAE: A Physics-Guided Cross-Modal Self-Supervised Learning Framework for Vision-Wireless Applications

Yubo Zhang and Yiyao Liu

Synchronized camera and wireless measurements observe the same scene through different physical channels. The central difficulty is that a representation learned in one deployment can fail when viewpoint, traffic, illumination, and propagation geometry change. This paper presents CM-MAE, a self-supervised vision–wireless pretraining framework for cross-scenario representation transfer. The evaluated real-data model uses only RGB frames and the measured 64-beam received-power vector available in DeepSense 6G; it does not use ray-traced paths, calibrated depth, or beam-index labels during pretraining. Its central pretraining term is a soft contrastive alignment loss. Instead of making the synchronized image–wireless pair the only positive pair, this loss builds a target distribution from similarities between measured beam-power profiles, so nonidentical samples with similar directional responses are not forced apart as false negatives. A masked joint decoder provides the complementary local objective by reconstructing hidden visual patches and wireless angular clusters under modality dropout. After pretraining, a differential-rate fine-tuning rule lets a new fusion head adapt quickly while the encoders move slowly. Under a sequence-disjoint DeepSense 6G protocol, adding the soft alignment loss improves a matched linear-probe transfer average from 24.88% to 29.49%. Mild fusion fine-tuning reaches 77.38% Top-1 accuracy on unseen Scenarios 6–8, and optional transductive normalization adaptation reaches 78.69%. Since the fusion setting uses the contemporaneous 64-beam power vector at inference, these results should be read as representation-transfer diagnostics, not as proactive beam-prediction or reduced-sweeping claims.
Read the full abstract

Computer Vision · Cross-listed from cs.LG

The Limits of Binding in Dual Encoders

Kin Ian Lo

Dual-encoder models such as CLIP score an image-caption pair by a single inner product of two independently computed unit vectors, and fail at binding, often scoring near chance when asked to distinguish "a red car and a blue dog" from "a blue car and a red dog". We give a mathematical account of when this failure is necessary and when it is contingent. Working within the ideal-encoder framework proposed by Kang et al., we first show the relevant axioms are satisfiable, so every impossibility must enter through an added, checkable hypothesis. We then prove three such obstructions. Depth: for recursive role-binding codes the swap margin obeys an exact law $m(D) = 2b^{-D}$ in the nesting depth D, with a finite-dimension version holding up to one explicitly flagged concentration estimate; the resolvable depth grows only logarithmically in the dimension and is single-digit at CLIP scale, the nesting depth of ordinary language. Objective: architecture-free throttle theorems showing that the contrastive objective's entire reward for binding is bounded by the rate at which training contrasts a caption against its own swap, a rate that vanishes at web scale, and that exactly reversed binding costs only that rate times the mean binding margin; both are verified in simulation. Geometry: a tight smoothness-binding frontier: the closer the two swap-related captions must embed to a shared paraphrase anchor, the smaller the binding margin can be, with an exact constant. Measuring its text-only diagnostic across 18 deployed text encoders, every model sits at roughly 25-35% of its ceiling, and the induced per-item ceiling tracks SugarCrepe's subset difficulty at r = 0.99. Binding failure in deployed dual encoders is thus not a dimension or smoothness limit today, but an incentive and code-structure limit, with a proved depth ceiling that remains once those are fixed.
Read the full abstract

Computer Vision

BagShift: Measuring How Patch Selection Changes the Evidence Seen by Whole-Slide MIL

Ruicheng Yuan, Zhenxuan Zhang, Liwei Hu et al. (+4 more)

Whole-slide multiple-instance learning (MIL) observes only the patches admitted by its selector. Deployment can alter this selector through compute limits, tissue masking, or regional workflows, even when the patch count is unchanged. We introduce BagShift, a paired protocol that changes the selector for the same case while holding its features and predictor fixed, thereby isolating selector response from case mix. With equal 128-patch budgets, sampling across the tissue or concentrating around one coordinate exposes markedly different evidence: on PANDA, the two views reduce quadratic weighted kappa by 1.57 and 17.96 points, respectively (QWK reported on the $\times100$ scale). On CAMELYON16, lesion annotations withheld from model development show that localized views retain tumor in only 10.0% of micrometastatic observations, and matched exposure does not consistently recover the loss. The same fixed-count stressor produces a much smaller response on external lung subtyping, although differences in relative coverage make cross-task severity descriptive. When repeated localized observations are available, unioning their patches before one nonlinear MIL pass improves PANDA QWK by 7.87 points over averaging regional predictions. Patch count specifies computation, not observed evidence; deployment evaluations should report both what a selector preserves and how repeated observations are aggregated.
Read the full abstract

All authors Ruicheng Yuan, Zhenxuan Zhang, Liwei Hu, Anbang Wang, Haijie Xu, Jiawei Luo, Guang Yang

Computer Vision · Cross-listed from cs.CL

SEER: Long-Context Reasoning via Selective Visual-Text Compression

Jiawei Xu, Zhilin Zhai, Jinrui Fang et al. (+6 more)

Long-context reasoning remains computationally expensive for large language models due to the quadratic complexity of attention over text tokens. Visual-text compression offers a promising alternative by rendering text into images and processing them with vision-language models, often reducing token usage. However, existing approaches apply uniform compression regardless of query relevance, potentially sacrificing precision where detailed extraction is required. We present SEER, a framework that learns to select query-relevant images through visual scanning and retrieve textual content only where needed, combining the efficiency of visual compression with the precision of text-based reasoning. Through supervised fine-tuning on tool-interaction trajectories, SEER learns adaptive tool invocation for selection and retrieval. Experiments on long-context benchmarks show that SEER improves extraction precision through selective text retrieval while retaining average prompt-token savings relative to full-text baselines. On LongBench, SEER achieves 51.11% average accuracy, outperforming the visual-text baseline Glyph-9B by 2.33 points and Qwen3-8B by 3.49 points. Code can be accessed at https://github.com/jiaweixu98/SEER
Read the full abstract

All authors Jiawei Xu, Zhilin Zhai, Jinrui Fang, Ruohan Xu, Mingfei Lu, Yi Zhang, Guanchu Wang, Tianlong Chen, Ying Ding

Computer Vision · Cross-listed from cs.GR

Differentiable Voxelization of Surface Representations

Tobias Djuren, Ugo Finnendahl, Markus Worchel et al. (+2 more)

Different shape representations facilitate different computations. Surface representations, in particular meshes, are often used for modeling, whereas volume representations are useful for spatial queries such as intersection or containment. Optimizing a surface representation based on a volumetric properties by gradient descent requires the derivatives of the volume relative to its bounding surface. We derive this gradient for winding numbers and show that it can be efficiently computed for volumetric values sampled on a regular grid (voxel representation) and surface parameters based on vertex sets (triangle meshes). This enables an efficient solution for a variety of optimization problems. We demonstrate the practical use of this approach at the examples of deforming meshes to resolve intersections, being manufacturable by cutting with a bandsaw from three directions, and creating shapes that are close to tiling 3D space.
Read the full abstract

All authors Tobias Djuren, Ugo Finnendahl, Markus Worchel, Hendrik Meyer, Marc Alexa

Computer Vision · Cross-listed from cs.GR

As-Rigid-As-Possible Regularization for Implicit Surfaces

Tobias Djuren, Markus Worchel, Ugo Finnendahl et al. (+1 more)

Implicit surface representations have regained popularity because of their use in machine learning. A common component in optimization is regularization, penalizing the deviation of the surface from its original shape. The popular as-rigid-aspossible (ARAP) energy strikes a good compromise between realistic deformation behavior and efficient computation, at least for piecewise linear meshes. We develop an approach for computing the ARAP energy of a deformation function based on point sampling of the surface. The implicit representation is exploited to provide differentials in each sample. The evaluation is efficient and exact in each sample (up to numerical precision). We demonstrate the general applicability of the method to neural shape processing in several applications and contrast its properties with alternatives from the literature.
Read the full abstract

All authors Tobias Djuren, Markus Worchel, Ugo Finnendahl, Marc Alexa