Computer Vision
Controllable Histopathology Image Synthesis with Training-free Structural Initialization and Textural Modulation
Yuheng Qiu, Jingyi Luo, Chenfei Ye et al. (+2 more)
Deep learning has demonstrated remarkable success in high-throughput histopathology image analysis. However, the performance of learning-based models critically depends on the quality and size of annotations by expert pathologists, which is a resource-intensive and time-consuming process. To address the limitations of data scarcity and annotation burden, several methods have been proposed to synthesize paired histopathology data. Nevertheless, these frameworks typically still require annotation data, albeit in reduced quantities, to impose structural constraints during training. In this work, we present CHIS, a plug-in framework that guides the sampling trajectory of a pretrained
diffusion model through two key stages: structural initialization at the start and textural modulation during generation. The initial noise state is refined by fusing the phase information from a prior mask with the amplitude of Gaussian noise in the frequency domain, yielding a structurally informed starting point. During the reverse
diffusion process, we adaptively modulate both coarse-grained and fine-grained textures at different wavelet decomposition levels. This enables a
diffusion model pretrained solely on unlabeled images to generate outputs that align with prior structural masks while preserving the reference tissue style. We conducted extensive experiments demonstrating the superiority of CHIS in generation fidelity and its substantial benefits for downstream
segmentation tasks. Code is available at https://github.com/IBIL-Code/CHIS.
Read the full abstract ▾
All authors Yuheng Qiu, Jingyi Luo, Chenfei Ye, Ting Ma, Jianfeng Cao
Computer Vision · Cross-listed from cs.AI
Verifiable Geometry Problem Solving: Solver-Driven Autoformalization and Theorem Proposing
Can Li, Ting Zhang, Junbo Zhao et al. (+1 more)
Geometry Problem Solving have increasingly adopt the neuro-symbolic paradigm, combining neural intuition with symbolic rigor. However, current frameworks suffer from severe bottlenecks in two core stages: autoformalization, which treats
multimodal translation as a static task decoupled from downstream solver compatibility, and
theorem prediction, where solvers frequently hit a deductive impasse due to fixed rule libraries. To address these, we propose SD-GPS, a solver-driven framework that treats the symbolic solver as an execution oracle throughout both formalization and deduction. First, Solver-Driven Autoformalization unifies supervised formal-
language adaptation and solvability-guided reinforcement learning into a single module built on QwenVL3-2B, making executability the central training signal. Second, Verified
Theorem Proposing introduces an impasse-aware
agent that proposes local auxiliary lemmas from current proof states, ensuring soundness by filtering all proposals through symbolic verification. Empirical evaluations on Geometry3K and PGPS9K demonstrate that SD-GPS consistently outperforms existing MLLM, neural, and neuro-symbolic methods across standard completion, multiple-choice, and cross-modal reference regimes, proving that closing the loop between
multimodal perception and symbolic execution significantly improves
geometric reasoning, offering profound insights into how neural
agents can be grounded by formal systems to achieve verifiable problem-solving capabilities.
Read the full abstract ▾
All authors Can Li, Ting Zhang, Junbo Zhao, Hua Huang
Computer Vision
Home3D 1.0: A High-Fidelity Image-to-3D Asset Generation System for Interior Design
Yiyun Fei, Guoqiu Li, Jin Song et al. (+11 more)
We present Home3D 1.0, a modular image-to-3D generation system that produces high-quality 3D assets from a single reference image, targeting interior design and e-commerce applications. Given a photograph of a furniture or decor item, the system outputs a mesh with physically-based rendering (PBR) materials, and the mesh can be decomposed into material-specific components. The pipeline is organized into four tightly coupled modules: Geometry reconstructs a watertight mesh through latent SDF modelling with a geometry VAE and a coarse-to-fine flow-matching DiT; Texture predicts multiview albedo observations, reprojects them onto the mesh, and completes unseen surface regions with a 3D texture field; Material uses MatWeaver to obtain component masks through
video-based
segmentation and UV-space voting, then retrieves and bakes PBR maps from a curated material library through hierarchical multi-modal matching; and Parts generates material-editable semantic part meshes with a PartVAE and PartDiT, decoding multi-head part-specific SDF fields in one pass. Each module is evaluated independently with dedicated metrics, highlighting both the current system capability and the remaining gaps toward broader deployment.
Read the full abstract ▾
All authors Yiyun Fei, Guoqiu Li, Jin Song, Chuqiao Wu, Delong Wu, Hong Wu, Ziru Zeng, Haohui Chen, Yindong Kong, Jing Li, Qi Wu, Feng Zhang, Jianan Jiang, Ruigao Yang
Computer Vision
Reflect-R1: Evidence-Driven Reflection for Self-Correction in Long Video Understanding
Shuimu Chen, Yuteng Chen, Yuanshen Guan et al. (+7 more)
Current
multimodal reflection mechanisms for long
video understanding predominantly rely on closed-loop self-reflection within internal parameters. Lacking objective external evidence, models are frequently trapped in blind confidence and often fail to correct errors. Furthermore, applying reinforcement learning to multi-stage reflection pipelines introduces severe policy coupling, which is exacerbated by a critical scarcity of dedicated training data. To address these limitations, this work proposes Reflect-R1, the first Evidence-Driven self-correction framework for long
video understanding. The framework constructs a three-stage pipeline consisting of intuition, verification, and arbitration. By dynamically retrieving objective visual evidence to verify initial intuitions and autonomously executing multiple temporal searches to resolve conflicts, it completely breaks the
hallucination loop. To overcome policy coupling, we design a stage-decoupled reinforcement learning
algorithm named SD-GRPO that independently computes advantage functions across different
reasoning stages. Concurrently, we construct a
dataset of 120K samples to bridge the training data gap. Extensive experiments on
benchmarks such as VideoMME and LongVideoBench demonstrate that Reflect-R1 achieves state-of-the-art performance. Our method significantly improves the genuine rectification rate and enables authentic self-correction strictly grounded in objective evidence.
Read the full abstract ▾
All authors Shuimu Chen, Yuteng Chen, Yuanshen Guan, Zebang Cheng, Zeyu Zhang, Shengqian Qin, Bin Xia, Jiaran Li, Wenming Yang, Fei Ma
Computer Vision
Every Step of the Way: Video-based Parkinsonian Turning Step Counting
Qiushuo Cheng, Jingjing Liu, Catherine Morgan et al. (+2 more)
As a prominent symptom of Parkinson's disease (PD), turning impairment is evaluated through parameters such as turning angle, duration, and particularly, the number of steps required to complete a turn, which directly reflects motor dysfunction. Accurate step counting is challenging due to variability in real-world turning movements and atypical shuffling patterns in parkinsonian gait. Existing methods are predominantly wearable-based, requiring users to wear and manage dedicated devices, which can be inconvenient for continuous daily use. To address this, we propose a passive,
video-based framework that estimates step count in a coarse-to-fine manner using diverse motion representations. Specifically, an initial step count is estimated from foot movement signals derived from 3D human mesh recovery, providing high-level motion structures. To incorporate fine-grained motion details, a motion encoder learns complementary gait dynamics from mesh and optical flow to refine the initial estimate. In this process, coarse foot movement signals query the pixel-level motion cues via cross
attention to capture subtle parkinsonian gait dynamics. To handle varying
video lengths, we partition each
video into clips and integrate clip-wise motion
embeddings via multiple instance learning (MIL) for step count residual prediction. Extensive experiments show our method consistently outperforms existing step counting methods on real-world PD turning
datasets.
Read the full abstract ▾
All authors Qiushuo Cheng, Jingjing Liu, Catherine Morgan, Alan Whone, Majid Mirmehdi
Computer Vision
There and Back Again: A Flexible-Frame Transformer for Multi-Exposure Fusion
Lishen Qu, Yao Liu, Shihao Zhou et al. (+4 more)
Multi-exposure fusion (MEF) brings the dynamic range of conventional cameras closer to that of human
vision, producing images with rich scene content. Given the large variability in scene luminance, exposure strategies often require different numbers of frames to capture the full radiance range faithfully. However, conventional MEF techniques are typically designed for a fixed number of inputs, forcing deployment systems to maintain separate models for different frame-count requirements, which undermines deployment efficiency. To address this limitation, we propose FreeMEF, the first flexible-frame
transformer for MEF that seamlessly accommodates varying numbers of input exposures without retraining or architectural changes. The proposed approach consists of two key modules. First, we introduce a recurrent state space module (RSSM) that sequentially fuses features from arbitrary sequences via adaptive
alignment and state-space recurrent modeling, thereby providing global information guidance for the subsequent restoration. Second, we devise a global feature guided block (GFGB) incorporating an extremity-aware hybrid
attention (EAHA) and an affine-injection feed-forward network (AFFN), which effectively resolves the similarity paradox while simultaneously optimizing contrast and brightness regulation. Extensive experiments on three
benchmark datasets demonstrate the effectiveness of our method, which performs favorably against state-of-the-art methods both quantitatively and qualitatively.
Read the full abstract ▾
All authors Lishen Qu, Yao Liu, Shihao Zhou, Jie Liang, Hui Zeng, Lei Zhang, Jufeng Yang
Computer Vision
Long-Term Prediction of Local and Global Human Motion with Occlusion Recovery
Qiaoyue Yang, Sven Heutger, Christopher Niemann et al. (+3 more)
Human motion describes the three-dimensional full-body movement of a person. Anticipating such motion holds significant relevance across a wide range of application domains such as human-
robot interaction, autonomous driving, animation, and healthcare. In recent research, spatial and temporal dependencies are modeled by bidirectional
attention mechanisms. These typically anticipate human motion in an autoregressive manner which could cause an accumulation of errors over time. As a consequence, they solely focus on local pose
forecasting. To address these limitations, we propose a non-autoregressive
transformer based on spatio-temporal
attention, and train it not only for local pose anticipation, but also for global motion prediction in space. Furthermore, to enhance its applicability in real-world scenarios, our model is also trained to recover missing joints due to occlusions, and is capable of processing varying lengths of history observations. Our code is publicly available at https://github.com/Q-Y-Yang/Prediction-of-Local-and-Global-Human-Motion.
Read the full abstract ▾
All authors Qiaoyue Yang, Sven Heutger, Christopher Niemann, Magnus Jung, Ayoub Al-Hamadi, Sven Wachsmuth
Computer Vision
A Multi-Attribute Latent Space for Visual Analysis of Watches
Kai Lawonn, Tobias Günther and Monique Meuschke
We present a design rationale,
embedding model, and interactive visual-analysis system for exploring large wristwatch collections through heterogeneous visual and semantic attributes. The system addresses a common limitation of catalog and e-commerce interfaces: users can filter by metadata, but they receive little support for open-ended exploration of visual similarity, stylistic alternatives, and mixed aesthetic-functional criteria. We therefore represent watches with separate attribute graphs for dial color and dial design, while using watch type as an explicit semantic organizer. Dials are segmented with a U-Net, watch types are predicted with a
Vision Transformer, colors are represented through a shared CIELAB reference palette, and dial structure is described with a gradient-based image descriptor. We extend UMAP by combining attribute-specific neighborhood graphs in a unified probabilistic objective and by adding a class-aware layout term that separates global type structure from local visual neighborhoods. The resulting map is exposed in an interactive interface with spatial
navigation, metadata filtering, detail inspection, and search-by-example insertion. We evaluate the approach through parameter analysis, runtime measurements, and a qualitative pilot study with watch experts and novices. The results suggest that the system supports discovery and comparison, while also revealing limitations in scalability assessment, search-by-example validation, and the need for broader domain studies. We explicitly discuss these limitations and derive design implications for multi-attribute latent-space visualization across heterogeneous visual collections.
Read the full abstract ▾
Computer Vision
OrthoTryOn: Geometric Orthogonalization for Conflict-Free Unified Fashion Generation
Zhaotong Yang, Ying Tai, Jiahui Zhan et al. (+3 more)
Unified fashion generation integrates tasks like virtual try-on and garment reconstruction into a single model to reduce task-specific adaptation costs. However, naive parameter sharing across semantically distinct tasks induces negative transfer through severe inter-task gradient conflict. We propose OrthoTryOn, a unified framework mitigating this interference within a shared Low-Rank Adaptation (LoRA) module. Its Orthogonal Subspace Projection (OSP) applies task-specific orthogonal rotations to bottleneck features, mapping them into decorrelated coordinate frames. To address residual semantic coupling at inference time, we further propose Fisher-guided Negative Guidance (FNG), a parameter-free strategy that utilizes diagonal Fisher information to quantify inter-task sensitivity overlap and explicitly repels generation trajectories from the most confusable task via Classifier-Free Guidance. Extensive experiments demonstrate that OrthoTryOn avoids the severe performance degradation typical of naive unified training and even surpasses independently trained task-specific models, achieving state-of-the-art results across multiple
benchmarks while generalizing robustly across diverse
diffusion backbones. Code is available at https://github.com/NJU-PCALab/OrthoTryOn.
Read the full abstract ▾
All authors Zhaotong Yang, Ying Tai, Jiahui Zhan, Yu Zheng, Jianjun Qian, Jian Yang
Computer Vision
SpatialUAV: Benchmarking Spatial Intelligence for Low-Altitude UAV Perception, Collaboration, and Motion
Haoyu Zhang, Meng Liu, Qianlong Xiang et al. (+3 more)
Spatial intelligence is essential for low-altitude unmanned aerial vehicle (
UAV) perception, collaboration, and
navigation. However, existing
UAV benchmarks often emphasize image-level recognition, single-view understanding, or narrow answer formats, leaving 3D spatial inference, multi-view collaboration, scene dynamics, and diverse task formulations insufficiently evaluated. To address these gaps, we introduce SpatialUAV, a real low-altitude
UAV benchmark comprising 4,331 curated instances across 14 fine-grained task types, covering semantic discrimination, spatial relation, aerial–aerial collaboration, aerial–ground collaboration, and motion understanding. SpatialUAV organizes all samples into a unified visual-input–question–answer schema, while supporting seven input configurations and nine answer formats, including option labels, region identifiers,
geometric values, cross-view correspondences, and free-form motion descriptions. To ensure reliable and grounded evaluation, our data construction pipeline integrates detector-assisted regions, depth supervision, metadata-derived rules, extensive manual annotation, blind filtering, and multi-turn human validation, together with task-specific metrics for heterogeneous outputs. Evaluating representative
vision-
language models across three categories, we show that current models remain far from human-level performance, with pronounced bottlenecks in cross-view association, structured grounding,
geometric reasoning, and temporal viewpoint understanding. These results offer empirical guidance for advancing low-altitude
UAV spatial intelligence. Code and data are available at https://github.com/Hyu-Zhang/SpatialUAV.
Read the full abstract ▾
All authors Haoyu Zhang, Meng Liu, Qianlong Xiang, Kun Wang, Yaowei Wang, Liqiang Nie
Computer Vision
A Unified Framework for Vision Transformers Equivariant to Discrete Subgroups of $\mathrm{O}(2)$
Tīkun Ông and Georg Bökman
Vision transformers have become a dominant architecture for visual recognition. However, standard models do not explicitly encode the planar
symmetries that arise in many
vision domains. We introduce a family of
vision transformers equivariant to arbitrary discrete subgroups of $\mathrm{O}(2)$, providing a unified framework that generalizes prior flipping- and $D_4$-equivariant
transformer architectures. Our construction yields equivariant analogues of the core
transformer components, together with expressivity guarantees for the resulting layers. In particular, we show that whenever $H \le G$, the class of $G$-equivariant ViTs embeds naturally into the class of $H$-equivariant ViTs. We also prove that, in the single-head setting, the corresponding equivariant self-
attention layer realizes every $G$-equivariant self-
attention map representable by ordinary self-
attention. We further construct a $D_6$-equivariant model based on hexagonal patches, making the architecture compatible with six-fold rotational
symmetries. We evaluate the resulting models on the PatternNet aerial image
dataset in artificially data-scarce regimes across subgroups of $D_4$ and $D_6$. Our experiments compare two equivariant
attention mechanisms and analyze how the choice of homogeneous-space configurations used in the nonlinearities affects performance. Preliminary results under matched parameter budgets indicate that equivariance can improve recognition accuracy, motivating further study of how discrete
symmetry groups shape
transformer-based visual recognition models.
Read the full abstract ▾
Computer Vision
ScaLe-INR: Scale and Learn Implicit Neural Representations
Buwaneka Epakanda, Athulya Ratnayake, Pandula Thennakoon et al. (+4 more)
Implicit Neural Representations (INRs) parameterized by multilayer perceptrons excel at modeling continuous signals. However, a key challenge persists as INRs fundamentally suffer from spectral bias and information cross-talk. When a single network attempts to capture multi-scale phenomena, high-frequency weight updates destructively interfere with the underlying low-frequency structural approximation. We introduce Scale and Learn INR (ScaLe-INR), a novel multi-branch architecture that resolves these limitations by explicitly matching the signal's frequency spectrum with the optimal operating region of the INR. Drawing upon the
Fourier inverse scaling
theorem we demonstrate that applying directional coordinate scaling expands a network's representational bandwidth along specific spatial axes. To mathematically enforce functional disentanglement and minimize task-specific information leakage between branches, we propose a Directional Edge Guidance Loss, a spatially-conditioned sparsity prior derived from ground-truth gradients. By constraining the high-frequency branches to act as strict, localized edge-filters, ScaLe-INR eliminates spectral cross-talk, accelerates convergence, and achieves high-fidelity signal reconstruction on complex multi-scale
topologies. We evaluate ScaLe-INR across diverse reconstruction and inverse tasks, demonstrating substantial performance gains over existing state-of-the-art (SOTA) methods. The proposed architecture improves upon the nearest baselines by +5.16 dB in image reconstruction and +0.65 dB in image denoising. Furthermore, it achieve an impressive figure of 50.02 dB on audio reconstruction and 0.999 IOU(Intersection Over Union) on 3D reconstruction which beats the all SOTA models.
Read the full abstract ▾
All authors Buwaneka Epakanda, Athulya Ratnayake, Pandula Thennakoon, Mario De Silva, Avishka Ranasinghe, Roshan Godaliyadda, Parakrama Ekanayake
Computer Vision
Hippocampus-DETR: An Explicit Memory Object Detection Framework Based on Hippocampus Modeling
Zhaoning Shi, Bo Ma, Hao Xu et al. (+2 more)
This paper addresses the lack of explicit
memory mechanisms in current object detection models and proposes Hippocampus-DETR, a novel detection framework based on biological hippocampal
memory modeling. This framework integrates a hippocampal
memory network module, HipNet, into the DETR architecture and systematically simulates the anatomical structure and functional organization of hippocampal subregions, including the entorhinal cortex, dentate gyrus, CA3, CA1, and subiculum. Through this design, Hippocampus-DETR realizes pattern separation, pattern completion, importance filtering, and information integration of visual encoding features. During training, different
memory submodules are optimized using a layer-wise training strategy, ultimately forming a
memory system with
memory retrieval and completion capabilities. Experimental results demonstrate that Hippocampus-DETR achieves higher detection accuracy than current mainstream models. More importantly, models equipped with this framework also exhibit excellent generalization ability and data efficiency in tasks such as few-shot image classification,
multimodal feature construction, and image restoration. Subsequent experiments further validate the functional necessity and internal interpretability of each
memory submodule. This study not only provides a novel object detection framework, but also offers a feasible technical pathway for integrating neurocognitive mechanisms with deep learning models, highlighting its significant value in improving model learning efficiency and task robustness. The project is available at https://github.com/2186cloud/hipnet.
Read the full abstract ▾
All authors Zhaoning Shi, Bo Ma, Hao Xu, Zepeng Yang, Bo Liang
Computer Vision
CSD: Content-aware Speculative Decoding for Efficient Image Generation
Mingcheng Wang, Junbo Qiao, Yunchen Li et al. (+8 more)
Speculative decoding (SD) has emerged as a key solution to accelerate the inference of autoregressive models. However, in the field of image generation, it faces the challenge of low acceptance rates, and directly relaxing its criteria leads to degradation in image quality. In this paper, we propose a novel content-aware speculative decoding
algorithm, termed CSD, which integrates an
entropy-based probability relaxation mechanism with an optimal resampling strategy to enhance the inference efficiency for autoregressive image generation. By leveraging the informational uncertainty inherent in different regions of an image, CSD dynamically adjusts the acceptance probability of candidate
tokens, increasing the acceptance rate in low-detail areas to accelerate generation. Moreover, a distribution
alignment filter is introduced to ensure the output distribution to be aligned with the target model, which significantly improves the generative quality. Experiments conducted on Lumina-mGPT and Janus-Pro demonstrate that the superiority of the proposed CSD. Our source code is available at https://github.com/aderfebr/CSD.
Read the full abstract ▾
All authors Mingcheng Wang, Junbo Qiao, Yunchen Li, Lingfu Jiang, Wei Li, Jie Hu, Jiao Xie, Zhou Yu, Xinghao Chen, Guixu Zhang, Shaohui Lin
Computer Vision
Video-MME-Logical: A Controlled Diagnostic Benchmark for Video Temporal-Logical Reasoning
Hohin Kwan, Hongyu Li, Ray Zhang et al. (+5 more)
Recent interest in
multimodal large
language models (MLLMs) raises a central question: can they reason over dynamic visual evidence rather than merely recognize objects or events in individual frames? This ability, which we refer to as
video temporal-logical
reasoning, requires models to maintain, update, and compose evidence as visual states evolve across frames. Existing
video benchmarks often conflate this capability with scene complexity, static recognition, or uncontrolled temporal variation. To isolate this capability, we introduce
Video-MME-Logical, a controlled
benchmark organized around five temporal-logical operations: state tracking, sequential counting, temporal ordering, dynamic spatiality, and structural composition. The
benchmark contains 25 fine-grained task categories generated with controlled object states, transitions, temporal dependencies, and logical compositions. It enables difficulty-controlled final-answer evaluation by varying temporal horizon and
reasoning complexity, and supports intermediate-state diagnostics by verifying whether models recover the required logical
reasoning trace before producing the final answer. Experiments with state-of-the-art MLLMs reveal a substantial human-model gap, especially as temporal-logical complexity increases. Supervised fine-tuning on up to 500K generated samples improves performance but remains insufficient to close the
reasoning gap, positioning
Video-MME-Logical as a scalable testbed for analyzing and improving temporal-logical
reasoning in MLLMs.
Read the full abstract ▾
All authors Hohin Kwan, Hongyu Li, Ray Zhang, Manyuan Zhang, Xianghao Kong, Anyi Rao, Jiahao Xie, Si Liu
Computer Vision
Scalable and Differentiable Point-Cloud Registration Using Maximum Mean Discrepancy
Rixon Crane, Fahira Afzal Maken, Nicholas Lawrance et al. (+4 more)
We present MMD-Reg, a novel correspondence-free approach to point-cloud registration that is differentiable and has linear computational complexity in the number of points. We model registration as a nonlinear least-squares problem based on the Maximum Mean Discrepancy, approximated using random
Fourier features. The resulting objective can be solved efficiently with standard methods such as Levenberg-Marquardt, and the solution is differentiable via the implicit function
theorem. This allows MMD-Reg to be used as a differentiable optimization layer within end-to-end trainable models, supporting registration under challenging conditions such as poor initial
alignment and partial overlap. We demonstrate this Neural MMD-Reg formulation by integrating the layer with a set
transformer, training the resulting model in supervised and unsupervised settings, and comparing its performance against recent learning-based methods. We also evaluate standalone MMD-Reg, comparing its accuracy and scalability against widely used non-learning-based registration methods.
Read the full abstract ▾
All authors Rixon Crane, Fahira Afzal Maken, Nicholas Lawrance, Stanislav Funiak, Kasra Khosoussi, Ming Xu, Russell Tsuchida
Computer Vision
Text as Illumination: Spatial Contrastive Retinex Learning for Language-guided Medical Image Segmentation
Jian Shi, Cheng Zhen, Pingping Zhang et al. (+6 more)
Language-guided Medical Image
Segmentation (LMIS) has shown great potential to improve the delineation of anatomical structures and lesions by integrating clinical textual information. Existing methods generally rely on either implicit interaction between textual and visual features or auxiliary coarse-grained supervision for cross-modal
alignment. However, these methods lack explicit and fine-grained constraints to ensure semantic consistency, causing a mismatch between
language and the
segmentation outputs. To address this issue, we propose Text-as-Illumination Retinex Network (TIRNet), a novel Retinex-inspired framework that treats text
embeddings as semantic illumination for feature modulation, thereby improving semantic consistency in LMIS. TIRNet introduces two key blocks integrated at each decoder stage: (1) the Retinex-inspired Text Modulation Block (RTMB), which employs positive and negative illumination maps to enhance text-relevant foreground features and suppress background interference; and (2) the Consistent Detail Compensation Block (CDCB), which selectively recovers high-frequency details via a consistency-gated mechanism conditioned on illumination reliability. Furthermore, we propose a Multi-Scale Illumination Supervision Loss (MSIS-Loss), comprising a Region-Grounded Contrastive Loss (RGC-Loss) that enforces cross-modal similarity to be concentrated in text-relevant foreground regions and suppressed in background regions, and a Background Suppression Loss (BS-Loss) that provides pixel-level supervision for negative illumination maps, jointly ensuring a precise cross-modal
alignment at each decoder stage. Extensive experiments on the MosMedData+ and QaTa-COV19
datasets demonstrate that TIRNet achieves state-of-the-art performance in LMIS. The code is available at: https://github.com/anaanaa/TIRNet.
Read the full abstract ▾
All authors Jian Shi, Cheng Zhen, Pingping Zhang, Rui Xu, Yanan Lv, Yili Ma, Huan Bi, Haojie Li, Huchuan Lu
Computer Vision
Improving Adversarial Robustness via Activation Amplification and Attenuation
Taïga Gonçalves, Yongsong Huang, Tomo Miyazaki et al. (+1 more)
The existence of
adversarial attacks is often attributed to the presence of non-robust features in neural networks. While prior defenses reduce their impact via
pruning, masking, or feature recalibration, we instead propose to jointly learn to amplify and attenuate these signals through a simple activation scaling mechanism. To this end, we introduce Activation Amplification and Attenuation (A3), a lightweight plug-in module that enhances
adversarial robustness with minimal modifications of the activations. A3 dynamically rescales the activations using a learnable mask and a scaling factor derived from the original activation magnitudes. The influence of
adversarial perturbations can be amplified or attenuated using the same learnable parameters by simply flipping the sign of the scaling operation. The amplified signals serve as negative references to construct novel contrastive and ranking loss functions. Experimental analysis shows that learning to degrade the predictions in amplification mode simultaneously improves
adversarial robustness in attenuation mode. Moreover, A3 relies on only a small number of learnable parameters, with most of its behavior being determined by the scaling mechanism rather than additional network capacity. Extensive experiments demonstrate that integrating A3 into different backbones,
datasets, and training methods consistently improves
adversarial robustness while introducing negligible computational and
memory overhead compared to existing plug-in modules. Code is available at: https://github.com/tgoncalv/A3.
Read the full abstract ▾
All authors Taïga Gonçalves, Yongsong Huang, Tomo Miyazaki, Shinichiro Omachi
Computer Vision
MindFlow: Harmonizing Cognitive Semantics and Acoustic Dynamics for Facial Animation Generation in Dyadic Conversations
Hejia Chen, Haoxian Zhang, Xu He et al. (+4 more)
Generating lifelike facial animation for dyadic conversations requires reconciling high-level cognitive intent with precise low-level motor reflexes, yet existing methods fall short in the semantic understanding of dialogue context and in precise dynamic control. In this paper, we propose MindFlow, a dual-pathway generative framework inspired by the Ventral-Dorsal pathway model in neuroscience, which decouples generation into two collaborative streams, thereby harmonizing deep semantic
reasoning with fine-grained control. In the Ventral module, we transform the conventional Sentence-Action approach into a novel Chunk-State approach that models raw acoustic streams as a context-aware, evolving emotional state chain, capturing subtle paralinguistic nuances and mid-utterance emotional shifts missed by sentence-level modeling. The Dorsal module features a conditional autoregressive flow matching network for high-fidelity facial motion, driven by high-frequency acoustic cues and modulated by
emotion states, plus a Selective Acoustic Injector for adaptive audio gating to ensure robustness in talking-and-listening dynamics without interference. Extensive experiments demonstrate that MindFlow achieves superior semantic appropriateness and motion naturalness compared to state-of-the-art baselines.
Read the full abstract ▾
All authors Hejia Chen, Haoxian Zhang, Xu He, Xiaoqiang Liu, Pengfei Wan, Shoulong Zhang, Shuai Li
Computer Vision
TRUST: Efficient Abdominal Trauma Recognition via Image-to-Ultrasound-Video Transfer Learning
Enguang Wang, Hao Zhou, Shuo Gao et al. (+2 more)
Abdominal ultrasound is indispensable for rapid, noninvasive trauma triage. However, interpreting the subtle dynamic cues embedded in continuous scanning is time-intensive and operator-dependent. Parameter-Efficient Image-to-
Video Transfer Learning (PEIVTL), which efficiently adapts pre-trained image models to the
video domain, notably through visual-textual
alignment, offers a promising paradigm for ultrasound
video analysis. Nevertheless, substantial spatiotemporal and semantic variations arising from physician-dependent scanning practices continue to limit the effectiveness and generalizability of this framework. We propose TRUST, a scan-aware PEIVTL framework that explicitly models fine-grained spatiotemporal variations to enable reliable ultrasound
video understanding. First, we introduce a Cross-Frequency Collaborative Adapter (CFCA) that establishes mutual constraints between low- and high-frequency components, enhancing discriminative spatial feature extraction under heavy speckle corruption. Second, we design a Multi-Granularity Motion-Aware (MGMA) module that integrates local temporal convolutions with motion-prior-guided global self-
attention, jointly capturing stable intra-view patterns and abrupt inter-view transitions to characterize complex scanning dynamics. Third, a Visual Query Semantic Aggregation (VQSA) module dynamically generates text prototypes conditioned on visual features, enabling adaptive visual-textual
alignment robust to intra-class variability under diverse scanning conditions. Experiments on in-house ultrasound trauma
datasets demonstrate that TRUST outperforms state-of-the-art methods by 9.63% with superior computational efficiency.
Read the full abstract ▾
All authors Enguang Wang, Hao Zhou, Shuo Gao, Tuo Liu, Guangquan Zhou
Computer Vision
ModaFlow: Modality-Aware Flow Matching for High-Fidelity Virtual Try-On
Xiangyu Sai, Meysam Madadi, Sergio Escalera et al. (+1 more)
Image-based virtual try-on has emerged as a compelling task in e-commerce and augmented reality, yet existing methods struggle to simultaneously preserve fine garment semantics and adapt to diverse person body geometries under large clothing-body deformations. We present ModaFlow, a modality-aware flow-matching based framework for high-fidelity virtual try-on that achieves precise
alignment between textual descriptions and garment appearance. Unlike prior methods that treat
multimodal conditions uniformly, ModaFlow introduces a modality-aware guidance scheme: visual garment
embeddings extracted by a pretrained image
prompt adapter provide deterministic, persistent structural guidance, while textual
embeddings generated from garment descriptions are controlled via classifier-free guidance (CFG) with adaptive scaling and zero-initialized velocity. To further enhance flow field accuracy, we propose two regularization losses, cosine similarity and perceptual flow discrimination, that jointly improve directional consistency and perceptual realism of the velocity field. Additionally, a mask manipulation strategy stochastically samples among box, transparent, and relaxed masks during training, simulating diverse occlusion scenarios and enabling robust inference under unpaired settings where only a box mask is available. Experiments show that ModaFlow achieves state-of-the-art results in both qualitative and quantitative evaluations, reducing FID by approximately 30% on paired and 20% on unpaired
benchmarks.
Read the full abstract ▾
All authors Xiangyu Sai, Meysam Madadi, Sergio Escalera, Yong Xu
Computer Vision
An Embedded Real-Time License Plate Recognition System for Complex Traffic Scenes
Anuki Pasqual, Dulan Lokugeegana, Manimohan Thiriloganathan et al. (+3 more)
Vehicle license plate recognition is an integral component of intelligent transportation systems. In this work, we present an embedded real-time license plate recognition system customized for developing countries. We address the challenge of handling complex, unstructured traffic scenes with diverse vehicle types while implementing the system on an embedded platform for low-cost deployment. Our method consists of license plate detection on a multi-vehicle image, followed by character recognition on the detected license plates. Both steps use lightweight convolutional neural networks to balance accuracy and efficiency. We also introduce the SL-LPR
dataset of Sri Lankan road images, which contains a variety of vehicle types and traffic conditions typically seen in developing countries. On this
dataset, the license plate detection and character recognition models achieved 93.6% mAP and 87.88% accuracy, respectively, and were competitive against larger models on several public
datasets. To achieve real-time performance in a resource-constrained embedded environment, we applied low-bitwidth quantization using the Brevitas library and implemented FPGA acceleration for the models using the FINN framework. The end-to-end system can operate at 11.5~FPS when implemented on the Xilinx Kria KV260 platform. These results demonstrate that our system is effective for real-time license plate recognition on an embedded device, even in complex traffic scenarios. The SL-LPR
dataset is available for research use at: https://github.com/sl-lpr-uom/SL-LPR.git.
Read the full abstract ▾
All authors Anuki Pasqual, Dulan Lokugeegana, Manimohan Thiriloganathan, Nuthya Rathnayake, Kithsiri Samarasinghe, Udaya S. K. P. Miriya Thanthrige
Computer Vision · Cross-listed from cs.LG
NormGuard: Reward-Preserving Norm Constraints in Flow-Matching Reinforcement Learning
Tianlin Pan, Lianyu Pang, Cheng Da et al. (+4 more)
Reinforcement learning (RL) post-training improves the reward
alignment of flow-based generators, but often degrades perceptual quality in ways that are not captured by the reward proxy. We identify a simple structural signature of this drift: across three post-training methods (NFT, AWM, DPO), RL fine-tuning inflates the per-step velocity norm $\|v_\theta\|$ by $5\%$ to $15\%$ relative to the reference. A form of norm inflation has been studied in classifier-free guidance (CFG), where rescaling the velocity back to a reference norm at inference time can mitigate the resulting artifacts. However, this inference-time correction does not transfer cleanly to RL: rescaling $v_\theta$ to match $\|v_{\text{ref}}\|$ at inference time neither improves reward nor fixes the quality degradation, because the inflation is co-adapted into the model weights. Furthermore, an adjoint sensitivity analysis shows that velocity magnitude rescaling carries no coherent first-order reward signal at the batch level, indicating that suppressing norm inflation is unlikely to remove a consistently reward-carrying component. Since inference-time renormalization fails while norm suppression carries no reward cost, training-time intervention is the appropriate strategy. Together, these findings motivate NormGuard, a hinge penalty that activates only when $\|v_\theta\|$ exceeds $\|v_{\text{ref}}\|$ and composes additively with any velocity-local base loss. Across two base models, three post-training methods, and two reward proxies, NormGuard consistently improves MLLM-judged image quality and forensic realism while preserving reward, with gains that amplify under few-step inference and are not explained by early stopping.
Read the full abstract ▾
All authors Tianlin Pan, Lianyu Pang, Cheng Da, Huan Yang, Changqian Yu, Kun Gai, Wenhan Luo
Computer Vision
PixelU: A U-Shaped Transformer for Efficient End-to-End Pixel Diffusion
Zipeng Guo, Lichen Ma, Yu He et al. (+4 more)
End-to-end pixel-space
diffusion models bypass the lossy compression of Latent
Diffusion Models (LDMs) but struggle to jointly model low-frequency semantics and high-frequency signals in high-dimensional space. Existing works heavily rely on complex pixel decoders to alleviate this issue. In this paper, we challenge this trend by revealing that these decoders primarily compensate for the optimization difficulties inherent to velocity prediction ($v$-prediction). Under the clean data paradigm ($x$-prediction), they are redundant. Motivated by this insight, we advocate for simplicity over complexity and introduce PixelU, a minimalist, single-stage U-shaped
Diffusion Transformer tailored for pixel space. PixelU abandons auxiliary decoders in favor of zero-cost skip connections, which provide an "information highway" that directly routes uncorrupted high-frequency spatial details from shallow to deep layers. To further enable the backbone to focus exclusively on modeling low-frequency semantics, we introduce a constant-channel spatial down-sampling mechanism as a natural low-pass filter, which compresses deep features into a compact, low-frequency semantic
manifold. Extensive experiments demonstrate that this decoupling of frequencies could outperform the strong baseline (JiT-G) with only about 1/3 of its computation cost. On ImageNet 256$\times$256 and 512$\times$512, PixelU achieves FID of 1.63 and 1.92 respectively, surpassing recent pixel-space methods and establishing a simple yet powerful new paradigm for end-to-end
diffusion models.
Read the full abstract ▾
All authors Zipeng Guo, Lichen Ma, Yu He, Xiaolong Fu, Jingling Fu, Junshi Huang, Yan Li
Computer Vision
Panoramic Scene Understanding: A Survey from Distortion-Aware Engineering to Sphere-Native Modeling
Qinfeng Zhu and Lei Fan
Panoramic images capture the full visual sphere in a frame, offering context unavailable to conventional cameras. Yet this completeness has an unavoidable
geometric cost: the 2-sphere cannot be faithfully mapped to the plane, and every projection introduces distortions that challenge standard
vision architectures. This survey traces panoramic scene understanding from projection-based adaptation and distortion-aware engineering to sphere-native modeling, reflecting increasing commitment to spherical geometry. Foundation models form a fourth family, geometry-aware tokenization, which adapts the input interface while reusing perspective-pretrained weights. We review these approaches across five task families: dense prediction, unified multi-task understanding, open-world perception,
vision-
language reasoning, and dynamic
video analysis. Across tasks, the same shift toward spherical geometry recurs. In practice, however, the field has converged not on the strongest sphere-native operators, which are exactly rotation-equivariant but cannot reuse perspective-pretrained backbones and thus have not scaled, but on a compatibility-preserving middle ground combining moderate
geometric awareness with large pretrained models. This commitment is uneven: deepest in dense prediction and shallowest in dynamic perception, where methods are spatially sphere-aware yet temporally planar. Foundation-model adaptation has advanced panoramic depth fastest, while layout, surface-normal, and
video-level understanding remain largely unexplored. No panoramic foundation model has yet been pretrained on spherical data. We identify five evaluation gaps: spherical-area-weighted metrics, seam-consistency tests, polar-robustness stratification, cross-projection generalization, and standardized open-world protocols. We conclude with a six-point roadmap toward general-purpose panoramic intelligence.
Read the full abstract ▾
Computer Vision
SIFT: Self-Imagination Fine-Tuning for Physically Plausible Motion in Video Diffusion Models
Ruoyu Wang, Jialun Liu, Huayang Huang et al. (+5 more)
Recent advances in
video diffusion models have greatly improved visual fidelity, yet their generated motions often violate physical plausibility. We observe a common kinematic failure, "motion
entanglement", the unintended coupling of independent motion sources, such as camera movement and object motion. We identify that this issue stems from data bias and the reconstruction-based training design of
diffusion models. Training on noisy
videos that still retain coarse motion cues inadvertently encourages the model to replicate existing motion without an incentive to learn how to model kinematically-grounded motions. To address this, we propose a Self-Imagination Fine-Tuning (SIFT) paradigm, which enables the model to learn from its own generated
videos rather than directly reconstructing real ones, breaking the reconstruction shortcut. We further employ motion-aware discriminative supervision and a progressive hard-case replay strategy to stabilize and accelerate learning. By leveraging freely-generated text
prompts, our method can densely cover a broad motion space, including rare or finely-disentangled scenarios that would be costly to collect as
video data. Extensive experiments demonstrate that our approach substantially improves the physical realism, motion disentanglement, and controllability of generated
videos.
Read the full abstract ▾
All authors Ruoyu Wang, Jialun Liu, Huayang Huang, Haibin Huang, Jiepeng Wang, Chi Zhang, Xuelong Li, Yu Wu
Computer Vision
Learning 1-Bit LiDAR-based Localization with Auxiliary Objective
Kaijie Yin, Zhiyuan Zhang, Tian Gao et al. (+3 more)
6-DoF
LiDAR-based localization is a fundamental capability for autonomous systems operating in large-scale outdoor environments. Many deep-learning-based localization methods have achieved promising performance so far. However, as one of the always-on modules competing for limited on-board computational resources, the localization module is expected to consume only a small portion of the overall compute budget. Most existing learning-based methods are still too heavy for this purpose. In contrast, binary neural networks (BNNs) offer an appealing solution, but the 1-bit compression causes severe information loss and performance drop. In this paper, we address this challenge by proposing Binarized
LiDAR-based Localization (BiLoc), the first binary neural network framework for 6-DoF
LiDAR localization. Specifically, we reinterpret the training of BNNs from the perspective of the information-bottleneck principle, aiming at retaining minimal yet sufficient representations for pose estimation while suppressing redundant variations. And we introduce an auxiliary objective that adaptively regulates information retention in the binary encoder, effectively mitigating the information loss caused by binarization. This auxiliary objective provides additional optimization signals that compensate for the limited representational capacity and the gradient mismatch inherent in BNNs. Extensive experiments on large-scale outdoor
LiDAR datasets demonstrate that BiLoc establishes a new state of the art for
LiDAR localization with BNNs.
Read the full abstract ▾
All authors Kaijie Yin, Zhiyuan Zhang, Tian Gao, Wentao Zhu, Cheng-zhong Xu, Hui Kong
Computer Vision
Scene and Human in One World: Reconstruction in a Feedforward Pass
Boao Shi, Qiao Feng, Yiming Huang et al. (+1 more)
Reconstructing humans in dynamic scenes from moving
monocular cameras remains challenging due to scale ambiguity, human-scene misalignment, and occlusion interference. Rather than treating human mesh recovery and scene reconstruction as separate tasks, we believe that accurate human-scene reconstruction requires the two tasks to mutually inform each other: parametric human models offer semantic structure and metric-scale priors, while scene geometry provides spatial context for human localization and
alignment. Built on this insight, we introduce SHOW, a mask-promptable human mesh recovery framework that couples feed-forward 3D scene reconstruction with Human Mesh Recovery in a unified metric space. SHOW injects human semantics and scale priors from parametric human models into normalized point-map prediction, enabling metric-scale scene reconstruction from inherently scale-ambiguous
monocular input. In turn, the recovered scene geometry constrains human mesh estimation, encouraging spatially consistent human placement and improved human-scene
alignment. To handle complex multi-person and cluttered scenes, SHOW further incorporates a promptable masking mechanism that enables flexible target-human selection while suppressing background distractions and occlusion interference. Through joint training, the model learns both human-aware
geometric features and geometry-constrained human features, producing aligned metric-scale reconstructions from
monocular human-centric
videos. Extensive experiments demonstrate that SHOW improves metric-scale consistency, human-scene
alignment, and reconstruction accuracy under challenging camera motion, occlusion, and cluttered backgrounds.
Read the full abstract ▾
All authors Boao Shi, Qiao Feng, Yiming Huang, Lingjie Liu
Computer Vision
MASS: Motion-Aligned Selective Scan for Refinement in Flow-Based Video Frame Interpolation
Jun-Sang Yoo and Seung-Won Jung
Video frame interpolation (VFI) remains a challenging task, particularly when dealing with large, non-linear motions and complex occlusions. While flow-based methods are prevalent, they often struggle with ambiguous correspondences. Recent VFI methods based on selective State Space Models (SSMs) are still limited by static grid-based scanning that misaligns with physical motion. In this paper, we propose Motion-Aligned Selective Scan (MASS), a novel framework that reformulates feature scanning from static spatial grids to dynamic motion trajectories. MASS builds a feature sequence along each pixel's flow-guided trajectory and aggregates it with an SSM. Specifically, we introduce a learnable non-linear path integration to approximate complex curved trajectories via residual velocity updates, and a velocity-aware SSM that dynamically adjusts the sampling budget and step size based on motion magnitude. This adaptive strategy allocates denser sampling to fast-motion regions while keeping static regions efficient. Furthermore, the aggregated states guide a refinement module to rectify intermediate flows and masks in an end-to-end manner. Extensive experiments indicate that MASS achieves highly competitive overall performance on standard
benchmarks, establishing state-of-the-art results particularly in challenging scenarios with large displacements and complex dynamics.
Read the full abstract ▾
Computer Vision
ZooClaw-FashionSigLIP2: Distilled Fine-tuning for Robust Fashion Retrieval
Siqiao Xue and Chunxue Xu
Adapting a foundation
vision-
language encoder to a specialized
retrieval task creates a fundamental tradeoff: gains on the target distribution come at the cost of the foundation model's broad generalization, and fashion
retrieval is a stringent instance of this problem. We present ZooClaw-FashionSigLIP2, a fashion-specialized SigLIP2-base model that resolves this tradeoff with a simple recipe – full fine-tuning with knowledge
distillation on curated in-domain data, followed by \wiseft~\citep{wortsman2022wiseft} weight interpolation with the base model – and outperforms LoRA, larger backbones (up to 1B parameters), and external training data. Under fair evaluation, ZooClaw-FashionSigLIP2 outperforms all baselines on every
benchmark in our suite. In addition, we release ZooClaw-Fashion, a new high-quality fashion
retrieval benchmark, and a systematic quality analysis of widely-used
benchmarks that exposes and mitigates structural biases in their public ground truth. We open-source the model weights and all evaluation artifacts to facilitate future research.
Read the full abstract ▾