Computer Vision
Camera trap classification with deep learning under ground truth uncertainty
Leonard Hockerts, Peter S. Stewart, Sarthak Arora et al. (+1 more)
Supervised deep learning methods enable the rapid processing of ecological image data, but depend on a costly annotation process. Consequently, training labels are commonly derived from volunteer citizen science projects. However, disagreement among volunteers introduces uncertainty in the "ground truth" data that are assumed to be correct for model training and validation. Using two
datasets containing camera trap images with associated volunteer and expert classifications, we investigated the effects of training under higher ground truth uncertainty. We observed improved overall test accuracy, particularly for images that were more difficult for volunteers. Species-level accuracy also generally improved, but generalisation to a different
dataset did not. The benefits of ground truth uncertainty were enhanced by pre-training on ImageNet. Pre-training also reduced the number of training epochs required; further reductions in computational cost, but not gains in accuracy, resulted from additional pre-training on other camera trap images. With unbalanced training data, we still observed a clear benefit of increased ground truth uncertainty for overall accuracy, especially on difficult images. Class imbalance improved accuracy for common species, reduced rare species accuracy, and changed patterns of misclassification to more closely resemble mistakes made by volunteers. Our findings have implications for applying deep learning across ecological image types with multiple labels. Practitioners can improve accuracy, especially on difficult examples, by including moderate levels of label disagreement during training and using models pre-trained on general image data. In addition to improving the use of citizen science-derived labels in model training, our study suggests avenues for more effectively integrating human and deep learning classifications in combined workflows. (abridged)
Read the full abstract ▾
All authors Leonard Hockerts, Peter S. Stewart, Sarthak Arora, Tiffany J. Vlaar
Computer Vision
PixelIR: Fidelity-Perception Decoupling via Pixel-Space Image-Residual Flow Matching for Efficient One-Step Real-World Super-Resolution
Bingtian Qiao, Yue Shi, Yong Guo et al. (+2 more)
Real-world image super-resolution (Real-ISR) aims to preserve structures supported by the degraded observation while reconstructing perceptually realistic details. However, existing Real-ISR methods largely optimize fidelity and perceptual quality within a shared network, causing the two objectives to interfere throughout training and making their balance difficult to control. Recent one-step methods reduce sampling steps, yet often inherit both this coupled optimization behavior and the expensive high-resolution backbone of their multi-step predecessors. We argue that efficient Real-ISR requires not only a shorter sampling trajectory, but also specialized modeling of faithful reconstruction and perceptual detail synthesis. Based on this insight, we propose PixelIR, a fidelity-perception decoupling framework built upon pixel-space image-residual flow matching. PixelIR first learns an image flow that maps the degraded observation to a faithful reconstruction. Then, a residual flow synthesizes the missing perceptual details from noise without repeatedly relearning or overwriting the complete restoration solution. We further distill the teacher into a deployment-oriented one-step student within a coarse-to-fine pyramid architecture. Extensive experiments show that PixelIR achieves leading PSNR, SSIM, and LPIPS on both RealSR and DRealSR. The final model completes pixel-space restoration in a single evaluation with only 32.9M parameters, 89.7G MACs, and 8.5ms latency, demonstrating a strong practical fidelity-perception-efficiency balance.
Read the full abstract ▾
All authors Bingtian Qiao, Yue Shi, Yong Guo, Wenjun Zhang, Jiezhang Cao
Computer Vision
CORAL: A Benchmark for Structure-aware and Brain-wide Neuron Reconstruction in Light Microscopy
Zekang Yang, Jiamin Li, Zhenghua Li et al. (+3 more)
Automatic
neuron reconstruction from light microscopy images is a central problem in computational neuroanatomy. While recent methods have achieved encouraging results on local image blocks, it remains unclear whether such progress translates to reconstruction that is both structurally accurate and scalable to the whole-
brain scale. We present CORAL, the first
benchmark for structure-aware evaluation of automatic
neuron reconstruction from light microscopy images at both local and whole-
brain scales. Built on a high-quality whole-
brain fMOST
dataset with carefully curated annotations, CORAL establishes two progressive tasks: block-level reconstruction, which evaluates reconstruction methods under limited spatial context, and
brain-wide reconstruction, which assesses complete
neuron reconstruction at the whole-
brain scale. To account for topological correctness beyond
geometric distance similarity, we introduce a structure-aware metric based on fiber prediction. To further achieve complete
neuron reconstruction across the entire
brain, we develop a
brain-wide
neuron tracing framework that extends arbitrary local reconstruction methods to the whole-
brain scale through an iterative local-to-global process. Using this
benchmark, we provide the first structure-aware comparison of mainstream methods for local
neuron reconstruction and further evaluate their performance in
brain-wide reconstruction. Our results underscore the importance of structure-aware evaluation and the need for more robust methods for complete
neuron reconstruction.
Read the full abstract ▾
All authors Zekang Yang, Jiamin Li, Zhenghua Li, Jiaqi Fan, Zengcai Guo, Xiaolin Hu
Computer Vision
ChessQueries: Toward Better Chess Board Recognition
Joël Seytre
Chess board recognition is the task of mapping the image of a chess board to the information of which piece is on which square. So far this task has two established
benchmarks: ChessCog is synthetic, and ChessReD comes from smartphone pictures of a single chess board setup. We introduce ChessQueries, a new method combining a ViT encoder with a DETR-style decoder, which outperforms existing methods. On the ChessReD
benchmark, we improve the state of the art from 15.3% to 99.2%, and demonstrate strong capabilities on out-of-distribution
datasets. Our method saturates the task on the two
datasets, with an average 0.01 wrong squares per board (vs. SotA: 3.4 / 0.15 respectively). We also share a new, harder public
dataset, parsed from broadcasted top-level chess tournaments. Code, model weights and the SLCC data will be released.
Read the full abstract ▾
Computer Vision
CheXGround: Anatomical Region Tokens for Grounded Longitudinal Chest X-ray Interpretation
Adonay Demewez Gebremedhin, Wessam Shehieb, Sara Alansari et al. (+4 more)
Recent radiology multi-modal
language models have made substantial progress in chest X-ray report generation, visual question answering, and temporal
reasoning. While longitudinal chest X-ray interpretation compares sequential examinations to describe change, visual grounding aims to connect clinical
language with localized image evidence. Although longitudinal modeling and visual grounding have each advanced radiology
language models, how localized visual evidence can support longitudinal interpretation remains under-explored. We introduce CheXGround, a region-grounded longitudinal chest X-ray
language model that represents paired studies through corresponding anatomical regions. CheXGround extracts anatomical regions from current and prior radiographs, encodes them as temporally enhanced Region-of-Interest (ROI)
tokens, and combines them with global temporal image context during generation. To connect these region
tokens with clinical text, we propose Temporal Region–Phrase
Alignment, a pretraining objective that aligns temporal anatomical representations with localized report phrases. We evaluate CheXGround on single-study and longitudinal Visual Question Answering (VQA), longitudinal findings generation, temporal grounded VQA, and anatomical grounding. Across these tasks, CheXGround improves clinical
language quality, temporal
reasoning, and localization accuracy over recent baselines. Our results suggest that organizing longitudinal evidence at the anatomical level is a strong representation for grounded radiology
language modeling. Project page: https://adonaydem.github.io/chexground-website
Read the full abstract ▾
All authors Adonay Demewez Gebremedhin, Wessam Shehieb, Sara Alansari, Mohamad Alansari, Muzammal Naseer, Sajid Javed, Naoufel Werghi
Computer Vision · Cross-listed from cs.AI
Autoregressive Mosaics: Probing 2D Spatial Reasoning in Text-Only Language Models
Ashwin Nedungadi, Stefan Oehmcke and Stefan Lüdtke
Large
language models (LLMs) trained only on text and code can sometimes generate programs that draw recognizable images. However, it is unclear whether this reflects an internal representation of 2D spatial layout or simply the ability to translate spatial descriptions into code. We introduce Autoregressive Mosaics (AM-Bench), a
benchmark that separates these factors: First, a translation task gives a model a fully specified geometry of a picture in words as a
prompt and asks for the code that produces it. Second, a layout task requires the model to compose an image from an underspecified
prompt. Across eight open-weight text-and-code-only models, all models reliably translate specified geometry into code, but their open-ended layout performance differs substantially, indicating that these differences are not explained by code-generation ability alone. An output-medium ablation further shows that the interface or medium of expression that the model uses matters: replacing procedural code with raw SVG improves layout scores across all models. Finally, probing model activations shows that a coarse layout plan is present before generation, but reflects only the layout implied by the
prompt. During generation, models track the evolving
geometric state instead of executing an initially fixed plan. Overall, these results show that 2D spatial performance in text-only LLMs depends on both the model and the output medium, and is not explained by code-generation ability alone.
Read the full abstract ▾
Computer Vision
RailGen: Improving Railway Intrusion Detection via Agent-Guided Small-Scale Foreign Object Generation
Quan Hao, Ziyang Tao, Chenxi Zhang et al. (+3 more)
Small-object detection under long-tailed data distributions is a fundamental yet challenging problem in multimedia. Railway Foreign Object Detection (RFOD) epitomizes this challenge with easily confused small intrusions and scarce samples. To address these issues, we propose a generative-augmented detection paradigm that leverages
multimodal image generation to enrich the feature space of rare and small objects. We first construct RailGen, a
multimodal image generation
agent based on large models. Under semantic constraints, RailGen automatically invokes tools to generate railway scenes, calibrate intrusion positions, extract foreign objects, and fuse them into realistic intrusion effects. This process produces high-quality synthetic samples that effectively densify the feature representations of tail classes and complete the small-object feature space. Within this paradigm, we further propose FocalDEIM, a detection framework designed to enhance training with generated data. FocalDEIM improves dense matching with Focal Modulation for better small-object discrimination and adopts Focal Loss to emphasize hard samples, thereby alleviating blurred inter-class boundaries in complex railway scenes. Experimental results demonstrate that RailGen can generate high-quality small-scale foreign objects, reducing the object pixel area by up to 58x and 13.85x on average. Equipped with these challenging samples, our paradigm surpasses the baseline DEIM by 5.6% and 7.5% in mAP@50 and mAP@(50-95), respectively, and outperforms existing state-of-the-art methods. Ablation studies verify RailGen's feature-space enrichment and FocalDEIM's boundary discrimination. The paradigm provides an effective
multimodal generative solution for long-tailed small-object detection in safety-critical applications.
Read the full abstract ▾
All authors Quan Hao, Ziyang Tao, Chenxi Zhang, Yudong Wang, Rui Shi, Liguo Zhang
Computer Vision · Cross-listed from cs.CL
SocialReasonBench: A Video-QA Benchmark for Social Reasoning with Counterfactual Narrative Videos
Zheyu Huang, Zijing Shi, Haozhe Luo et al. (+4 more)
Recent advances in Large
Multimodal Models (LMMs) have greatly improved
video understanding, yet their ability to reason about human-centered social situations remains limited. Existing
benchmarks typically rely on
videos with a single observed trajectory, making it difficult to determine whether models truly understand social dynamics or merely exploit recurring narrative patterns. We introduce SocialReasonBench, a
video multiple-choice QA
benchmark for evaluating socially grounded
reasoning in scenarios derived from interactive narratives. Built from gameplay
videos of Detroit: Become Human, the
benchmark leverages branching storylines where player decisions lead to alternative social outcomes that can be checked against the game's own script, flowchart, and recorded branches. We develop a multi-
agent curation pipeline that localizes socially meaningful clips, grounds answer labels in game-state signals, and generates theory-guided questions with diagnostic distractors. SocialReasonBench covers seven
reasoning dimensions, including intent recognition, emotional empathy, moral dilemma, counterfactual
reasoning, and causal antecedent. Experiments on contemporary LMMs show that models perform reasonably well on basic social understanding but struggle with counterfactual and causal
reasoning. Further ablation and diagnostic error analyses reveal that models often depend on incomplete modality cues and fall into
reasoning traps such as visual shortcuts, highlighting a gap between observable event recognition and deeper
reasoning over latent social states.
Read the full abstract ▾
All authors Zheyu Huang, Zijing Shi, Haozhe Luo, Huadong Tang, Mingyu Liu, Meng Fang, Ling Chen
Computer Vision
SegWave: Wavelet-Driven Segmentation of Tampered Regions
Siddhi Pravin Lipare, Vishesh Kumar and Akshay Agarwal
Verifying image authenticity is increasingly difficult, posing serious risks across journalism, law enforcement, and political domains. Most existing forensic methods rely on high-level visual artifacts and treat frame detection as a simple binary task. To address this, we propose SegWave, a hybrid framework that jointly leverages spatial and frequency-domain cues for image tampering detection. SegWave integrates a
transformer-based architecture with the Discrete Wavelet Transform (DWT) to capture localized, multi-scale frequency inconsistencies indicative of manipulation. To further improve localization effectiveness, we introduce an Adaptive Sub-band
Attention module (ASA) that dynamically highlights the informative high-frequency wavelet components. Extensive experiments on multiple
benchmark datasets demonstrate that SegWave consistently outperforms state-of-the-art tampering detection methods in challenging evaluation settings.
Read the full abstract ▾
Computer Vision
RailSyn: Diagnosis-Guided Image Generation for Traceable Data Completion in Railway Foreign Object Detection
Quan Hao, Chenxi Zhang, Ziyang Tao et al. (+6 more)
Railway foreign object detection (RFOD) is critical to safe railway operation, yet scarce real positive samples incompletely represent task-relevant variations in object scale, intrusion relation, railway scene, illumination, and adverse weather. Existing synthetic augmentation can improve RFOD detection, but its gains lack an explicit account of the task-relevant deficiencies complemented by the generated data. We therefore introduce RailSyn, a diagnosis-guided framework comprising a real-referenced Inspector and a requirement-aligned Generator. The Inspector constructs a variable-radius empirical cover from finite real observations to localize candidate completion regions and profile synthetic pools. The resulting audit identifies railway-context, intrusion-semantic, and visual-consistency requirements; the Generator addresses them through domain adaptation,
agent-planned placement and physical contact relations, and plan-consistent conditional refinement. Using the Inspector, we further trace representation-space changes across generation variants; the complete system attains a local-shell occupation of $C_{gap}$ to 13.64%, which measures generated coverage of real-derived completion regions. Extensive experiments show AP50–95 gains of up to 4.9 points and consistent improvements across nine mainstream detectors, demonstrating broad cross-architecture utility.
Read the full abstract ▾
All authors Quan Hao, Chenxi Zhang, Ziyang Tao, Yuyuan Zhou, Yudong Wang, Rui Shi, Lechuan Xu, Changhao Liu, Liguo Zhang
Computer Vision
VisLens: Single-Pass Interpretable Visual Search for Multimodal LLMs
Jingyi He, Sanghwan Kim and Zeynep Akata
Multimodal large
language models (MLLMs) struggle with fine-grained Visual Search, the task of locating small or rare objects in high-resolution images. Existing remedies fall into two families: (1) Training-free methods based on
attention or confidence scores are accurate but slow, since they require multiple MLLM queries per example. (2) Reinforcement Learning (RL) trained tool-use models are faster at inference but opaque, since their tool calls remain uncontrollable and hard to interpret. To overcome this, we propose VisLens (Visual Focus via Logit Lens), a Visual Search method built on the logit lens, which decodes the semantics held in a hidden state by projecting it through the LLM head. VisLens further uses a lightweight tuned-lens that maps early hidden states into the final hidden state space, so visual
tokens can be read out from early layers. These
tokens are matched to target words in the query to generate a crop of the relevant region, which is fed back in alongside the original image to produce the final answer. The whole process, from decoding to the final answer, completes in a single forward pass without repeated queries. VisLens matches or exceeds prior baselines while delivering a substantial latency advantage, running $8.5$–$9.9\times$ faster than Thyme and up to $22.2\times$ faster than training-free multi-pass search methods.
Read the full abstract ▾
Computer Vision
TUE-Detector: A Tool-Using Expert MLLM-Based Detector for AI-Generated Videos
Yichen Wu, Haoxuan Qu, Yongxing Dai et al. (+5 more)
AI-generated
video detection, which aims to distinguish AI-generated
videos from real ones, has recently received increasing research
attention. To perform this task reliably, a key challenge lies in accurately identifying subtle-yet-measurable unnatural artifacts. In this work, we address this challenge from a novel perspective of tool-mediated evidence discovery and propose Tool-Using Expert MLLM-based AI-generated
Video Detector (TUE-Detector), a novel framework for AI-generated
video detection. TUE-Detector trains a general MLLM into a task-tailored tool-using expert detector that learns to invoke suitable tools, collect concrete evidence of unnaturalness, and reason over the evidence for reliable detection. Meanwhile, TUE-Detector further introduces novel designs to equip the expert detector with high-quality and suitable tools. Extensive experiments demonstrate the effectiveness of our framework.
Read the full abstract ▾
All authors Yichen Wu, Haoxuan Qu, Yongxing Dai, Yan Bai, Yihang Lou, Yuqi Lin, Hossein Rahmani, Jun Liu
Computer Vision
Can Video World Models Track Unobserved World States?
Joonghyuk Shin, Yicong Hong, Jaesik Park et al. (+1 more)
Video world models are increasingly used as simulators, but visual fidelity alone does not show that a model maintains the hidden state of the world. We examine this difference with an action-conditioned
video Shell Game, a visual analogue of $S_5$ state tracking that separates visual rendering from compositing the unobserved world state. Trained on 5-swap chains, standard backbones (e.g., bidirectional and autoregressive
Transformers, Mamba, and linear
attention) render plausible
videos and predict the correct ball location up to 5 swaps. However, they fail to learn the rule and generalize to longer swap chains, even with more denoising steps. As the pixel-based
diffusion loss does not force the generated frames to hold the unseen ball position, output
tokens cannot carry it, and the state has to live within the architecture. In a causal
Transformer, this implicit state is an append-only KV cache, which is written once and never revised, so the model must re-compose the swaps at every chunk. Tracking $S_5$ this way requires depth to grow with sequence length, which no fixed-depth
Transformer provides. We study what enables learning the rule, and find that length generalization requires a revisable state carried across chunks and an update expressive enough to apply a swap. Linear
attention can achieve this by allowing negative transition
eigenvalues, and autoregressive
Transformers can do so with nonlinear TTT fast weights (e.g., SwiGLU) whose online updates change the feature map used to read their state. We further examine
Memory Maze and Block World, where the state is not fixed by the input action stream alone and must be corrected from observations or keeps changing out of view, and discuss the implications for building stateful
video world models.
Read the full abstract ▾
All authors Joonghyuk Shin, Yicong Hong, Jaesik Park, Xun Huang
Computer Vision
Failure or Drift? Evaluating Monocular SLAM under Synthetic and Real-World Corruptions
Abhay Skaria Thomas, Shashank Agnihotri and Margret Keuper
Visual SLAM is commonly evaluated on clean trajectories, although deployment failures are often caused by adverse weather, illumination, blur, and sensor artifacts. Controlled corruptions are attractive because they isolate such factors, but a synthetic stress test is useful only when it leads to the same engineering conclusion as the condition it is intended to approximate. This work examines that question for
monocular SLAM. We evaluate a classical feature-based system and two learned trackers under image-space, geometry-aware, and compound corruptions, and compare their behavior with adverse conditions from 4Seasons. Rather than reducing robustness to a single trajectory error, the evaluation separates explicit tracking failure from drift accumulated by methods that remain active. The results show that learned trackers largely replace catastrophic loss with sustained, and sometimes severe, drift. More importantly, the apparent ordering of the learned systems changes with the physical fidelity of the corruption: structured rain and fog proxies preserve the real-world ordering, whereas a simple illumination proxy does not. Code is available at: https://github.com/abhaythomas/master_thesis_vslamlab_robustness.
Read the full abstract ▾
Computer Vision
CANVAS: Consistency-Aware Navigation via Visual Adaptive Sampling for Long-Context Text-to-SVG Generation
Yichen Wu, Haoxuan Qu, Yihang Lou et al. (+2 more)
Autoregressive large models have recently advanced Text-to-SVG generation from simple icons to complex, long-context graphics, yet standard autoregressive decoding often fails to maintain global consistency across geometry, layout, occlusion, and composition. We introduce CANVAS (Consistency-Aware
Navigation via Visual Adaptive Sampling), a training-free, render-aware inference framework that combines power-sharpened trajectory likelihood with visual feedback from rendered futures and derives a stroke-wise
navigation rule. It effectively estimates each candidate stroke's future value under a limited generation and rendering budget and adaptively allocates samples according to candidate uncertainty, decision influence, and rollout cost. Experiments across multiple autoregressive SVG backbones and complementary
benchmarks demonstrate improvements in global consistency, which includes sound
geometric relationships, spatial layouts, occlusion ordering, and overall composition, without additional training, demonstrating the effectiveness and generalization ability of our framework.
Read the full abstract ▾
All authors Yichen Wu, Haoxuan Qu, Yihang Lou, Hossein Rahmani, Jun Liu
Computer Vision
UFPR-PEs: A Brazilian Face Recognition Benchmark with Self-Declared Race/Color Labels
Alexandre Diano, Bernardo Biesseck, Gabriel Polo et al. (+4 more)
While face recognition systems are widely deployed, ensuring their demographic reliability and robustness under uncontrolled visual conditions remains a critical challenge. To bridge this gap, we present UFPR-PEs, a
benchmark for face recognition bias evaluation using public
videos of elected Brazilian politicians annotated with official self-declared race/color categories. The
dataset adopts the Brazilian census taxonomy, including the parda category, which has no direct equivalent in the U.S.- or Europe-centric schemas commonly used in prior
benchmarks. Our
benchmark is built from compressed public
video and preserves difficult samples so that performance can be analyzed under realistic conditions. We describe the construction pipeline, report
dataset statistics, and evaluate face recognition performance across verification and (closed- and open-set) identification settings, including subgroup analysis by race/color and difficulty level. The results show that recognition performance varies substantially with image quality, and that subgroup gaps must be interpreted jointly with visual difficulty rather than in isolation. Overall, UFPR-PEs provides a reproducible and demographically grounded setting for studying face recognition bias under challenging public
video conditions.
Read the full abstract ▾
All authors Alexandre Diano, Bernardo Biesseck, Gabriel Polo, Vinicius Gregorio, Laura Lopes, Diego Addan, David Menotti
Computer Vision
InfraOcc: An Infrastructure Occupancy Benchmark with Static-to-Dynamic Reasoning
Lei Yang, Xiaokai Bai, Boqi Li et al. (+8 more)
Fixed-viewpoint infrastructure sensors repeatedly observe the same traffic space, making roadside 3D occupancy structurally different from ego-vehicle perception: a near-persistent static scaffold is overlaid with sparse, short-lived dynamic events. Existing occupancy
benchmarks and methods, however, are built around moving ego vehicles and neither measure nor exploit this structure, instead treating occupancy as flat one-shot voxel classification. We address this gap from both data and model perspectives. We build InfraOcc, to our knowledge, the first real-world infrastructure-side semantic occupancy
benchmark, with dense voxel annotations for 290 multi-modal sequences in a fixed roadside frame, a static-dynamic decoupled annotation pipeline, unified camera-only,
LiDAR-only, and multi-modal evaluation, and diagnostics for static and dynamic occupancy. InfraOcc shows that static infrastructure fills 97.3% of occupied voxels and persists across frames, whereas dynamic participants have a median occupied-frame ratio of only 1.8% per location, revealing a structural static-dynamic asymmetry beyond semantic long-tailedness. We further propose ProSD-Occ, which reformulates occupancy as progressive static-to-dynamic evidence
reasoning: it explains persistent layout, exposes residual dynamic evidence under static-confidence guidance, and recomposes static, dynamic, and free-space evidence into a unified field. ProSD-Occ ranks first in overall, dynamic, static, and
geometric occupancy on every track, e.g., a 23.5% relative camera-only dynamic-mIoU gain over the strongest baseline and 65.87 multi-modal overall mIoU, establishing fixed-viewpoint roadside occupancy as a distinct problem with its own
reasoning paradigm. The
benchmark and code will be publicly available at https://github.com/yanglei18/InfraOcc
Read the full abstract ▾
All authors Lei Yang, Xiaokai Bai, Boqi Li, Chunmian Lin, Li Wang, Ziying Song, Jiahuan Zhang, Enhui Ma, Haibao Yu, Jiaqi Ma, Kaicheng Yu
Computer Vision
APT: Anchor-aligned Perturbations for Tamper Localization in Fully Regenerated Images
Suhyeon Ha, Woo Jae Kim, Joonsung Jeon et al. (+2 more)
Proactive tamper localization embeds an imperceptible signal into an image prior to distribution, enabling pixel-level manipulation detection. Existing methods assume a spliced (SP) setting, where synthesized regions are composited onto the original background, leaving embedded signals intact. However, real-world
diffusion-based inpainting operates in a fully regenerated (FR) setting, where the entire image undergoes denoising, disrupting background signals and rendering existing frameworks ineffective. We propose APT, a semi-fragile latent-space perturbation that embeds a dense, vector-wise localization signal. By aligning each spatial feature vector toward a fixed anchor direction, APT localizes tampering via the
alignment disparity between synthesized foreground and anchor-aligned background features after inpainting. The proposed hard negative mining loss and noisy perturbation branch further enforce uniform
alignment. Experiments on COCO demonstrate that APT achieves an FR IoU of 0.92, outperforming the strongest baseline (WAM, 0.84), while existing methods collapse to near-random performance (AUC 0.5), establishing APT as a practical forensic framework generalizable across tampering types unknown at test time.
Read the full abstract ▾
All authors Suhyeon Ha, Woo Jae Kim, Joonsung Jeon, Sooel Son, Sung-eui Yoon
Computer Vision
Joonki Min, Chaeyun Kim, Hyungwook Choi et al. (+4 more)
Multimodal Large
Language Models (MLLMs) are increasingly deployed in multi-image scenarios requiring complex
reasoning across visual contexts. However, current MLLMs remain fundamentally limited by object
hallucination-generating plausible yet factually inconsistent descriptions about objects. Existing
benchmarks, designed primarily for single-image settings or providing only high-level multi-image assessments, cannot systematically diagnose how visual complexity and
reasoning demands trigger
hallucination. To address this gap, we introduce MIOH, a fine-grained multi-image object
hallucination benchmark that systematically evaluates object
hallucination across four foundational tasks (existence, counting, attribute, position) through three multi-image
reasoning patterns (comprehensive, comparative, selective) under three controlled
adversarial pressures (visual context scale, perceptual difficulty, contextual bias). Through evaluation of 29 models, we reveal that even state-of-the-art systems like GPT-5 and Gemini-2.5-Pro exhibit distinct failure patterns across different
reasoning patterns and tasks. Our evaluation reveals that
hallucination stems not merely from perceptual failures but from integration-stage limitations when maintaining object representations across multiple images. MIOH provides a controlled framework for analyzing multi-image object
hallucination and serves as a critical evaluation tool for developing more reliable
multimodal AI systems.
Read the full abstract ▾
All authors Joonki Min, Chaeyun Kim, Hyungwook Choi, Yejin Kim, Kihyun Kim, Yohan Jo, Joonseok Lee
Computer Vision · Cross-listed from cs.CL
Where Identity Lives: Localized, Retain-Free Identity Unlearning in Multimodal Large Language Models
Kangwook Ko, Jaehyuk Jang, Wonjun Lee et al. (+2 more)
Removing a specific individual's information from
multimodal large
language models (MLLMs) is often needed after deployment, but existing methods rely on a retain set, which is hardest to obtain at that point, and rebuilding it recreates the
privacy exposure that unlearning aims to remove. Forgetting from the forget set alone instead damages the shared visual-
language computation, harming perception. We cast retain-free unlearning as a localization problem: causal tracing, weight transplant, and Fisher overlap all point to early-to-mid decoder MLPs as the layers where identity information is stored and, unlike other module families, can be modified without substantially disrupting
vision. We turn this into Pathway-Aware Visual-attribute Anchoring (PAVA), which confines updates to these layers and pairs a forget loss with a visual-attribute anchor that preserves image-grounded behavior by distilling the model's own pre-unlearning answers from the forget images alone. On MLLMU-Bench and ReMem, PAVA gives the strongest forget-retain trade-off among forget-set-only methods and remains competitive with retain-based baselines.
Read the full abstract ▾
All authors Kangwook Ko, Jaehyuk Jang, Wonjun Lee, Hee-Seon Kim, Changick Kim
Computer Vision
Quantum-Grassmann-Plucker Token Mixing for Deep Learning-Based Post-Disaster Damage Assessment
Kooroush Farahkhah, Umut Lagap, Taha Rezaei et al. (+1 more)
Timely post-disaster building damage assessment from satellite imagery is a critical engineering decision support task, yet it remains constrained by class imbalance, ambiguous intermediate damage states, and limited cross-event transferability. This study presents, to our knowledge, the first application of Grassmann-Plucker (GP)
token mixing to computer
vision and introduces two extensions for image classification: the
Quantum-inspired Grassmann-Plucker (QGP) head and the Hybrid
Quantum Machine Learning Grassmann-Plucker (HQML-GP) head. The GP head represents multiscale relationships among image patch
tokens by encoding subspaces formed by
token pairs with Plucker coordinates; QGP enriches these coordinates with amplitude-derived probability features, whereas HQML-GP incorporates expectation values generated by a simulated
quantum circuit into the
geometric token representation. Paired pre- and post-event image patches from the xBD tornado
dataset were processed using a frozen six-channel
Vision Transformer base encoder with 16 x 16-pixel patches. The three GP-based heads were compared with multilayer perceptron and
Transformer baselines under identical training, checkpoint selection, and evaluation protocols. Joplin and Moore tornado samples were used for model development and seen-event testing, while Tuscaloosa was reserved for unseen-event evaluation. QGP led both test sets in accuracy and macro-F1: 83.46% and 64.50% for the seen events, and 66.45% and 52.70% for the unseen event. Although HQML-GP obtained the highest validation macro-F1 of 65.63%, it did not surpass QGP on either test set and required substantially more training time per epoch. These results establish GP
token mixing as a competitive
attention-free alternative to conventional
Transformer-based
token mixing for paired satellite image damage classification.
Read the full abstract ▾
All authors Kooroush Farahkhah, Umut Lagap, Taha Rezaei, Saman Ghaffarian
Computer Vision
Cost-efficient Active Learning for Referring Image Segmentation and Grounding
Junbeom Hong, Seonghoon Yu, Hyung Rok Jung et al. (+2 more)
Collecting natural-
language referring expressions along with region annotations, such as masks or boxes, is a major bottleneck in visual grounding (VG), as annotators must write descriptions that distinguish target regions from visually similar ones. We tackle this by formulating active learning (AL) for VG under the realistic setting where only raw images are available without accompanying text. Since ground-truth text is unavailable, sample selection must estimate which images contain ambiguous regions that would require discriminative referring expressions. To address this, we generate auxiliary region-text pairs using foundation models, and introduce Referred Region Ambiguity, a new acquisition function that measures whether the model's confidence collapses onto a single region or disperses across multiple candidates. It allows our method to prioritize images with strong cross-region competition, which are more informative due to their visual ambiguity. We also design a referring-expression annotation interface that helps annotators quickly focus on writing discriminative
language with a few clicks. Experiments on RIS and REC
benchmarks show that our AL framework consistently outperforms several AL baselines, while a user study shows up to 1.6X faster description labeling of ours.
Read the full abstract ▾
All authors Junbeom Hong, Seonghoon Yu, Hyung Rok Jung, Sundong Kim, Jeany Son
Computer Vision
Keno Moenck and Thorsten Schüppstuhl
Transformer-based decoders for 3D instance
segmentation typically commit to a fixed number of queries and positional modeling calibrated on the training distribution rather than on the scene at hand. Indoor scans vary widely in spatial extent and object count, so a fixed query set over-initializes small scenes and under-initializes large ones, while learned absolute and relative encodings are bound to the training scenes' extents and can saturate. We present AQ3D, which is designed to handle scenes of various sizes during training and inference. Queries are instantiated at a fixed ratio of the scene's superpoints, forming an overcomplete set whose background rejection is entirely left to the decoder. Positional information is encoded using 3D RoPE over quantized metric coordinates, replacing learned bounded lookup tables of prior decoders. Further, we improve the decoder itself by using attribution-based superpoint pooling, a mask refinement branch, and a cosine classifier for background rejection. Experiments show our method sets a new state-of-the-art on validation and hidden test splits across the
datasets ScanNetV2, ScanNet200, and ScanNet++V2 among decoder methods trained without additional data augmentation. Code is available at \href{https://github.com/kenomo/aq3d}{github.com/kenomo/aq3d}.
Read the full abstract ▾
Computer Vision
RealCAD: Towards Real-World Image-to-CAD Reconstruction under Domain Shift and Parameter Bias
Yihe Sun, Ziyu Lu, Kaihua Tang et al. (+1 more)
Reconstructing editable Computer-Aided Design (CAD) models from images is essential for downstream modification, manufacturing, and design reuse. However, existing image-to-CAD methods are developed predominantly on synthetic renderings and face two coupled obstacles: a substantial appearance domain gap between synthetic and real images, and a previously overlooked parameter bias in widely used CAD data. We show that the local normalization adopted by DeepCAD concentrates several
geometric parameters around a few discrete values while encoding substantial information in a single scale factor. Consequently, a model can achieve deceptively high parameter accuracy by exploiting these frequent values rather than inferring geometry from the input image. In this paper, we propose RealCAD, a unified framework that addresses these limitations at the representation, image, and feature levels. At the representation level, we redistribute scale information to the corresponding
geometric parameters, producing less concentrated parameter distributions in a shared scale space. At the image level, geometry-constrained translation converts synthetic renderings toward the real-image domain while conditioning on object contours. At the feature level, a multi-positive contrastive objective aligns representations of the same CAD model across viewpoints and image domains, enabling CAD sequence prediction from each individual view. We further introduce OpenRealCAD, comprising four-view photographs of 392 3D-printed objects paired with ground-truth command sequences. Experiments show that the revised representation substantially reduces the accuracy attainable from parameter-frequency priors, making parameter accuracy a more reliable measure of image-conditioned
geometric inference. RealCAD further improves real-domain command and parameter accuracy, while retaining competitive synthetic-domain performance.
Read the full abstract ▾
All authors Yihe Sun, Ziyu Lu, Kaihua Tang, Xian-Sheng Hua
Computer Vision
OCR-Based Field Extraction for Archaeological Pottery Metadata: The CENTURIA Dataset
Gissu Valentina Naghavi, Dominik Hagmann, Martin Kampel et al. (+1 more)
Pottery is a primary source for reconstructing the chronological and economic dimensions of past societies. Archaeologists often document ceramic finds through technical drawings and handwritten metadata. This metadata is critical for dating, provenance attribution, and cross-site comparison, but remains inaccessible to computational analysis, requiring manual transcription of every record. We investigate whether state-of-the-art document analysis models can address this task, and introduce CENTURIA, a
dataset of 507 pottery records from the Roman site of Carnuntum, providing transcriptions, bounding boxes, and structured field-level labels across seven metadata categories. Benchmarking five OCR models reveals a substantial domain gap: zero-shot transcription error reaches 15-32% SpACER-M, far exceeding rates on printed archival documents, with domain-specific fields recovered in fewer than 3% of cases. LoRA fine-tuning on just 57 samples, reflecting a realistic archival annotation budget, closes this gap, reducing transcription error to below 1.5% and recovering overall field-level accuracy above 87%. Our results show that a small expert-validated fine-tuning set suffices to convert handwritten pottery documentation into structured, searchable metadata ready for archaeological databases.
Read the full abstract ▾
All authors Gissu Valentina Naghavi, Dominik Hagmann, Martin Kampel, Irene Ballester
Computer Vision
DiffSAC: Diffusion-guided Sampling for Consensus-based Robust Estimation
Chang Nie, Guangming Wang, Zhe Liu et al. (+1 more)
Robust estimation is a core computer
vision task frequently tackled using sample consensus. However, traditional methods suffer from inefficient sampling as they struggle to identify effective minimum sets before hypothesis evaluation. To address these challenges, we propose a novel
Diffusion-guided Sampling for Consensus-based Robust Estimation (DiffSAC) framework. DiffSAC introduces a
diffusion model to learn the distribution of effective minimum sets. It refines the confidence for each data point, indicating whether it belongs to a good minimum set, rather than ranking the data points as in previous work. This significantly reduces the need to process numerous bad sets. To constrain the refinement direction,
geometric features are incorporated as conditions within our
diffusion model. Consequently, DiffSAC outputs a small number of high-quality minimum sets, enabling identification of the best hypothesis via consensus evaluation. Notably, compared to previous works requiring evaluating over ten thousand hypotheses, DiffSAC achieves state-of-the-art performance with only dozens, significantly boosting efficiency. Extensive experiments across five classic computer
vision tasks demonstrate the superiority of DiffSAC. The
diffusion model's sampling accelerators enable real-time operation, and DiffSAC can be used as a plug-and-play module to improve existing sample consensus methods.
Read the full abstract ▾
All authors Chang Nie, Guangming Wang, Zhe Liu, Hesheng Wang
Computer Vision
Learning Compositional Spatio-Temporal Video Grounding with Synthetic Curriculum
Xingjian Wang, Shijian Wang, Yibo Wang et al. (+4 more)
Despite the impressive progress of recent MLLMs on spatio-temporal
video grounding (STVG), existing evaluations and training data focus primarily on simple queries. They largely overlook the compositional queries prevalent in real-world scenarios, where a target must be disambiguated by jointly
reasoning about its attributes and relations to other entities. To bridge this gap, we propose Compositional Spatio-Temporal
Video Grounding (CompSTVG), a task that requires models to process complex textual queries where every intertwined attribute and relational cue is essential for disambiguation. To facilitate this task at scale, we build a synthetic data engine that leverages a spatio-temporal scene graph as a difficulty measure and casts difficulty-controlled query synthesis as a constraint programming problem, producing difficulty-graded data for both evaluation and training. Built on this engine, we introduce STVG-CompBench, a
benchmark stratified by explicit difficulty levels that jointly capture temporal complexity and spatial interference. Evaluating 11 representative STVG models on STVG-CompBench reveals that current models perform poorly on compositional queries, exhibiting a sharp performance drop that is typically obscured by overall
dataset-level averages. We further construct synthetic training data and propose CurrSTVG, a curriculum reinforcement learning framework that delivers consistent gains, with the largest improvements observed on the most challenging compositional queries.
Read the full abstract ▾
All authors Xingjian Wang, Shijian Wang, Yibo Wang, Zihao Yu, Runhao Fu, Xuelian Cheng, Zongyuan Ge
Computer Vision
Modality Disentangled Learning for Incomplete Multimodal Emotion Recognition: A Primitive Memory Distillation Perspective
Jiaqi Zhang, Zheng Pang, Mengting Li et al. (+9 more)
Multimodal Emotion Recognition (MER) systems often suffer from missing modalities in real-world scenarios. Existing methods usually generate, align, or distill missing modalities as a whole, overlooking the heterogeneous nature of the information carried by each modality. Such holistic treatment mixes inferable shared semantics with uncertain modality-specific details, yielding unstable representations and degrading robustness. To address this issue, we propose the Primitive
Memory Distillation (PriMD) framework. Unlike existing methods, PriMD takes an intra-modal perspective and focuses on how different types of information within a modality differ in recoverability within each modality. PriMD first disentangles cross-modal shared semantics from modality-specific representations, and then discretizes the latter into learnable semantic primitives to construct modality-specific
memory banks. When modalities are missing, PriMD is a teacher-student framework that the student model uses the shared semantics of available modalities as queries to dynamically retrieve primitives. It compensates for missing modality-specific information within a constrained
memory space and aligns with the teacher model. Extensive experiments on IEMOCAP, CMU-MOSI, and CMU-MOSEI demonstrate that PriMD achieves state-of-the-art performance and consistently stronger robustness across a wide range of missing-modality settings, while mitigating the instability caused by holistic feature inference. Our code and project website are available at https://github.com/JiaqiZhang-Sengoku/PriMD and https://jiaqizhang-sengoku.github.io/PriMD/, respectively.
Read the full abstract ▾
All authors Jiaqi Zhang, Zheng Pang, Mengting Li, Yiqi Wang, Guangyuan Dong, Chao Xue, Yusen Wu, Zihao Li, Huy Phan, Sicheng Zhao, Björn W. Schuller, Jiachen Luo
Computer Vision · Cross-listed from cs.AI
GarmentWeaver: Schema-Aware Structured Synthesis for Multimodal Sewing Patterns
Yinwen Lu, Weihao Luo and Yueqi Zhong
Multimodal Sewing pattern generation aims to infer executable sewing patterns from design cues such as sketches and textual descriptions. As an interpretable and simulation-compatible representation, sewing patterns are particularly valuable for digital garment creation. However, existing methods often model garment specifications as flat long sequences, which entangles garment structure with detailed parameters and leads to redundant components, inaccurate local details, and poor simulation compatibility. In this paper, we present GarmentWeaver, a schema-aware framework for
multimodal Sewing pattern generation. GarmentWeaver constructs compact hierarchical targets by activating garment-relevant structural branches and predicts executable Sewing patterns in a structured manner. Specifically, we introduce a schema-aware target construction strategy, build the generator on top of a pretrained
vision-
language model for
multimodal garment understanding, and impose feasibility-aware regularization to encourage structurally valid and simulation-compatible outputs. Extensive experiments show that GarmentWeaver produces more accurate and more executable sewing patterns than strong baselines, while also yielding better simulation results. These findings demonstrate the effectiveness of schema-aware structured generation for reliable
multimodal Sewing pattern prediction.
Read the full abstract ▾
Computer Vision
Minerals in the Wild: A Hyperspectral-XRF Dataset for Elemental Composition Estimation
Eleftheria Tetoula-Tsonga, George Arvanitakis and Theodoros Giannakas
Rapid mineral characterization is essential for applications ranging from mineral exploration to industrial ore processing. To this end, Hyperspectral Imaging (HSI) has emerged as a promising sensing modality thanks to its fine spectral resolution, enabling mineral discrimination in both close-range and remote sensing settings. However, the scarcity of publicly available
datasets with reliable ground-truth labels hinders the development and evaluation of HSI-based mineral identification methods. We release Minerals in the Wild, a multi-purpose
dataset comprising 1,132 rock specimens collected across Europe. For each specimen, we provide an HSI acquisition together with an elemental characterization obtained via an XRF sensor. We define the task of elemental characterization on our
dataset and propose a
pruning mechanism that removes distant signatures from the USGS dictionary prior to a convex optimization approach for matching HSI pixels with USGS spectral signatures. Finally, we empirically show that our approach outperforms simpler baselines.
Read the full abstract ▾