Computer Vision · Cross-listed from cs.LG
MT-ProtBERT: Multi-task Learning ProtBERT for Intrinsically Disordered Proteins Classification with Scarce Data
Jian Sun, Kingshuk Ghosh, Lilianna Houston et al. (+1 more)
Intrinsically disordered
proteins (IDPs) differ from folded
proteins in that they are dynamic, lack a stable three-dimensional conformation, and have low sequence similarity between similar
proteins. The conformational heterogeneity of IDPs - while beneficial for their diverse functions - limits the use of traditional experimental tools to determine their conformation. The experimental difficulty, along with low sequence similarity, results in data scarcity, and makes it difficult to classify/detect IDPs that are similar or dissimilar, a task relevant to understand biology and
evolution. We address this challenge using Multi-task ProtBERT (MT-ProtBERT), a multi-task extension of ProtBERT tailored for low-data regimes. MT-ProtBERT integrates Dynamic Window Masking, a Multi-Scale 1D Convolutional classifier (MS-Conv1D), and auxiliary objectives that jointly optimize masked
language modeling and biochemistry-informed tasks. We evaluate this framework on two tasks under limited data: (i) phosphorylation site prediction (S/T/Y) in short sequences and small
datasets, and (ii)
protein compaction prediction on two small
datasets (684 and 530 sequences), including sequences comparable in length to typical disordered regions. MT-ProtBERT consistently outperforms PARROT, an RNN-based IDP-specific model, across all tasks. These results demonstrate that combining self-supervised and biochemistry-informed tasks, and multi-scale learning enables robust modeling of unstructured
proteins under data scarcity.
Read the full abstract ▾
All authors Jian Sun, Kingshuk Ghosh, Lilianna Houston, Mohammad H. Mahoor
Computer Vision
MirrorDistill: Illumination-Aware Latent Distillation for Efficient Low-Light Restoration
Farida Mohsen, Tala Zaim, Nurul Izni Rusli et al. (+3 more)
Low-light image enhancement (LLIE) is an im- portant component of visual sensing systems operating under degraded illumination, including nighttime surveillance, au- tonomous
navigation, remote sensing, and inspection in poorly lit industrial environments. Most LLIE methods rely on output- level reconstruction losses that supervise only the final restored image, leaving the intermediate feature recovery process weakly constrained. This paper proposes MirrorDistill, an illumination- aware latent
distillation framework that links the low-light and clean domains through feature mirroring. During training, a shared encoder and an exponential-moving-average teacher decoder process the clean reference image to generate clean- domain latent targets. These targets supervise the low-light student at two levels: raw encoder features and standardized multi-scale decoder projections. The
alignment is applied layer by layer, while a proposed illumination-aware weighting scheme gives greater emphasis to underexposed regions. The teacher and reference branches are used only during training, so inference requires only the lightweight student encoder-decoder and in- troduces no teacher-side computational cost. Under evaluation on the standard LOL
benchmarks, MirrorDistill outperforms the state-of-the-art methods on the real-captured LOL-v2-Real set, while having the lowest compute complexity (GMACs) and while remaining competitive on the LOL-v1 and LOL-v2-Synthetic
datasets. Ablation studies further show the contributions of the encoder mirror, decoder mirror, and illumination-aware weighting. Finally, we release our code as open-source for the benefit of future research.
Read the full abstract ▾
All authors Farida Mohsen, Tala Zaim, Nurul Izni Rusli, Ali Al-Zawqari, Ali Safa, Samir Brahim Belhaouari
Computer Vision
Uncertainty-Aware 3D Residual Wavelet Diffusion for Ultra Low-Field MRI Super-Resolution
Rui W. Yeow, Millie Beament, Fred Dick et al. (+6 more)
Ultra low-field MRI expands global access to neuroimaging but produces scans with low signal-to-noise ratio, reduced contrast, and thick slices. While regression-based super-resolution can recover anatomical detail for
segmentation, it returns a single deterministic estimate that gives no indication of regions where the low-field input leaves anatomy underdetermined. Generative
diffusion models offer an alternative by sampling the posterior distribution of plausible high-field images, quantifying this anatomical ambiguity. However, applying them to 3D whole-
brain MRI is restricted by
memory bottlenecks, slow sampling, and scanner domain shifts. We propose a 3D residual wavelet
diffusion model that combines three ideas to overcome these hurdles. A lossless wavelet reparameterisation shrinks the spatial grid to fit a whole
brain on a single GPU, residual shifting accelerates sampling by starting from the low-field input, and domain randomisation promotes scanner generalisation without paired training data. As the high-field reference is not a voxel-aligned ground truth, we evaluate downstream volumetric agreement. On a healthy cohort (n=19) imaged at 0.064T and 3T, our method matches a leading general-purpose regression approach in volumetric accuracy while additionally generating per-voxel uncertainty maps highlighting underdetermined regions. Furthermore, on a pilot
dataset (n=11) of participants with cognitive impairment, disease-relevant atrophy is preserved rather than normalised towards a healthy prior. Our framework brings whole-
brain posterior sampling to low-field super-resolution without sacrificing volumetric accuracy.
Read the full abstract ▾
All authors Rui W. Yeow, Millie Beament, Fred Dick, Raha Razin, Martina Bocchetta, David L. Thomas, Henry F. J. Tregidgo, Daniel C. Alexander, James H. Cole
Computer Vision · Cross-listed from cs.RO
Beyond the Flat Seafloor: A Closed-Form Two-View Constraint to Aid Sidescan Sonar Reconstruction
Kalin Norman and Joshua G. Mangelson
Sidescan sonar is a common sensor for both manned and autonomous marine exploration and mapping, yet very few methods build upon or exploit the
geometric projection model of the sensor. As sidescan sonar is limited to a 1D range measurement, many approximations are frequently used, including the long-standing assumption of a flat seafloor. Rather than make similar approximations, this paper focuses on a multi-view geometry based approach and formalizes a two-view
geometric constraint and proves that a shared feature is constrained to a locus within the intersection of a sphere and a plane. In addition, we characterize what governs the size of the ambiguity locus through Monte Carlo simulation that is grounded in real aperture and mounting geometry for both a surface vessel and an
underwater vehicle. We translate additional simulations of relative trajectories for both vehicle platforms into concrete survey-planning guidance. Our results show that locus length is strongly governed by elevation misalignment, and peaks at a moderate oblique crossing angle of approximately 20 degrees, with minimal locus lengths obtained at near parallel and anti-parallel passes.
Read the full abstract ▾
Computer Vision
RULER: Instance-aware Rubric Rewards for SVG Generation
Hangyu Ran, Yuhao Zheng, Yingying Zhang et al. (+2 more)
Generating Scalable Vector Graphics (SVG) code from natural-
language instructions is an open-ended task without absolute visual ground truth, leaving both evaluation and policy optimization without a faithful signal. Scalar metrics (CLIP, Aesthetic) calibrated on natural images transfer poorly to stylized vector content, and reusing them as RL rewards triggers reward hacking. We address both limitations with rubric-based scoring. We first establish empirically that prompting a
vision-
language judge with a multi-axis rubric correlates with human judgments far better than scalar metrics, both across samples and within instructions. Building on this finding, we introduce RULER (Instance-aware Rubric Rewards for Reinforcement Learning), which converts each instruction into an instance-aware rubric of six items spanning semantic, visual, and stylistic axes; a judge VLM scores rendered rollouts item-by-item, and the weighted satisfactions form a fine-grained reward optimized via Group Relative Policy Optimization. Because the rubric is derived from text alone, RULER requires neither paired SVG ground truth nor human preference labels. On MMSVG-Illustration and MMSVG-Icon, RULER lifts the rubric score from 0.432/0.395 to 0.693/0.683, surpassing dedicated SVG specialists and matching the substantially larger DeepSeek-V3, with ablations identifying rubric design as the active lever for RL on open-ended SVG generation. The project page is available at https://hangyuran.github.io/RULER/.
Read the full abstract ▾
All authors Hangyu Ran, Yuhao Zheng, Yingying Zhang, Kevin Qinghong Lin, Han Peng
Computer Vision
ImIR: Image-Instruction Tuning for All-in-One Image Restoration
Süleyman Aslan, Görkay Aydemir, M{\i}sra Yavuz et al. (+5 more)
Degradations vary widely across images, so a practical restoration system has to handle many degradation types with one model. A recent and effective recipe adapts a large pretrained image-editing model to restoration using a small low-rank adapter with a text
prompt. We replace that
prompt with an instruction derived from the degraded image itself. The image reaches the editor through two paths: its structure comes from the model's VAE, and its semantic instruction comes from a lightweight
token mapper that shifts the degraded image's
vision-
language embedding toward the
embedding a clean image would produce. Because the instruction is a continuous vector, scaling it yields a family of valid restorations for tasks whose target is not unique, such as low-light enhancement. We adapt one Qwen-Image-Edit model to six tasks with a single adapter trained in about three hours on one GPU. The image instruction outperforms text conditioning under a matched comparison, and it supports task agnostic restoration without a degradation label, which the text variant does not.
Read the full abstract ▾
All authors Süleyman Aslan, Görkay Aydemir, M{\i}sra Yavuz, Yunus Bilge Kurt, Nasrin Rahimi, Ahmet Rasim Emirdağ{\i}, Burak Can Biner, M. Ak{\i}n Y{\i}lmaz
Computer Vision
Geometric and Semantic Coupling for Interaction Understanding in 3D Scenes
Hanyang Kong and Xingyi Yang
Understanding interaction in a 3D scene requires recovering movable parts, their motion, and where they can be operated. These quantities are related, and their predictions can inform one another. A closed cabinet door, for instance, reveals a movable surface but may leave the hinge side ambiguous; its handle helps resolve this ambiguity, while the part provides context for localizing and interpreting the small handle. Building on this observation, we present SEGMENT-SNAP, which combines
geometric and semantic evidence through part-handle coupling. Three independently trained predictors recover movable parts, dense handles, and part-associated handle proposals. We couple their outputs in two directions. For part motion, a training-free
geometric decoder fits predicted part surfaces under explicit physical priors and uses detected handles to select candidate hinge lines. For handle prediction, a part-conditioned branch proposes additional handles, while standalone part classes refine their rotation/translation labels, with dense-handle labels as a fallback. Each transfer is applied once, without iterative feedback. On the Articulate3D validation set, handle guidance raises motion-gated AP from 13.74 to 40.98 under fixed masks and axes. Additional handle proposals raise handle AP from 24.63 to 29.65, and full contextual class correction raises it to 30.99 in the reference configuration. Fixed-input controls, retraining ablations, learned-decoder comparisons, and paired visualizations together characterize the benefits and limits of this coupling. Our system also achieved first place in the Articulate3D Challenge.
Read the full abstract ▾
Computer Vision · Cross-listed from stat.ME
Calibration Count Reuse: Validity Does Not Determine Efficiency
Rudra Chopra
Calibration count reuse raises separate validity and efficiency questions. We give a validity criterion for general count-dependent nonconformity scores: transferring one count from another class to the scored class must not improve its conformity. A leave-self-out full conformal reference proves the criterion without requiring normalization or preservation of same-class score order. For a common separable transformation, universal exchangeable validity is equivalent to being nondecreasing in the count, provided $K\alpha \geq 1$; normalized multiplicative weights obey the complementary nonincreasing condition. Additive penalties are covered under the stated information restrictions. Efficiency has no parallel ordering: two iid constructions make the same valid rule improve or worsen expected size at unchanged coverage. An expanded 55-rule study finds no resolved advantage from selected live-count rules over uniform weights. Image studies identify undercoverage under iid resampling, including at numerical convergence. Separately, execution of the released Conf-OT pipeline on its DTD and Aircraft
benchmark subsets produces near-nominal median coverage under fixed stratified counts. The native results are reported separately from the iid analyses, without treating a
benchmark observation as a universal guarantee. The findings separate validity, classifier confidence, numerical convergence, and population-specific efficiency.
Read the full abstract ▾
Computer Vision
GameHorizon Suite: Multi-Horizon Data and Evaluation in Gameplay
Yiran Wang, Xingyilang Yin, Junfu Pu et al. (+12 more)
Modern
video games provide a measurable testbed for AI models, combining abilities of visual understanding, instruction decomposition, goal planning, and precise action control over multiple temporal horizons. Existing
datasets and
benchmarks, however, either cover a narrow range of games, lack
language instructions, or rely on high-variance online rollouts. To address these challenges, we introduce GameHorizon, a unified data and evaluation suite that measures gameplay capabilities at different horizons for diverse model families. GameHorizon Suite consists of three components. First, GameHorizon-Annotator is a scalable and automated annotation pipeline for multi-horizon instructions. Second, utilizing the pipeline, we construct GameHorizon-Data, the first large-scale AAA gameplay
dataset with temporally aligned
videos, player actions, and multi-horizon instructions. It comprises 5,000 hours of recordings from 21 games, collected by 100 human expert players. Third, we build GameHorizon-Bench with reproducible offline and stepwise online testing. The offline track enables reproducible evaluation using thousands of standardized questions organized into three primary tasks and a series of diagnostic variants, while the online track tests whether offline scores reflect actual gameplay capabilities and localizes failures to specific steps within long-horizon gameplay. Based on our GameHorizon Suite, we evaluate 47 models through more than one million model invocations, revealing a meaningful hierarchy of task difficulty and pronounced differences in model capabilities. Our work can provide a standardized yardstick for evaluating gameplay capabilities across horizons and model families. We will release our
dataset, annotator, and
benchmark to facilitate future research.
Read the full abstract ▾
All authors Yiran Wang, Xingyilang Yin, Junfu Pu, Guangzhi Wang, Kaifeng Li, Mingyu Ouyang, Huiqiang Sun, Lingen Li, Cheng Cheng, Wangbo Yu, Honghao Chen, Xiaodong Cun, Chi-Man Pun, Zhiguo Cao, Ying Shan
Computer Vision
VideoGen-Agent: Reinforcing Video Generation Agents
Binxu Li, Haoyi Duan, Yuhui Zhang et al. (+9 more)
Recent advances in
video generative models have enabled high-fidelity, temporally coherent
video generation. However, these models often struggle to satisfy
prompts requiring specialized knowledge, specific identities, physical consistency, or ordered events. In this paper, we present VideoGen-
Agent, a
multimodal agent trained through multitask agentic reinforcement learning to use external tools for
video generation. The
agent coordinates augmentation, generation, and verification tools through multi-turn interactions, using the
prompt and intermediate observations to guide its decisions. We train a shared policy on a category-balanced
dataset spanning six tasks. Supervised fine-tuning on teacher-generated trajectories establishes tool-use behavior, which is then refined through reinforcement learning. A category-aware hybrid reward evaluates tool-call validity, task-appropriate tool use, and generated
video quality. We further introduce VABench, a held-out
benchmark of 600
prompts covering procedural knowledge, single- and multi-entity identity preservation, physical consistency, scene composition, and multi-shot temporal structure. On VABench, VideoGen-
Agent improves over its base text-to-
video generator by 19.1 points, from 56.5 to 75.6. Upgrading the generation tools further raises the score to 86.1 without additional
agent training. Human raters prefer the upgraded configuration over the strongest standalone baseline in 84.3% of comparisons. These results support learning tool use across
video-generation tasks and show that the trained
agent can benefit from subsequent advances in generation tools. Project page: https://andyca111.github.io/VideoGen_
Agent/
Read the full abstract ▾
All authors Binxu Li, Haoyi Duan, Yuhui Zhang, Yaohui Zhang, Zihao Lin, Kaituo Feng, Suozhi Huang, Xiangyi Li, Yu Li, Chunyuan Li, Shilong Liu, Mengdi Wang
Computer Vision
WorldCrafter: Consistent Video World Model with Implicit 3D-aware Memory
Wangbo Yu, Kunhao Liu, Wenbo Hu et al. (+8 more)
Video world models enable interactive exploration of dynamic environments, yet struggle to respect prior observations over long horizons and across viewpoints. We present WorldCrafter, a
video world model that learns a camera-queryable implicit 3D-aware
memory for this purpose. The key insight is to let the requested viewpoint shape how multi-view evidence is compressed into the
video generator's limited
token budget. Trained jointly with the
video generator, a
memory encoder and pose-conditioned readout module integrate historical observations into a fixed set of target view-specific
tokens before denoising, without explicit depth-based correspondences. By combining this
memory with recent temporal context and few-step
distillation, WorldCrafter enables streaming scene exploration from a single input image or text
prompt. Experiments across static and dynamic scenes show substantial gains in long-horizon consistency and camera-control accuracy while preserving visual quality during minute-scale exploration.
Read the full abstract ▾
All authors Wangbo Yu, Kunhao Liu, Wenbo Hu, Shenghai Yuan, Chaoran Feng, Haiyang Zhou, Yukun Huang, Yiran Wang, Wang Zhao, Yingmin Luo, Ying Shan
Computer Vision
GAE: Learning a Geometry-Native Latent Space for 3D-Consistent World Generation
Jiahao Lu, Minghao Yin, Wenbo Hu et al. (+5 more)
We present a compact geometry-native latent space as a shared foundation for perception and generation. Visual generators can produce photorealistic frames without preserving a consistent 3D scene. We argue that this is not only a modeling problem but also a representation problem: generators typically evolve appearance-centric latents, while perception models recover geometry in a semantically rich space that encodes cross-view structure. Rather than adding geometry as another output, we reparameterize a geometry foundation model's features into a compact latent space for generation. We realize this shift with the geometry-native autoencoder (GAE), whose latent is jointly decodable to appearance, depth, cameras, and point maps. With this state, a standard conditional flow supports diverse generation tasks. In controlled comparisons that hold the generator and training protocol fixed, replacing the latent with GAE improves both visual quality and independently measured 3D coherence: FVD falls by $12.7\%$ and $23.1\%$ on RealEstate10K and DL3DV, and camera-trajectory error is halved on RealEstate10K. Together, these results show that the latent space is central to geometry-consistent generation and can serve as a shared interface between perception and generation.
Read the full abstract ▾
All authors Jiahao Lu, Minghao Yin, Wenbo Hu, Hengyu Liu, Wang Zhao, Sai-Kit Yeung, Ying Shan, Yuan Liu
Computer Vision · Cross-listed from cs.RO
DexTacWAM: A Visuo-Tactile World-Action Model for Dexterous Manipulation
Haoran Yuan, Zekai Wang, Boning Shao et al. (+4 more)
Dexterous manipulation depends on contact dynamics that are often only partially observable from
vision. Recent World-Action Models (WAMs) couple predictive
video world modeling with action generation, but remain largely
vision-centric and therefore cannot directly model these contact dynamics. We present DexTacWAM, a visuo-tactile WAM that encodes each fingertip independently, aggregates the resulting features through a finger- and pose-aware tactile compressor, and injects the tactile latent into a
video diffusion world model for joint visuo-tactile world modeling. Across six contact-rich dexterous manipulation tasks on a 22-DoF bimanual platform, DexTacWAM achieves the highest score on every task, averaging 70.6 versus 38.0 for the strongest baseline. Ablations attribute the gain to modeling contact
evolution as part of the predicted world state rather than tactile conditioning alone: removing tactile world modeling reduces the four-task mean from 74.7 to 26.6 while keeping the same tactile features and action expert. After four hours of tactile-encoder adaptation with a frozen pretrained
vision VAE, our continual
vision-to-touch learning extends the pretrained
video model to touch using roughly 100 demonstrations per task without tactile midtraining, while retaining visual prediction quality within 0.5 dB of
vision-only counterparts. The compressor retains 89.4% of pre-fusion contact recall while enabling 2.26x faster training and 1.29x faster inference. Together, these results show that pretrained
video priors can be extended to distributed multi-finger contact dynamics in a data- and compute-efficient manner.
Read the full abstract ▾
All authors Haoran Yuan, Zekai Wang, Boning Shao, Haoran Lu, Trevor Darrell, Ismini Lourentzou, Wei Zhan
Computer Vision
Anatomy-Decomposed Chest Computed Tomography (CT) Projections as Scalable Supervision for Bone Suppression in Chest Radiographs
Mrunmay Angaitkar, Piyush Kumar, Aarjav Satia et al. (+4 more)
Bone overlap can obscure abnormalities in chest radiographs, while scarce paired training data limit supervised bone suppression. We address this challenge with a digitally reconstructed radiograph (DRR) framework that converts chest computed
tomography (CT) into paired supervision for component suppression. A novel bone
segmentation algorithm enables CT decomposition into bone, non-lung soft-tissue, and lung components, which are projected separately. Their weighted combination yields synthetic radiographs with pixel-registered component images that sum exactly to the full DRR. Models trained on these data suppress bone or lung components by predicting the target component and recovering the remainder by subtraction, transferring to real radiographs without real paired training data. As an extension, their outputs on real radiographs provide target domains for unpaired, component-wise DRR translation, reducing the appearance gap while retaining anatomical details. Across multiple public
datasets, downstream detection experiments demonstrate the utility of bone suppression, with gains concentrated on abnormalities with substantial bone overlap. Compared with open-source DRR engines applied to the same CTs, our unmodified DRRs achieve comparable realism and preservation of label-relevant anatomy, while translated DRRs achieve the best Fréchet inception distance (FID), lung-field sharpness, and agreement with source-CT anatomy among the evaluated methods. Models and inference code: https://huggingface.co/qureaiorg/bone-suppression ; Translated projections: https://huggingface.co/
datasets/qureaiorg/ct2xr-projections.
Read the full abstract ▾
All authors Mrunmay Angaitkar, Piyush Kumar, Aarjav Satia, Pranav Rao, Ashish Mittal, Manoj Tadepalli, Preetham Putha
Computer Vision
Yongsheng Yu, Wei Xiong, Yichen Sheng et al. (+2 more)
Recent advances in pixel-space
diffusion models have narrowed the image quality gap with latent-space
diffusion, but still converge more slowly and lag behind in final image quality. We argue that a key reason is the lack of an explicit representation prior: unlike latent
diffusion, which usually denoises in a compact and structured latent space, pixel
diffusion needs to learn denoising-friendly representations and pixel generation simultaneously from raw RGB space. To address this problem, we propose PixelDiT2, an end-to-end pixel-space
diffusion model designed to decouple representation learning from pixel generation without introducing an autoencoder or latent reconstruction bottleneck. We propose representation grounding that uses a frozen pretrained
vision foundation model to provide explicit per-patch representation guidance throughout denoising, allowing the pixel
diffusion transformer to focus more on pixel generation. On ImageNet-256x256, PixelDiT2 achieves an FID of 1.46 after 600 epochs; at 512x512 resolution, PixelDiT2 achieves an FID of 1.48 after 680 epochs. Project page: https://pixeldit.github.io/pixeldit2/
Read the full abstract ▾
All authors Yongsheng Yu, Wei Xiong, Yichen Sheng, Shiqiu Liu, Jiebo Luo
Computer Vision
SLICEChat: Progressive In-Encoder Token Pruning for Whole-Slide Pathology Language Models
Ali Kerem Bozkurt, Baris Cem Bakay, Ibrahim Kulac et al. (+3 more)
Whole-slide pathology images (WSIs) contain gigapixel-scale visual content, creating a major scalability challenge for slide-level
multimodal large
language models (MLLMs). Existing approaches process thousands of patch
tokens and typically apply compression only after slide encoding, leaving
multimodal attention computationally expensive. We introduce SLICEChat, a slide-level MLLM that integrates progressive
token pruning within a hybrid Mamba–
Transformer slide encoder. Mamba layers enable efficient long-range propagation, while
Transformer layers preserve global interactions as the sequence is progressively shortened. Between stages,
language-supervised, region-aware
pruning removes spatially coherent low-utility regions under a controlled keep-rate schedule, producing compact slide representations before
multimodal fusion. On SlideBench VQA, SLICEChat achieves 79.84% accuracy on TCGA and 59.09% on BCNB cohorts, outperforming prior slide-level pathology MLLMs, and achieves the highest overall WSI-Bench metrics. It also provides competitive
memory usage and the inference latency among the evaluated models. These results demonstrate accurate and computationally efficient
multimodal reasoning over gigapixel WSIs.
Read the full abstract ▾
All authors Ali Kerem Bozkurt, Baris Cem Bakay, Ibrahim Kulac, Cigdem Gunduz-Demir, Erkut Erdem, Aykut Erdem
Computer Vision
Generating Chest X-Ray Counterfactuals by Specialising Foundation Image Models
Xiaodan Xing, Rajat R. Rasal, Julia A. Meister et al. (+3 more)
Counterfactual image generation answers questions about how a subject would have looked under retrospective, hypothetical scenarios. Recent methods have improved perceptual quality, identity preservation and faithfulness to an underlying causal model, but their adoption in healthcare is limited by scarce annotated data, distribution shift between
datasets, and mismatches between pretrained generative models and those required for counterfactual inference. We propose specialisation, a data and parameter-efficient framework for adapting pretrained, non-causal generative models into causal mechanisms under distribution shift. Based on this framework, we train a radiology counterfactual image generation model, called RadCF, using latent flow matching. We validate our approach on three chest X-ray
datasets spanning different
dataset shifts, data volumes, and counterfactual questions, associated with challenging, highly-localised interventions. Our results show that RadCF and specialisation improve counterfactual soundness over existing methods while being data and parameter efficient, and that the resulting counterfactuals can detect and mitigate shortcut learning in a downstream medical classifier. Code is available at https://github.com/GSK-AI/RadCF/.
Read the full abstract ▾
All authors Xiaodan Xing, Rajat R. Rasal, Julia A. Meister, Sara Ghorayeb, Galvin Khara, Jessica Schrouff
Computer Vision
SPHQuant: Efficient extreme low bit weight quantization for Vision-Language Models
Kewei Zhang, Zheng Chen, Haotong Qin et al. (+1 more)
Recent foundation models are moving toward native
multimodal Vision-
Language Models (VLMs), making VLMs a central form of next-generation foundation models. However, their large
language backbones make edge deployment difficult due to high
memory footprint and
memory-bound autoregressive decoding. Weight-only post-training quantization is a practical solution, but pushing VLMs to extreme low bit-widths remains challenging: existing rotation-free methods suffer from outliers at 2-3 bits, while rotation-based methods improve accuracy at the cost of additional runtime overhead. We propose SPHQuant, a rotation-free spherical weight-only quantization framework for VLMs. Instead of quantizing weights directly in Cartesian coordinates, SPHQuant decomposes each 8D weight vector into coordinate signs, radius, and a positive unit direction. This representation isolates outlier magnitude into the radius while keeping directions bounded and statistically regular. Based on this insight, SPHQuant allocates extra precision to the radius to mitigate accuracy degradation induced by outliers. It further uses a compact positive-direction codebook and fine-tunes codebook entries through angular parameterization to preserve the unit-sphere constraint. We also design a hardware-friendly GEMV kernel that keeps the direction codebook small enough for shared-
memory lookup and packs radial bits efficiently. Experiments show that SPHQuant matches the performance of state-of-the-art extreme low-bit quantization methods while improving decode throughput over QTIP by 30.3% on RTX A6000. Code will be released in https://github.com/Pushazf/SPHQuant.
Read the full abstract ▾
All authors Kewei Zhang, Zheng Chen, Haotong Qin, Yulun Zhang
Computer Vision
DTKDP: A Dual Teacher Knowledge Distillation and Pruning Framework for Lightweight Oriented SAR Ship Detection
Yuming Li, Fan Zhang and Alin M. Achim
Two-stage oriented detectors achieve high localization accuracy in synthetic aperture radar (SAR) ship detection, but their large backbones, feature pyramids, proposal modules, and heavy region of interest (RoI) heads hinder deployment. Existing lightweight SAR ship detectors typically use one-stage frameworks that lack proposal-level refinement for precise rotated localization. This paper presents a dual-teacher knowledge
distillation and
pruning (DTKDP) framework for lightweight oriented SAR ship detection. DTKDP introduces learnable gates into convolutional, normalization, and linear layers to prune convolutional channels and RoI-head
neurons. Rotated proposal
alignment (RPA) distills teacher and student predictions in a shared teacher-generated rotated proposal space, while a dual-teacher scheme combines classification and regression guidance from a homogeneous main teacher with complementary classification cues from a heterogeneous auxiliary teacher. Experiments on the SAR Ship Detection
Dataset (SSDD) and Rotated Ship Detection
Dataset in SAR Images (RSDD-SAR) show that DTKDP reduces the parameters of Oriented Region-based Convolutional Neural Network (Oriented R-CNN) and RoI
Transformer equipped with ResNet-50 backbones by 87.5-91.8% and their floating-point operations (FLOPs) by 75.6-79.9%. In terms of average precision (AP) and mean average precision (mAP), the resulting Oriented R-CNN-slim and RoI
Transformer-slim retain accuracy close to their full-scale counterparts. Relative changes across $\mathrm{AP}_{50}$, $\mathrm{AP}_{75}$, $\mathrm{mAP}_{50:75}$, and $\mathrm{mAP}_{50:95}$ range from a 2.38% decrease to a 0.65% improvement. Compared with RTMDet-tiny, they improve all four metrics on both
datasets by 0.52-27.55% and consistently surpass representative
distillation methods, demonstrating a favorable accuracy-efficiency trade-off.
Read the full abstract ▾
Computer Vision
Revisiting Multi-View Stereo: A Sequence-to-Sequence Formulation
Aoxiang Fan, Corentin Dumery, Nicolas Talabot et al. (+1 more)
Computing accurate geometry from multi-view images is a fundamental problem in computer
vision. Recent feed-forward (FF) models jointly estimate 3D geometry and camera parameters, but they typically suffer from geometry distortion caused by reconstruction ambiguity, even when ground-truth camera parameters are supplied. In this paper, we study the multi-view stereo (MVS) problem with known camera parameters and propose a novel approach that bridges conventional MVS and FF methods. Rather than casting MVS as a sequence-to-one mapping that predicts depth only for a single reference view, we reformulate it as a sequence-to-sequence task, akin to FF models, that jointly predicts geometry for all input views. We introduce a global
transformer-based architecture with two components that explicitly exploit camera-induced priors: ray-map
embeddings that inject camera parameters into image patch
tokens, making the
transformer camera-aware, and a unified global cost volume that replaces conventional per-view cost volumes to jointly capture 3D structure across all views. Extensive experiments on multiple public
benchmarks show our approach achieves state-of-the-art performance, surpassing both MVS and FF reconstruction baselines.
Read the full abstract ▾
All authors Aoxiang Fan, Corentin Dumery, Nicolas Talabot, Pascal Fua
Computer Vision
When Wider Views Fail: Stress-Testing Feed-Forward 3D Reconstruction
Daisy Li, Kyle Gao, Quanyun Wu et al. (+3 more)
Feed-forward 3D reconstruction models enable efficient geometry estimation from sparse images, but their pretrained nature can make them vulnerable to distribution shifts beyond their training data. Identifying these failure modes is important for understanding when such models can be reliably deployed in unconstrained imaging settings. We investigate viewpoint variation as a controlled distribution shift by varying the angular span of sparse image inputs while keeping the input budget fixed. Across multiple feed-forward reconstruction models, we observe substantial degradation as viewpoint span increases, with wide spans producing both incomplete surface coverage and geometry unsupported by the observed imagery. These results reveal that viewpoint variation can induce failure modes beyond conventional reconstruction incompleteness, highlighting the need to evaluate pretrained feed-forward models under distribution shifts that challenge their learned
geometric priors.
Read the full abstract ▾
All authors Daisy Li, Kyle Gao, Quanyun Wu, Hanna Chomko, John S. Zelek, Jonathan Li
Computer Vision
ZVeC: A Zero-Shot Framework for Instance-Level Vehicle Extraction and Generative Point Cloud Completion
Daisy Li, Kyle Gao, Quanyun Wu et al. (+3 more)
LiDAR point clouds acquired in underground environments exhibit severe
geometric incompleteness due to occlusions and limited sensor viewpoints, making reliable point cloud completion challenging without large supervised
datasets. We propose ZVeC, a zero-shot, instance-driven framework that reformulates scene-level completion as compositional object-level reconstruction. By decomposing a scene into semantic object instances, ZVeC reduces reconstruction ambiguity in cluttered environments while eliminating the need for scenario-specific training. Each segmented vehicle is completed independently using a depth- and 3D Gaussian-conditioned
diffusion model that exploits generalized
geometric priors before the reconstructed instances are recomposed into the original scene. To evaluate our approach, we construct a real-world dense
LiDAR benchmark of underground parking environments. Experimental results demonstrate consistent improvements over representative scene-level baselines in both quantitative metrics and visual quality. The completed point cloud differs substantially from the measured input (average KL divergence ~ 2.1), yet reducing the input to only 1% of the original
LiDAR measurements changes the completed reconstruction only marginally (KL divergence < 0.50). This demonstrates that ZVeC produces geometrically consistent completions even under extreme input sparsity.
Read the full abstract ▾
All authors Daisy Li, Kyle Gao, Quanyun Wu, Boris Jutzi, John S. Zelek, Jonathan Li
Computer Vision · Cross-listed from eess.IV
When is a closed-form RGB->S/P ratio adequate? A hyperspectral characterization on natural scenes for mesopic display
Naoyuki Uchida
Mesopic and low-light display transforms require, as their driving signal, a per-pixel scotopic-to-photopic luminance ratio (S/P); the exact spectral S/P is unavailable for ordinary RGB content, so a low-cost closed form that estimates S/P from a linear-RGB triplet is used in its place. Such closed forms exist but have been characterized only on narrowband / LED sources, i.e. spectrally sparse spectra, where a relative error of ~41% has been reported for a three-channel projection. Display content, however, is natural and broadband. We ask whether the same closed form is adequate there, using per-pixel spectral S/P from hyperspectral imagery as ground truth. On a daylight radiance time-series, a six-scalar closed form (three photopic and three scotopic channel weights) reproduces spectral S/P with a median error of ~0.07 that is time-invariant once the RGB input is chromatically adapted to D65; evaluated in un-adapted sRGB the error instead carries a color-temperature tilt across illuminants (~0.19), so adaptation is the enabling step for this use case. The result generalizes to an independent fifty-scene set (pooled median 0.024; 45/50 scenes within a pre-registered 0.10 band), with the few exceedances concentrated in saturated, spectrally-peaky surfaces that approach the narrowband regime (floral close-ups in this set). The scotopic weight vector is shown to be primary-model dependent, but the value used here is corroborated by a primary-free XYZ projection, and the median error stays within the band across all principled coefficient choices. We do not claim observer-validated appearance fidelity or adequacy on narrowband sources; both are out of scope. Both outcomes follow from the same three-channel projection: it is overwhelmed by spectrally sparse inputs and adequate on spectrally smooth ones.
Read the full abstract ▾
Computer Vision · Cross-listed from cs.RO
Uranus: Building the Next-Generation Simulation Infrastructure for Embodied AI
Wenkang Qin, Yukun Zhou, Noah Shen et al. (+5 more)
Scalable simulation is essential for
robot data generation, policy training, evaluation, and safe iteration, yet real-world interaction is costly and conventional simulators require labor-intensive construction. We present Uranus, a data-driven
robot simulator built around a joint-trajectory-conditioned autoregressive
diffusion model. Uranus offers three key capabilities: (1) streaming, open-ended rollout, which receives future joint-position trajectories online and autoregressively generates one latent frame per step, corresponding to four RGB frames, without a fixed horizon; (2) low-latency generation, achieving 24 FPS after inference optimization; and (3) scalable, extensible
robot control, providing a unified interface for synchronized multi-view generation across diverse
robot embodiments and camera configurations. We conduct comprehensive quantitative and qualitative evaluations on both in-distribution and out-of-distribution data, providing an objective assessment of Uranus and clearly identifying its current limitations. We release the code and model weights to empower the community with practical tools and insights.
Read the full abstract ▾
All authors Wenkang Qin, Yukun Zhou, Noah Shen, Jisong Cai, Dongxiao Mao, Baicheng Li, Yue Zhang, Wei Sui
Computer Vision
Mobile Imaging Solutions for Medical Diagnosis: Trends and Applications
Syed Muhammad Ibne Zulfiker, Fariha Tabassum Islam, Md Sultanul Arifin et al. (+7 more)
Advances in processing power, camera technologies, and mobile image analysis have made smartphones and other mobile devices, such as laptops, increasingly suitable for medical diagnosis and healthcare applications. Researchers have developed low-cost solutions for the early detection and monitoring of various health conditions, including eye and ENT diseases, malnutrition, heart rate variability, skin and oral conditions, and injuries, using images captured by non-medical devices such as smartphones and webcams. This survey examines existing research on mobile image-based medical diagnosis, with an emphasis on its potential to enable low-cost and accessible healthcare. We comparatively analyze state-of-the-art solutions across different healthcare application categories, examining their advantages and limitations. Based on this analysis, we identify desirable characteristics of mobile image-based diagnostic tools and highlight areas where existing approaches have made progress as well as areas requiring further research. We also discuss application-specific and common challenges and outline directions for future research. Overall, this study provides a comprehensive overview of mobile image-based healthcare solutions and their potential to support low-cost disease diagnosis and monitoring, particularly for underserved populations in remote and resource-constrained settings.
Read the full abstract ▾
All authors Syed Muhammad Ibne Zulfiker, Fariha Tabassum Islam, Md Sultanul Arifin, Khandker Aftarul Islam, Nishat Anjum Bristy, Faria Huq, Priyeta Saha, Syeda Nahida Akter, Arpita Saha, Tanzima Hashem
Computer Vision
INTCORT: Training-Free Spatial Reasoning Enhancement for Vision-Language Models via Input Transformations and Confidence Routing
Haoran Sun, Jingqi Xu, Yanhui Li et al. (+3 more)
Vision-
Language Models (VLMs) have demonstrated remarkable capabilities in
multimodal tasks, yet they still exhibit poor ability in spatial
reasoning. Existing training-dependent and training-free enhancement methods suffer from high computational costs with catastrophic forgetting and internal mechanism interference that compromises general capabilities, respectively. In this work, we first verify two key hypotheses: appropriate
geometric image transformation and query-reversal transformation can recover incorrect spatial predictions, and correct predictions exhibit higher relation-
token confidence than incorrect ones. Based on these findings, we propose INTCORT, a training-free spatial
reasoning enhancement framework that constructs multiple inference views through input transformations and aggregates their predictions via relation-
token confidence routing, without modifying the VLM's internal mechanisms. Experimental results on several commonly-used
benchmarks demonstrate that INTCORT substantially improves spatial
reasoning accuracy across diverse VLMs, achieving an average improvement of 10.01% over all models and
benchmarks. Compared with prior works, INTCORT achieves superior performance with improvements of up to 25.01%.
Read the full abstract ▾
All authors Haoran Sun, Jingqi Xu, Yanhui Li, Enci Liu, Kaidi Xu, Yanwei Liu
Computer Vision
Streaming Video Editing with Easy Adaptation
Yujia Hu, Jiajun Li, Zihao He et al. (+1 more)
In this paper, we propose SVEET, a framework that requires merely training on a pretrained bidirectional
video diffusion model but supports high-quality streaming
video editing in an auto-regressive fashion. To tackle this problem, we first systematically revisit existing
video-to-
video diffusion approaches and identify two key principles for such streaming adaptation: backbone feature disentanglement and conditional frame independence. Building on these insights, we develop a novel paradigm for controllable
video generation. At its core, an auxiliary model branch encodes source
video inputs with temporally independent self-
attention, and the intermediate features are injected into the corresponding backbone blocks for streaming-compatible control. Moreover, to bridge the discrepancy between the feature spaces of bidirectional and streaming models, we propose a decoupled training scheme that explicitly enforces the orthogonality between the optimization directions of
video controllability and model causality. Such disentanglement ensures compatibility between the two objectives at inference and facilitates smooth zero-shot knowledge transfer across heterogeneous backbone architectures. Extensive experiments demonstrate that SVEET achieves superior editing quality while maintaining real-time performance, attaining 15 FPS on a single H100 GPU 17 without any auxiliary acceleration techniques. Codes are available at https://github.com/YujiaHu1109/SVEET.
Read the full abstract ▾
All authors Yujia Hu, Jiajun Li, Zihao He, Songhua Liu
Computer Vision
Toward a foundation model for forest point clouds
Yuanwen Yue, Stefano Puliti, Damien Robert et al. (+7 more)
Forest inventories increasingly rely on artificial intelligence (AI) models to derive forest attributes from large-scale 3D point clouds. Current models are typically specialized to a single task, sensor, and forest type, making adaptation expensive in terms of annotations, computation, and expertise. We ask whether a single pretrained model can instead learn transferable representations across diverse forest inventory settings. Inspired by recent developments in
language modelling and computer
vision, we take a step toward a foundation model (FM) for 3D forestry. Using LitePT as backbone, we first establish a strong supervised baseline that sets a new state of the art on forest semantic and instance
segmentation, tree species classification, and age regression
benchmarks. We then curate a large-scale unlabelled corpus spanning airborne,
UAV, and mobile
laser scanning across diverse forest ecosystems, and pretrain the same backbone using self-supervised learning. We systematically evaluate representation learning strategies by comparing training from scratch, supervised pretraining, and self-supervised pretraining across four representative forestry tasks, under varying annotation budgets. Compared with training from scratch, self-supervised pretraining accelerates model convergence and consistently improves performance when annotations are scarce. Compared with task-specific supervised pretraining, self-supervised pretraining yields more transferable representations across downstream forestry tasks. These findings identify the practical regime in which pretrained representations are most valuable and suggest that instance discrimination, rather than forest semantics, is the main remaining obstacle to a general-purpose 3D forest foundation model. Code and models are available at: https://github.com/prs-eth/ForPT.
Read the full abstract ▾
All authors Yuanwen Yue, Stefano Puliti, Damien Robert, Atakan Topaloğlu, Binbin Xiang, Maciej Wielgosz, Jan Dirk Wegner, Rasmus Astrup, Christian Rupprecht, Konrad Schindler
Computer Vision
Virtual neural networks: hundreds of souls in a body
Petr Hurtik, Marek Vajgl, Zahra Alijani et al. (+1 more)
A new concept, termed virtual neural networks, is introduced, where the count of trainable parameters is kept constant, and scalability is attained purely through computational resources. This concept is an abstract framework that can be realized using any standard convolutional neural network. It merges siamese neural networks with a deep ensemble technique by generating numerous virtual models that share weights derived from a small set of physical models. The ensemble comprises up to hundreds of trained models simultaneously. All virtual networks take the same input, and their interconnected structure induces an internal distortion that boosts the entire ensemble robustness. The accuracy of the ensemble improves as the number of virtual networks increases, without changing the capacity. Virtual neural networks outperform larger capacity models, typical deep ensembles, and contemporary approaches like SWA and Masksembles. Additionally, the highest-performing individual model from the ensemble surpasses other models trained individually, even those with a greater number of parameters. Code: gitlab.com/EnginCZ/virtual-models-public
Read the full abstract ▾
All authors Petr Hurtik, Marek Vajgl, Zahra Alijani, Vojtech Molek
Computer Vision
Brain Metastases Segmentation for BraTS 2026 Task 1: A Multi-Architecture Comparison
Mahdi Islam and Musarrat Tabassum
Brain metastases are the most common intracranial malignancy, occurring in roughly 30% of patients with primary solid tumors and carrying a median survival near 5.9 months. Automated
segmentation is critical for treatment planning and volumetric monitoring, but metastases are frequently small, numerous, and heterogeneous in size within a single patient. We compare a plain nnU-Net baseline, a Residual Encoder Large (ResEncL) variant, region-based training, and a Primus
transformer model for BraTS-METS 2026 Task 1, using patient-grouped cross-validation to prevent leakage from the longitudinal UCSD subset. Primus (label-based) is our strongest individual model by aggregate DSC/NSD, achieving 0.710/0.761 (ET), 0.742/0.785 (TC), 0.683/0.689 (WT), and 0.531/0.436 (RC). ResEncL trails Primus on aggregate DSC/NSD but achieves substantially higher lesion-wise F1 (e.g. ET: 0.452 vs. 0.052); a probability-averaging ensemble of the two only partially preserves ResEncL's F1 advantage (ET lesion-wise F1: 0.064). We further report three postprocessing and label-reconstruction pitfalls we believe generalize beyond this challenge. Code is available at https://github.com/mahdiislam79/BraTS_METS_2026.
Read the full abstract ▾