Computer Vision
FogDrive: A Multi-Modal Synthetic Driving Dataset for Perception under Graded Fog
Vansh Panwar
Perception under adverse weather remains a critical bottleneck for reliable autonomous driving, yet existing
benchmarks lack the systematic multi-modal
alignments needed to evaluate robust sensor fusion. Real-world weather
datasets suffer from uncontrolled collection and single-level, uncalibrated conditions, while synthetic alternatives either target camera-only restoration or lack the paired clean-and-foggy structure needed to
benchmark "defog-then-detect" pipelines. We present FogDrive, a rigorously calibrated, multi-modal autonomous-driving
dataset bridging data-centric engineering and robust machine learning. Built with the CARLA simulator, FogDrive contains 660 scenes (~133k fully annotated frames, 50:50 day/night) across four synchronized cameras (RGB, depth, semantic
segmentation), a
LiDAR and semantic-
LiDAR pair, and front radar. Physically consistent fog is modeled independently on camera channels (Koschmieder model) and
LiDAR channels (Beer-Lambert law) at three calibrated visibility densities (160m, 100m, 50m). Every scene ships in four matched variants (clean plus three graded fog levels) with cross-calibrated 2D and 3D bounding boxes. A semantic-
segmentation-based quality audit over 8k images validates annotations at 95.1% precision and over 99% recall for vehicles within 40m. We establish baseline
benchmarks with state-of-the-art architectures (TransFusion, BEVFusion, YOLOv8-m) across two paradigms: 3D multi-modal fusion and 2D image restoration. These yield critical data-centric insights: mixing multi-density fog during training tightens 3D bounding-box geometry without added data-scaling cost, while in 2D pipelines image-quality metrics (PSNR, SSIM) prove poor predictors of downstream detection performance. FogDrive will be fully open-sourced alongside our data-generation framework to accelerate robust, multi-modal research.
Read the full abstract ▾
Computer Vision · Cross-listed from cs.AI
Test-Time Coverage: Test-Conditioned Data Curation for Deployment-Aware Learning
Nadine Chang, Maying Shen, Shizhe Diao et al. (+6 more)
Deployed AI systems are often trained from broad candidate data pools, necessitating data curation towards the deployment test distribution. However, standard data curation methods score training-side criteria rather than directly optimizing deployment match. We introduce TTCov (Test-Time Coverage), a data-level test-conditioned curation method that uses test-side information before training instead of updating model weights at inference. TTCov decomposes deployment-conditioned curation into coverage and distribution. To represent coverage, it builds a task Atlas, a collection of LLM-based atomic propositions (APs) describing deployment-relevant concepts, seeded from open task knowledge and expanded with unmatched APs extracted from unlabeled deployment samples. To represent distribution, it instantiates the matched deployment APs with their frequencies, yielding a Knowledge Atlas (K-Atlas) that operationalizes the deployment distribution as a curation target. TTCov then selects a budgeted training set whose deployment APs distribution approximates this target. We apply TTCov towards autonomous driving (AD), keeping adaptation off the inference path while selecting data with greater deployment-relevant coverage, closer K-Atlas matching, and stronger downstream end-to-end driving performance than data-curation baselines, including seamless adaptability to novel domains via city-to-city expansion.
Read the full abstract ▾
All authors Nadine Chang, Maying Shen, Shizhe Diao, Jialiang Wang, Jingde Chen, Thomas Breuel, Pavlo Molchanov, Rafid Mahmood, Jose M. Alvarez
Computer Vision · Cross-listed from cs.LG
CT-Merging: Consensus Directions and Task-Specific Scaling for LoRA Adapter Merging
Keumseo Ryum and Joonhyuk Kang
LoRA merging methods increasingly operate on the low-rank structure of task updates, yet how the common subspace is estimated and how coefficients are assigned after recomposition are rarely compared directly. We propose CT-Merging, which estimates common directions from averaged task subspace projectors and assigns a separate residual scale to each task. Projector averaging selects directions supported across task subspaces without weighting them by singular magnitude, while task-specific scaling removes component-wise magnitude variation and preserves scale differences across tasks. On the released
KnOTS CLIP adapters, CT-Merging achieves the best average and worst-task normalized accuracy on both backbones, improving over the strongest baseline by up to 2.56 and 6.65 points, respectively. On the DC-Merge adapter
benchmark, it achieves the best average normalized accuracy in eight of nine backbone and task-count settings. Ablations show that projector averaging outperforms summed-update SVD and that task-specific scaling improves worst-task accuracy over global isotropic scaling.
Read the full abstract ▾
Computer Vision
Hazard or Anomaly? Evaluating VLMs for Understanding Dangers and Discrepancies
Murali Indukuri, Mohammad Eskandari, Sree Nitya Kollu et al. (+2 more)
Modern safety-critical systems increasingly rely on human-
robot interaction to reduce disaster risk and support decision-making during emergencies.
Vision-
Language Models (VLMs) are promising for these settings because they can interpret complex scenes and communicate safety-relevant information, but they still require careful evaluation to ensure reliable safety
reasoning. In particular, current evaluations often frame danger recognition as a binary decision (Safe/Unsafe), making it unclear whether a model is identifying true physical hazards or merely reacting to unusual scene elements. We address this limitation by introducing an explicit distinction between hazard and anomaly, and by separately recognizing hazardous and anomalous states. We evaluate several state-of-the-art VLMs across two
datasets and multiple prompting strategies to test whether this distinction changes model behavior. Our results show that VLMs frequently misinterpret anomalousness as hazardousness, revealing an over-reliance on contextual irregularity as a proxy for danger. We further show that explicitly separating anomaly from hazard provides a more informative evaluation of VLM safety
reasoning and exposes failure modes that binary safety judgments can obscure. Our public
dataset is available on Roboflow https://app.roboflow.com/vlm-in-context-anomaly-and-hazard-detection/camera-ready-roman-ds.
Read the full abstract ▾
All authors Murali Indukuri, Mohammad Eskandari, Sree Nitya Kollu, Stephanie Lukin, Cynthia Matuszek
Computer Vision
Automated Cardiac Adipose Tissue Segmentation in Computed Tomography: A Literature Review
Andreas W. Aspe, Jonas Jalili Pedersen, Andreas Ohrt Johansen et al. (+4 more)
This review provides an overview of recent advancements in automated
segmentation methods on Computed
Tomography (CT) for two types of cardiac fat: Epicardial adipose Tissue (EAT) and Pericardial Adipose Tissue (PAT). These fat deposits, separated by the pericardium, have been linked to various cardiovascular diseases, with EAT receiving the most research
attention. Their complex anatomical context makes manual quantification highly time-consuming and prone to considerable inter-observer variability. Automated methods effectively address these complications, offering a more efficient and consistent solution. This study encompasses a broad range of methods, spanning AI as well as non-AI approaches. Additionally, it presents the remaining challenges, including the need for larger annotated public
datasets and optimized attenuation thresholds for contrast-enhanced CT. It is demonstrated that automated methods are able to achieve
segmentation results comparable to the quality of human annotation, proving their potential as a clinical tool for discovering new biomarkers and enhancing patient outcomes.
Read the full abstract ▾
All authors Andreas W. Aspe, Jonas Jalili Pedersen, Andreas Ohrt Johansen, Klaus Fuglsang Kofoed, Kristine Aavild Sørensen, Rasmus Reinhold Paulsen, Josefine Vilsbøll Sundgaard
Computer Vision
When Physical Preferences Meet Semantic Constraints: Physical and Semantic Direct Preference Optimization for Text-to-Video Generation
Siwei Meng, Yawei Luo, Shu Zhang et al. (+1 more)
Text-to-
video (T2V) generation models have achieved strong visual realism, but improving physical plausibility can come at the cost of semantic consistency with the input text. This tension arises because physical preference is typically determined by comparing dynamics between two
videos, without accounting for whether either
video faithfully depicts the scene specified by the
prompt, making physical-semantic conflict a systematic tendency under this supervision paradigm. We formulate this challenge as a constrained preference optimization problem and propose Physical and Semantic Direct Preference Optimization (PSDPO), which modulates each preference pair's contribution based on the agreement between its physical and semantic signals. A gradient-level analysis shows that PSDPO bounds the semantic drift from conflicting pairs to a controllable residual, and further motivates a staged optimization protocol that provably reduces cumulative drift. The resulting method operates entirely within the standard DPO framework, requiring no auxiliary models or additional loss terms. Experiments show that PSDPO improves physical plausibility by up to $2\times$ over the baseline on VideoPhy-2, while maintaining strong semantic consistency on VBench, achieving a more reliable balance than existing preference-based methods.
Read the full abstract ▾
All authors Siwei Meng, Yawei Luo, Shu Zhang, Ping Liu
Computer Vision · Cross-listed from cs.GR
Points as Tori: Fast Pointwise Signed Distance for Point Clouds
Nicole Feng, Ioannis Gkioulekas and Keenan Crane
We describe a method for computing signed distance to point clouds that allows fast pointwise evaluation at arbitrary spatial resolution. As input, our method takes a point cloud with normals; as output, it provides an analytical parameterization that allows queries of signed distance to the approximate underlying surface at arbitrary points - simultaneously providing reconstruction and distance. Our key idea is to reconstruct shapes by locally fitting point clouds with
tori, which have closed-form signed distance functions.
Tori are fitted in a feed-forward manner, using a pre-trained network to output per-point
curvature and shift parameters. Importantly, our method does not require costly global optimization or spatial discretization, and is easily parallelizable. Underlying our method is a new theory that unifies signed distance with the classic reconstruction methods of winding numbers and Poisson surface reconstruction. We use our method to compute signed distance to point clouds arising from photogrammetry, meshes, 3D Gaussians, and neural implicits. Our method allows point clouds to be used directly in applications, without explicit surface reconstruction: as examples, we take offsets of point clouds, apply morphological and Boolean operations, and directly visualize offset surfaces using sphere tracing.
Read the full abstract ▾
Computer Vision · Cross-listed from cs.LG
Investigation of Polycystic Ovary Syndrome (PCOS) Diagnosis Using Machine Learning Approaches
Al Zadid Sultan Bin Habib, Md Asif Bin Syed, Md. Ekramul Islam et al. (+1 more)
Polycystic Ovarian Syndrome (PCOS) is a widespread hormone problem for women of childbearing age. Women with PCOS may not ovulate; they might have high levels of androgens and have many small cysts on the ovaries. It can cause missed or irregular menstrual periods, excess hair growth, acne, infertility, and weight gain. Machine Learning (ML) can effectively diagnose this disease at an earlier stage as tons of medical data are available now. Traditional approaches to detect PCOS encompass a combination of clinical evaluation, medical history assessment, physical examination, and laboratory tests. These approaches aim to identify the characteristic symptoms and hormonal imbalances associated with PCOS. Physical examination requires good resources and costs time and money. In recent times, data-driven techniques have substantially advanced disease prediction within the medical field. We aim to utilize ML approaches, incorporating unique feature selection
algorithms, to predict PCOS. This paper introduces a data-driven approach to PCOS diagnosis, combining Feature Engineering and ML. Several feature selection approaches have been considered to select sets of features for training the ML model, including CatBoost, Extreme Gradient Boosting (XGBoost), Light Gradient Boosting Machine (LGBM), AdaBoost, Random Forest (RF). Results demonstrate that AdaBoost, with ten features selected by RF Feature Importance and Highest Correlation (HC), provides the highest test accuracy.
Read the full abstract ▾
All authors Al Zadid Sultan Bin Habib, Md Asif Bin Syed, Md. Ekramul Islam, Tanpia Tasnim
Computer Vision
What Do They See? Interpreting Complex Road Scenarios Through the Eyes of Vision-Language-Action Models for Safe and Trustworthy Autonomous Vehicle Learning
Kalpana Panda, Wesley Maia, Vinti Agarwal et al. (+1 more)
End-to-end autonomous driving models are now able to navigate complex road scenarios, mapping raw sensor observations directly to observed paths for open-loop evaluation and often effective driving in closed-loop evaluation. Yet the internal logic of these safety-critical systems remains largely opaque, due to the complexity of traffic scenes. We propose a counterfactual ablation framework called Counterfactual
Vision Action Analysis (CVAA) that systematically removes individual detected objects from front-camera images using photorealistic generative inpainting to prepare counterfactual sets to evaluate the difference in the model's response. This isolates the causal effect of each object's presence on the model's planning behaviour. Applied to the Alpamayo 1 trajectory predictor across 210 nuScenes driving scenes, we create a
dataset Counter -nuScenes, using which we see that vehicles and pedestrians within the model's 'path' dominate causal influence as expected, while traffic lights, as expected, exert disproportionate effect relative to their image footprint. However, we also find cases where the model responds strongly to objects a human driver would consider irrelevant. This brings forth a deeper question: does the model itself view the scene as a sum of individual objects influencing the outcome, or does it encode an entirely different set of internal features that do not correspond to human-legible scene elements? To further understand this, we compare intermediate representations of original and inpainted image pairs using mechanistic interpretability techniques and examine the effect of the removal through the various model layers. Together, these two stages offer a path from behavioral
auditing to representational understanding, creating explainable driving systems and solidifying human-AI trust.
Read the full abstract ▾
All authors Kalpana Panda, Wesley Maia, Vinti Agarwal, Ross Greer
Computer Vision
Pediatric Bone Age Prediction Using Deep Learning
Al Zadid Sultan Bin Habib, Md. Ekramul Islam, Md Asif Bin Syed et al. (+2 more)
Pediatric bone age prediction is a crucial task in clinical practice that can help diagnose endocrine disorders and provide insight into a child's growth and development. However, conventional bone age prediction methods are often labor-intensive and require specialized radiological expertise. This paper presents a Deep Learning (DL)-based approach to pediatric bone age prediction using EfficientNet with Additive
Attention, a state-of-the-art neural network architecture for image classification and regression tasks. The method utilizes over 12,000 X-ray images from the RSNA bone age
dataset. It involves image preprocessing, transforming them into three-channel images, and training a Convolutional Neural Network (CNN) to automatically learn the features of hand bone images. This approach provides a more effective and accurate solution for predicting bone age, which is critical in diagnosing pediatric endocrine diseases. This work uses two variations of the EfficientNet model (B0 and B4), where EfficientNetB4 is also finetuned with the Additive
Attention mechanism. These three models predict the age for the original age, and their comparison is shown in curves. The predicted ages depict that in most cases, EfficientNetB4 and EfficientNetB4 with Additive
Attention (EN-AA) successfully predicted the bone ages more accurately regarding the original age, and their performance was better than the EfficientNetB0. Specific performance metrics are provided to underscore this improvement. Learning curves for training and validation loss confirm effective learning without overfitting or underfitting, further validating our approach's efficacy in pediatric endocrine disease diagnosis.
Read the full abstract ▾
All authors Al Zadid Sultan Bin Habib, Md. Ekramul Islam, Md Asif Bin Syed, Md Younus Ahamed, Tanpia Tasnim
Computer Vision · Cross-listed from physics.ao-ph
C3DIR: A Deep Learning 3-Dimensional Cloud Property Retrieval Scheme for Passive Satellite Imagers
Charles H. White, Yoo-Jeong Noh, John M. Haynes et al. (+1 more)
We develop the Cloud 3-Dimensional Imager
Retrieval (C3DIR), a deep learning model that estimates 3-D cloud properties for multiple passive satellite imagers trained to match
retrievals from the Earth Cloud Aerosol and Radiation Explorer(EarthCARE) ACM-CAP product. This work is aimed towards moving AI/ML 3-D cloud
algorithms closer towards operational use. C3DIR predicts the occurrence water content of ice, cloud
liquid, and rain along the imager line-of-sight and uses a voxel-level collocation approach to account for the misaligned viewing geometries of passive imagers and active profiling instruments. This precise collocation methodology allows for constructing vertical profiles using voxels contained by multiple imager pixels to facilitate comparisons with active profiling instruments. Qualitative case studies show that C3DIR can accurately depict multiple distinct overlapping cloud layers, albeit with some smoothing. Quantitative evaluations illustrate that C3DIR overall excels at hydrometeor detection which intuitively tends to be a function of water content. However, detection of voxels classified as
liquid cloud remains difficult due to the their small
geometric thickness, finer horizontal scale, and frequent tendency to be obscured or embedded within ice clouds. In general, water content estimation is reasonably accurate, yielding the best results in ice clouds but uncertainties remain for
liquid and rain water content. Column-integrated water paths are in tighter agreement with EarthCARE. Comparisons with the
algorithms underpinning current NOAA operational products highlight several areas where C3DIR may offer improvement. Overall, these results demonstrate the potential for C3DIR to provide flexible 3-D output depicting vertically resolved cloud structure which can offer broader utility for aviation applications, numerical weather modeling, and
climate research.
Read the full abstract ▾
All authors Charles H. White, Yoo-Jeong Noh, John M. Haynes, Imme Ebert-Uphoff
Computer Vision
Splat-based 3D Scene Reconstruction with Extreme Motion-blur
Hyeonjoong Jang, Dongyoung Choi, Donggun Kim et al. (+2 more)
We propose a splat-based 3D scene reconstruction method from RGB-D input that effectively handles extreme motion blur, a frequent challenge in low-light environments. Under dim illumination, RGB frames often suffer from severe motion blur due to extended exposure times, causing traditional camera pose estimation methods, such as COLMAP, to fail. This results in inaccurate camera pose and blurry color input, compromising the quality of 3D reconstructions. Although recent 3D reconstruction techniques like Neural Radiance Fields and Gaussian Splatting have demonstrated impressive results, they rely on accurate camera trajectory estimation, which becomes challenging under fast motion or poor lighting conditions. Furthermore, rapid camera movement and the limited field of view of depth sensors reduce point cloud overlap, limiting the effectiveness of pose estimation with the ICP
algorithm. To address these issues, we introduce a method that combines camera pose estimation and image deblurring using a Gaussian Splatting framework, leveraging both 3D Gaussian splats and depth inputs for enhanced scene representation. Our method first aligns consecutive RGB-D frames through optical flow and ICP, then refines camera poses and 3D geometry by adjusting Gaussian positions for optimal depth
alignment. To handle motion blur, we model camera movement during exposure and deblur images by comparing the input with a series of sharp, rendered frames. Experiments on a new RGB-D
dataset with extreme motion blur show that our method outperforms existing approaches, enabling high-quality reconstructions even in challenging conditions. This approach has broad implications for 3D mapping applications in robotics, autonomous
navigation, and augmented reality. Both code and
dataset are publicly available on https://github.com/KAIST-VCLAB/gs-extreme-motion-blur.
Read the full abstract ▾
All authors Hyeonjoong Jang, Dongyoung Choi, Donggun Kim, Woohyun Kang, Min H. Kim
Computer Vision
Pedestrian Archetypes Extension – More Pedestrian Models for Autonomous Vehicle Safety Testing
Taorui Huang, Namita Gaidhani, Ritvik Bansal et al. (+6 more)
In our prior work, Pedestrian Archetypes, we defined pedestrian archetypes as collections of behaviors that uniquely identify a specific type of pedestrian. The first paper proposed 12 pedestrian archetypes, including the Wanderer, Drunk, Distracted, Flash, Indecisive, Blind, Flock, Jaywalker, Elderly, Kid, Eventful, and Parked Pedestrian. These archetypes were introduced to move beyond single behavior labels and provide a more natural way to describe how dangerous pedestrians actually behave progressively in real-world traffic scenarios. However, upon further annotation of YouTube dash-cam
videos, we identified 7 additional pedestrian archetypes with observable and significant behavioral differences from the previously proposed ones. These new archetypes capture pedestrian behavior patterns that could not be fully explained by the original taxonomy. In this pre-print, we introduce each new archetype, define its essential and optional behaviors, explain how it differs from previously proposed archetypes, and provide
video-frame evidence showing the archetype in action.
Read the full abstract ▾
All authors Taorui Huang, Namita Gaidhani, Ritvik Bansal, S M Jubaer, Regina Lim, Rhett Zhao, Gavin Rafael Selin, Sunnie Deng Gao, Hasnain N Syed
Computer Vision
Cross-Branch Conflict as a Shield: Safeguarding Facial Identities in Unified Multimodal Image Editing
Weiwei Tan, Junxian Li, Rui Wang et al. (+3 more)
Unified
multimodal models (UMMs) have recently demonstrated powerful instruction-based image editing capabilities, while also raising serious concerns about the unauthorized manipulation of personal portraits. We investigate a novel and practical problem: protecting facial identities against unauthorized editing of UMMs. Existing
diffusion-based and VLM-based protection methods often become ineffective because they typically disrupt only a single visual branch. To understand this limitation, we conduct a feature-level analysis of the understanding and generation branches in unified image editing models. Our observations show that the structural agreement between these two branches is closely related to successful image editing. When only one branch is distorted, the model may still recover identity information from the other branch. Based on this, we propose Cross-Branch Conflict as a Shield (CCS), a unified
adversarial protection framework. CCS jointly drives the ViT and VAE representations away from their clean counterparts. It also uses a linear Centered Kernel
Alignment (CKA) objective to disrupt the structural consistency between the two branches. By degrading reliable identity information in both visual pathways and inducing incompatible cross-branch representations, CCS effectively prevents UMMs from recovering consistent facial identity cues during editing. Extensive experiments suggest that CCS consistently provides stronger protection in suppressing identity-preserving edits. Codes are in the supplementary material.
Read the full abstract ▾
All authors Weiwei Tan, Junxian Li, Rui Wang, Zhenhua Xu, Yanjun Zhang, Yu Leo Zhang
Computer Vision
Transferable Low-Rank Convolutional Bases for Onboarding Unseen Medical Imaging Modalities
Ranat Das Prangon, Istiaque Ahmed, Shajid Hasan Naim et al. (+2 more)
Deploying a medical imaging model that must later accommodate a modality it has never seen is a recurring practical problem: retraining the shared representation is expensive and destroys performance on the modalities already in service. We study this onboarding problem under a strict leave-one-domain-out protocol, in which a convolutional backbone is pre-trained on source modalities (Kidney CT and
Brain MRI), frozen permanently, and then required to accommodate an unseen modality (Chest X-ray). Under this protocol we establish three findings. First, decision-layer parameter-efficient fine-tuning is insufficient when the backbone has never observed the target modality: a linear probe and fully-connected LoRA both fall well short, whereas convolutional LoRA recovers most of the achievable accuracy, showing that adaptation must reach the convolutional features. Second, and centrally, the low-rank convolutional basis learned on the source modalities transfers: freezing that basis and training only its up-projections onboards the unseen modality using just $0.78\%$ of full fine-tuning's parameters, at an accuracy $6.11$ percentage points above a random basis of identical size, while an equivalent decision-layer basis exhibits no reliable transfer. Third, adapter-based onboarding leaves source-modality accuracy exactly unchanged ($\Delta = 0.00$ pp), whereas full fine-tuning reaches the highest target accuracy only by catastrophically degrading the source modalities. A Mahalanobis score on frozen backbone features detects the unseen modality with high sensitivity at a strict source-retention threshold, providing a practical trigger for when onboarding is required. All results are reported over three seeds with paired bootstrap confidence intervals.
Read the full abstract ▾
All authors Ranat Das Prangon, Istiaque Ahmed, Shajid Hasan Naim, Waseem Mustak Zisan, Hossain Md Shakhawat
Computer Vision
InfoDense: Density-Aware Regional Decisive Replay for Memory-Efficient Incremental Face Forgery Detection
Jikang Cheng, Hao Shen, Xueyi Zhang et al. (+4 more)
The rapid
evolution of face forgery techniques has introduced an increasing variety of manipulations. Incremental Face Forgery Detection (IFFD), which incrementally adds new forgery data to fine-tune previously trained models, has emerged as a promising approach to handle evolving forgery threats. However, conventional replay-based IFFD methods suffer from catastrophic forgetting. Storing full historical images under limited
memory often either fails to preserve subtle forgery cues or introduces domain bias, reducing the model's ability to learn intrinsic and transferable manipulation characteristics. In this paper, we propose a Density-Aware Regional Decisive replay strategy, termed InfoDense, to address these challenges. InfoDense prioritizes artifact-dense and forgery-critical regions, significantly reducing storage requirements while maintaining high-fidelity forgery evidence. We first introduce InfoDense Cut to localize decisive patches using CLIP-based
embeddings. Then, InfoDense Select ranks candidate segments by combining latent-space representativeness and decisive patch counts, ensuring both diversity and information density in the replay buffer. Finally, InfoDense Fuse reconstructs unbiased training inputs by adaptively merging stored segments with current-task samples, enhancing knowledge retention and generalization. Extensive experiments on challenging incremental deepfake
benchmarks demonstrate that InfoDense effectively mitigates catastrophic forgetting while improving cross-domain generalization.
Read the full abstract ▾
All authors Jikang Cheng, Hao Shen, Xueyi Zhang, Guangcheng Wang, Zhongyuan Wang, Renye Yan, Baojin Huang
Computer Vision
InLiER: Learning-Free Heterogeneous LiDAR Place Recognition via Intermediate Mixed-Radix Structural Keypoint Tokenization
Nikolaos Stathoulopoulos and George Nikolakopoulos
LiDAR place recognition supports loop closure, relocalization, and multi-
agent map management. As robotic platforms increasingly combine
LiDARs with different fields of view, resolutions, and scanning patterns, existing descriptors degrade because they are tightly coupled to sensor-specific characteristics. We present InLiER, a learning-free pipeline based on an intermediate tokenization step. Height-sliced keypoints from structural elements receive mixed-radix
token IDs encoding height, radial distance, local shape, and azimuth from local 3D geometry, in a compact sub-2KB representation. The same vocabulary is reorganized across three
retrieval stages: height-ceiling histogram intersection for fast rotation-invariant shortlisting, binary bitmask
alignment for yaw estimation and reranking, and
token-guided
geometric verification for 6-DoF pose estimation. InLiER achieves state-of-the-art performance on the HeLiPR
dataset and in real-world field experiments, among modern handcrafted methods and outperforms the learning-based baseline on most cross-sensor configurations.
Read the full abstract ▾
Computer Vision
Haoru Tan, Wang Wang, Sitong Wu et al. (+5 more)
We revisit
dataset distillation from an outcome-centric perspective. Rather than aligning process surrogates (per-step gradients or training trajectories), Influence Matching (Inf-Match) aligns the final outcome of training: it learns a compact synthetic set whose effect on the converged parameters matches that of the full
dataset. Concretely, we introduce a fully differentiable, sample-level influence
estimator that quantifies parameter shifts from adding or removing data, without time-consuming inverse-Hessian products or convexity assumptions. The
estimator runs in linear time by unrolling the optimization dynamics and applying a first-order Taylor approximation. We then learn the synthetic set by minimizing the mismatch between its influence and that of the real
dataset, yielding outcome
alignment rather than heuristic process imitation. Inf-Match delivers the best accuracy across standard classification
benchmarks. For instance, on Tiny-ImageNet (IPC=10), Inf-Match attains 31.5%, a +4.7% improvement over NCFM. Beyond classification, Inf-Match scales to
vision-
language distillation on Flickr30K, outperforming strong process-matching baselines. For instance, with 200 to 1000 synthetic samples, our method achieved a leading impressive average on image/text
retrieval tasks, higher than NCFM by 2.5%. The code will be released via https://github.com/hrtan/infmatch.
Read the full abstract ▾
All authors Haoru Tan, Wang Wang, Sitong Wu, Xiuzhe Wu, Yangtian Sun, Chirui Chang, Shaofeng Zhang, Xiaojuan Qi
Computer Vision
Look Clearly Before Answering: Mitigating Hallucinations in LVLMs via Saliency-Driven Perceptual Realignment
Pengxu Chen, Yao Zhu, Guangming Zhu et al. (+4 more)
Large
vision-
language models (LVLMs) have demonstrated remarkable capabilities in
multimodal understanding. However, they remain prone to
hallucinations, generating responses that are inconsistent with the visual evidence. Existing mitigation methods largely address
language-prior bias or cross-modal imbalance, while progressive visual degradation across perception and
memory remains underexplored. In this work, we propose Saliency-Driven Perceptual Realignment (SDPR), a training-free framework that mitigates the degradation of visual awareness throughout inference. Specifically, we first introduce saliency-driven
attention redistribution to release
attention hijacked by non-semantic sink
tokens, thereby recovering critical visual evidence. Second, we identify spatial distortion in the KV cache and propose saliency-driven cache
alignment to preserve query-relevant visual features during generation. Finally, we introduce prior-constrained contrastive decoding to penalize unfaithful predictions induced by dominant
language priors. Our proposed SDPR is robust against
hallucinations due to its holistic
alignment of visual awareness across the entire generative trajectory. Extensive experiments across diverse LVLM architectures show that SDPR outperforms state-of-the-art methods on both
hallucination and general-purpose
benchmarks, requiring no additional training and incurring minimal runtime overhead. The code is available \href{https://github.com/PengSyuChen/SDPR}{\color{blue}{here}}.
Read the full abstract ▾
All authors Pengxu Chen, Yao Zhu, Guangming Zhu, Jun Sheng, Jincai Huang, Xiangyang Ji, Liang Zhang
Computer Vision
TopoGS: Planar Reconstruction via Topology-aware 3D Gaussian Splatting
Shanshan Pan, Jiale Chen, Yilin Liu et al. (+1 more)
Extracting structured, parametric 3D representations from raw images remains a fundamental challenge in computer
vision and graphics. While recent advancements in the 3D Gaussian Splatting (3DGS) pipeline integrate planar primitives to yield compact and editable geometry, these approaches typically treat planes as isolated, discrete sets. This lack of topological connectivity hinders robust
geometric reasoning, leading to fragmented reconstructions and misaligned boundaries that fall short of the precision for rigorous spatial analysis and professional design workflows. To address this, we introduce TopoGS, the first 3DGS framework to explicitly integrate both planar and topological constraints for coherent 3D reconstruction. Specifically, we extract global 2D topological relationships from multi-view image
segmentations and anchor Gaussian primitives to these structural elements. This formulation enables the joint optimization of plane parameters, rendering fidelity, and topological adjacency. By enforcing strict multi-view consistency alongside these topological constraints, our method significantly mitigates
geometric misalignments and produces connected, structured 3D models. Extensive evaluations on the ScanNet++
dataset demonstrate that TopoGS achieves state-of-the-art performance, providing a highly robust solution for generating accurate, topologically sound, and visually faithful scene representations.
Read the full abstract ▾
All authors Shanshan Pan, Jiale Chen, Yilin Liu, Hui Huang
Computer Vision
Robust PnP on a Neuromorphic Processor for Object Pose Estimation
Tam Ngoc-Bang Nguyen, Mohsi Jawaid and Tat-Jun Chin
Neuromorphic computing is gaining
attention in robotic perception due to its higher energy efficiency. While neural network-based methods can more readily exploit the distributed and parallelized structure of neuromorphic computers, crafting neuromorphic solutions for non-learning tasks is less straightforward. This hampers the usage of neuromorphic computing for perception pipelines that depend on both learning and non-learning components, such as object pose estimation (OPE) where state-of-the-art methods use a deep network to predict 2D landmarks and nonlinear optimization to solve perspective-n-point (PnP). In this paper, we propose a novel neuromorphic-deployable formulation for robust PnP, where given outlier-prone 2D-3D correspondences, the object pose with the largest number of inliers is determined. Underpinning our method is a distributed
algorithm for robust least squares estimation of rigid body pose that can be executed on a neuromorphic processor. We also design a spiking neural network (SNN) to predict 2D landmarks from event data, where the main layers of the SNN were designed according to the principles of spiking
neurons. Overall, our work enables neuromorphic treatment of the major stages of an OPE pipeline, from event sensing and learned landmark prediction, to
geometric optimization for robust PnP. Results on neuromophic hardware (Intel Loihi 2) indicate the higher energy efficiency our neuromorphic robust PnP, while achieving competitive accuracy.
Read the full abstract ▾
Computer Vision
UniNDM: A Unified Noise-driven Detection and Mitigation Framework Against Sexual Content in Text-to-Image Generation
Yao Huang, Yitong Sun, Huanran Chen et al. (+8 more)
Despite the impressive generative capabilities of text-to-image
diffusion models, they remain vulnerable to implicit sexual
prompts, where subtle cues disguised as benign terms or
adversarial tokens unexpectedly generate the inappropriate content due to model biases or latent correlations in training data. Existing safety mechanisms face fundamental limitations: detection methods primarily identify explicit content and fail to capture implicit malicious intent, while mitigation approaches rely on static negative
prompts inadequate for diverse implicit scenarios. To address these challenges, we propose UniNDM, a unified noise-driven framework that rethinks safety mechanisms through the lens of noise dynamics in
diffusion processes. Our key insight is that early-stage predicted noise exhibits inherent separability between normal and sexually explicit content, which we theoretically demonstrates quadratically increasing semantic concentration with timestep. Leveraging this property, we develop a lightweight noise-based detector achieving superior accuracy with virtually no computational overhead. For mitigation, we introduce noise-enhanced adaptive negative guidance: dynamically generating context-specific negative
prompts via large
language models to handle diverse implicit content, while optimizing initial noise by suppressing
attention concentration on explicit
tokens to provide comprehensive protection. Besides the U-Net-based
diffusion models, we further extend our framework to emerging
Diffusion Transformer architectures through region-constrained semantic guidance tailored for their unified
multimodal attention. Comprehensive experiments across U-Net models and DiT models on both natural and
adversarial datasets demonstrate substantial improvements over state-of-the-art methods, including SLD, UCE, Safree, etc. Our code is publicly available at https://github.com/Aries-iai/UniNDM.
Read the full abstract ▾
All authors Yao Huang, Yitong Sun, Huanran Chen, Ruochen Zhang, Shouwei Ruan, Ranjie Duan, Maoxun Yuan, Yinpeng Dong, Hui Xue, Xiaochun Cao, Xingxing Wei
Computer Vision
Spatially-Aware Class-Agnostic Object Counting
Robert Wijaya, Md. Tanvir Hossain, Amanda Kau et al. (+1 more)
Generalised object counting aims to estimate the number of instances of an arbitrary object category from a single image, but many recent methods can struggle on structurally complex objects due to limited spatial modelling. We present UpCount, a class-agnostic counter designed to better preserve spatial structure. UpCount strengthens the visual representation by extracting multi-layer features from a ViT-B/16 encoder and reassembling them into a refined multi-scale pyramid that is spatially refined using Dense Prediction
Transformers and FeatUp, yielding features with improved structural and spatial sensitivity; a proposal–verification counting head then identifies repeated patterns and produces a density map for the final count. On FSC-147, UpCount achieves 12.39 MAE and 100.89 RMSE on the test set, and it transfers effectively to vehicle counting on CARPK (6.27 MAE, 8.79 RMSE). Code: https://github.com/r28112072-rgb/upcount
Read the full abstract ▾
All authors Robert Wijaya, Md. Tanvir Hossain, Amanda Kau, Ngai-Man Cheung
Computer Vision
Test-Time Registers as Global Priors for Tokenized Image Generation
Cheng-Yao Hong, Yifan Wang, Yuewei Lin et al. (+1 more)
Attention-based models often develop
attention sinks, where a small number of
tokens repeatedly attract
attention and accumulate unusually large activations. In
vision transformers, these outliers are closely related to registers, which have been diagnostically linked to global, low-frequency image structure. Existing work has largely studied registers through interpretability analyses and linear probes, leaving open whether they can be operationalized as plug-and-play signals for generation without retraining. We revisit this question in tokenized image generation. Using OpenCLIP and DINOv2 on ImageNet, we find that test-time register features exhibit stronger low-frequency concentration than both [CLS] readouts and patch-mean features, and show a consistent (albeit moderate) correlation with pixel-space DCT low-frequency energy. Motivated by these diagnostics, we introduce RegToken, a training-free procedure that converts register structure into a small set of global prior
tokens by (i) NFN-based layer localization, (ii) TokenRank-guided subspace extraction, and (iii) a projection-and-conservation update on the register subspace. Inserted into a frozen compact 1D
token generation pipeline, RegToken improves ImageNet generation and
alignment metrics (e.g., FID-5k 20.5 to 20.1, SigLIP 3.6 to 3.9) without modifying pretrained weights, and accelerates test-time optimization (Steps@$\tau$ 74 to 52). Overall, our results suggest that structures often viewed as
attention artifacts can be repurposed as lightweight global priors for tokenized generation.
Read the full abstract ▾
All authors Cheng-Yao Hong, Yifan Wang, Yuewei Lin, Chenyu You
Computer Vision
Scene-SAM3D: Multi-View Scene Asset Generation Without Fine-Tuning
Yuqi Zhang, Yadan Luo, Xiangyu Sun et al. (+3 more)
High-quality 3D scene assets are critical for
embodied applications such as robotic manipulation,
navigation, and simulation. Despite their strong object priors, recent single-image 3D generation models such as SAM3D remain insufficient for real-world scenes, where severe occlusions, redundant observations, and cross-view inconsistencies make reliable scene generation challenging. We introduce Scene-SAM3D, a training-free framework that extends SAM3D from single-view object generation to calibrated multi-view scene asset generation. Scene-SAM3D selects a compact set of complementary views, reducing observation redundancy while providing additional evidence for regions occluded in individual views. Based on the selected views, it performs step-efficient latent velocity fusion to integrate multi-view evidence and suppress cross-view conflicts in canonical space. Finally, a lightweight rigid-object Gaussian optimization refines the scene layout within 200 iterations while preserving the generated object geometry. Experiments on Replica and ScanNet++ demonstrate consistent improvements at both instance and scene levels, with our method reducing scene-level CD by 43.8% on Replica and 30.9% on ScanNet++, while cutting flow-model sampling FLOPs and wall-time latency by nearly 20% under the same multi-view setting. Code will be released at https://github.com/xibi777/Scene-SAM3D.
Read the full abstract ▾
All authors Yuqi Zhang, Yadan Luo, Xiangyu Sun, Fengyi Zhang, Zi Huang, Xin Tan
Computer Vision · Cross-listed from cs.LG
Value-Monotonicity Matters: A Concordance Loss for Deep Survival Prediction
Meixu Chen, Kai Wang and Jing Wang
Deep survival models are evaluated almost exclusively by the concordance index (C-index), yet they are commonly trained using likelihood objectives such as the Cox partial likelihood, discrete-time negative log-likelihood, and DeepHit likelihood. This mismatch is usually considered acceptable because the C-index can be recomputed on validation data during training. However, for end-to-end training of high-capacity encoders on small, heavily censored oncology cohorts, frequent C-index evaluation is computationally expensive, making the loss value itself an important signal for monitoring, early stopping, and model selection. We show that likelihood losses are unreliable for this purpose and propose a value-monotone concordance loss. We prove that every strictly proper survival likelihood admits directions where the loss decreases while the C-index remains unchanged, causing the loss value to decouple from ranking performance. We then study a sigmoid concordance loss (SCL), whose value approximates one minus the C-index up to a temperature term, ensuring that lower loss corresponds to higher C-index during optimization. The loss is architecture agnostic and reduces to a convex survival ranking support vector machine for linear models. Across eighteen
datasets from four modalities using a unified five-fold cross-validation protocol, SCL achieves discrimination comparable to standard likelihood losses and is the best or within one standard deviation of the best C-index. Unlike likelihood losses, SCL maintains a strong correlation between loss value and C-index during training, with rank correlations of 0.96 to 0.99 compared with -0.03 to 0.53 for likelihood losses. Calibration measured by the integrated Brier score is comparable. SCL provides a value-monotone optimization objective whose value can serve as a reliable surrogate for the C-index during expensive end-to-end training.
Read the full abstract ▾
Computer Vision
HTT-Net: Hierarchical Text-guided Transition Modeling for Surgical Video Phase Recognition
Kunjie Deng, Jinghui Zhang, Weidong Chen et al. (+4 more)
Surgical video phase recognition is a fundamental task in computer-assisted intervention, supporting workflow understanding, intraoperative guidance, and
surgical quality assessment. Although recent visual-temporal models have achieved promising progress, accurate and temporally coherent phase recognition remains challenging due to local visual ambiguity, transient prediction noise, and insufficient use of procedural semantics. To address these challenges, we propose HTT-Net, a Hierarchical Text-guided Transition modeling Network for
surgical video phase recognition. The key idea is to introduce structured
surgical semantic knowledge into phase-aware segment construction and semantic refinement. Specifically, we construct a hierarchical
surgical semantic
memory with intra-phase descriptions, inter-phase transition descriptions, and fine-grained semantic units. Based on this
memory, the proposed Transition-Aware Segment Construction (TAS-Con) organizes frame-level evidence into coherent segment representations and handles boundary clips with inter-phase transition descriptions. Furthermore, we introduce Transition-Aware Segment Calibration (TAS-Calib), which calibrates phase-aware segment representations through hierarchical
surgical semantics and improves discrimination under visual ambiguity without dense frame-level
vision-
language fusion. Experiments on Cholec80 and LCRS-100 demonstrate the effectiveness of HTT-Net for robust
surgical video phase recognition.
Read the full abstract ▾
All authors Kunjie Deng, Jinghui Zhang, Weidong Chen, Ganbin Li, Xiangjun Lyu, Zhendong Mao, Yingchi Yang
Computer Vision
Spatiotemporal Facial Action Unit Detection using Twin Cycle Autoencoders for Driver Monitoring
Sai Sidharth D
Driver monitoring systems (DMS) increasingly rely on facial cues to infer drowsiness, distraction, and cognitive load in real time. Facial Action Units (AUs), grounded in the Facial Action Coding System (FACS), provide an objective and interpretable representation of such states, but their automatic detection in the driving context is complicated by low and variable illumination, partial occlusion, head-pose variation, and the subtlety and short duration of relevant AU activations. Existing AU detectors largely treat spatial appearance and temporal dynamics separately, limiting their ability to exploit self-supervisory signal from abundant unlabeled driving
video. We propose the Twin Cycle Autoencoder (TCA), a spatiotemporal architecture composed of two coupled cycle-consistent autoencoder branches: a Spatial Cycle Autoencoder that disentangles AU-relevant appearance from identity through image-level cycle consistency, and a Temporal Cycle Autoencoder that enforces forward-backward consistency over latent AU trajectories to capture onset-apex-offset dynamics. The two branches are coupled through a cross-branch latent
alignment loss and fused via an
attention module before multi-label AU classification. We evaluate TCA on the DISFA and BP4D
benchmarks and on an in-cabin naturalistic driving
dataset, and observe consistent improvements over CNN-RNN, 3D-CNN, and graph-based AU baselines, particularly for low-intensity and rapidly transitioning AUs relevant to fatigue (AU45, AU43) and yawning (AU26). We further show the model sustains real-time throughput on an embedded Jetson Xavier NX platform, supporting its use in production-grade advanced driver assistance systems (ADAS).
Read the full abstract ▾
Computer Vision
Hybrid Machine Learning for Articulation Angle Estimation of Truck-Semitrailer Combinations
Qixuan Zhang, Jonas Boettcher, Simon F. G. Ehlers et al. (+1 more)
Accurate articulation angle estimation of trucks with trailers is critical for autonomous driving and advanced driver assistance system (ADAS). Existing methods either require manual initialization, additional sensors, or prior knowledge and signals from trailers, or they lack real-world validation, limiting practical deployment. This paper presents multiple learning-based models to directly estimate articulation angles from visual and kinematic inputs, eliminating the need for dedicated driving maneuvers for initialization, bounding box annotations, trailer-mounted sensor signals, or prior knowledge of trailer parameters. Two learning-based models are integrated with a kinematic model within an extended Kalman filter (EKF) framework, and an adaptive weighting scheme based on uncertainty quantification is applied for measurements involving visual input. Extensive real-world experiments with different trailer types demonstrate the approaches' robustness and generalization under out-of-domain conditions, including new trailers, varying colors, and lighting conditions. Results show that the hybrid method achieves accurate and reliable articulation angle estimation while maintaining reduced implementation requirements and practical deployment advantages.
Read the full abstract ▾
All authors Qixuan Zhang, Jonas Boettcher, Simon F. G. Ehlers, Marvin Stuede
Computer Vision
VisionAssist: An Open-Source Smartphone Assistant for AI-Based Visual Accessibility
Ayşe Özlem Çal{\i}şkan and Jordi Sanchez-Riera
People with low
vision often face challenges in performing everyday tasks that require interpreting visual information. We present VisionAssist, an open-source mobile application designed to improve independence by providing AI-powered visual assistance through a smartphone. The application integrates three complementary functionalities within a single interface. First, it enables users to locate specific objects by analyzing the live camera feed. Second, it generates spoken descriptions of captured images, allowing users to identify visual content such as food labels, documents, and everyday objects. Third, it integrates with the smartphone's contacts and calendar to facilitate emergency calls and provide voice-based reminders. The application supports hands-free interaction through voice commands and delivers all feedback using text-to-
speech synthesis, making it fully accessible to users with visual impairments. By combining multiple assistive services into a unified platform and releasing the project as open-source software, the proposed solution aims to encourage community contributions and accelerate the development of accessible technologies. The source code is publicly available at: https://github.com/AOzlemC/LowVisionProject.git
Read the full abstract ▾