Computer Vision
Making Implicit Preservation Intent Explicit in Conversational Image Editing
Soomin Han, Jihyung Ahn, Bumsoo Kim et al. (+1 more)
Conversational image editing requires preserving not only visible content, but also content that temporarily disappears across turns. When newly added or modified content occludes a previously visible region, that region should reappear if it was never semantically changed. However, existing systems often fail to recover such occluded-but-unchanged content, producing inconsistent or hallucinated results. We introduce OCCUR-Bench, a diagnostic
benchmark for temporal preservation in conversational image editing. OCCUR-Bench provides diverse occlusion-and-revelation scenarios with historical restoration references, enabling evaluation of faithful restoration rather than plausible regeneration. We also propose ReSpec, a training-free framework that makes implicit preservation explicit by pairing restoration-aware instructions with historical visual references. Given an editing history, ReSpec identifies what should persist, selects the historical image state that provides missing visual evidence, and conditions an in-context editor on the resulting instruction and reference image. Experiments show that ReSpec improves restoration fidelity and temporal consistency on OCCUR-Bench, highlighting the need to ground preservation in editing history rather than only the current image.
Read the full abstract ▾
All authors Soomin Han, Jihyung Ahn, Bumsoo Kim, Buru Chang
Computer Vision · Cross-listed from eess.IV
From Data Completeness to Data Sufficiency: A Task-Driven Imaging Framework for Intraoperative CBCT under Quality-Time-Dose Trade-offs
Yi Jia, Rongjun Ge, Yang Chen et al. (+2 more)
Mobile C-arm cone-beam computed
tomography (CBCT) has been widely used for real-time intraoperative 3D imaging. However, current practice often mechanically applies the fan-beam CT criterion of "180{\deg} plus fan angle" in pursuit of "data completeness" in reconstruction. This review argues that, under the single circular trajectory of three-dimensional cone-beam geometry, complete data are mathematically unattainable; moreover, blindly increasing sampling may exacerbate the trade-off among intraoperative image quality (Q), imaging time (T), and radiation dose (D). Against this background, this review reframes the evaluation of intraoperative CBCT around "data sufficiency" rather than "data completeness." This perspective moves beyond the excessive pursuit of absolute mathematical and analytic accuracy, and instead emphasizes task-specific minimum image-quality thresholds required for clinical decision-making. By synthesizing evidence from multiple clinical scenarios, this review suggests that approximation errors can be acceptable when clinical decision-making requirements are satisfied, thereby achieving a Q-T-D balance.
Read the full abstract ▾
All authors Yi Jia, Rongjun Ge, Yang Chen, Yan Xi, Wenjun Xia
Computer Vision
Counterfactual Context Auditing for Glioma Segmentation Failure Detection Across MRI Datasets
Nguyen Linh Dan Le, Tran Dang Khoi and Nguyen Pham Hoang Le
Glioma
segmentation models can lose accuracy across MRI
datasets, while identifying the cases that require correction remains difficult. In this study, we introduce counterfactual context transport (CCT), which replaces a scan's feature statistics with those from training scans and measures the resulting changes in tumor masks. We combine this sensitivity with prediction uncertainty to rank cases for review without retraining or additional
segmentation networks. Experiments on BraTS and UTSW show that context sensitivity provides evidence of
segmentation failure that uncertainty in the original prediction alone does not consistently capture. Combining both signals gives the strongest average failure detection among the evaluated methods, including an ensemble of separately trained networks. These findings support using changes in tumor predictions under context replacement to help identify cases that require expert review.
Read the full abstract ▾
Computer Vision
AnchorPrune: Relevance-Anchored Contextual Expansion for Visual Token Pruning
Kyuan Oh and Bumsoo Kim
Large
vision-
language models incur substantial inference costs because high-resolution inputs introduce thousands of visual
tokens, many of which are redundant for a given query. Existing
pruning methods often combine query relevance and
token diversity, yet these objectives can conflict under aggressive compression: relevance-driven selection may overconcentrate the budget on correlated local evidence, while diversity-driven selection may suppress indispensable
tokens or retain distinct but uninformative regions. We introduce AnchorPrune, a training-free framework that first constructs a protected relevance anchor and then expands it with complementary visual context. AnchorPrune adaptively determines the anchor size from the novelty profile of relevance-ranked
tokens, preserving a compact set of query-critical evidence, and allocates the remaining budget through importance-weighted novelty to recover informative, non-redundant context relative to the anchor. This ordered design prevents contextual expansion from displacing indispensable query cues while improving overall visual coverage. AnchorPrune is lightweight, architecture-aware, and requires neither retraining nor model modification. Across image and
video vision-
language models and
benchmarks, it consistently improves the accuracy-efficiency trade-off over training-free baselines, particularly under severe compression. On LLaVA-NeXT-7B, AnchorPrune preserves 97.6% of full-
token performance using only 160 of 2,880 visual
tokens. These results establish relevance-anchored contextual expansion as an effective principle for efficient
multimodal inference. Code is available at https://github.com/MULTI-cau/AnchorPrune.
Read the full abstract ▾
Computer Vision · Cross-listed from cs.LG
Latent graph encoding of multimodal neuroimaging features with generative AI architectures
Ishaan Batta, Meenu Ajith and Vince Calhoun
While generative models enable encoding of complex neuroimaging data for feature generation and reconstruction, developing optimal architectural frameworks with appropriate encoding and latent space processes is crucial for studying structural and functional properties of the
brain. We design a
multimodal generative framework for structural and functional
magnetic resonance imaging (MRI) features through systematic evaluation of encoding strategies, latent
multimodal fusion, and generative model selection. Using structural gray matter volume (GMV) and static functional network connectivity (sFNC) features from a large neuroimaging
dataset, we analyze generative frameworks involving variational autoencoders (VAEs),
transformers, generative
adversarial networks (GANs), and
diffusion models. Architectures that employ modality-aware graph encoding of functional connectivity into a lower-dimensional latent space outperform vectorized encoders or direct data space approaches. The proposed
multimodal graph VAE (gMMVAE) surpasses alternative generative variants across multiple metrics for generation fidelity, reconstruction quality, efficiency, and latent space discriminability, highlighting its potential for robust
multimodal neuroimaging analysis.
Read the full abstract ▾
Computer Vision
SHTA: Semantic Hard Token Correction and Center Alignment for Semi-Supervised Medical Image Segmentation
Zhuoru Zhang, Yiheng Zhong, Zimu Zhang et al. (+1 more)
Recent advances in semi-supervised medical image
segmentation have achieved remarkable performance through prediction consistency, pseudo-label supervision, and hard-region supervision. However, these methods primarily improve supervision quality rather than explicitly enforcing semantic consistency in the learned representations of hard regions. Consequently, even under increasingly stronger prediction-level supervision, difficult regions exhibiting unstable semantic assignment often fail to establish semantically consistent representations during training, thereby limiting further
segmentation improvement. To address this issue, we propose SHTA (Semantic Hard
Token Correction and Center
Alignment), a lightweight training-time semantic representation branch. Instead of introducing additional prediction supervision, SHTA refines intermediate semantic representations through Semantic Assignment, Hard
Token Refinement, and Semantic Center
Alignment, thereby improving semantic consistency in hard regions while preserving the original prediction pathway and introducing no additional inference cost. We integrate SHTA into representative semi-supervised
segmentation frameworks, including GA-CPS, CPS, URPC, and MagicNet, and conduct evaluations on the Synapse and AMOS
datasets. Experimental results demonstrate that SHTA delivers consistent paired improvements across frameworks, with especially clear gains in
segmentation accuracy, weak-organ recovery, and semantic ambiguity reduction, while incurring only training-time overhead. The code is available at https://anonymous.4open.science/r/release_SHTA-42D5/.
Read the full abstract ▾
All authors Zhuoru Zhang, Yiheng Zhong, Zimu Zhang, Xiaofeng Liu
Computer Vision
Ego-Human Motion Prediction with 3D-Aware LLM
Yujin Bae, Jaewoo Jeong, Hyeonseong Kim et al. (+1 more)
Anticipating human motion from an
egocentric perspective is fundamental for proactive assistance in AR/VR, human-
robot collaboration, and
embodied AI. While recent works incorporate
language as a semantic prior to reduce the ill-posed nature of
egocentric forecasting, they largely neglect the 3D spatial and semantic context that governs how motion unfolds, and treat pose and
language prediction as separate inference streams. We introduce Ego3DLM, built on two core principles: accurate motion
forecasting requires explicit spatial and semantic understanding of the 3D environment, and pose and
language must be predicted holistically in a single pass, since motion is inherently tied to the semantic interpretation of actions being performed. Given three-point tracking, 3D scene features, and
egocentric video, Ego3DLM simultaneously decodes past pose, future pose, past narration, and future narration in a single autoregressive pass, grounding predicted poses and descriptions in one another to enforce cross-modal and temporal consistency. We adopt a three-stage training scheme: (1) spatial-semantic scene awareness pretraining; (2) holistic instruction tuning over all four outputs in a single pass; and (3) GRPO-based reinforcement finetuning with intra- and inter-modal rewards that directly optimize pose-
language fidelity. Experiments on the Nymeria
benchmark demonstrate that Ego3DLM achieves state-of-the-art performance across future motion prediction, past motion tracking, and motion description, showing that 3D scene grounding and holistic cross-modal prediction yield physically plausible and semantically coherent motion forecasts. The project page is available at https://jaewoo97.github.io/Ego3DLM/.
Read the full abstract ▾
All authors Yujin Bae, Jaewoo Jeong, Hyeonseong Kim, Kuk-Jin Yoon
Computer Vision
EdgeCompress: Coupling Multidimensional Model Compression and Dynamic Inference for EdgeAI
Hao Kong, Di Liu, Shuo Huai et al. (+5 more)
Convolutional neural networks (CNNs) have demonstrated encouraging results in image classification tasks. However, the prohibitive computational cost of CNNs hinders the deployment of CNNs onto resource-constrained embedded devices. To address this issue, we propose EdgeCompress, a comprehensive compression framework to reduce the computational overhead of CNNs. In EdgeCompress, we first introduce dynamic image cropping (DIC), where we design a lightweight foreground predictor to accurately crop the most informative foreground object of input images for inference, which avoids redundant computation on background regions. Subsequently, we present compound shrinking (CS) to collaboratively compress the three dimensions (depth, width, and resolution) of CNNs according to their contribution to accuracy and model computation. DIC and CS together constitute a multidimensional CNN compression framework, which is able to comprehensively reduce the computational redundancy in both input images and neural network architectures, thereby improving the inference efficiency of CNNs. Further, we present a dynamic inference framework to efficiently process input images with different recognition difficulties, where we cascade multiple models with different complexities from our compression framework and dynamically adopt different models for different input images, which further compresses the computational redundancy and improves the inference efficiency of CNNs, facilitating the deployment of advanced CNNs onto embedded hardware. Experiments on ImageNet-1K demonstrate that EdgeCompress reduces the computation of ResNet-50 by 48.8% while improving the top-1 accuracy by 0.8%. Meanwhile, we improve the accuracy by 4.1% with similar computation compared to HRank, the state-of-the-art compression framework. The source code and models are available at https://github.com/ntuliuteam/edge-compress
Read the full abstract ▾
All authors Hao Kong, Di Liu, Shuo Huai, Xiangzhong Luo, Ravi Subramaniam, Christian Makaya, Qian Lin, Weichen Liu
Computer Vision
HPR-SAM: Hierarchical Probabilistic Representation Learning for Prompt-free SAM-based Medical Image Segmentation
Yingzhen Hu, Yiheng Zhong, Keying Zhu et al. (+5 more)
Prompt-free adaptation of the Segment Anything Model (SAM) has emerged as a promising paradigm for automatic medical image
segmentation. Existing methods mainly focus on
prompt generation, while overlooking that
prompt quality is fundamentally constrained by the expressiveness of anatomical representations. However, deterministic prototypes or semantic
tokens are insufficient to jointly capture global anatomical priors, intra-structure diversity, and local structural reliability. To address this limitation, we propose the Hierarchical Probabilistic Representation (HPR) framework, which learns complementary anatomical representations through Distributional Anatomical Representation (DAR), Multi-component Anatomical Representation (MAR), and Local Reliability Representation (LRR), and integrates their predictions via Hierarchical Prediction Fusion (HPF) while remaining compatible with the original SAM decoder. Experiments on the Synapse, LA, and PROMISE12
datasets demonstrate that HPR-SAM achieves state-of-the-art performance on Synapse and the best performance under few-shot settings on LA and PROMISE12, validating the effectiveness of the proposed hierarchical probabilistic representation learning framework for
prompt-free medical image
segmentation. Code is available at https://anonymous.4open.science/r/HPR-SAM-E4AF.
Read the full abstract ▾
All authors Yingzhen Hu, Yiheng Zhong, Keying Zhu, Zimu Zhang, Zihan Ye, Sifan Song, Jionglong Su, Xiaofeng Liu
Computer Vision
SpiS-GAN: Spiral-Modulated Handwriting Synthesis with Star Operation
Nguyen Duy Hieu, Dang Hoai Nam, Pham Hoang Giap et al. (+2 more)
Training robust handwriting recognition (HTR) systems requires massive amounts of annotated data, which is often difficult to acquire. While synthetic handwriting generation offers a practical solution to expand training sets, existing models struggle with several core issues. First, previous approaches, even MLP-based models fail to effectively trace cursive handwriting due to fixed-grid spatial receptive field. Second, their CNN-relied discriminators usually lose structural details through aggressive downsampling, making broken connections difficult to detect. Third, existing architectures are either limited to linear feature interactions or too expensive for high-resolution synthesis. Finally, existing approaches lack explicit edge constraints, often resulting in blurred stroke boundaries. To address these challenges, this study proposes a Spiral-Modulated Handwriting Synthesis framework based on Generative
Adversarial Networks (SpiS-GAN). Our generator employs
Star-Spiral Blocks combining proposed Modulated Elliptical SpiralFC with the
star operation to capture spatial relationships and efficiently follow complex handwriting stroke trajectories, while a Spiral-Modulated discriminator is introduced for multi-domain flaws detection. Additionally, we introduce a Sobel-Regularized Edge Reconstruction Loss that provides edge guidance, ensuring every character remains clear and legible. Evaluations on the English and Vietnamese
datasets demonstrate that SpiS-GAN significantly outperforms current state-of-the-art models. The generated images are highly authentic, accurately preserve the original writer's style across
languages, and successfully lower error rates when training downstream HTR systems.
Read the full abstract ▾
All authors Nguyen Duy Hieu, Dang Hoai Nam, Pham Hoang Giap, Quang Huu Hieu, Vo Nguyen Le Duy
Computer Vision
Self-Supervised Pretraining Improves Cross-Site and Cross-Scale Robustness of Point Cloud Leaf-Wood Segmentation
Heeju Mun, Tackang Yang, Yunsoo Nam et al. (+1 more)
The accuracy of existing leaf-wood
segmentation methods for tree point clouds varies across forest types and sites. Self-supervised learning (SSL) on point clouds has improved the generalization of deep learning models for forestry point cloud tasks, including biomass regression and individual tree
segmentation, but its applicability to leaf-wood
segmentation remains untested. In this study, we pretrained Point-M2AE, a widely used SSL architecture for point clouds, on ShapeNet-55 augmented with 2,400 individual tree point clouds. For fine-tuning and inference, we used recursive voxel subdivision to handle the wide variation in point density across inputs, allowing the same model to operate at both individual-tree and plot scales without architecture change. Compared to the model without pretraining, the pretrained model improved wood IoU from 60.5% to 70.0% for needleleaf and from 69.7% to 76.3% for broadleaf trees. On a
benchmark spanning four countries across three climatic zones, the pretrained model achieved the smallest cross-site variation and highest overall performance among compared methods (LeWos, CWLS, and PointTransformer). Plot-level
segmentation maintained accuracy comparable to individual-tree performance, with mIoU of 84.7% for broadleaf and 77.7% for needleleaf plots, showing that the model generalizes across scales without additional finetuning. As a downstream test in tropical forests, where dense canopies make
segmentation challenging, we applied our model and a quantitative structure model to estimate wood volume for 28 trees from Guyana, Indonesia, and Peru to assess whether the
segmentation improvements from SSL pretraining translate into improved downstream performance. The resulting volume estimates achieved the lowest error among all methods tested (MAE = 2.40 m$^3$), less than half that of algorithmic baselines (LeWos: 5.94 m$^3$; CWLS: 5.27 m$^3$).
Read the full abstract ▾
All authors Heeju Mun, Tackang Yang, Yunsoo Nam, Changhyun Choi
Computer Vision
General Incomplete Multimodal Learning via Dynamic Quality Perception
Xiangyu Meng and Shicai Wei
Multimodal learning robust to missing modalities is essential for real-world applications. Existing methods mainly focus on inter-modality missing, where entire modalities are absent, while overlooking intra-modality degradation, where modalities are present but severely corrupted. In practice, these two types of missing often coexist, making existing approaches ineffective. To address this limitation, we propose General Incomplete
Multimodal Learning (GIML), a unified framework that simultaneously handles both inter-modality missing and intra-modality degradation through dynamic quality perception. Specifically, GIML models heterogeneous missing patterns as continuous modality information degradation, enabling degradation-aware adaptive fusion. To achieve reliable quality perception, we introduce a Noise-aware Quality
Estimator that learns the mapping from corrupted features to noise intensity through controlled noise injection. Furthermore, we propose a Noise-Semantic Decoupled module that separates semantic information from noise interference. This improves robustness and generalization to unseen corruption patterns. Extensive experiments across
datasets with diverse modality types demonstrate the effectiveness and generality of GIML. Code is available at: https://github.com/Yu-Five/GIML.
Read the full abstract ▾
Computer Vision
Bi-PT: Bidirectional Cross-Attention Point Transformers for Four-Chamber Heart Reconstruction from Sparse Cardiac MRI Data
Chenchuhui Hu, Shaoming Pan, Leon Axel et al. (+1 more)
We propose Bi-PT, a pipeline for reconstructing 3D four-chamber human heart meshes from clinical sparsely sampled cardiac
magnetic resonance imaging (CMR) data. This work addresses the error-prone generation of 3D cardiac shape from a sparse point cloud (SPC) extracted from 2D long-axis and short-axis views used in routine clinical CMR protocols. Bi-PT enables accurate inference of the four-chamber heart mesh from the SPC by learning robust point features via bidirectional point cross-
attention between an atlas and the SPC, together with per-point semantic labels that improve correspondence estimation. We formulate the deformation field as a Neural Ordinary Differential Equation (NODE) parameterized by a per-point affine transformation and translation to deform the atlas toward the target heart shape. By learning such a NODE, we can guarantee the deformation field to be a locally affine diffeomorphic deformation. We also integrate a semantic label loss into the Chamfer distance to encourage label-consistent correspondences and add a smoothness regularization to stabilize and improve the learning of the deformation field. Extensive experiments demonstrate that Bi-PT achieves accurate and robust performance compared to baselines.
Read the full abstract ▾
All authors Chenchuhui Hu, Shaoming Pan, Leon Axel, Meng Ye
Computer Vision · Cross-listed from cs.LG
Latency-Constrained DNN Architecture Learning for Edge Systems using Zerorized Batch Normalization
Shuo Huai, Di Liu, Hao Kong et al. (+4 more)
Deep learning applications have been widely adopted on edge devices, to mitigate the
privacy and latency issues of accessing cloud servers. Deciding the number of
neurons during the design of a deep neural network to maximize performance is not intuitive. Particularly, many application scenarios are real-time and have a strict latency constraint, while conventional neural network optimization methods do not directly change the temporal cost of model inference for latency-critical edge systems. In this work, we propose a latency-oriented neural network learning method to optimize models for high accuracy while fulfilling the latency constraint. For efficiency, we also introduce a universal hardware-customized latency predictor to optimize this procedure to learn a model that satisfies the latency constraint by only a one-shot training process. The experiment results reveal that, compared to state-of-the-art methods, our approach can well-fit the 'hard' latency constraint and achieve high accuracy. Under the same training settings as the original model and satisfying a 34 ms latency constraint on the ImageNet-100
dataset, we reduce GoogLeNet's latency from 40.32 ms to 34 ms with a 0.14% accuracy reduction on the NVIDIA Jetson Nano. When coupled with quantization, our method can be further improved to only 0.04% drop for GoogLeNet. On the NVIDIA Jetson TX2, we compress VGG-19 from 119.98 ms to 34 ms and even improve its accuracy by 0.5%, and we scale GoogLeNet up from 20.27 ms to 34 ms and achieve higher accuracy by 0.78%. We also open source this framework at https://github.com/ntuliuteam/ZeroBN
Read the full abstract ▾
All authors Shuo Huai, Di Liu, Hao Kong, Weichen Liu, Ravi Subramaniam, Christian Makaya, Qian Lin
Computer Vision
Compass: Prostate Cancer Detection Needs Multi-View Context
Paul F.R. Wilson, Mohamed Harmanani, Zhuoxin Guo et al. (+6 more)
Artificial intelligence (AI) analysis of micro-ultrasound ($\mu$US) has shown promise for prostate cancer (PCa) detection. However, most existing AI methods focus on the analysis of single $\mu$US images in isolation. By contrast, expert $\mu$US readers typically assess a full recorded
video study, which provides three-dimensional context, to improve PCa detection compared to single-frame analysis. Inspired by this clinical workflow, we propose Compass, a novel AI methodology which models a $\mu$US study as a stream of 2D images. Compass jointly integrates rotational sweep
videos of the prostate with $\mu$US frames acquired at the moment of biopsy, and performs evidence aggregation across the study using a
transformer conditioned on the probe's rotational angle. Finally, a decoder head predicts frame-level and study-level risk scores for the patient. The model is trained and evaluated using a multi-center clinical trial
dataset of $\mu$US studies, including continuous rotational scans of the prostate and
videos captured during biopsy acquisition. We compare the proposed method to baseline AI methods from the literature and to risk scores provided by clinical experts. Our framework shows strong performance, highlighting the value of multi-view context for $\mu$US PCa detection, and providing a potentially powerful tool to complement human expertise in $\mu$US-based PCa diagnosis. Our code is available at: https://github.com/mharmanani/Compass.
Read the full abstract ▾
All authors Paul F.R. Wilson, Mohamed Harmanani, Zhuoxin Guo, Obed K. Dzikunu, Hannes Cash, Adam Kinnaird, Brian Wodlinger, Purang Abolmaesumi, Parvin Mousavi
Computer Vision
LoCA: Spatially-Aware Low-Rank Convolutional Adaptation of Vision Foundation Models
Sojung An, Junha Lee, Sujeong You et al. (+2 more)
Pre-trained
Vision Foundation Models (VFMs) provide strong visual representations for diverse downstream tasks. The key challenge of VFM adaptation stems from the prohibitive costs of full fine-tuning and catastrophic forgetting. To address this, Low-Rank Adaptation (LoRA) has emerged as the prevailing paradigm for Parameter-Efficient Fine-Tuning (PEFT). However, LoRA is typically designed for
transformer self-
attention layers parameterized by 2D
matrices. Since convolutional kernels inherently couple spatial and channel information within a 4D tensor, forcing them into a monolithic 2D
matrix disrupts the inherent spatial
topology. In this paper, we propose Low-Rank Convolutional Adaptation (LoCA), a convolution-aware PEFT framework that addresses spatial-channel
entanglement by decoupling channel and spatial adaptation. LoCA introduces a low-rank channel adaptation for dense cross-channel mixing and refines spatial bases extracted from pre-trained kernels via Singular Value Decomposition (SVD). Experimental results show that LoCA preserves pre-trained spatial priors and achieves competitive or state-of-the-art performance across fine-grained classification, domain-generalized semantic
segmentation, and generative
benchmarks.
Read the full abstract ▾
All authors Sojung An, Junha Lee, Sujeong You, Nam Ik Cho, Donghyun Kim
Computer Vision
Smart Scissor: Coupling Spatial Redundancy Reduction and CNN Compression for Embedded Hardware
Hao Kong, Di Liu, Shuo Huai et al. (+5 more)
Scaling down the resolution of input images can greatly reduce the computational overhead of convolutional neural networks (CNNs), which is promising for edge AI. However, as an image usually contains much spatial redundancy, e.g., background pixels, directly shrinking the whole image will lose important features of the foreground object and lead to severe accuracy degradation. In this paper, we propose a dynamic image cropping framework to reduce the spatial redundancy by accurately cropping the foreground object from images. To achieve the instance-aware fine cropping, we introduce a lightweight foreground predictor to efficiently localize and crop the foreground of an image. The finely cropped images can be correctly recognized even at a small resolution. Meanwhile, computational redundancy also exists in CNN architectures. To pursue higher execution efficiency on resource-constrained embedded devices, we also propose a compound shrinking strategy to coordinately compress the three dimensions (depth, width, resolution) of CNNs. Eventually, we seamlessly combine the proposed dynamic image cropping and compound shrinking into a unified compression framework, Smart Scissor, which is expected to significantly reduce the computational overhead of CNNs while still maintaining high accuracy. Experiments on ImageNet-1K demonstrate that our method reduces the computational cost of ResNet50 by 41.5% while improving the top-1 accuracy by 0.3%. Moreover, compared to HRank, the state-of-the-art CNN compression framework, our method achieves 4.1% higher top-1 accuracy at the same computational cost. The codes and data are available at https://github.com/ntuliuteam/smart-scissor
Read the full abstract ▾
All authors Hao Kong, Di Liu, Shuo Huai, Xiangzhong Luo, Weichen Liu, Ravi Subramaniam, Christian Makaya, Qian Lin
Computer Vision
Seeing What Matters: Lesion-Aware High-Resolution Patch Discovery and Fusion for Chest X-ray Report Generation
Yingshu Li, Yunyi Liu, Zhenghao Chen et al. (+5 more)
Despite rapid advances in chest X-ray (CXR) foundation models, most radiology report generation (RRG) systems still rely on heavily downsampled inputs (e.g., 256x256) due to the fixed visual
token budgets of pretrained
vision encoders, suppressing subtle yet clinically important cues present in native-resolution images. However, enabling high-resolution (high-res) perception remains challenging: naive tiling causes prohibitive
token inflation, while global compression suppresses subtle lesions and degrades diagnostic fidelity. Inspired by radiologists' workflow, localizing suspicious regions before detailed high-res assessment. We propose Lesion-Aware High-Resolution Patch Discovery and Fusion for Chest X-ray Reporting (LePaX), the first RRG framework that enables efficient high-res CXR perception (up to 1920x1920) without increasing the
vision-
token count. LePaX formulates high-res perception as a constrained spatial resolution allocation problem under a fixed
token budget and introduces two key components: Learnable Spatial Resolution Allocation (LSRA), which learns a spatial utility map that adaptively allocates limited high-res capacity to diagnostically relevant regions, enabling targeted extraction of high-res patches from native CXRs; and Global-Regional Fusion (GRF), which performs
token-preserving region-to-global refinement by projecting high-resolution regional evidence back onto the global feature grid through spatially aligned resolution write-back, avoiding
token inflation. Experiments on multiple CXR
benchmarks demonstrate that LePaX consistently improves both clinical and linguistic metrics while enabling native-resolution CXR perception with over 10x fewer visual
tokens than naive high-res tiling.
Read the full abstract ▾
All authors Yingshu Li, Yunyi Liu, Zhenghao Chen, Tong Chen, Zailong Chen, Lingqiao Liu, Lei Wang, Luping Zhou
Computer Vision · Cross-listed from cs.RO
Dynamic Object Detection and Tracking in Construction: A Fisheye Camera and LiDAR Sensor Fusion Model
Yilong Chen, Huili Huang and Yong K. Cho
Robust dynamic object detection and tracking are essential for enabling
robots to operate safely and effectively alongside humans in complex environments such as construction sites. While
LiDAR-based SLAM and occupancy grid methods offer viable solutions for detecting and tracking motion, many state-of-the-art 3D
vision approaches rely heavily on pre-trained neural networks and require additional post-processing to identify moving objects. Sensor fusion techniques, combining the precision of
LiDAR with the semantic richness of RGB imagery, offer a promising alternative. In this work, we present a novel framework that enhances a quadruped
robot equipped with a
LiDAR sensor and an upward-facing fisheye camera for real-time dynamic object detection and tracking. After identifying moving objects within a registered point cloud, our method assigns semantic labels by projecting 3D coordinates onto a 2D cylindrical panorama, aligning with real-time image-based detections for observation update of the Kalman filter. The proposed system demonstrates high precision, simplicity, and robustness, particularly in handling objects transitioning between dynamic and static states, thus it is well-suited for deployment in real-world construction environments.
Read the full abstract ▾
Computer Vision
ReMoDEx: A Local-to-Global Relevance-Based Model Decision Explainability Framework for large-Scale Image Datasets
Abhay Kumar Pathak, Mrityunjay Chaubey and Manjari Gupta
Deep learning image classifiers achieve strong predictive performance yet remain opaque in how decisions are formed. A model may predict correctly while relying on irrelevant cues, shortcut associations, peripheral structures, or device level artifacts instead of task relevant regions. On large scale
datasets this opacity is especially problematic, since inspecting heatmaps one sample at a time cannot scale to thousands of predictions. We propose Relevance Based Model Decision Explainability (ReMoDEx), a framework for systematic,
dataset scale assessment of model decision behaviour in image classification. ReMoDEx defines a stepwise pipeline: model inference, target class selection, relevance map generation, heatmap standardisation, similarity based grouping of patterns, cluster level interpretation, and spatial relevance assessment. Local methods GradCAM++, Integrated Gradients, Occlusion Sensitivity, and Layerwise Relevance Propagation are each combined independently with a single global module that summarises an entire set of relevance maps into a few decision strategy clusters, replacing sample by sample inspection with an automatic, scalable summary. To demonstrate ReMoDEx, we applied it to a VGG16 based classifier distinguishing COVID-19, Normal, Lung Opacity, and Viral Pneumonia. The classifier showed stable performance (86.27% test accuracy, 0.9624 test AUC). However, each explainer combined with the global module consistently produced two recurring strategies: central thoracic region decisions and border/corner sensitive decisions, indicating possible shortcut learning that conventional metrics could not reveal. Masked image validation confirmed that model confidence and predicted class changed when central or peripheral regions were occluded. ReMoDEx thus provides a scalable relevance based decision assessment framework and an essential complement to accuracy based evaluation.
Read the full abstract ▾
Computer Vision
Video2Reaction: Mapping Video to Audience Reaction Distribution in the Wild
Trang Nguyen, Sidong Zhang, Shiv Shankar et al. (+4 more)
Understanding and
forecasting audience reactions to
video content are crucial for improving content creation, recommendation systems, and media analysis. To enable audience reaction prediction and other content engagement applications, we introduce $\textbf{Video2Reaction}$, a
multimodal dataset that maps short movie segments to a distribution of $\textit{induced emotions}$ of viewers in the wild, as expressed through social media. $\textbf{Video2Reaction}$ spans more than 10,000
videos and serves as a reliable
benchmark as well as a training resource for audience reaction prediction. To enable cost-effective continuous annotations as reactions may change over time, we develop a two-stage multi-
agent pipeline using only open-source LLMs, achieving 86% correctness under blind human verification despite the inherently noisy and subjective nature of the task. We establish the first
benchmark for
video-to-reaction-distribution prediction in the wild and show that pretrained foundation
video models fail in zero-shot settings, while finetuning transforms them into state-of-the-art predictors capable of modeling both full reaction distributions and dominant responses from
video alone. However, the task remains challenging: even the strongest methods achieve only 77% Top-3 F1 in dominant reaction prediction (LLaVA-Next), highlighting a substantial gap in modeling collective audience reaction. \modification{
Dataset and code are available at our project page: https://information-fusion-lab-umass.github.io/
video2reaction-bench.github.io
Read the full abstract ▾
All authors Trang Nguyen, Sidong Zhang, Shiv Shankar, Gauri Jagatap, Deepak Chandran, Andrea Fanelli, Madalina Fiterau
Computer Vision
Ensemble Deep Learning Approaches for AI-Altered Video Detection
Laiba Khan, Hung-Mao Wu, Wei Lin et al. (+3 more)
The increasing accessibility of artificial intelligence has led to a rapid rise in AI-generated
videos, making it more difficult to distinguish between real and manipulated content. Many existing detection methods rely on a single model and often struggle to generalize across different types of deepfakes. In this work, we developed a
multimodal deepfake detection system that combines both audio and visual analysis using an ensemble of models. The system includes AASIST for audio-based detection, and EfficientNet, XceptionNet, and MesoNet for analyzing visual features in
video frames. The pipeline takes a
video as input, separates the audio, and extracts face frames using MTCNN. Each model produces a score indicating the likelihood of the input being fake. These scores are then combined using ensemble strategies, including mean averaging and stacking. Mean fusion provides a simple and stable baseline, while stacking uses a trained meta-model to learn how to combine predictions more effectively. Results show that while individual models perform well on the
datasets they were trained on, their performance drops when tested on more diverse
datasets. The ensemble approach helps improve overall robustness by combining predictions from multiple models, leading to more consistent performance across different types of deepfakes. This suggests that using both audio and visual information together is a more reliable approach for deepfake detection. Our results highlight generalization to unseen manipulations as the central open challenge, with average accuracy around 70%.
Read the full abstract ▾
All authors Laiba Khan, Hung-Mao Wu, Wei Lin, Frank Bi, Yousef Abdelhadi, Joshua Jung
Computer Vision
Geometric Collapse: When Vision Models Fail to Verify Physical Causality
Wentao Zhang, Jinhu Qi, Weiqiang Jin et al. (+3 more)
Recent progress in large-scale self-supervised learning has improved dense
geometric prediction, but it remains unclear whether such scaling yields inference-time physical plausibility checks. We propose Scrambled Edges, a controlled counterfactual that injects salient edge-like cues while violating surface continuity, illumination coherence, and occlusion ordering. With energy-matched and structure-matched controls, we isolate the effect of unsupported edge evidence from high-frequency energy and edge sparsity. Across CNN/ViT/SSL depth predictors on NYU Depth v2 and KITTI, Scrambled Edges induce up to 3.2x larger deviation from clean predictions than energy-matched noise; additional
diffusion and flow-matching depth
estimators show attenuated but still significant collapse. The resulting
Geometric Collapse propagates globally: even with oracle knowledge of the corrupted region, output-level repair recovers only 47%, with substantial error outside the mask. These findings provide controlled behavioral evidence that current dense predictors lack reliable mechanisms to quarantine physically unsupported edge cues, motivating explicit plausibility scoring and selective cue integration.
Read the full abstract ▾
All authors Wentao Zhang, Jinhu Qi, Weiqiang Jin, Yifei Zhang, Chan-Tong Lam, Irwin King