Computer Vision
Decoder Generates Manufacturable Structures: A Framework for 3D-Printable Object Synthesis
Abhishek Kumar
This paper presents a novel decoder-based approach for generating manufacturable 3D structures optimized for additive manufacturing. We introduce a deep learning framework that decodes latent representations into geometrically valid, printable objects while respecting manufacturing constraints such as overhang angles, wall thickness, and structural integrity. The methodology demonstrates that neural decoders can learn complex mapping functions from abstract representations to valid 3D geometries, producing parts with significantly improved manufacturability compared to naive generation approaches. We validate the approach on diverse object categories and demonstrate practical 3D printing of decoder-generated structures.
Read the full abstract ▾
Computer Vision
A Unified Attention U-Net Framework for Cross-Modality Tumor Segmentation in MRI and CT
Nishan Rai and Pushpa R. Dahal
This study presents a unified
Attention U-Net architecture trained jointly on MRI (BraTS 2021) and CT (LIDC-IDRI)
datasets to investigate the generalizability of a single model across diverse imaging modalities and anatomical sites. Our proposed pipeline incorporates modality-harmonized preprocessing,
attention-gated skip connections, and a modality-aware Focal Tversky loss function. To the best of our knowledge, this study is among the first to evaluate a single
Attention U-Net trained simultaneously on separate MRI (BraTS) and CT (LIDC-IDRI) tumor
datasets, without relying on modality-specific encoders or domain adaptation. The unified model demonstrates competitive performance in terms of Dice coefficient, IoU, and AUC on both domains, thereby establishing a robust and reproducible baseline for future research in cross-modality tumor
segmentation.
Read the full abstract ▾
Computer Vision
TIR-Flow: Active Video Search and Reasoning with Frozen VLMs
Hongbo Jin, Siyi Xie, Jiayu Ding et al. (+2 more)
While Large
Video-
Language Models (
Video-LLMs) have achieved remarkable progress in perception, their
reasoning capabilities remain a bottleneck. Existing solutions typically resort to a heavy "data engineering" paradigm-synthesizing large-scale Chain-of-Thought (CoT)
datasets followed by Supervised Fine-Tuning (SFT) and Reinforcement Learning (RL). This pipeline primarily optimizes probability sampling efficiency and aligns output distributions, but fails to activate the intrinsic intelligence required for dynamic visual exploration. In this work, we propose TIR-Flow, a novel framework that shifts the paradigm from passive processing to active
video searching and
reasoning without additional data or parameter updating. Concretely, our framework operates through three synergistic modules: HDD decomposes complex queries into a set of verifiable sub-tasks; HAP actively directs visual
attention to gather high-resolution evidence for hypothesis validation; EBA maintains a persistent workspace to accumulate and update the discovered clues for logical
reasoning. Extensive experiments on seven
benchmarks demonstrate that TIR-Flow significantly outperforms recent strong baselines, delivering an average performance boost of 5.9%, with gains reaching 10.5% on Egoschema. Our analysis confirms that empowering frozen VLMs with System-2-like active perception is a scalable path toward solving long-horizon
video reasoning.
Read the full abstract ▾
All authors Hongbo Jin, Siyi Xie, Jiayu Ding, Kuanwei Lin, Ge Li
Computer Vision · Cross-listed from eess.IV
Deep Joint Source-Channel Coding for Wireless Video Transmission with Asymmetric Context
Xuechen Chen, Junting Li, Chuang Chen et al. (+2 more)
In this paper, we propose a high-efficiency deep joint source-channel coding (JSCC) method for
video transmission based on conditional coding with asymmetric context. The conditional coding-based neural
video compression requires to predict the encoding and decoding conditions from the same context which includes the same reconstructed frames. However in JSCC schemes which fall into pseudo-analog transmission, the encoder cannot infer the same reconstructed frames as the decoder even a pipeline of the simulated transmission is constructed at the encoder. In the proposed method, without such a pipeline, we guide and design neural networks to learn encoding and decoding conditions from asymmetric contexts. Additionally, we introduce feature propagation, which allows intermediate features to be independently propagated at the encoder and decoder and help to generate conditions, enabling the framework to greatly leverage temporal correlation while mitigating the problem of error accumulation. To further exploit the performance of the proposed transmission framework, we implement content-adaptive coding which achieves variable bandwidth transmission using
entropy models and masking mechanisms. Experimental results demonstrate that our method outperforms existing deep
video transmission frameworks in terms of performance and effectively mitigates the error accumulation. By mitigating the error accumulation, our schemes can reduce the frequency of inserting intra-frame coding modes, further enhancing performance.
Read the full abstract ▾
All authors Xuechen Chen, Junting Li, Chuang Chen, Hairong Lin, Yishen Li
Computer Vision
Think Bright, Diffuse Nice: Enhancing T2I-ICL via Inductive-Bias Hint Instruction and Query Contrastive Decoding
Zhiyong Ma, Zhenpeng Li, Yuanjie Shi et al. (+3 more)
Text-to-Image In-Context Learning (T2I-ICL) enables customized image synthesis via interleaved text-image examples but faces two mutually reinforcing bottlenecks, compliance failure and prior-dominated
hallucination, that form a vicious cycle degrading generation quality. Existing methods rely on tailored training, which limits flexibility and raises deployment costs. To address these challenges effectively, we propose TBDN, a training-free framework integrating two complementary closed-loop mechanisms: Hint Instruction (HI) and Query Contrastive Decoding (QCD). HI injects task-aware inductive bias via lightweight
prompt engineering to anchor models on contextual mapping rules, thereby mitigating compliance failure. QCD adjusts the decoding distributions of
language models by contrasting full-input and query-omitted distributions, suppressing prior-dominated
hallucination. TBDN achieves State-of-the-Art performance on CoBSAT and Text-to-Image Fast Mini-ImageNet, with robust generalization across model backbones,
prompt designs, and hyperparameters. It also maintains promising performance in concept preservation and
prompt following on Dreambench++. By breaking the two bottlenecks, TBDN establishes a simple yet effective framework for efficient and reliable T2I-ICL.
Read the full abstract ▾
All authors Zhiyong Ma, Zhenpeng Li, Yuanjie Shi, Zhengping Li, Jiahao Chen, Qingyuan Chuai
Computer Vision
Analyzing the Structure of Handwritten Digits: A Comparative Study of PCA, Factor Analysis, and UMAP
Jyotiraditya Gupta
Handwritten digit images lie in a high-dimensional pixel space but exhibit strong
geometric and statistical structure. This paper investigates the latent organization of handwritten digits in the MNIST
dataset using three complementary dimensionality reduction techniques: Principal Component Analysis (PCA), Factor Analysis (FA), and Uniform
Manifold Approximation and Projection (UMAP). Rather than focusing on classification accuracy, we study how each method characterizes intrinsic dimensionality, shared variation, and nonlinear geometry. PCA reveals dominant global variance directions and enables high-fidelity reconstructions using a small number of components. FA decomposes digits into interpretable latent handwriting primitives corresponding to strokes, loops, and
symmetry. UMAP uncovers nonlinear
manifolds that reflect smooth stylistic transitions between digit classes. Together, these results demonstrate that handwritten digits occupy a structured low-dimensional
manifold and that different statistical frameworks expose complementary aspects of this structure.
Read the full abstract ▾
Computer Vision
B-FIRE: Binning-Free Diffusion Implicit Neural Representation for Hyper-Accelerated Motion-Resolved MRI
Di Xu, Hengjie Liu, Yang Yang et al. (+11 more)
Accelerated dynamic volumetric
magnetic resonance imaging (4DMRI) is essential for applications relying on motion resolution. Existing 4DMRI produces acceptable artifacts of averaged breathing phases, which can blur and misrepresent instantaneous dynamic information. Recovery of such information requires a new paradigm to reconstruct extremely undersampled non-Cartesian k-space data. We propose B-FIRE, a binning-free
diffusion implicit neural representation framework for hyper-accelerated MR reconstruction capable of reflecting instantaneous 3D abdominal anatomy. B-FIRE employs a CNN-INR encoder-decoder backbone optimized using
diffusion with a comprehensive loss that enforces image-domain fidelity and frequency-aware constraints. Motion binned image pairs were used as training references, while inference was performed on binning-free undersampled data. Experiments were conducted on a T1-weighted StarVIBE liver MRI cohort, with accelerations ranging from 8 spokes per frame (RV8) to RV1. B-FIRE was compared against direct NuFFT, GRASP-CS, and an unrolled CNN method. Reconstruction fidelity, motion trajectory consistency, and inference latency were evaluated.
Read the full abstract ▾
All authors Di Xu, Hengjie Liu, Yang Yang, Mary Feng, Jin Ning, Xin Miao, Jessica E. Scholey, Alexandra E. Hotca-cho, William C. Chen, Michael Ohliger, Martina Descovich, Huiming Dong, Wensha Yang, Ke Sheng
Computer Vision
What Users Leave Unsaid: Under-Specified Queries Limit Vision-Language Models
Dasol Choi, Guijin Son, Hanwool Lee et al. (+7 more)
Current
vision-
language benchmarks predominantly feature well-structured questions with clear, explicit
prompts. However, real user queries are often informal and underspecified. Users naturally leave much unsaid, relying on images to convey context. We introduce HAERAE-
Vision, a
benchmark of 653 real-world visual questions from Korean online communities (0.76% survival from 86K candidates), each paired with an explicit rewrite, yielding 1,306 query variants in total. Evaluating 39 VLMs, we find that even state-of-the-art models (GPT-5, Gemini 2.5 Pro) achieve under 50% on the original queries. Crucially, query explicitation alone yields 8 to 22 point improvements, with smaller models benefiting most. We further show that even with web search, under-specified queries underperform explicit queries without search, revealing that current
retrieval cannot compensate for what users leave unsaid. Our findings demonstrate that a substantial portion of VLM difficulty stem from natural query under-specification instead of model capability, highlighting a critical gap between
benchmark evaluation and real-world deployment.
Read the full abstract ▾
All authors Dasol Choi, Guijin Son, Hanwool Lee, Minhyuk Kim, Hyunwoo Ko, Teabin Lim, Ahn Eungyeol, Jungwhan Kim, Seunghyeok Hong, Youngsook Song
Computer Vision
Forget-It-All: Multi-Concept Machine Unlearning via Concept-Aware Neuron Masking
Kaiyuan Deng, Bo Hui, Gen Li et al. (+4 more)
The widespread adoption of text-to-image (T2I)
diffusion models has raised concerns about their potential to generate copyrighted, inappropriate, or sensitive imagery. As a practical solution, machine unlearning aims to erase unwanted concepts without retraining from scratch. While most existing methods are effective for single-concept unlearning, they often struggle when removing multiple concepts, causing significant challenges in unlearning effectiveness, generation quality, and sensitivity to hyperparameters and
datasets. We take a unique perspective on multi-concept unlearning by leveraging model sparsity and propose the Forget It All (FIA) framework. FIA first introduces Contrastive Concept Saliency to quantify each weight connection's contribution to a target concept. It then identifies Concept Sensitive
Neurons by combining temporal and spatial information, ensuring that only
neurons consistently responsive to the target concept are selected. Finally, FIA constructs masks from the identified
neurons and fuses them into a unified multi-concept mask, where Concept Agnostic
Neurons that broadly support general content generation are preserved while concept-specific
neurons are pruned to remove the targets. FIA is training-free and requires minimal hyperparameter tuning for new tasks, enabling plug-and-play use. Extensive experiments across three distinct unlearning tasks demonstrate that FIA achieves more reliable multi-concept unlearning, improving forgetting effectiveness while maintaining generation fidelity and quality. Code is available at https://github.com/kaiyuan02415/Forget-It-All
Read the full abstract ▾
All authors Kaiyuan Deng, Bo Hui, Gen Li, Jie Ji, Minghai Qin, Geng Yuan, Xiaolong Ma
Computer Vision
UniDrive-WM: Unified Understanding, Planning and Generation World Model for Autonomous Driving
Zhexiao Xiong, Xin Ye, Burhan Yaman et al. (+5 more)
World models have become central to autonomous driving, where accurate scene understanding and future prediction are crucial for safe control. Recent work has explored using
vision-
language models (VLMs) for planning, yet existing approaches typically treat perception, prediction, and planning as separate modules. We propose UniDrive-WM, a unified VLM-based world model that jointly performs driving-scene understanding, trajectory planning, and trajectory-conditioned future image generation within a single architecture. UniDrive-WM's trajectory planner predicts a future trajectory, which conditions a VLM-based image generator to produce plausible future frames. These predictions provide additional supervisory signals that enhance scene understanding and iteratively refine trajectory generation. We further compare discrete and continuous output representations for future image prediction, analyzing their influence on downstream driving performance. Experiments on the challenging Bench2Drive
benchmark show that UniDrive-WM produces high-fidelity future images and improves planning performance by 7.3% in L2 trajectory error and 10.4% in collision rate over the previous best method. These results demonstrate the advantages of tightly integrating VLM-driven
reasoning, planning, and generative world modeling for autonomous driving. The project page is available at https://unidrive-wm.github.io/UniDrive-WM.
Read the full abstract ▾
All authors Zhexiao Xiong, Xin Ye, Burhan Yaman, Sheng Cheng, Yiren Lu, Jingru Luo, Nathan Jacobs, Liu Ren
Computer Vision
Addressing Overthinking in Large Vision-Language Models via Gated Perception-Reasoning Optimization
Xingjian Diao, Zheyuan Liu, Chunhui Zhang et al. (+6 more)
Large
Vision-
Language Models (LVLMs) have exhibited strong
reasoning capabilities through chain-of-thought mechanisms that generate step-by-step rationales. However, such slow-thinking approaches often lead to overthinking, where models produce excessively verbose responses even for simple queries, resulting in test-time inefficiency and even degraded accuracy. Prior work has attempted to mitigate this issue via adaptive
reasoning strategies, but these methods largely overlook a fundamental bottleneck: visual perception failures. We argue that stable
reasoning critically depends on low-level visual grounding, and that
reasoning errors often originate from imperfect perception rather than insufficient deliberation. To address this limitation, we propose Gated Perception-
Reasoning Optimization (GPRO), a meta-
reasoning controller that dynamically routes computation among three decision paths at each generation step: a lightweight fast path, a slow perception path for re-examining visual inputs, and a slow
reasoning path for internal self-reflection. To learn this distinction, we derive large-scale failure attribution supervision from approximately 790k samples, using teacher models to distinguish perceptual
hallucinations from
reasoning errors. We then train the controller with multi-objective reinforcement learning to optimize the trade-off between task accuracy and computational cost under uncertainty. Experiments on five
benchmarks demonstrate that GPRO substantially improves both accuracy and efficiency, outperforming recent slow-thinking methods while generating significantly shorter responses.
Read the full abstract ▾
All authors Xingjian Diao, Zheyuan Liu, Chunhui Zhang, Weiyi Wu, Keyi Kong, Lin Shi, Kaize Ding, Soroush Vosoughi, Jiang Gui
Computer Vision
CRUNet-MR-Univ: A Foundation Model for Diverse Cardiac MRI Reconstruction
Donghang Lyu, Marius Staring, Hildo Lamb et al. (+1 more)
In recent years, deep learning has attracted increasing
attention in the field of Cardiac MRI (CMR) reconstruction due to its superior performance over traditional methods, particularly in handling higher acceleration factors, highlighting its potential for real-world clinical applications. However, current deep learning methods remain limited in generalizability. CMR scans exhibit wide variability in image contrast, sampling patterns, scanner vendors, anatomical structures, and disease types. Most existing models are designed to handle only a single or narrow subset of these variations, leading to performance degradation when faced with distribution shifts. Therefore, it is beneficial to develop a unified model capable of generalizing across diverse CMR scenarios. To this end, we propose CRUNet-MR-Univ, a foundation model that leverages spatio-temporal correlations and
prompt-based priors to effectively handle the full diversity of CMR scans. Our approach consistently outperforms baseline methods across a wide range of settings, highlighting its effectiveness and promise.
Read the full abstract ▾
All authors Donghang Lyu, Marius Staring, Hildo Lamb, Mariya Doneva
Computer Vision
From Preoperative CT to Postmastoidectomy Mesh Construction: Mastoidectomy Shape Prediction for Cochlear Implant Surgery
Yike Zhang, Eduardo Davalos, Dingjie Su et al. (+2 more)
Cochlear Implant (CI) surgery treats severe hearing loss by inserting an electrode array into the cochlea to stimulate the auditory nerve. An important step in this procedure is mastoidectomy, which removes part of the mastoid region of the temporal bone to provide
surgical access. Accurate mastoidectomy shape prediction from preoperative imaging improves pre-
surgical planning, reduces risks, and enhances
surgical outcomes. Despite its importance, there are limited deep-learning-based studies regarding this topic due to the challenges of acquiring ground-truth labels. We address this gap by investigating self-supervised and weakly-supervised learning models to predict the mastoidectomy region without human annotations. We propose a hybrid self-supervised and weakly-supervised learning framework to predict the mastoidectomy region directly from preoperative CT scans, where the mastoid remains intact. Our hybrid method achieves a mean Dice score of 0.72 when predicting the complex and boundary-less mastoidectomy shape, surpassing state-of-the-art approaches and demonstrating strong performance. The method provides groundwork for constructing 3D postmastoidectomy surfaces directly from the corresponding preoperative CT scans. To our knowledge, this is the first work that integrating self-supervised and weakly-supervised learning for mastoidectomy shape prediction, offering a robust and efficient solution for CI
surgical planning while leveraging 3D T-distribution loss in weakly-supervised medical imaging.
Read the full abstract ▾
All authors Yike Zhang, Eduardo Davalos, Dingjie Su, Ange Lou, Jack Noble
Computer Vision
3D-Agent:Tri-Modal Multi-Agent Collaboration for Scalable 3D Object Annotation
Jusheng Zhang, Yijia Fan, Zimo Wen et al. (+2 more)
Driven by applications in autonomous driving robotics and augmented reality 3D object annotation presents challenges beyond 2D annotation including spatial complexity occlusion and viewpoint inconsistency Existing approaches based on single models often struggle to address these issues effectively We propose Tri MARF a novel framework that integrates tri modal inputs including 2D multi view images textual descriptions and 3D point clouds within a multi
agent collaborative architecture to enhance large scale 3D annotation Tri MARF consists of three specialized
agents a
vision language model
agent for generating multi view descriptions an information aggregation
agent for selecting optimal descriptions and a gating
agent that aligns textual semantics with 3D geometry for refined captioning Extensive experiments on Objaverse LVIS Objaverse XL and ABO demonstrate that Tri MARF substantially outperforms existing methods achieving a CLIPScore of 88 point 7 compared to prior state of the art methods
retrieval accuracy of 45 point 2 and 43 point 8 on ViLT R at 5 and a throughput of up to 12000 objects per hour on a single NVIDIA A100 GPU
Read the full abstract ▾
All authors Jusheng Zhang, Yijia Fan, Zimo Wen, Jian Wang, Keze Wang
Computer Vision
Performance Analysis of Image Classification on Bangladeshi Datasets
Mohammed Sami Khan, Fabiha Muniat and Rowzatul Zannat
Convolutional Neural Networks (CNNs) have demonstrated remarkable success in image classification tasks; however, the choice between designing a custom CNN from scratch and employing established pre-trained architectures remains an important practical consideration. In this work, we present a comparative analysis of a custom-designed CNN and several widely used deep learning architectures, including VGG-16, ResNet-50, and MobileNet, for an image classification task. The custom CNN is developed and trained from scratch, while the popular architectures are employed using transfer learning under identical experimental settings. All models are evaluated using standard performance metrics such as accuracy, precision, recall, and F1-score. Experimental results show that pre-trained CNN architectures consistently outperform the custom CNN in terms of classification accuracy and convergence speed, particularly when training data is limited. However, the custom CNN demonstrates competitive performance with significantly fewer parameters and reduced computational complexity. This study highlights the trade-offs between model complexity, performance, and computational efficiency, and provides practical insights into selecting appropriate CNN architectures for image classification problems.
Read the full abstract ▾
Computer Vision · Cross-listed from cs.GR
Radiant Foam Rendering on a Graph Processor
Zulkhuu Tuya, Ignacio Alzugaray, Nicholas Fry et al. (+1 more)
Many emerging many-core accelerators replace a single large device
memory with hundreds to thousands of lightweight cores, each owning only a small local SRAM and exchanging data via explicit on-chip communication. This organization offers high aggregate bandwidth, but it breaks a key assumption behind many volumetric rendering techniques: that rays can randomly access a large, unified scene representation. Rendering efficiently on such hardware therefore requires distributing both data and computation, keeping ray traversal mostly local, and structuring communication into predictable routes. We present a fully in-SRAM, distributed renderer for the Radiant Foam Voronoi-
cell volumetric representation on the Graphcore Mk2 IPU(Intelligence Processing Unit), a many-core accelerator with tile-local SRAM and explicit inter-tile communication. Our system shards the scene across tiles and forwards rays between shards through a hierarchical routing overlay, enabling ray marching entirely from on-chip SRAM with predictable communication. On Mip-NeRF~360 scenes, the system attains near-interactive throughput of approximately 1 fps at 640x480 with image and depth map quality close to the original GPU-based Radiant Foam implementation, while keeping all scene data and ray state in on-chip SRAM. Beyond demonstrating feasibility, we analyze routing,
memory, and scheduling bottlenecks that inform how future distributed-
memory accelerators can better support irregular, data-movement-heavy rendering workloads.
Read the full abstract ▾
All authors Zulkhuu Tuya, Ignacio Alzugaray, Nicholas Fry, Andrew J. Davison
Computer Vision
Few-Shot LoRA Adaptation of a Flow-Matching Foundation Model for Cross-Spectral Object Detection
Maxim Clouser, Kia Khezeli and John Kalantari
Foundation models for
vision are predominantly trained on RGB data, while many safety-critical applications rely on non-visible modalities such as infrared (IR) and synthetic aperture radar (SAR). We study whether a single flow-matching foundation model pre-trained primarily on RGB images can be repurposed as a cross-spectral translator using only a few co-measured examples, and whether the resulting synthetic data can enhance downstream detection. Starting from FLUX.1 Kontext, we insert low-rank adaptation (LoRA) modules and fine-tune them on just 100 paired images per domain for two settings: RGB to IR on the KAIST
dataset and RGB to SAR on the M4-SAR
dataset. The adapted model translates RGB images into pixel-aligned IR/SAR, enabling us to reuse existing bounding boxes and train object detection models purely in the target modality. Across a grid of LoRA hyperparameters, we find that LPIPS computed on only 50 held-out pairs is a strong proxy for downstream performance: lower LPIPS consistently predicts higher mAP for YOLOv11n on both IR and SAR, and for DETR on KAIST IR test data. Using the best LPIPS-selected LoRA adapter, synthetic IR from external RGB
datasets (LLVIP, FLIR ADAS) improves KAIST IR pedestrian detection, and synthetic SAR significantly boosts infrastructure detection on M4-SAR when combined with limited real SAR. Our results suggest that few-shot LoRA adaptation of flow-matching foundation models is a promising path toward foundation-style support for non-visible modalities.
Read the full abstract ▾
Computer Vision · Cross-listed from cs.LG
Aligned explanations in neural networks
Corentin Lobet and Francesca Chiaromonte
As artificial intelligence increasingly drives critical decisions, the ability to genuinely explain how neural networks make predictions is essential for trust. Yet, most current explanation methods offer post-hoc rationalizations rather than guaranteeing a true reflection of the model's
reasoning. We introduce the notion of explanatory
alignment, a requirement that explanations directly construct predictions rather than rationalize them. To achieve this in complex data domains, we present Pointwise-interpretable Networks (PiNets), a pseudo-linear architecture that forms linear models instance-wise. Evaluated on image classification and
segmentation tasks, PiNets demonstrate that their explanations are deeply faithful across four criteria: meaningfulness,
alignment, robustness, and sufficiency (MARS). Our contributions pave the way for promising avenues: by reconciling the predictive power of deep learning with the interpretability of linear models, PiNets provide a principled foundation for trustworthy AI and data-driven scientific discovery.
Read the full abstract ▾
Computer Vision
Combining Facial Videos and Biosignals for Stress Estimation During Driving
Paraskevi Valergaki, Vassilis C. Nicodemou, Iason Oikonomidis et al. (+2 more)
Reliable stress recognition is critical in applications such as medical monitoring and safety-critical systems, including real-world driving. While stress is commonly detected using physiological signals such as perinasal perspiration and heart rate, facial activity provides complementary cues that can be captured unobtrusively from
video. We propose a
multimodal stress estimation framework that combines facial
videos and physiological signals, remaining effective even when biosignal acquisition is challenging. Facial behavior is represented using a dense 3D Morphable Model, yielding a 56-dimensional descriptor that captures subtle expression and head-pose dynamics over time. To study how stress modulates facial motion, we perform extensive experiments alongside established physiological markers. Paired hypothesis tests between baseline and stressor phases show that 38 of 56 facial components exhibit consistent, phase-specific stress responses comparable to physiological markers. Building on these findings, we introduce a
Transformer-based temporal modeling framework and evaluate unimodal, early-fusion, and cross-modal
attention strategies. Cross-modal
attention fusion of 3D-derived facial features with physiological signals substantially improves performance over physiological signals alone, increasing AUROC from 52.7% and accuracy from 51.0% to 92.0% and 86.7%, respectively. Although evaluated on driving data, the proposed framework and protocol may generalize to other stress estimation settings.
Read the full abstract ▾
All authors Paraskevi Valergaki, Vassilis C. Nicodemou, Iason Oikonomidis, Antonis Argyros, Anastasios Roussos
Computer Vision · Cross-listed from physics.optics
End-to-end differentiable design of geometric waveguide displays
Xinge Yang, Zhaocheng Liu, Zhaoyu Nie et al. (+4 more)
Geometric waveguides are a promising architecture for optical see-through augmented reality displays, but their performance is severely bottlenecked by the difficulty of jointly optimizing non-sequential light transport and polarization-dependent multilayer thin-film coatings. Here we present the first end-to-end differentiable optimization framework for
geometric waveguide that couples non-sequential Monte Carlo polarization ray tracing with a differentiable transfer-
matrix thin-film solver. A differentiable Monte Carlo ray tracer avoids the exponential growth of deterministic ray splitting while enabling gradients backpropagation from eyebox metrics to design parameters. With
memory-saving strategies, we optimize more than one thousand layer-thickness parameters and billions of non-sequential ray-surface intersections on a single multi-GPU workstation. Automated layer
pruning is achieved by starting from over-parameterized stacks and driving redundant layers to zero thickness under discrete manufacturability constraints, effectively performing
topology optimization to discover optimal coating structures. On a representative design, starting from random initialization within thickness bounds, our method increases light efficiency from 4.1% to 33.5% and improves eyebox and FoV uniformity by $\sim$17$\times$ and $\sim$11$\times$, respectively. Furthermore, we jointly optimize the waveguide and an image preprocessing network to improve perceived image quality. Our framework not only enables system-level, high-dimensional coating optimization inside the waveguide, but also expands the scope of differentiable optics for next-generation optical design.
Read the full abstract ▾
All authors Xinge Yang, Zhaocheng Liu, Zhaoyu Nie, Qingyuan Fan, Zhimin Shi, Jim Bonar, Wolfgang Heidrich
Computer Vision
PackCache: A Training-Free Acceleration Method for Unified Autoregressive Video Generation via Compact KV-Cache
Kunyang Li, Mubarak Shah and Yuzhang Shang
A unified autoregressive model is a
Transformer-based framework that addresses diverse
multimodal tasks (e.g., text, image,
video) as a single sequence modeling problem under a shared
token space. Such models rely on the KV-cache mechanism to reduce
attention computation from O(T^2) to O(T); however, KV-cache size grows linearly with the number of generated
tokens, and it rapidly becomes the dominant bottleneck limiting inference efficiency and generative length. Unified autoregressive
video generation inherits this limitation. Our analysis reveals that KV-cache
tokens exhibit distinct spatiotemporal properties: (i) text and conditioning-image
tokens act as persistent semantic anchors that consistently receive high
attention, and (ii)
attention to previous frames naturally decays with temporal distance. Leveraging these observations, we introduce PackCache, a training-free KV-cache management method that dynamically compacts the KV cache through three coordinated mechanisms: condition anchoring that preserves semantic references, cross-frame decay modeling that allocates cache budget according to temporal distance, and spatially preserving position
embedding that maintains coherent 3D structure under cache removal. In terms of efficiency, PackCache accelerates end-to-end generation by 1.7-2.2x on 48-frame long sequences, showcasing its strong potential for enabling longer-sequence
video generation. Notably, the final four frames - the portion most impacted by the progressively expanding KV-cache and thus the most expensive segment of the clip - PackCache delivers a 2.6x and 3.7x acceleration on A40 and H200, respectively, for 48-frame
videos.
Read the full abstract ▾
Computer Vision · Cross-listed from cs.RO
UNIC: Learning Unified Multimodal Extrinsic Contact Estimation
Zhengtong Xu and Yuki Shirai
Contact-rich manipulation requires reliable estimation of extrinsic contacts-the interactions between a grasped object and its environment which provide essential contextual information for planning, control, and policy learning. However, existing approaches often rely on restrictive assumptions, such as predefined contact types, fixed grasp configurations, or camera calibration, that hinder generalization to novel objects and deployment in unstructured environments. In this paper, we present UNIC, a unified
multimodal framework for extrinsic contact estimation that operates without any prior knowledge or camera calibration. UNIC directly encodes visual observations in the camera frame and integrates them with proprioceptive and tactile modalities in a fully data-driven manner. It introduces a unified contact representation based on scene affordance maps that captures diverse contact formations and employs a
multimodal fusion mechanism with random masking, enabling robust
multimodal representation learning. Extensive experiments demonstrate that UNIC performs reliably. It achieves a 9.6 mm average Chamfer distance error on unseen contact locations, performs well on unseen objects, remains robust under missing modalities, and adapts to dynamic camera viewpoints. These results establish extrinsic contact estimation as a practical and versatile capability for contact-rich manipulation. The overview and hardware experiment
videos are at https://youtu.be/xpMitkxN6Ls?si=7Vgj-aZ_P1wtnWZN
Read the full abstract ▾
Computer Vision
Comparative Analysis of Custom CNN Architectures versus Pre-trained Models and Transfer Learning: A Study on Five Bangladesh Datasets
Ibrahim Tanvir, Alif Ruslan and Sartaj Solaiman
This study presents a comprehensive comparative analysis of custom-built Convolutional Neural Networks (CNNs) against popular pre-trained architectures (ResNet-18 and VGG-16) using both feature extraction and transfer learning approaches. We evaluated these models across five diverse image classification
datasets from Bangladesh: Footpath
Vision, Auto Rickshaw Detection, Mango Image Classification, Paddy Variety Recognition, and Road Damage Detection. Our experimental results demonstrate that transfer learning with fine-tuning consistently outperforms both custom CNNs built from scratch and feature extraction methods, achieving accuracy improvements ranging from 3% to 76% across different
datasets. Notably, ResNet-18 with fine-tuning achieved perfect 100% accuracy on the Road Damage BD
dataset. While custom CNNs offer advantages in model size (3.4M parameters vs. 11-134M for pre-trained models) and training efficiency on simpler tasks, pre-trained models with transfer learning provide superior performance, particularly on complex classification tasks with limited training data. This research provides practical insights for practitioners in selecting appropriate deep learning approaches based on
dataset characteristics, computational resources, and performance requirements.
Read the full abstract ▾
Computer Vision
SCAR-GS: Spatial Context Attention for Residuals in Progressive Gaussian Splatting
Diego Revilla, Pooja Suresh, Ooi Wei Tsang et al. (+1 more)
Recent advances in 3D Gaussian Splatting have allowed for real-time, high-fidelity novel view synthesis. Nonetheless, these models have significant storage requirements for large and medium-sized scenes, hindering their deployment over cloud and streaming services. Some of the most recent progressive compression techniques for these models rely on progressive masking and scalar quantization techniques to reduce the bitrate of Gaussian attributes using spatial context models. While effective, scalar quantization may not optimally capture the correlations of high-dimensional feature vectors, which can potentially limit the rate-distortion performance. In this work, we introduce a novel progressive codec for 3D Gaussian Splatting that replaces traditional methods with a more powerful Residual Vector Quantization approach to compress the primitive features. Our key contribution is an auto-regressive
entropy model, guided by a multi-resolution hash grid, that accurately predicts the conditional probability of each successive transmitted index, allowing for coarse and refinement layers to be compressed with high efficiency.
Read the full abstract ▾
All authors Diego Revilla, Pooja Suresh, Ooi Wei Tsang, Anand Bhojan
Computer Vision
Mohsen Ghafoorian and Amirhossein Habibian
Recent advances in
video diffusion models have shifted towards
transformer-based architectures, achieving state-of-the-art
video generation but at the cost of quadratic
attention complexity, which severely limits scalability for longer sequences. We introduce ReHyAt, a Recurrent Hybrid
Attention mechanism that combines the fidelity of softmax
attention with the efficiency of linear
attention, enabling chunk-wise recurrent reformulation and constant
memory usage. Unlike the concurrent linear-only SANA
Video, ReHyAt's hybrid design allows efficient
distillation from existing softmax-based models, reducing the training cost by two orders of magnitude to ~160 GPU hours, while being competitive in the quality. Our light-weight
distillation and finetuning pipeline provides a recipe that can be applied to future state-of-the-art bidirectional softmax-based models. Experiments on VBench and VBench-2.0, as well as a human preference study, demonstrate that ReHyAt achieves state-of-the-art
video quality while reducing
attention cost from quadratic to linear, unlocking practical scalability for long-duration and on-device
video generation. Project page is available at https://qualcomm-ai-research.github.io/rehyat.
Read the full abstract ▾
Computer Vision
Unified Text-Image Generation with Weakness-Targeted Post-Training
Jiahui Chen, Philippe Hansen-Estruch, Xiaochuang Han et al. (+7 more)
Unified
multimodal generation architectures that jointly produce text and images have recently emerged as a promising direction for text-to-image (T2I) synthesis. However, many existing systems rely on explicit modality switching, generating
reasoning text before switching manually to image generation. This separate, sequential inference process limits cross-modal coupling and prohibits automatic
multimodal generation. This work explores post-training to achieve fully unified text-image generation, where a model autonomously transitions from textual
reasoning to visual synthesis within a single inference process. We study this on BAGEL, a 14B mixture-of-
transformers model that pairs autoregressive text generation with flow-matching image synthesis. We examine the impact of joint text-image generation on T2I performance and the relative importance of each modality during post-training. We additionally explore different post-training data strategies, showing that a targeted
dataset addressing specific limitations achieves superior results compared to broad image-caption corpora or
benchmark-aligned data. Using offline, reward-weighted post-training with fully self-generated synthetic data, our approach enables improvements in
multimodal image generation across four diverse, independent T2I
benchmarks, demonstrating the effectiveness of reward-weighting both modalities and strategically designed post-training data.
Read the full abstract ▾
All authors Jiahui Chen, Philippe Hansen-Estruch, Xiaochuang Han, Yushi Hu, Emily Dinan, Amita Kamath, Michal Drozdzal, Reyhane Askari-Hemmat, Luke Zettlemoyer, Marjan Ghazvininejad
Computer Vision
Embedding Textual Information in Images Using Quinary Pixel Combinations
A V Uday Kiran Kandala
This paper presents a novel technique for
embedding textual data into images using quinary combinations of pixel intensities in RGB space. Existing methods predominantly rely on least and most significant bit (LSB & MSB) manipulation, Pixel Value Differencing (PVD), spatial perturbations in RGB channels, transform domain based methods, Quantization methods, Edge and Region based methods and more recently through deep learning methods and generative AI techniques for hiding textual information in spatial domain of images. Most of them are dependent on pixel intensity flipping over multiple pixels, such as LSB and combination of LSB based methodologies, and on transform coefficients, often resulting in the form of noise. Encoding and Decoding are deterministic in most of the existing approaches and are computationally heavy in case of higher models such as deep learning and gen AI approaches. The proposed method works on quinary pixel intensity combinations in RGB space, where five controlled different pixel intensity variations in each of the R, G, and B channels formulate up to one hundred and twenty five distinct pixel intensity combinations. These combinations are mapped to textual symbols, enabling the representation of uppercase and lowercase alphabetic characters, numeric digits, whitespace, and commonly used special characters. Different metrics such as MSE, MAE, SNR, PSNR, SSIM, Histogram Comparison and Heatmap analysis, were evaluated for both original and encoded images resulting in no significant distortion in the images. Furthermore, the method achieves improved
embedding efficiency by encoding a complete textual symbol within a single RGB pixel, in contrast to LSB and MSB based approaches that typically require multiple pixels or multi-step processes, as well as transform and learning based methods that incur higher computational overhead.
Read the full abstract ▾
Computer Vision
Beyond Binary Preference: Aligning Diffusion Models to Fine-grained Criteria by Decoupling Attributes
Chenye Meng, Zejian Li, Zhongni Liu et al. (+9 more)
Post-training
alignment of
diffusion models relies on simplified signals, such as scalar rewards or binary preferences. This limits
alignment with complex human expertise, which is hierarchical and fine-grained. To address this, we first construct a hierarchical, fine-grained evaluation criteria with domain experts, which decomposes image quality into multiple positive and negative attributes organized in a tree structure. Building on this, we propose a two-stage
alignment framework. First, we inject domain knowledge to an auxiliary
diffusion model via Supervised Fine-Tuning. Second, we introduce Complex Preference Optimization (CPO) that extends DPO to align the target
diffusion to our non-binary, hierarchical criteria. Specifically, we reformulate the
alignment problem to simultaneously maximize the probability of positive attributes while minimizing the probability of negative attributes with the auxiliary
diffusion. We instantiate our approach in the domain of painting generation and conduct CPO training with an annotated
dataset of painting with fine-grained attributes based on our criteria. Extensive experiments demonstrate that CPO significantly enhances generation quality and
alignment with expertise, opening new avenues for fine-grained criteria
alignment.
Read the full abstract ▾
All authors Chenye Meng, Zejian Li, Zhongni Liu, Yize Li, Changle Xie, Kaixin Jia, Ling Yang, Huanghuang Deng, Shiying Ding, Shengyuan Zhang, Jiayi Li, Lingyun Sun
Computer Vision · Cross-listed from cs.LG
ArtCognition: A Multimodal AI Framework for Affective State Sensing from Visual and Kinematic Drawing Cues
Behrad Binaei-Haghighi, Nafiseh Sadat Sajadi, Mehrad Liviyan et al. (+3 more)
The objective assessment of human affective and psychological states presents a significant challenge, particularly through non-verbal channels. This paper introduces digital drawing as a rich and underexplored modality for affective sensing. We present a novel
multimodal framework, named ArtCognition, for the automated analysis of the House-Tree-Person (HTP) test, a widely used psychological instrument. ArtCognition uniquely fuses two distinct data streams: static visual features from the final artwork, captured by computer
vision models, and dynamic behavioral kinematic cues derived from the drawing process itself, such as stroke speed, pauses, and smoothness. To bridge the gap between low-level features and high-level psychological interpretation, we employ a
Retrieval-Augmented Generation (RAG) architecture. This grounds the analysis in established psychological knowledge, enhancing explainability and reducing the potential for model
hallucination. Our results demonstrate that the fusion of visual and behavioral kinematic cues provides a more nuanced assessment than either modality alone. We show significant correlations between the extracted
multimodal features and standardized psychological metrics, validating the framework's potential as a scalable tool to support clinicians. This work contributes a new methodology for non-intrusive affective state assessment and opens new avenues for technology-assisted mental healthcare.
Read the full abstract ▾
All authors Behrad Binaei-Haghighi, Nafiseh Sadat Sajadi, Mehrad Liviyan, Reyhane Akhavan Kharazi, Fatemeh Amirkhani, Behnam Bahrak
Computer Vision
Choreographing a World of Dynamic Objects
Yanzhe Lyu, Chen Geng, Karthik Dharmarajan et al. (+4 more)
Dynamic objects in our physical 4D (3D + time) world are constantly evolving, deforming, and interacting with other objects, leading to diverse 4D scene dynamics. In this paper, we present a universal generative pipeline, CHORD, for CHOReographing Dynamic objects and scenes and synthesizing this type of phenomena. Traditional rule-based graphics pipelines to create these dynamics are based on category-specific heuristics, yet are labor-intensive and not scalable. Recent learning-based methods typically demand large-scale
datasets, which may not cover all object categories in interest. Our approach instead inherits the universality from the
video generative models by proposing a
distillation-based pipeline to extract the rich Lagrangian motion information hidden in the Eulerian representations of 2D
videos. Our method is universal, versatile, and category-agnostic. We demonstrate its effectiveness by conducting experiments to generate a diverse range of multi-body 4D dynamics, show its advantage compared to existing methods, and demonstrate its applicability in generating robotics manipulation policies. Project page: https://yanzhelyu.github.io/chord
Read the full abstract ▾
All authors Yanzhe Lyu, Chen Geng, Karthik Dharmarajan, Yunzhi Zhang, Hadi Alzayer, Shangzhe Wu, Jiajun Wu