Computer Vision
An Empirical Study of Handcrafted Feature Learning and Convolutional Neural Networks for Facial Expression Recognition
Chethiya Galkaduwa
Facial expression recognition is an important computer
vision task with applications in human–computer interaction, mental health monitoring, driver alert systems, and behavioral analysis. While convolutional neural networks (CNNs) dominate modern facial expression recognition, handcrafted feature descriptors such as Histogram of Oriented Gradients (HOG) and Local Binary Patterns (LBP) remain useful classical baselines. This study compares HOG with Support Vector Machine (SVM), LBP with Logistic Regression, and a lightweight CNN across three facial expression
datasets: FER-2013, CK+, and KDEF. The results show that CNNs achieve the best overall performance, particularly on more complex data, while HOG performs strongly in controlled environments. LBP performs poorly across all
datasets. The study highlights that
dataset complexity significantly affects performance and that robust feature learning is essential for real-world facial expression recognition.
Read the full abstract ▾
Computer Vision · Cross-listed from cs.LG
CARPRT: Class-Aware Zero-Shot Prompt Reweighting for Black-Box Vision-Language Models
Ruijiang Dong, Zesheng Ye, Jianzhong Qi et al. (+4 more)
Pre-trained
vision-
language models (VLMs) enable zero-shot image classification by computing the similarity score between an image and textual descriptions, typically formed by inserting a class label (e.g., "cat") into a
prompt (e.g., "a photo of a"). Since the score for a given image-class pair is sensitive to the choice of
prompt, existing studies ensemble multiple
prompts using a weighting vector to aggregate scores across different
prompts. Yet, in current strategies, the weighting vector assigned to each
prompt is shared across all classes, implicitly assuming that
prompts are conditionally independent of classes, which often does not hold in practice, as a
prompt like "an aerial view of" might be apt for "airport" but ill-suited for "apple". To address this, we propose class-aware zero-shot
prompt reweighting (CARPRT). This scoring scheme adjusts the weighting vector for each class label by capturing the class-specific relevance of different
prompts in a training-free manner. For each class label and every available
prompt, we quantify their class-specific relevance by averaging image-text relevance scores over images predicted to that class under the given
prompt. These estimates are then normalized to derive class-specific weights. Evaluations on standard image classification
benchmarks show that CARPRT outperforms existing class-independent reweighting methods, confirming that modeling
prompt-class dependencies is crucial for effective zero-shot prediction and even broader VLM-based application settings that rely on
prompt ensembling. Our code is available at https://github.com/tmlr-group/CARPRT.
Read the full abstract ▾
All authors Ruijiang Dong, Zesheng Ye, Jianzhong Qi, Lei Feng, Feng Liu, Gang Niu, Masashi Sugiyama
Computer Vision
Memory-Augmented LSTM Autoencoder for Unsupervised Activity Recognition with IMU Sensor Fusion
Saeid Arabzadeh, Farshad Almasganj and Mohammad Mahdi Ahmadi
HAR using Inertial Measurement Unit (IMU) sensors is vital for healthcare monitoring and rehabilitation. Despite deep learning advancements, major challenges remain: reliance on labeled data, multi-sensor fusion complexity, and the limited ability of unsupervised methods to capture spatiotemporal dependencies. These issues are pronounced in real-world scenarios with noisy data, overlapping activities, and missing labels. We propose a fully unsupervised spatiotemporal feature fusion framework using a
memory-augmented autoencoder. It enhances activity representations via short temporal windows of multi-sensor IMU data, enabling real-time applications. Our framework extracts hierarchical static features via a Stacked Autoencoder, fusing them within and across sensors. A sequence-to-sequence LSTM Autoencoder then temporally refines these features, incorporating historical motion patterns without labels. We analyze key hyperparameters to identify configurations that maximize feature separability under short-window constraints. Evaluated on DaLiAc and PAMAP2 using realistic inter-class window
segmentation, our method achieves 96.6% and 98.4% accuracy, respectively, surpassing supervised baselines and unsupervised approaches. Our method improves feature separability by up to 9% despite shorter temporal windows. While our realistic inter-class
segmentation reduces accuracy by ~7%, it was intentionally adopted to better reflect real-world activity transitions and practical relevance.
Read the full abstract ▾
Computer Vision · Cross-listed from cs.RO
Rotation-Aware Point-Cloud Embeddings for Vision-Based In-Hand Reorientation
Yashom Dighe and Karthik Dantu
Point-cloud goals provide a direct way to specify dexterous in-hand reorientation: instead of defining an object-specific pose frame or estimating 6D pose at test time, the policy is given the desired 3D geometry of the object. Yet raw point-cloud goal conditioning is poorly conditioned for policy learning. Current and goal clouds are unordered, independently sampled, and often visibility-dependent, so their discrepancy entangles object rotation with permutation, resampling, and unstable correspondence structure. For this reason, prior point-cloud manipulation methods typically add structure outside the representation itself, such as explicit pose or relative-pose inputs, dense flow features, or
distillation from privileged teachers. We close this gap by learning a rotation-aware point-cloud
embedding whose Euclidean latent distance is calibrated to the SO(3) geodesic error between object orientations. The resulting representation turns current-goal comparison into a smooth control signal, allowing a model-free RL policy to act from current and goal point-cloud
embeddings, proprioception, and centroid metadata, without object pose, relative pose, dense flow, or teacher-action supervision. In in-hand reorientation experiments, this interface matches privileged-state and
distillation-based baselines while avoiding brittle test-time computation of structured pose or flow inputs. These results suggest that point-cloud goals become practical for this task when the representation, rather than an external module, encodes the task-relevant geometry of rotation. We also show evidence that generic visual point-cloud pretraining is insufficient for such a current-goal comparison because it discards the task-relevant state and preserves only shape features.
Read the full abstract ▾
Computer Vision
Motion-Aware Reinforcement Learning For Object Localization
Prithvi Raj Singh and Satyendra Singh
We present MARLNet (Motion-Aware Reinforcement Learning Network), a PPO-based bounding-box refinement
agent that incorporates a constant-velocity motion prior into the observation state and an action smoothness penalty into the reward function. The
agent operates on 268-dimensional observations encoding the current proposal, a kinematic prediction, the previous action, and a 256-dimensional EfficientNet-B0 crop feature, and learns a five-dimensional policy controlling coordinate adjustments and a binary termination trigger. Evaluated on Pascal VOC 2012 and VisDrone 2019, MARLNet trains stably across all regularization strengths tested and achieves consistent gains in detection success rate at $\text{IoU} \geq 0.5$: up to $+0.011$ on VOC ($\lambda_\text{phys}{=}0.10$), where the motion prior prevents the overshooting that causes plain PPO to regress on this metric, and $+0.007$ on VisDrone ($\lambda_\text{phys}{=}0.70$), where unconstrained PPO achieves a larger gain ($+0.025$) owing to the weaker base detector. Through reward design ablations and training dynamics analysis, we identify a reward interference in which combining a constant-velocity deviation penalty with an absolute IoU term causes trigger collapse, and show that replacing it with the action smoothness penalty resolves this failure. We further characterize a representational ceiling facing crop-feature refinement
agents that share a backbone with their base detector, confirmed through a global-plus-local observation ablation. Project page: https://prithviraj97.github.io/marl-net
Read the full abstract ▾
Computer Vision
From Gradient Clipping to Structural Refinement: Improving DPSGD for Medical Image Segmentation
Shiva Parsarad, Parth Shandilya and Isabel Wagner
Medical image
segmentation is widely used for disease detection but relies on sensitive data, raising
privacy concerns as trained models can leak information. Differential
privacy, typically implemented via Differential Private
Stochastic Gradient Descent (DPSGD), provides a solution, though at the cost of reduced utility. Recent DPSGD variants, including Automatic clipping (Auto-S), Normalised SGD with perturbation (NSGD), and Per-sample adaptive clipping (PSAC), have shown promise in image classification, but their behavior in medical
segmentation remains underexplored. We evaluate these methods across binary and multi-class tasks and analyze gradient
alignment, showing that prior assumptions, particularly for PSAC, do not consistently hold. We further demonstrate that combining clipping strategies with morphological refinement improves
segmentation quality under
privacy constraints. Finally, we propose an adaptive DP-Morph variant that captures class-specific structures and enhances performance in multi-class settings.
Read the full abstract ▾
Computer Vision · Cross-listed from eess.IV
Scaling up fine-grained intracranial vessel annotations in computed tomography angiography
Chu-Hsuan Lin, Alberto Mario Ceballos-Arroyo, Jisoo Kim et al. (+4 more)
In this work, we present SemanticVessel, a
dataset for fine-grained
brain vessel
segmentation in computed
tomography angiography scans. Based on the detailed contrast provided by dynamic 4D-CTA scans, we generate
segmentation traces for arteries and veins. We then use intensity-guided region growing to obtain
segmentations of the majority of vascular territories in the human
brain, which are refined and annotated with 20 unique arterial classes by an expert radiologist. Unlike existing
datasets, where minor arteries are discarded as background content, we merge these minor arteries into a generic arterial class. Due to the multiple-phase acquisition of dynamic 4D-CTA, labels for a single phase can be re-used for other phases in the same series, greatly increasing the size of our
dataset with no additional annotation cost. The results show that models trained with the additional generic artery class produce better fine-grained
segmentations across the board. We will make our code, annotation GUI, and model weights available to the scientific community. Code, weights, and data will be made available on https://github.com/alceballosa/robust-vessel-
segmentation
Read the full abstract ▾
All authors Chu-Hsuan Lin, Alberto Mario Ceballos-Arroyo, Jisoo Kim, Shrikanth M. Yadav, Huaizu Jiang, Lei Qin, Geoffrey S. Young
Computer Vision · Cross-listed from cs.GR
Scene-Level Heterogeneous Physics Simulation with 3D Gaussian Splats
Xiaoyang Liu, Shangzhe Wu and Kai Han
3D Gaussian Splatting (3DGS) has achieved state-of-the-art photorealistic rendering, but the representation gap prevents these assets from being physically interactive. Production-grade physics engines do not understand the 3DGS representation, while prior physics-for-3DGS methods are monolithic silos. These prior works are fundamentally limited, demonstrating only object-centric physics in isolated environments, such as on an ideal plane. They are incapable of interacting with complex static collision geometry or heterogeneous assets. We propose a novel framework that, for the first time, bridges this gap by enabling 3DGS assets to participate in scene-level, heterogeneous, multi-solver physical simulations. Our core contribution is a Representation Abstraction Framework that translates all diverse assets, including 3DGS, virtual meshes, and fluids, into a unified physical particle set. This abstraction is key to enabling complex behaviors, such as the non-rigid deformation of 3DGS assets, within a unified physics pipeline. This particle set, along with the static scene collision boundaries derived from scene capture, is processed within a solver-agnostic physics kernel. The physical results are then mapped back to drive each asset's specific visual reconstruction. This architecture unlocks capabilities impossible with prior art. We demonstrate complex, two-way interactions between deformable 3DGS assets, standard CG assets such as fluids and meshes, and large-scale captured static environments, showcasing realistic coupled phenomena that were previously unattainable.
Read the full abstract ▾
Computer Vision · Cross-listed from eess.IV
Configurable Algorithms for Histopathologic Cancer Detection on Quantum Hardware
Nandika Goyal, Glen Uehara and Andreas Spanias
Histopathologic cancer detection is challenging due to tissue variability, staining differences, and subtle visual distinctions between disease classes. We propose two
quantum algorithms for this task: a configurable dual-gradient CSWAP
circuit (DG-CSWAP) that computes multi-directional edge responses in a single execution via per-pixel local Ry encoding, and a hardware-efficient destructive swap
circuit (DG-DST) natively matched to
quantum processing unit (QPU) gate sets at substantially lower
circuit complexity. We prove algebraic equivalence between DG-CSWAP and DG-DST, enabling a two-
circuit QPU validation strategy. A three-stage NISQ mitigation pipeline, including readout error correction, bias subtraction, and slope regression, reduces single-pixel hardware MSE by ~8x. Validated on five
quantum processors via Amazon Braket, the method achieves inter-platform Pearson r ~ 0.93-0.94 across all local-simulator pairs. Compared to a prior
Quantum Fourier Transform (QFT) based amplitude-encoding baseline requiring 12-
qubit global state preparation and a three-model ensemble (85.55% on PatchCamelyon), the proposed method uses shot-based measurements, executes on real
quantum hardware, and achieves 79.80% accuracy with a single ResNet-50. A Lite configuration delivers a 17x preprocessing speedup at a 2.59% accuracy cost. To the best of our knowledge, this is the first
quantum hardware implementation study with noise mitigation for histopathologic image classification.
Read the full abstract ▾
Computer Vision
Quantile Adaptive Temperature Scaling for Confidence Calibration
Omprakash Chakraborty, Leo Fillioux, Ismail Ben Ayed et al. (+1 more)
Deep neural networks often produce poorly calibrated confidence estimates, overstating their certainty even when predictions are incorrect. Temperature Scaling remains the most widely used posthoc calibration method due to its simplicity and effectiveness, yet its global, uniform rescaling of logits fails to correct the highly heterogeneous structure of miscalibration observed across the confidence spectrum. In particular, the largest correctness confidence discrepancies arise in different
quantile regions depending on the setting, low confidence predictions, where uncertainty matters most, tend to exhibit the largest correctness confidence discrepancies, which standard TS leaves largely unaddressed. We introduce
Quantile Adaptive Temperature Scaling (QaTS), a simple and efficient post hoc calibration method that adapts the temperature as a function of a predictions empirical confidence
quantile. By mapping confidences into the
quantile space, QaTS normalizes the calibration problem, makes the structure of miscalibration explicit and enables a monotone temperature function that adapts across
quantiles while leaving well calibrated high confidence predictions largely unchanged. preserving high confidence behavior. This
quantile aware formulation aligns naturally with a reparameterized Expected Calibration Error (ECE) objective and yields a sample wise temperature that is robust across a variety of challenging scenarios, such as class imbalance and distributional shifts. Across a broad range of
datasets, architectures, evaluation scenarios and diverse tasks, QaTS consistently, and substantially, outperforms state of the art post hoc calibration methods, delivering more reliable and trustworthy confidence estimates without modifying model predictions.
Read the full abstract ▾
All authors Omprakash Chakraborty, Leo Fillioux, Ismail Ben Ayed, Jose Dolz
Computer Vision
Adversarial Domain Prompt Tuning and Generation for Single Domain Generalization
Zhipeng Xu, De Cheng, Xinyang Jiang et al. (+3 more)
Single domain generalization (SDG) aims to learn a robust model, which could perform well on many unseen domains while there is only one single domain available for training. One of the promising directions for achieving single-domain generalization is to generate out-of-domain (OOD) training data through data augmentation or image generation. Given the rapid advancements in AI-generated content (AIGC), this paper is the first to propose leveraging powerful pre-trained text-to-image (T2I) foundation models to create the training data. However, manually designing textual
prompts to generate images for all possible domains is often impractical, and some domain characteristics may be too abstract to describe with words. To address these challenges, we propose a novel Progressive
Adversarial Prompt Tuning (PAPT) framework for pre-trained
diffusion models. Instead of relying on static textual domains, our approach learns two sets of abstract
prompts as conditions for the
diffusion model: one that captures domain-invariant category information and another that models domain-specific styles. This
adversarial learning mechanism enables the T2I model to generate images in various domain styles while preserving key categorical features. Extensive experiments demonstrate the effectiveness of the proposed method, achieving superior performances to state-of-the-art single-domain generalization approaches.
Read the full abstract ▾
All authors Zhipeng Xu, De Cheng, Xinyang Jiang, Nannan Wang, Dongsheng Li, Xinbo Gao
Computer Vision
HPP: Hierarchical Programmatic Probing for Long Video Understanding by Decoupling Perception and Reasoning
Awais Rauf, Ahmed Hasssan and Greg Slabaugh
Understanding long
videos requires fine-grained perception and multi-step, higher-order
reasoning over complex, long-range spatio-temporal dynamics.
Vision-
language models (VLMs) encode
video frames into visual
tokens and attempt to perform both perception and multi-step planning latently, within a single forward pass. This coupled formulation, however, is bottlenecked by the LLM's limited capacity to discover and execute multi-step strategies in its latent representations. To address this bottleneck, we propose Hierarchical Programmatic Probing (HPP), a framework that decouples semantic perception from higher-order temporal
reasoning by reformulating long
video understanding as iterative, programmatic exploration of a hierarchically segmented
video. Specifically, a coding-capable LLM plans and executes a multi-step strategy in an interactive coding environment, probing the
video for information and invoking a VLM for localized perception on demand. To make probing tractable over long
videos, we introduce three components: information-density-aware hierarchical
segmentation, late-interaction semantic
retrieval, and structured probing functions for coarse-to-fine temporal localization. We validate HPP on LongVideoBench, which requires both fine-grained perception and long-range relational
reasoning, and show that decoupling the two via iterative programmatic probing yields substantial gains. Further results on EgoSchema, VideoMME, and MLVU demonstrate the effectiveness of our approach across diverse long-
video benchmarks.
Read the full abstract ▾
Computer Vision · Cross-listed from physics.med-ph
Edge-Aligned Beam Placement in Scanning Probe Tomography via Reconstruction-Free Sequential Design of Experiments
San Dinh, Zichao Wendy Di and Matt Menickelly
In X-ray scanning probe
tomography, reconstruction quality generally improves with larger numbers of projections. However, additional projections increase experiment costs, acquisition time, and the radiation dose imparted to the sample. One mitigation to these trade-offs is to adopt a sequential design of experiments, in which each subsequent measurement is determined as a function of previously acquired data in order to maximize information gain. In scanning probe
tomography, a widely used heuristic to maximize information is to align beams with the edges of the sample. A key challenge, however, is that the true sample is unknown, so identifying edge-aligned beams typically requires reconstructing the sample based on available measurements. This work proposes a novel sequential design method that identifies edge-aligned measurements directly from the sinogram, bypassing any reconstruction, thereby improving computational efficiency and reducing the experimental design's susceptibility to reconstruction errors. Our method dynamically selects the next set of measurement beams by maximizing an acquisition function that balances exploration and exploitation over the domain of all possible measurements, improving reconstruction quality while reducing measurement redundancy.
Read the full abstract ▾
Computer Vision
Structural Assessment for Understanding and Guiding Dataset Distillation in Discrete Token Space
Yue Cao, Jianyang Gu, Vyacheslav Kungurtsev et al. (+4 more)
Dataset distillation (DD) has proven to reduce training cost while preserving accuracy. While promising, the factors that make one distilled
dataset more effective than another remain poorly understood. In this work, we investigate this question through the lens of discrete visual tokenizers. Whereas many prior DD efforts emphasize matching global data distributions, we suggest that the effectiveness depends on which semantic concepts are captured and how they are composed. Discrete visual tokenizers provide a finite vocabulary that enables direct statistical analysis of such compositional structure. Through quantitative analysis of
token-level statistics, we introduce the structural score to measure the adequacy of
token compositions. We observe that distilled
datasets with balanced
token composition yield higher validation performance. On the other hand, divergence from the original data does not necessarily harm performance. We further show that samples with high structural scores in the discrete
token space can effectively guide
diffusion-based DD. Our findings highlight the importance of
token composition in
dataset effectiveness, offering a principled complement to distributional similarity considerations in DD.
Read the full abstract ▾
All authors Yue Cao, Jianyang Gu, Vyacheslav Kungurtsev, Yu Hu, Jozsef Hamari, Zheng Liu, Mohsen Zardadi
Computer Vision
VT-DUDA: Visual Token Conditioning for Diffusion-guided Unsupervised Domain Adaptation
Xuan Qi, Daniele Berardini, Dario Serez et al. (+2 more)
Unsupervised domain adaptation (UDA) aims to learn a target-domain classifier from labeled source data and unlabeled target data under distribution shift. Recent
diffusion-based UDA methods approach this problem by synthesizing labeled target-style images and training on the resulting synthetic data. However, their performance depends heavily on the conditioning design: class
prompts provide only coarse guidance, while domain adaptation modules mainly control appearance, which may leave target-style synthesis insufficiently specified. We propose VT-DUDA, a visual-
token conditioning framework for
diffusion-guided UDA. Instead of relying only on text
prompts, VT-DUDA uses source images to provide additional instance-level visual context for target-style synthesis. Specifically, VT-DUDA maps each source image to a compact sequence of visual
tokens and forms a hybrid conditioning context by concatenating these
tokens with the corresponding text
embeddings along the cross-
attention context dimension of a latent
diffusion model. This provides instance-dependent conditioning beyond text alone, while synthesis is performed with the target-domain adapter branch. Because guidance is represented explicitly as a
token sequence, the same interface also permits inference-time manipulation of the conditioning signal through
token selection and
token-strength adjustment. The proposed method preserves the standard
diffusion objective and can be integrated into existing adapter-based
diffusion frameworks without modifying the backbone. Across Office-31, Office-Home, and VisDA-2017, VT-DUDA improves average target-domain accuracy over strong discriminative and
diffusion-based UDA baselines. The results suggest that, in generation-based UDA, a stronger conditioning interface can improve the downstream usefulness of synthetic target-style data.
Read the full abstract ▾
All authors Xuan Qi, Daniele Berardini, Dario Serez, Vito Paolo Pastore, Vittorio Murino
Computer Vision
Enlight: Fast Low-Light Image Enhancement via Multi-Objective Optimization and Shadow-Aware Refinement
Nirjhor Datta and M. Sohel Rahman
We present ENLIGHT, a fast and training free framework for low-light image enhancement based on direct optimization of a perceptual objective. Unlike deep learning approaches that require large scale training data and supervision, ENLIGHT operates in a zero-shot manner by optimizing image quality at inference time. The method employs a two stage global to local optimization strategy. In the first stage, ENLIGHT performs global illumination adjustment to improve visibility while maintaining structural consistency and avoiding excessive noise enhancement. In the second stage, a shadow aware refinement selectively improves low-intensity regions through masked local optimization, enhancing visibility without overexposure. To balance quality and efficiency, we introduce two modes: Fast, which uses a multi-objective formulation combining
entropy, gradient preservation, and noise regularization, and Ultrafast, which reduces computational cost via a lightweight approximation of the same objective. The framework is optimizer agnostic and supports both evolutionary and lightweight local search methods. Experiments on BAID, Backlit300, LIME, MEF, NPE, and DICM demonstrate that ENLIGHT achieves competitive perceptual quality (MUSIQ, NIQE, BRISQUE) with significantly lower inference time. Qualitative results further show improved contrast, preserved structural details, and controlled noise amplification, making ENLIGHT a practical and interpretable alternative to learning based methods.
Read the full abstract ▾
Computer Vision
UnityShots: Memory-Driven Multi-Shot Audio-Video Generation with Boundary-Aware Gating
Jiehui Huang, Yuechen Zhang, Bin Xia et al. (+7 more)
Generating a coherent multi-shot
video requires structured cross-shot
memory. Subject appearance, scene context, and speaker identity must persist across cuts. Existing approaches either train end-to-end over fixed-length sequences and cannot scale, generate shot-by-shot with
memory banks that grow linearly, or orchestrate pretrained generators under an LLM planner without a multi-shot-aware backbone. We present UnityShots, a
memory-driven multi-shot audio-
video generation system built on LTX-2.3, trained on annotated cinematic and music-
video shots. The
video stream maintains two fixed-size slots, a long-term
memory (LTM) slot anchored to the opening shot and a short-term
memory (STM) slot holding the immediately preceding tail, both updated at every cut by a boundary-conditioned gate that fuses visual cut probability and beat-tracker signals. The audio stream injects a reference speaker
token at every shot to preserve vocal timbre without a sliding audio bank. A discrete cut-type prior, learned through AdaLN, becomes an inference-time control knob over transition strength. We release a
benchmark of $200$ multi-cultural multi-shot sequences spanning six ethnic regions and ten or more
languages, with per-shot reference identities, reference audio, and per-boundary transition labels. Evaluated across I2V, T2V, and R2V conditioning modes, UnityShots leads open-source baselines on every cross-shot coherence metric and matches the strongest closed-source system on the multi-shot axes.
Read the full abstract ▾
All authors Jiehui Huang, Yuechen Zhang, Bin Xia, Jiahao Wang, Xu He, Zhenchao Tang, Meng Chu, Xin Tao, Pengfei Wan, Jiaya Jia
Computer Vision
Chehre: An Emoji-Prompted Dataset to Explore Perceptual Flexibility in Video Language Models
Bita Azari, Zoe Stanley, Avneet Batra et al. (+4 more)
Do people perceive the same facial expression in the same way? Should we expect
vision models to be flexible in how they perceive facial expressions? Facial expressions are nonverbal social signals used in human interaction, but facial expression recognition
datasets often focus on a single deterministic annotation per sample. We introduce Chehre, an emoji-prompted
video dataset with a wide range of dynamic facial expressions for exploring perceptual variation. In Chehre, 203 participants were prompted to express and record 40 facial emojis. Later, their facial motions were transferred onto synthetic faces to preserve
privacy. A separate group annotated the
videos, resulting in 2,111
videos annotated by 1,242 perceivers, with ~30 annotators per
video. Chehre enables us to define a new task: "distributional expression recognition", which tests whether a model can reproduce the variation observed across annotator responses. We test a selection of
video language models on our task. Interestingly, we find that persona prompting can act as a controllable way to shift model perception while helping models better capture the variation observed across human annotators. The
dataset and code are available at https://chehre-
dataset.github.io/.
Read the full abstract ▾
All authors Bita Azari, Zoe Stanley, Avneet Batra, Poorvi Bhatia, Hali Kil, Manolis Savva, Angelica Lim
Computer Vision · Cross-listed from eess.IV
PaaF: Raising the perceived quality of INR-Based Image Compression
Lorenzo Catania and Dario Allegra
Implicit Neural Representations (INRs) have recently emerged as a promising paradigm for image compression, offering a fundamentally different approach from traditional and learned codecs. Nevertheless, INR-based methods for image compression suffer from long encoding times and a consistent performance gap in classic quality metrics such as PSNR. In this work, we explore the potential of purely INR-based compression methods and we propose PaaF (Picture as a Function), a novel INR-based image codec that introduces improved architectural design, adaptive quantization, and an efficient
entropy coding scheme. These components are designed to enhance rate-distortion performance while preserving the simplicity and parallelizability of INR-based decoding. Experimental results demonstrate consistent improvements over existing INR-based methods in both quantitative metrics and perceptual quality. These findings highlight the potential of INR-based approaches and contribute to narrowing the gap between functional representations and more established compression paradigms.
Read the full abstract ▾
Computer Vision
A DVDrive Approach for doScenes Instructed Driving Challenge
Zijian Fu, Xiangyang Chu, Mengshi Qi et al. (+3 more)
Instruction-conditioned trajectory prediction is an emerging problem in autonomous driving, where a model predicts the future ego trajectory not only from visual scene context and historical motion, but also from a natural-
language maneuver instruction. This paper presents our submission to the doScenes Instructed Driving Challenge, built upon OmniDrive, a
vision-
language-action driving
agent with 3D perception,
reasoning, and planning capabilities. We adapt OmniDrive to the doScenes setting by training it on instruction-annotated nuScenes scenes and generating a 6-second ego trajectory represented by 12 future waypoints. To improve multi-view visual grounding, we further introduce a DVPE-style divided-view perception module into the OmniDrive perception head. Instead of attending globally to all camera features, the proposed module groups query features and image
tokens into divided local view spaces and performs visibility-aware cross-
attention within each view. This design reduces irrelevant cross-view interference and helps the model better align
language instructions with local driving-relevant visual evidence. The code is publicly available at: https://github.com/feel12348/doscenes-omnidrive.
Read the full abstract ▾
All authors Zijian Fu, Xiangyang Chu, Mengshi Qi, Huadong Ma, Guanghao Zhang, Wei Li
Computer Vision
Cross-Modal Corroboration for Annotation-Free Wildlife Monitoring
Bharath Pillai, Varun Viswapriyan, Christopher Stewart et al. (+2 more)
Scaling wildlife monitoring for real-world conservation deployments requires automated analysis of smart sensors that operate under severe annotation scarcity. We propose leveraging expert knowledge of species activity patterns as an annotation-free validation signal for
multimodal monitoring pipelines. We operationalize agreement as the
alignment of independently derived hourly activity curves both with each other and with published behavioral priors-a three-way convergence that rules out shared-data confounds and
dataset-internal correlation as alternative explanations. Our
vision pipeline combines zero-shot species detection via BioCLIP 2, sliced inference to handle deployment-constrained camera positioning, and geometry-based geographic localization from camera trap imagery. Our acoustic pipeline detects species vocalizations via a fine-tuned classifier. We validate the pipeline on a breeding herd of Milu deer and demonstrate that both modalities independently recover activity patterns consistent with known deer behavioral ecology with minimal manual annotation. The framework applies to species detectable in both visual and acoustic modalities for which behavioral priors are documented in the literature, suggesting a practical path toward self-validating wildlife-monitoring pipelines at conservation scale.
Read the full abstract ▾
All authors Bharath Pillai, Varun Viswapriyan, Christopher Stewart, Tanya Berger-Wolf, Jenna Kline
Computer Vision
CurvSegFlow: Time-Conditioned Flow Matching for Robust Segmentation of Curvilinear Structures in Noisy Biomedical Images
Sidi Mohamed Sid'El Moctar, Achraf Ait Laydi, Alexandre Beber et al. (+4 more)
Accurate
segmentation of curvilinear structures remains challenging in biomedical imaging due to their thin geometry, complex
topology, and sensitivity to noise. This is particularly critical for microscopy images of cytoskeletal network, where low signal-to-noise ratios and dense filament crossings often lead to fragmented or inaccurate
segmentation. In this work, we propose CurvSegFlow, a
segmentation framework based on time-conditioned flow matching. Instead of predicting a
segmentation mask in a single pass, the method models
segmentation as a dynamic process that progressively refines a noisy initialization into the target structure through a learned velocity field. The proposed model combines a U-Net backbone with triple-term loss function and temporal
embeddings to guide the refinement process across reconstruction stages. This formulation enables gradual error correction and improves the continuity of thin structures. CurvSegFlow is evaluated on multiple synthetic and real microtubule
datasets, as well as on public
benchmarks of retinal vessels, corneal nerves and coronary arteries. Across
datasets, the method achieves competitive or superior performance compared to established
segmentation models, with consistent improvements in precision and structural continuity, particularly under low signal-to-noise conditions. These results show that flow-based iterative refinement provides a robust and general framework for curvilinear structure
segmentation. Overall, the proposed approach improves
segmentation quality in challenging imaging conditions and generalizes effectively across modalities without architectural changes.
Read the full abstract ▾
All authors Sidi Mohamed Sid'El Moctar, Achraf Ait Laydi, Alexandre Beber, Marcus Braun, Zdenek Lansky, Yousef El Mourabit, Helene Bouvrais
Computer Vision
T-MOR: Learning Motion-Aware Skeleton Representations for Human Action Recognition
Di Yang, Mahmoud Ali, Quan Kong et al. (+2 more)
Vision-
language models such as CLIP have recently achieved strong performance on a wide range of visual understanding tasks. However, most existing models rely primarily on appearance-level supervision from images or
videos, and do not explicitly model human motion, which is essential for fine-grained and human-centric action recognition task as actions are defined by temporally structured and physically grounded body movements. To address this problem, we propose Transferable skeleton MOtion Representation (T-MOR), a motion-aware framework that learns transferable action representations from skeleton sequences with the aid of
video and
language supervision during training. T-MOR adopts a multi-modal contrastive learning scheme that aligns skeleton motion with visual and textual representations, while performing inference using only lightweight skeleton inputs. To support large-scale pre-training, we construct PoseCap-1M, a new
dataset that contains over one million synchronized
video, skeleton, and text triplets covering diverse human activities. We evaluate T-MOR on a range of human-centric action recognition
benchmarks, including action classification and frame-wise temporal detection. Experimental results show that T-MOR consistently improves performance across multiple
datasets, such as Toyota Smarthome, Penn Action,
UAV-Human, TSU, and Charades. In addition, T-MOR demonstrates strong generalization ability in few-shot and zero-shot settings, highlighting the effectiveness of motion-centric and
embodied representations for transferable action understanding.
Read the full abstract ▾
All authors Di Yang, Mahmoud Ali, Quan Kong, Gianpiero Francesca, Francois Bremond
Computer Vision
$\mu$Match: Foundation Models for Semi-supervised Learning and Domain Adaptation in EM
Marei Freitag, Olesia Korchevaia, Luca Freckmann et al. (+2 more)
Vision foundation models have substantially advanced computer
vision, enabling state-of-the-art performance in zero- and few-shot settings. They have been successfully applied to biomedical imaging tasks ranging from organ
segmentation in computed
tomography to
cell segmentation in light microscopy. Electron microscopy (EM) is a central modality for analyzing cellular ultrastructure due to its nanometer-scale resolution. However, the application of foundation models in EM has so far been limited to specific organelles, such as mitochondria, largely due to the diversity of
segmentation tasks and the scarcity of comprehensively annotated data. As a result, EM
segmentation still predominantly relies on supervised learning, requiring extensive manual annotation and limiting ultrastructural analysis. To address this gap, we propose $\mu$Match, a framework for semi-supervised learning and domain adaptation that leverages foundation models. We implement state-of-the-art student-teacher-based methods and evaluate multiple foundation models (SAM, SAM2, $\mu$SAM, DINOv2/v3) on challenging EM tasks, including mitochondrion,
nucleus, and neurite
segmentation. Our results demonstrate consistent improvements over strong baselines and highlight a path toward substantially reducing the annotation effort in EM.
Read the full abstract ▾
All authors Marei Freitag, Olesia Korchevaia, Luca Freckmann, Anwai Archit, Constantin Pape
Computer Vision · Cross-listed from eess.IV
Deep Unrolled Networks in Representation Space Applied to MRI Reconstruction
Efe Il{\i}cak, Baris Imre, Chloé Najac et al. (+4 more)
Deep unrolled networks (DUNs) integrate physical forward models with learned regularization in cascaded network architectures, achieving exceptional performance in inverse problems while maintaining interpretability. While most DUNs operate in the object domain (e.g., image space), recent variants explored representation spaces for improved information flow. However, these methods rely on heuristic methods for data consistency (DC), sacrificing fidelity with measurements. In this work, we introduce DUNE (Deep Unrolled Networks in rEpresentation space), a framework that maintains exact adherence to physical measurements while operating in learned representation spaces. By deriving the DC gradient via the chain rule and implementing it through the Vector-Jacobian Product (VJP), we enable exact backpropagation of measurement residuals into the representation space. This formulation supports diverse architectural backbones, including pre-trained encoders to guide the iterative process. We assess DUNE against state-of-the-art baselines on accelerated MRI reconstruction tasks, demonstrating that exact VJP-based gradients yield superior reconstruction quality and structural fidelity across both single-channel portable low-field and multi-channel clinical high-field MRI acquisitions. The code will be available upon publication at https://github.com/EfeIlicak/DUNE.
Read the full abstract ▾
All authors Efe Il{\i}cak, Baris Imre, Chloé Najac, Ruben van den Broek, Beatrice Lena, Andrew Webb, Marius Staring
Computer Vision
$\phi$-Scene: Physically Grounded Image-to-3D Scene Reconstruction
Haodong Li, Lulu Shao, Haolin Lu et al. (+4 more)
Recent image-to-3D scene methods recover high-fidelity 3D objects with plausible arrangements, but often leave floatings and interpenetrations that limit physical validity and downstream use in interactive environments. We present $\phi$-Scene, a physically grounded approach for open-vocabulary and compositional image-to-3D scene reconstruction that treats a scene not merely as a set of objects with predicted poses, but as a globally stable physical system. $\phi$-Scene formulates reconstruction as
topology-driven physical assembly: given an initial scene of reconstructed objects, it infers how objects support one another and settles them one by one in topological order. For each object, SDF-based optimization first resolves penetrations against the already-settled support context, and rigid-body simulation then settles the object into a stable
equilibrium under real-world physical constraints. The resulting scene stays aligned to the reference image, with every object resting at a physically valid, stable contact configuration. On the 3D-Front
benchmark, $\phi$-Scene achieves the strongest overall performance among out-of-domain methods and remains highly competitive with in-domain baselines on standard reconstruction metrics. Human and MLLM studies prefer $\phi$-Scene in visual quality, reference
alignment, and physical plausibility. Dedicated physical metrics show that it substantially reduces penetration artifacts and yields much lower post-simulation drift. To our knowledge, $\phi$-Scene is among the first image-to-3D scene reconstruction methods that explicitly reaches dynamic rigid-body
equilibrium while preserving reference
alignment.
Read the full abstract ▾
All authors Haodong Li, Lulu Shao, Haolin Lu, Yu Fu, Yen-Ru Chen, Seemandhar Jain, Manmohan Chandraker
Computer Vision
Boundary-by-Mask: Few-Shot Instance Segmentation with Mask-Conditioned Boundary Learning for Texture-Poor Industrial Parts
Yutaka Yoshinaga, Naoya Chiba and Koichi Hashimoto
Recent advances in large pre-trained models have led to remarkable progress in instance
segmentation on general images. However, industrial scenarios remain challenging. Instance definitions are often application-specific and inconsistent, and the domain gap from general imagery is substantial due to weak textures and limited contextual cues. Consequently, a direct application of existing models is unreliable. We propose Boundary-by-Mask, a few-shot instance
segmentation framework that supervises boundaries instead of interior appearance. Given a few RGB images and corresponding instance masks, the method extracts rich visual features using a foundation-model encoder and trains a lightweight Signed Distance Function (SDF) head to predict boundary-aware distance maps.
Segmentation masks are obtained through an SDF-to-mask reconstruction process. By explicitly estimating contours, the framework achieves reliable instance separation even on low-texture and color-uniform surfaces. The instance definition is conditioned by the instance mask. Replacing the mask specifies the
segmentation target, such as the whole object or a sub-part. A pixel-wise shallow MLP head enables rapid training. Experiments on industrial parts and food items with ambiguous boundaries show strong few-shot generalization, robustness in feature-poor conditions, and precise control over mask-level targets.
Read the full abstract ▾
Computer Vision
Radial Basis Function Networks as Projection Heads in Self-Supervised Learning
Andreas Schliebitz, Heiko Tapken and Martin Atzmueller
Self-supervised learning (SSL) typically relies on a backbone encoder followed by a small multilayer perceptron (MLP) projection head, which is conventionally discarded after training, while backbone quality is assessed via costly linear probing on labeled data. We argue that this approach including discarding the projector is rather computationally wasteful. Instead, we propose replacing the MLP head with a radial basis function network (RBFN), whose interpretable center and shape parameters can be exploited to judge representation quality without labels or a separate classifier. To this end, we introduce Scale-Normalized Separation (SNS), a novel label-free quality metric derived solely from the kernel centers and shapes learned during training. Across five canonical SSL architectures (MoCo, SimCLR, BYOL, SwAV and SimSiam) and four image classification
datasets, we show that RBFN projection heads are competitive drop-in replacements for standard MLP projectors. We recommend constructing them with three RBF layers activated by the Gaussian radial basis function. Moreover, SNS exhibits strong to very strong positive correlation with established logistic regression metrics, demonstrating that a trained RBFN projector can act as a reliable proxy for backbone representation quality. We additionally publish a novel PyTorch compatible image classification
dataset based on Google's Open Images V7 to facilitate reproducible research into representation learning.
Read the full abstract ▾