Computer Vision
ParCo-SDF: Learning Prior-Free Partial-to-Complete Signed Distance Fields of Deformable Objects
Deokmin Hwang, Minseok Song and Daehyung Park
This study addresses the partial-to-complete geometry reconstruction of deformable objects (DOs) from point-cloud observations toward precise DO manipulation. Recent DO reconstruction approaches often adopt implicit neural representations (INRs) to model continuous surfaces as well as capture structural variability. However, these methods typically rely on object-specific shape priors that improve training stability and limit generalization. To figure it out, we introduce ParCo-SDF, a two-stage partial-to-complete signed distance field (SDF) reconstruction framework consisting of temporal geometry encoding followed by FiLM-conditioned SDF prediction. The temporal encoder captures structural similarity across DO sequence, enabling prior-free stable training. FiLM-based conditioning preserves reconstruction expressivity while reducing network complexity. We evaluate the proposed method against a state-of-the-art DO surface reconstruction baseline on a rubber band manipulation
dataset, demonstrating robust and high-fidelity reconstruction under severe occlusions.
Read the full abstract ▾
Computer Vision · Cross-listed from cs.RO
3DVLA: Enhancing Vision-Language-Action Models via 3D Spatial and Instance Understanding
Zhongyu Xia, Yousen Tang, Bingqing Wei et al. (+1 more)
Vision-
Language-Action models have achieved remarkable progress in robotic manipulation, yet they suffer from a critical limitation: a lack of 3D scene understanding. This deficiency manifests as three intertwined challenges: weak extraction of 3D spatial positions without enforcing multi-view consistency, inadequate 3D instance understanding, and fragile
reasoning under occlusion. Although mature 3D perception methods exist, their direct integration into VLA pipelines is hindered by architectural incompatibility and by heavy reliance on costly instance-level annotations. To address the above challenges, we propose 3DVLA, a plug-and-play framework that injects robust 3D
reasoning into pretrained VLAs without requiring extra manual labels or discarding VLM priors. Specifically, 3DVLA tackles the three challenges through: (1) pervasive 3D feature encoding with explicit multi-view consistency constraints across all modalities and a Spatially-Conditioned Geometry Aggregation method, (2) an instance estimation module with high-level instance
tokens for 3D instance awareness, and (3) a masked self-supervised 3D encoding branch that retains its predictor for visual
token completion to handle occlusions. We integrate 3DVLA with multiple VLA baselines and evaluate on LIBERO-Plus and RoboTwin 2.0. Results show consistent and significant gains in manipulation performance, validating both the effectiveness and plug-and-play compatibility of our approach.
Read the full abstract ▾
All authors Zhongyu Xia, Yousen Tang, Bingqing Wei, Yongtao Wang
Computer Vision · Cross-listed from eess.IV
Constructing efficient channels for ideal observers using the conjugate gradient method
Weimin Zhou
Task-based assessment of image quality (IQ) is critically important for the design and optimization of medical imaging systems. Ideal observers, including the
Bayesian Ideal Observer (IO) and the ideal linear observer, i.e., the Hotelling observer (HO), provide objective figures of merit (FOMs) that quantify system performance on signal detection tasks. However, the application of ideal observers to high-dimensional image data is often computationally intractable. Channel mechanisms provide an effective framework for dimensionality reduction that can facilitate the computation of ideal observers. This work presents a conjugate gradient (CG)-based method to construct efficient channels for approximating the IO and HO performance.
Read the full abstract ▾
Computer Vision
Semantic and Visual Evidence for Efficient Long-Video Reasoning: A Solution for the HD-EPIC VQA Challenge
Yinsong Xu, Wei Jing, Liuxin Zhang et al. (+2 more)
Understanding long-form
egocentric videos remains challenging for
multimodal large
language models (MLLMs) due to limited context length and insufficient grounding of fine-grained visual details. The recently proposed HD-EPIC
benchmark highlights these limitations: even strong long-context models achieve relatively low performance across diverse
video question answering tasks. In this paper, we propose a unified framework that decouples long-
video reasoning into two complementary forms of evidence: semantic evidence and visual evidence. Semantic evidence captures global procedural structure through a coarse-to-fine extraction pipeline, while object-centric visual evidence preserves fine-grained grounding through bounding boxes and visual
embeddings. During inference, we formulate
reasoning as a query-conditioned evidence
retrieval and integration process, dynamically selecting relevant information from both sources. Our approach achieves competitive performance in the HD-EPIC-VQA Challenge across multiple task categories. More broadly, our results demonstrate that explicitly structuring, retrieving, and integrating semantic and visual evidence is critical for effective long-
video understanding with MLLMs.
Read the full abstract ▾
All authors Yinsong Xu, Wei Jing, Liuxin Zhang, Wanjun Lv, Hui Li
Computer Vision
Orthogonal Negative Guidance in Attention Feature Space for Text-to-Image Generation
Jungmin Ko, Jungwon Park, Jimyeong Kim et al. (+3 more)
Text-to-image (T2I) models have become increasingly capable of generating high-quality images. Yet, enforcing the explicit absence of a specified object or attribute remains a fundamentally challenging problem. Existing approaches, including
prompt negation, post-hoc editing, and negative guidance, remain insufficient for explicit concept suppression, often failing to remove the target concept or degrading overall image quality. To this end, we propose Orthogonal Negative Guidance in
attention feature space, a training-free method that operates in the
attention output space of MM-DiT-based T2I
transformers. Our method orthogonalizes negative-
prompt attention features with respect to positive-
prompt features and subtracts only the orthogonal component, suppressing unwanted concepts while preserving desired semantics. Experiments on FLUX-dev and FLUX-schnell show that our method achieves favorable trade-offs between concept suppression,
prompt alignment, and image quality. In human evaluation, our method outperforms the second-best baseline by 18.78%. We further show that our method supports multi-concept suppression and adjustable concept suppression.
Read the full abstract ▾
All authors Jungmin Ko, Jungwon Park, Jimyeong Kim, Changin Choi, Wonseok Lee, Wonjong Rhee
Computer Vision · Cross-listed from cs.LG
TRACER: Persistent Regularization for Robust Multimodal Finetuning
Hesam Asadollahzadeh, Feng Liu, Christopher Leckie et al. (+1 more)
Mainstream strategies for finetuning pretrained
multimodal models often degrade out-of-distribution (OOD) robustness, a phenomenon known as catastrophic forgetting. In this paper, we develop a theoretical framework for
multimodal contrastive finetuning, yielding closed-form solutions and a
geometric decomposition for each strategy. This framework shows that self-
distillation is more effective than other regularization approaches to retain the knowledge of the pretrained model. Our analysis reveals a largely overlooked limitation: standard Exponential Moving Average (EMA) teachers, widely used in robust finetuning, suffer from collapse. To solve this, we prove that a Weighted Moving Average (WMA) teacher maintains a persistent regularizing force over finite horizons and yields bias-free convergence in the task subspace while preserving orthogonal knowledge. These insights motivate **TRACER** (**T**rajectory-**R**obust **A**nchoring for **C**ontrastive **E**ncoder **R**egularization), which combines contrastive learning with WMA-guided multi-perspective
distillation. Extensive experiments on CLIP finetuning demonstrate consistent OOD accuracy and calibration gains across three backbone architectures, and comprehensive ablations confirm that TRACER is both principled and robust to hyperparameter choices. Code is available at [https://github.com/HesamAsad/TRACER](https://github.com/HesamAsad/TRACER).
Read the full abstract ▾
All authors Hesam Asadollahzadeh, Feng Liu, Christopher Leckie, Sarah M. Erfani
Computer Vision · Cross-listed from cs.CR
DeepFake Forensics AI: A Multi-Modal Detection and Blockchain-Anchored Evidence Management Platform
Naisha Minnah
The proliferation of AI-generated synthetic media poses a critical threat to the integrity of digital evidence in legal and forensic contexts. Existing deepfake detection systems typically address a single modality and provide no mechanism for tamper-proof evidence preservation. We present DeepFake Forensics AI, a unified platform that detects synthetic media across image,
video, and audio modalities, identifies generative architecture fingerprints, and anchors forensic evidence immutably on the Ethereum blockchain. Our system trains four independent neural networks from scratch: an EfficientNet-B4 image detector (AUC = 0.9868), a Bidirectional LSTM
video detector (AUC= 0.9628), an ECAPA-TDNN audio detector (EER = 18.63%), and a novel GAN fingerprinting module (accuracy = 99.88%) that identifies the generative architecture behind a fake image. Evidence files are hashed with SHA-256, stored on IPFS via Pinata, and registered on-chain via a Solidity smart contract with role-based access control. The platform provides a React frontend and FastAPI backend suitable for deployment in forensic and legal workflows. To our knowledge, this is the first system to unify multi-modal deepfake detection with blockchain-based chain-of custody management.
Read the full abstract ▾
Computer Vision
WorldMemArena: Evaluating Multimodal Agent Memory Through Action-World Interaction
Chengzhi Liu, Yuzhe Yang, Sophia Xiao Pu et al. (+14 more)
Multimodal large
language models are increasingly deployed as long-horizon
agents, where
memory must do more than recall: it must track an evolving world, revise what has gone stale, and surface the right evidence at decision time. Existing
benchmarks measure recall over static dialogue, collapse
memory into a single end-of-task accuracy, and reduce visual observations to captions, leaving us unable to localize failures to writing, maintenance,
retrieval, or use. The rise of
agent harnesses that author their own
memory sharpens this gap, since we have no principled way to compare hand-designed pipelines with self-managing alternatives. To close these gaps, we formulate
multimodal agent memory as an Action-World Interaction Loop with an observable four-stage lifecycle, and instantiate it in WorldMemArena: 400 multi-session
multimodal tasks spanning Lifelong
Evolution (evolving personal and task states) and Agentic Execution (
memory from real observations, actions, and feedback), annotated with gold
memory points, updates, distractors, and evidence chains for stage-level diagnosis. This enables the first head-to-head comparison of long-context, manually designed (RAG and external
memory systems), and harness-based
memory agents. Results show that: (1) better
memory writing and storage do not guarantee better performance; (2)
multimodal memory still struggles to fully use visual evidence; (3) systems are unstable across domains and degrade on realistic agentic trajectories; and (4) harness
memory is more flexible but remains costly and less reliable.
Read the full abstract ▾
All authors Chengzhi Liu, Yuzhe Yang, Sophia Xiao Pu, Yepeng Liu, Lin Long, Yichen Guo, Nuo Chen, Zhaotian Weng, Elena Kochkina, Simerjot Kaur, Charese Smiley, Xiaomo Liu, James Zou, Sheng Liu, Yuheng Bu, Songyou Peng, Xin Eric Wang
Computer Vision
Junzhe Zhang, Huixuan Zhang, Guirong Wang et al. (+5 more)
With the rapid advancement of
multimodal large
language models (MLLMs), models have demonstrated increasingly powerful
multimodal capabilities. However, whether MLLMs trained through statistical learning can truly understand the causal relationships underlying the real world remains a key research question. In recent years, numerous
multimodal causal
reasoning datasets have been proposed. Nevertheless, these
datasets are either limited in scale or constructed from synthetic images and
videos, cartoon-based content, or other non-realistic
multimodal sources. To address these limitations, we collect real-world
videos and construct DMC-CF-Static, a large-scale
benchmark for
multimodal causal counterfactual
reasoning. Furthermore, to mitigate issues such as data contamination in traditional static evaluation, we represent causal events using causal graphs and propose the Dynamic Graph Intervention (DGI) framework to build the dynamic evaluation
benchmark DMC-CF-Dynamic from DMC-CF-Static. Experimental results on the overall DMC-CF, which includes both static and dynamic evaluation
benchmarks, demonstrate that the
multimodal causal
reasoning capabilities of current
multimodal large
language models in real-world scenarios still require substantial improvement.
Read the full abstract ▾
All authors Junzhe Zhang, Huixuan Zhang, Guirong Wang, Xingyao Zhang, Pei Liu, Lin Qu, Hu Wei, Xiaojun Wan
Computer Vision
Rethinking FID Through the Geometry of the Reference Dataset
Yunghee Lee and Byeonghyun Pak
Fréchet Inception Distance (FID) is widely used to evaluate image generators, yet lower FID does not always correspond to better sample quality. We show that this mismatch depends in part on the geometry of the reference
dataset. In a controlled study across six
datasets, distributional density and effective rank significantly explain how FID changes as sample quality improves. Concentrated
datasets tend to yield more favorable FID trends, whereas more dispersed
datasets can make FID worsen despite better samples. Attribution to precision and recall and ablations with alternative feature spaces and distances support the same conclusion. These results suggest that distributional metrics should be interpreted together with the geometry of the reference
dataset for more reliable benchmarking.
Read the full abstract ▾
Computer Vision
EarthShift: a benchmark for measuring robustness to real-world distribution shifts in Earth observation
Kelsey Doerksen and Hannah Kerner
Current Earth observation
benchmarks focus on measuring performance on diverse tasks and applications, typically measuring generalization in-distribution. But when models are deployed, they must generalize to myriad out-of-distribution scenarios, such as new time periods, geographies, scales, and sensors. We introduce EarthShift: the first public testbed for benchmarking robustness across multiple realistic distribution shifts encountered in remote sensing. EarthShift enables users to measure distributional robustness by comparing performance in- and out-of-distribution using
datasets from paired
datasets from different sources, temporal windows, geographic locations, and sensors. Our experiments on 8 geospatial foundation models (GFMs) and 11 tasks covering 5 shift types show that GFMs consistently perform 15-20% worse out-of-distribution on average regardless of model architecture, size, pre-training or fine-tuning strategy. We show that GFM robustness is similar to that of generic
vision foundation models, and even fully-supervised models. This highlights a need for future research to strive for improvements in distributional robustness, not just performance, which can be benchmarked using EarthShift. We release our code and
datasets to provide a testbed to guide future work to create foundation models that are robust and reliable in real-world applications. Code and data for EarthShift are available at: https://earthshift.github.io
Read the full abstract ▾
Computer Vision
Multi-Stage VLM Pipeline for Zero-Shot Traffic Accident Understanding
Fumiya Tatematsu and Fumihiko Takahashi
We present the 1st-place solution to the ACCIDENT challenge at the CVPR 2026 AUTOPILOT Workshop, which asks for zero-shot prediction of accident timing, impact centroid, and collision type from CCTV footage. On a frozen Qwen3-VL-32B-Instruct checkpoint we build a three-stage pipeline (full-
video joint prediction, time refinement, and single-frame grounding of the impact centroid), run the same pipeline a second time on a 235B Mixture-of-Experts sibling, blend the two outputs 9:1, and finally snap each predicted point onto the nearest vehicle detection. The final system reaches Public LB 0.55469 / Private LB 0.57080, roughly +0.21 over the strongest host baseline (Molmo-7B, 0.358) and wins the challenge. We ablate each component, report the negative results that shaped the final design, and release the code at https://github.com/fuumin621/cvpr2026-accident-1st-place-solution.
Read the full abstract ▾
Computer Vision · Cross-listed from cs.CL
STAMP: Training Explicit Memory for Mobile GUI Agents in Controllable and Scalable Virtual Environments
Junyang Wang, Haiyang Xu, Xi Zhang et al. (+4 more)
Mobile GUI
agents excel at immediate reactive control but frequently fail in realistic, long-horizon tasks that require
memory. This failure stems from a fundamental conflict between limited context windows and
token-heavy screenshots. To save the limited context,
agents must progressively discard older visual history, permanently losing crucial transient information. Furthermore, existing action-centric
datasets fail to teach
agents what or when to explicitly memorize, and augmenting static real-world data is prohibitively expensive and lacks interactive verification. To resolve this, we present STAMP, a framework that trains explicit
memory in mobile
agents through controllable virtual environments, where deterministic
memory variables are programmatically injected into synthesized tasks to control what must be memorized, when it should be encoded, and when it must later be retrieved, thereby producing verifiable supervised data at scale and enabling online reinforcement learning through environment-driven reward feedback. Evaluated on our newly introduced
Memory-World
benchmark, the resulting Stamp-GUI
agent achieves state-of-the-art performance among GUI-specialized models and sets a new high watermark on our
Memory-World
benchmark, demonstrating exceptional
memory accuracy and task resilience while maintaining strong general mobile
navigation capabilities.
Read the full abstract ▾
All authors Junyang Wang, Haiyang Xu, Xi Zhang, Zhaoqing Zhu, Ming Yan, Jieping Ye, Jitao Sang
Computer Vision · Cross-listed from cs.GR
FreeForm: Reduced-Order Deformable Simulation from Particle-Based Skinning Eigenmodes
Donglai Xiang, Vismay Modi, Rishit Dagli et al. (+5 more)
We present a novel formulation for mesh-free, reduced-order simulation of deformable hyperelastic objects. Existing work in reduced-order elastodynamic simulation represents the input geometry by either meshes, which can be difficult to obtain due to challenges in scanning and triangulating complex shapes, or by neural fields that require per-shape optimization. We propose to adopt a Reproducing Kernel Particle Method (RKPM) representation, which enables the construction of reduced-order skinning weights by solving a generalized eigensystem on the Hessian
matrix of the elastic energy. We demonstrate that this formulation not only leads to a 40x training speedup compared with the per-shape optimization of neural fields, but also achieves lower simulation error when evaluated against the converged results of finite element method. We show our simulation results on a wide variety of objects in different representations including meshes and Gaussian splats, as well as the application of our method in the downstream task of
robot simulation.
Read the full abstract ▾
All authors Donglai Xiang, Vismay Modi, Rishit Dagli, Ty Trusty, Gilles Daviet, Anka He Chen, Nicholas Sharp, David I.W. Levin
Computer Vision
CapTalk: Text-Guided Stylization and Speech-Driven 3D Head Animation
Xuangeng Chu, Yuan Gan, Ziteng Cui et al. (+4 more)
Audio-driven 3D facial animation aims to generate synchronized lip movements and vivid facial expressions from arbitrary audio clips. While existing methods can produce synchronized lip motions, they often rely on predefined identity or style latent features, which limits users' ability to freely control speaking styles. Moreover, applying a fixed style or identity to an entire audio segment typically results in facial animation styles that do not adapt to the emotional content of the audio. To address these challenges, we revisit the
entanglement between style and
emotion, construct a large-scale
dataset with textual descriptions of both style and
emotion, and propose a novel talking head generation framework that enables separate control over style and
emotion. Our model takes as input both textual descriptions of speaking style and character
emotion, as well as the driving audio stream, enabling real-time generation of highly synchronized lip movements and facial expressions that match the provided descriptions. Furthermore, our model supports dynamic
emotion control during inference, allowing it to handle scenarios where the target
emotion changes throughout the
speech.
Read the full abstract ▾
All authors Xuangeng Chu, Yuan Gan, Ziteng Cui, Shuhong Liu, Jian Wang, Bing Zhou, Tatsuya Harada
Computer Vision
ViASNet: A Video Ad Saliency Network for Predicting Dynamic Saliency and Viewer Engagement
Jianping Ye and Michel Wedel
The digital media landscape has seen a pervasive shift toward short-form
video advertising on TV, social media and e-commerce platforms. The present study focuses on deep saliency prediction for short-form
video advertising. Deep saliency models have been used to generate predictions of human eye fixation patterns with the purpose of enhancing user interaction with digital technology and optimizing its design. For
video ads, dynamic saliency maps capture where and when viewers are looking, revealing why
video ads are effective, and how their content should be optimized. We develop and test a new deep dynamic saliency prediction model called ViASNet (
Video Ad Saliency Network), which has an architecture founded on the 3D U-Net, and accommodates the influence of audio and the semantic meaning of scenes. We assess the model's performance on 151
video ads, each seen by about 20 viewers wile their eye movements were tracked, and explore the critical factors influencing model performance through ablation experiments. We calculate the
entropy of the predicted saliency maps frame-by-frame as a diagnostic tool to identify ads and scenes that fail to engage viewers, and illustrate its use on test data of 15 unseen ads. Our study reveals that ad design and testing can be sped up considerably through automated systems built on deep saliency models such as ViASNet.
Read the full abstract ▾
Computer Vision
Pocket-Dentist: On-Device Dental Image Understanding via Efficient Multimodal Large Language Models
Kai Bian, Xucheng Guo, Bin Chen et al. (+4 more)
Evaluations of dental
vision-
language models remain fragmented across
datasets, task definitions and metrics, and often ignore their computational cost. This limits their widespread deployment for dental screening outside specialist centres, where timely inference, limited hardware, and local handling of patient images are vital for practical,
privacy-preserving clinical prescreening. Here we present Pocket-Dentist, an efficiency-aware
benchmark for dental
multimodal question answering that brings together three
datasets spanning approximately 1,159 patients from BRAR and MetaDent, five task types and seven metrics. Across 14 typical VLMs, our results reveal an interesting observation: compact VLMs, such as 2B-parameter models, become competitive with much larger VLMs on most metrics after lightweight adaptation while requiring substantially lower computational costs in dental image understanding. Deployed locally on an iPhone 17 Pro, our finetuned compact VLM Pocket-Dentist-2B processed each sample in 4.31 s, reducing latency by 4.9x and
memory use by 2.3x compared with a 7B baseline. Our project page is available at https://2026-icml.github.io/pocket-dentist-icml.
Read the full abstract ▾
All authors Kai Bian, Xucheng Guo, Bin Chen, Lingyan Ruan, Yiran Shen, Ting Dang, Hong Jia
Computer Vision
Turbulence-Robust Dynamic Object Segmentation with Multi-Signal Priors and SAM2 Refinement
Bolian Peng, Ying Tang, Xu Liu et al. (+2 more)
This technical report presents our solution for the CVPR 2026 UG2+ Challenge Track 3: Dynamic Object
Segmentation in
Turbulence (DOST). We design a training-free multi-signal
segmentation pipeline that combines pretrained motion estimation, self-supervised semantic priors, background anomaly modeling, manually calibrated proposal fusion, and SAM2-based mask refinement. The method uses RAFT for dense motion responses, DINOv2 for semantic objectness priors, ViBe for training-free background modeling, and pretrained SAM2 for box-
prompt mask refinement. Instead of optimizing an end-to-end
segmentation network, our system operates entirely in inference mode. This design is suitable for the DOST setting, where severe atmospheric
turbulence produces pseudo-motion, blur, and intermittent target visibility, making a single motion cue unreliable. The final submitted masks are evaluated by the official leaderboard, which reports 0.425041 mIoU and 0.457206 mDice. Since no task-specific model training or fine-tuning is performed, stronger learned temporal association, adaptive proposal selection, or task-specific adaptation may further improve the system.
Read the full abstract ▾
All authors Bolian Peng, Ying Tang, Xu Liu, Long Sun, Xiaoqiang Lu
Computer Vision · Cross-listed from cs.IR
UniNote: A Unified Embedding Model for Multimodal Representation and Ranking
Jinghan Zhao, Wenwei Jin, Anqi Li et al. (+5 more)
Item-to-Item (I2I)
retrieval is a fundamental part of modern content platforms, supporting critical industrial workflows from recommendation engines to content
auditing. While
multimodal embedding methods have advanced general
retrieval, they often falter in I2I scenarios due to the challenges of balancing global content representation with fine-grained local
retrieval, the systemic inefficiency of decoupled
embedding-and-ranking pipelines, and the inherent trade-offs between model precision and serving latency. To solve these issues, we propose UniNote, a unified
embedding model designed for industrial I2I
retrieval. Tailored
retrieval strategies are introduced to support representation learning over complex,
multimodal content at varying granularities. To operationalize these strategies, UniNote employs a two-stage training paradigm: the first stage leverages contrastive SFT to establish robust base
embeddings, while the second stage refines ranking quality through a reinforcement learning (RL) process that aligns the model with content relevance. Our results show that UniNote achieves SOTA performance across diverse I2I tasks. Deployed at Xiaohongshu and integrated with Matryoshka Representation Learning (MRL), UniNote achieved significant improvements in
retrieval quality and cost efficiency in large-scale applications.
Read the full abstract ▾
All authors Jinghan Zhao, Wenwei Jin, Anqi Li, Jintao Tong, Luya Mo, Jiawei Li, Bin Li, Yao Hu
Computer Vision
Deep Psychovisual Image Representations
Wendi Ma, Aryaman Sharma, Wei Dai et al. (+1 more)
Psychovisual models suggest human
vision decouples low-level feature extraction from higher cognition by first forming intermediate abstractions. In contrast, deep learning-based
vision models routinely extract and aggregate features using homogeneous stacks of spatial layers, rendering their decision-making processes opaque. In this paper, we propose Deep Visual Coding, a learned frequency-domain representation inspired by 1990s image codes that quantised perceptually salient frequencies, which together with complex-valued image representations produces psychovisual-style abstractions. This approach enables the first psychovisual-based deep learning framework, utilizing data-driven spectral filters that learn to encode task-relevant semantic structures within distinct frequency sub-bands. Salience analyses reveal that our psychovisual models extract highly interpretable object parts compared to the amorphous regions produced by regular Convolutional Neural Networks (CNNs). Furthermore, we find that our models are less depth dependent than CNNs for model scaling, since our complex-valued representations and learned abstractions subsume the role of the deep spatial layers. Together, these findings demonstrate that psychovisual coding provides a promising path toward more efficient and transparent
vision models.
Read the full abstract ▾
All authors Wendi Ma, Aryaman Sharma, Wei Dai, Shekhar S. Chandra
Computer Vision
Toward a More Ethical Facial Age Estimation: A Generalized Zero-Shot Benchmark Without Training on Children's Data
Caio Petrucci, Leo Sampaio Ferraz Ribeiro and Sandra Avila
Age estimation from facial images typically relies on training data that includes images of minors, a practice that raises ethical, legal, and
privacy concerns and that child-data governance frameworks explicitly advise against. While the task remains relevant (e.g., for detecting child sexual abuse imagery), we advocate against using data from minors entirely and quantify what the exclusion costs in accuracy. We formalize age estimation without children's training data as a generalized zero-shot learning (GZSL) problem: age intervals present during training are seen classes and withheld intervals are unseen, with models evaluated jointly on both. The generalized setting, rather than conventional zero-shot evaluation on unseen classes alone, is the appropriate one here because a deployed
estimator must operate across the entire lifespan, not only on the interval withheld from it. Revisiting six widely used
datasets, we introduce standardized splits with strict age-group separation. For
datasets with identity annotations, subject-age-exclusive splits prevent identity leakage across the seen/unseen boundary. Evaluating nine state-of-the-art age estimation methods under this protocol reveals that all of them fail to generalize to unseen age groups, suffering substantial degradation — on average 46.4%, and up to 52.8% — relative to the supervised baseline. Moreover, models do not simply degrade: they systematically anchor predictions for unseen ages to nearby seen classes, a manifestation of the well-known seen-class bias in generalized zero-shot learning.
Read the full abstract ▾
Computer Vision
An Approach for Thyroid Nodule Analysis Using Thermographic Images
J. R. González, É. O. Rodrigues, C. P. Damião et al. (+6 more)
Thyroid cancer is said to be the second most common type of cancer in female individuals and the third in males by 2030, according to projections. In general, detecting cancer in its early stages improves the chance of survival of the individual. Thermography is a diagnostic tool that has been increasingly used to detect cancer and abnormalities, including that of thyroid. Various methods to segment and detect hot regions in thermograms and, consequently, to detect suspicious tissues present in these images have been proposed. It is well known that medical diagnosis yields a great deal of information. Thus, physicians have to comprehensively analyse and evaluate this information in a short period of time, which is infeasible in most cases. In this work, we perform a general review of thermography , focusing on the thyroid analysis. We propose protocols for image acquisiton and an autonomous registration for thyroid images. We also perform analyses of the image data, which include feature extraction, image processing, and a possible approach for classification of healthy or unhealthy patients. In summary, this work presents a pilot project for detection of tumors in our university hospital, which is part of an effort to support preventive medical actions in our endocrinology department. Under some future adjustments, this project will be submitted for approval by the ethics and research committee of Hospital Universitário Antonio Pedro at Universidade Federal Fluminense (HUAP-UFF) and to the Brazilian Ministry of Health Ethical committee under the name: Evaluation of the importance of thermography to aid diagnosis of thyroid nodules of patients in HUAP-UFF (in Portuguese: Avaliação da importância da termografia no auxílio à investigação diagnóstica de nódulos tireoidianos em pacientes acompanhados no HUAP-UFF).
Read the full abstract ▾
All authors J. R. González, É. O. Rodrigues, C. P. Damião, C. A. P. Fontes, A. C. Silva, A. C. Paiva, H. Li, C. Du, A. Conci
Computer Vision
Motion-guided sparse correction enables expert-quality point tracking across diverse microscopy regimes
Leonidas Zimianitis, Pasindu Thenahandi, Kai Buckhalter et al. (+10 more)
Tracking the dynamics of non-canonical biological systems in microscopy
videos remains a persistent challenge. Both classical and learning-based trackers depend on expert-reviewed data to be evaluated and adapted, yet exhaustive manual annotation rarely scales to the
videos where these tools are needed most. We developed RIPPLE (Refinement Interpolation Platform for Point Location Estimation), which recasts annotation as sparse correction: a user clicks a starting point, RIPPLE proposes a full trajectory, and the user intervenes only where the trajectory drifts. We tested RIPPLE on five challenging microscopy
datasets from our laboratories, four from the transparent jellyfish Clytia hemisphaerica and one tracking landmarks on rapidly moving sperm. Across these, RIPPLE matched the quality of exhaustive manual annotation while reducing manual clicks by 3 to 25 times across
datasets. RIPPLE thereby fills a missing layer between manual annotation and fully automated tracking, enabling immediate quantification of biological dynamics, method benchmarking, and the production of the gold-standard data needed to adapt future automated microscopy trackers.
Read the full abstract ▾
All authors Leonidas Zimianitis, Pasindu Thenahandi, Kai Buckhalter, Dineth Jayakody, Julian O. Kimura, Xinyue Liang, Karen Cunningham, Azeem Ahmad, Balpreet S. Ahluwalia, Sampath Jayarathna, Nikos Chrisochoides, Brandon Weissbourd, Dushan N. Wadduwage
Computer Vision
SalsaAgent: A multimodal embodied language model for interactive dance generation
Payam Jome Yazdian, Zoe Stanley and Angelica Lim
Embodied interaction with
humanoids depends on bidirectional nonverbal reactivity, coordination, and synchrony to convey cues and move with a partner. For socially interactive
embodied agents, reactive motion generation requires expressive full-body motion that remains contextually appropriate while maintaining spatial and temporal synchrony. We present SalsaAgent, a
language model that generates expressive, full-body salsa follower motions in reaction to a human leader and music. We formulate partner interaction as nonverbal
token passing, extending the vocabulary of a large
language model (LLM) to process discrete motion
tokens, pairwise relation
tokens, and audio
tokens. Our method introduces full-body and pairwise-relation tokenizers, aligns
language and motion
tokens with automatically derived text descriptions of skeleton dynamics, and applies a two-stage
token-to-
diffusion pipeline. Subjective and objective evaluations show improved motion quality, two-person spatial coordination, and music and partner coordination relative to prior baselines.
Read the full abstract ▾
Computer Vision
Towards the automated segmentation of epicardial and mediastinal fats: A multi-manufacturer approach using intersubject registration and random forest
É. O. Rodrigues, A. Conci, F. F. C. Morais et al. (+1 more)
The amount of fat on the surroundings of the heart is correlated to several health risk factors such as carotid stiffness, coronary artery calcification, atrial fibrillation, atherosclerosis, cancer incidence and others. Furthermore, the cardiac fat varies unrelated to the overall fat of the subject, and, therefore, it reinforces the quantitative analysis of these adipose tissues as being essential. Clinical decision support systems are computer programs capable of evaluating information and providing a corresponding diagnosis or data to complement the physicists' analyses. The aim of this work is to propose a method capable of fully automatically segmenting two types of cardiac adipose tissues that stand apart from each other by the pericardium on CT images obtained by the standard acquisition protocol used for coronary calcium scoring. Much effort was devoted to promote minimal user intervention and ease of reproducibility. The methodology proposed in this work consists of a registration, which will roughly adjust input images to a standard, an extraction of features related to pixels and their surrounding area and a
segmentation step based on data mining classification
algorithms that define if an incoming pixel is of a certain type. Experimentations showed that the achieved mean accuracy for the epicardial and mediastinal fats was 98.4% with a mean true positive rate of 96.2%. In average, the Dice similarity index was equal to 96.8%.
Read the full abstract ▾
All authors É. O. Rodrigues, A. Conci, F. F. C. Morais, M. G. Pérez
Computer Vision
MetaRanker: Human-in-the-loop Active Ranking for Metalens Image Quality
Yujin Park, Haejun Chung and Ikbeom Jang
Image quality in modern imaging systems emerges from the coupled effects of the sensor, optics, and computational reconstruction. Ultra-thin metalenses offer a path toward substantial miniaturization of optical modules, but practical designs often exhibit pronounced chromatic and field-dependent aberrations that necessitate computational reconstruction. In current metalens pipelines, reconstruction models are commonly trained and selected using distortion-based fidelity objectives, such as PSNR, yet these proxies can be weakly correlated with human preference and downstream utility, reflecting the well-known perception–distortion trade-off. We introduce MetaRanker, a human-in-the-loop active ranking framework that formalizes metalens image quality in terms of semantic interpretability, defined as the degree to which humans can reliably recognize objects and structures in the presence of optical artifacts. MetaRanker combines a probabilistic preference model with uncertainty-aware query selection, and leverages
vision–
language models to provide lightweight semantic priors. Importantly, these priors are used only to guide the sampling of informative comparisons; human judgments remain the primary supervision signal throughout. Across real-world and synthetic metalens
datasets with distinct degradation profiles, MetaRanker produces rankings that align most closely with human assessments, while reducing the number of pairwise annotations required by approximately 80% relative to exhaustive pairwise evaluation. Finally, we show that standard image quality assessment metrics exhibit limited
alignment with human interpretability in the metalens domain, positioning MetaRanker as a practical step toward perceptually grounded metalens evaluation and co-design.
Read the full abstract ▾
Computer Vision
Guidance Contrastive Token Credit Assignment for Discrete Policy Optimization
Shufan Li, Konstantinos Kallidromitis, Akash Gokul et al. (+2 more)
Group-advantage-based reinforcement learning methods, such as GRPO and DAPO, have demonstrated strong performance across diverse domains, including mathematical
reasoning and text-to-image generation. However, their reliance on sample-level rewards introduces a key limitation as uniform credit assignment across all
tokens fails to capture fine-grained,
token-level contributions. To address this issue, we propose Guidance Contrastive Policy Optimization (GCPO), a novel
algorithm that enables per-
token credit assignment by contrasting model predictions under positive and negative
prompts. Rather than uniformly broadcasting sample-level advantages, GCPO assigns
token-level advantages proportional to the difference between these contrastive predictions, allowing more precise and informative learning signals. Empirically, we find that GCPO emphasizes semantically relevant regions such as visual areas aligned with textual
prompts in text-to-image generation, and critical keywords within
reasoning traces for chain-of-thought tasks. Through extensive experiments, GCPO consistently outperforms GRPO and DAPO baselines on both text-to-image generation and chain-of-thought
reasoning benchmarks, demonstrating its effectiveness as a general and scalable optimization strategy for discrete policy learning.
Read the full abstract ▾
All authors Shufan Li, Konstantinos Kallidromitis, Akash Gokul, Yuta Kyuragi, Aditya Grover