Papers

Heat Kernel Textures (HKTex) represent surface appearance with intrinsic anisotropic kernels, but evaluate them using 50 global Laplace-Beltrami eigendecompositions and a resident basis of shape [50,V,256]. We study two complementary ways to remove this bottleneck while leaving the trainer, GeodesicOpt, density control, and compositing unchanged. LocalHK exploits the measured locality of trained kernels and replaces spectral evaluation by radius-bounded hinge unfolding and an analytic log-map kernel. On 10 Objaverse meshes and an 8-mesh low-poly holdout, it changes mean view PSNR from 31.35 to 32.00 and from 29.82 to 30.76 dB, respectively, while reducing initialization by 40.5 times and enabling a 749,570-vertex proxy-backed run where the spectral baseline fails. ThermalRF instead preserves the discrete anisotropic heat semigroup: GPU sparse Chebyshev actions and randomized range finding construct global low-rank heat factors without mesh-sized eigenvectors, and a compiled evaluator mixes four neighboring thermal responses. On spot and a thin-stem challenge, ThermalRF reduces end-to-end preprocessing, initialization, and 5,000-step optimization by 29.3% and 24.4%, with every surface, atlas, or view PSNR change within 0.12 dB and training allocation reduced by about 90%. The two routes expose a useful design choice: maximal locality and scale versus fidelity to the thermal PDE. Broader thermal-feature evaluation and real large scenes remain future work.

Radiomic texture features are commonly extracted from anisotropic CT and MRI acquisitions, where identical voxel offsets may represent different physical distances. We implemented and validated a voxel-spacing-aware extension of PyRadiomics that incorporates spacing information without generating interpolated gray levels. The framework operates across the Python frontend, C wrapper, and computational backend. GLCM uses anisotropy-relative feature-level angular aggregation, NGTDM uses anisotropy-relative weighted neighborhood averaging, and GLRLM, GLDM, and GLSZM are computed on a finite-volume zero-order-hold representation derived from the native anisotropic grid. Synthetic 3D phantoms were used for software validation. The modified implementation reproduced standard PyRadiomics exactly when spacing-aware mode was disabled and remained equivalent under isotropic spacing across 75 texture features. Under anisotropic spacing, the method selectively modified texture families and was numerically distinct from nearest-neighbor, linear, and B-spline resampling. Computational profiling showed moderate runtime and memory increases, while sensitivity analyses quantified finite-volume rounding effects and confirmed that spacing-aware differences persisted across binWidth settings. The framework provides a backward-compatible technical basis for future evaluation of spacing-aware radiomics in heterogeneous medical imaging datasets.

Fashion Image Captioning (FIC) plays a vital role in enhancing user experience and product search in e-commerce platforms. Unlike natural scene image captioning, FIC requires fine-grained visual reasoning and knowledge of domain-specific terminology to capture subtle attributes such as neckline and closure types, graphic patterns, and dress silhouettes. Moreover, as fashion inventories evolve rapidly with new trends, styles, and frequently emerging vocabulary, developing training-free captioning solution becomes essential for scalability and real-world adaptability. Instruction-tuned vision-language models (VLMs) offer a promising solution to fashion image captioning dueto their strong zero-shot capabilities and natural language fluency. However, these general-purpose models often lack attribute-level coverage and precision, and tend to hallucinate or misidentify fine-grained fashion details, making them less suitable for high-fidelity applications like product cataloging or personalized recommendations. To address this, we propose RA-CoA (Retrieval-Augmented Chain-of-Attributes), a novel, training-free framework that disentangles fashion image captioning into two interpretable stages: (i) retrieval of relevant attribute sets from a product knowledge base, and (ii) attribute-level reasoning to generate the final caption. RA-CoA is a model-agnostic approach that works with frozen VLMs to improve fine-grained attribute precision in product captions without the need for fine-tuning. Extensive evaluations across diverse VLM model families under different prompting paradigms demonstrate that RA-CoA significantly improves caption quality, achieving an average gain of 26.3% METEOR score over zero-shot captioning. We make our code publicly available.

Subtomogram classification in cryo-electron tomography (cryo-ET) is a challenging problem due to the scarcity of labeled examples. While cryo-ET simulators can be adopted to generate unlimited synthetic data, the substantial domain gap between synthetic and real subtomograms hinders its practical utilization. In this work, we propose a novel synthetic-to-real adaptation framework with a learnable transformation module, bridging this gap at both the input and feature levels. Extensive experiments demonstrate that our method consistently outperforms existing transfer learning baselines in few-shot settings.

Promptable models such as microSAM segment electron microscopy (EM) images from point prompts, but automation requires generating prompts without user input. We ask whether open-weight multimodal large language models (MLLMs) can generate them from natural-language requests by returning coordinates to a frozen segmenter. To that end, we convert masks from three mitochondria datasets into training examples, pairing images and instructions with centroid coordinates, then train LoRA adapters while freezing the MLLM backbone and microSAM. We find that Qwen3-VL reaches segmentation AP$_{50}$ $0.736$ after supervised fine-tuning and reward optimization, up from $0.247$ without adaptation, while automatic prompt generation (APG) achieves $0.773$. In addition, two other MLLMs improve, reaching or exceeding APG. When compared with a supervised centroid-heatmap detector that reaches AP$_{50}$ $0.904$ for this mitochondria task, Qwen3-VL more closely matches the annotated point set and instance counts. Moreover, training on two public datasets transfers to an unseen third, while training on all three transfers to an independent EM volume. Robustness tests show stable performance under unseen formulations of the natural-language request, while the coordinates can be reused by a second segmenter. To our knowledge, this is the first feasibility study of open-weight MLLMs as EM point generators, providing an inspectable, language-directed link between localization and mask decoding.

Counterfactual world models (CWM) extract motion from pretrained video predictors by comparing factual and intervened predictions, but uniform aggregation weights responses equally without explicitly incorporating physical priors. Our key insight is to incorporate physical priors into candidate reliability learning, motivating LPA-CWM with a lightweight Learned Physical Adjudicator (LPA). Trained on dense MOVi-F trajectories, the 3.0M-parameter LPA compares visual context and response structure across an unordered candidate set to predict relative weights; windowed localization and one paired re-evaluation recover motion with the CWM frozen. Existing video-level benchmarks do not directly assess motion correspondence, where low localization error can conceal missing trajectory segments. We introduce Completeness-aware Motion Correspondence (CMC), a ground-truth-anchored protocol jointly measuring localization, completeness, visibility, and continuity, counting missing predictions as failures on visible dynamic points. Across DAVIS, Kinetics, and RoboTAP, LPA-CWM improves all main CMC measures over Uniform CWM, with relative gains of 18.1%–60.0% in average Dynamic Correspondence Accuracy ($\mathrm{DCA}_{\mathrm{avg}}$), and improves TAP-Vid First tracking accuracy (overview: https://LPA-CWM.github.io).

Although existing multi-agent Retrieval-Augmented Generation (RAG) systems have demonstrated promise on complex multimodal reasoning tasks, they remain fundamentally limited in reasoning depth and memory structure, suffering from inadequate retrieval and state blindness when answering knowledge-intensive questions. To address these limitations, we propose GraMRAG, a graph memory-guided multi-agent RAG framework that integrates a dynamic multimodal memory graph to enable stable, multi-step multimodal reasoning. We introduce a vision-text bridged reasoning paradigm that unifies multi-scale entity cropping with a ReAct-style visual toolchain, enhancing the long-horizon cross-modal reasoning capability. We further construct a multimodal memory graph that formalizes agent reasoning as a dynamic directed acyclic graph (DAG), explicitly modeling action-observation dependencies to mitigate state blindness and suppress redundant retrieval. Moreover, we propose Topology-Aware Policy Optimization (TAPO) that leverages graph topology for critical path identification and targeted node pruning, enabling fine-grained credit assignment across multi-step reasoning trajectories. Extensive experiments on challenging multimodal benchmarks demonstrate that our approach consistently outperforms existing baselines and achieves state-of-the-art performance on complex long-horizon reasoning tasks.

Forward-looking sonar object detection is essential for underwater perception, yet deployment on embedded platforms requires highly compact models. To address this challenge, we explore quantum computing and introduce Quantum-Gated LiteSSD, a parameter-efficient hybrid quantum–classical detector that reformulates QuCNet-style multi-circuit quantum processing as an identity-centered channel-gating mechanism for spatial feature modulation. Experiments on the Marine Debris Watertank dataset and UATD forward-looking sonar benchmarks demonstrate an effective parameter–accuracy trade-off. The proposed detector achieves 90.84% $\mathrm{mAP}_{50}$ on Watertank with approximately $62\times$ fewer parameters than YOLO26s and $164.3\times$ fewer parameters than SSD-VGG16. On UATD, the model achieves 70.37% $\mathrm{mAP}_{50}$ with only 0.150M parameters, making it approximately $4.1\times$ smaller than SSGA-YOLO while retaining meaningful multi-class detection capability.

We present CirrGuide, a deep cascaded framework for cirrhotic liver segmentation and severity classification. Cirrhosis causes progressive structural changes in the liver and can lead to serious clinical complications, making severity assessment important for disease monitoring and treatment planning. However, severity classification is challenging because imaging patterns are often subtle, spatially variable, and similar across adjacent stages. CirrGuide addresses this by explicitly linking localization with classification. A ResNet50 encoder with an Attention U-Net decoder first predicts a soft cirrhotic liver mask, which is then used as an anatomical prior in a ResNet50-based classification branch. This branch combines global multi-scale features with mask-guided attention-pooled regional features to classify Mild, Moderate, and Severe cirrhosis. On the official CirrMRI600+ T2-weighted (T2W) 2D split, CirrGuide achieves 89.83% Dice and 84.14% mIoU for segmentation, 69.58% accuracy and 61.55% macro F1-score for severity classification. Compared with segmentation-only, classification-only, and multi-task baselines, CirrGuide improves both localization and severity classification, demonstrating the benefit of using predicted cirrhotic liver masks as anatomical priors for cirrhosis analysis.

Reconstructing 3D objects from brain signals offers a promising avenue for understanding human visual cognition. While prior work has shown initial success using EEG signals for 3D reconstruction, existing methods typically employ a uniform diffusion decoder, overlooking the evolving semantic granularity of both EEG representations and the diffusion denoising process. In this paper, we propose Mind2Cloud, a novel EEG-to-point-cloud generation framework based on two-granularity diffusion decoding. The core of Mind2Cloud is a time-aware decoder that integrates a global Transformer branch and a local Point-Voxel CNN (PVCNN) branch across diffusion timesteps through a learnable fusion mask. Specifically, Transformer layers are incorporated into the early upsampling stages to capture global object structure under high uncertainty, while PVCNN modules are used in later stages to refine local geometric details. Inspired by the hierarchical nature of EEG-based visual representations, this design dynamically adapts its spatial granularity in accordance with the coarse-to-fine trajectory of diffusion denoising. We further introduce an adversarial refinement module to enhance geometric realism and semantic consistency. Extensive experiments on the EEG-3D dataset across all 12 subjects demonstrate that Mind2Cloud outperforms prior work in both geometric accuracy and semantic alignment, setting a new benchmark for EEG-to-point-cloud generation. Our source code is available at https://github.com/duasoi/Mind2Cloud.

Vision-Language-Action (VLA) models combine a pretrained vision encoder, a language backbone, and an action head, but their relative contribution has not been established under controlled, latency-paired conditions. We fix the backbone families (SigLIP2 and Qwen2.5) and the training pipeline, sweep action-head design and module scale, and pair each configuration with measured on-device latency. The study yields three findings. First, action-head performance is governed primarily by initialization rather than decoder architecture, loss, or inference budget: copying the last transformer layers of the language backbone into the head is the single largest lever, at no latency cost, and the only axis that helps at every module scale. Alignment also explains the other axes: flow matching and a heavier decoder pay off only while the head is misaligned and reverse once it is aligned, and extra inference passes give no measurable benefit; expressiveness appears to substitute for missing alignment. We read this as representation transfer: the aligned head keeps attending to the instruction's object nouns and stays close to the backbone in weight space rather than relearning to act from scratch. Because we reach alignment only through initialization, we offer this as the account that best organizes the measurements, not a demonstrated cause, and name the control that would settle it. Second, capacity pays only after alignment: the aligned action head is the highest-return module to scale. Third, those returns diminish sharply near the size today's $\pi$-series VLAs already use, so further growth buys little in-domain accuracy for its latency. These specify EffVLA, a compact model matching the strongest open-source VLAs on standard LIBERO, leading on most LIBERO-Plus perturbation axes at lower latency, and transferring to a real SO-ARM101 arm with the recipe unchanged.

Ptychographic phase reconstruction is commonly formulated as an iterative inverse problem, requiring repeated object-probe updates and resulting in substantial computational cost for large-scale 4D-STEM data. We present a direct local-to-global learning framework that reconstructs full-field phase maps from diffraction measurements without iterative refinement during inference. The proposed network predicts local wrapped-phase patches from individual diffraction patterns using a sine-cosine representation, and the predictions are assembled into a full-field reconstruction using calibrated scan positions and Gaussian-weighted stitching. To evaluate generalization beyond the training domain, the model is trained on one material and directly applied to another in a zero-shot setting without target-domain fine-tuning. Experiments on AuPd and MoS$_2$ demonstrate consistent cross-material transfer in both directions, with the proposed method achieving the best full-field MSE, PSNR, and MS-SSIM among the evaluated learning-based methods. Compared with the iterative ePIE approach, the proposed direct local-to-global pipeline reduces end-to-end reconstruction time by approximately 10x, demonstrating its potential for efficient and transferable ptychographic reconstruction.

Video highlight detection aims to identify temporally important segments that capture the most informative or engaging events in a video. Reliable prediction therefore requires not only discriminative segment representations but also preservation of the temporal relationships among neighboring and distant segments. The information bottleneck principle has proven effective for learning compact and task-relevant representations, yet it has not been explored for video highlight detection, and applying conventional formulations directly would overlook inter-segment relational structure and distort highlight relevant temporal organization during compression. We therefore introduce the Sliced Gromov-Monge Gap (SGMG), a structure aware regularizer that measures the excess relational distortion induced by a prescribed source-to-bottleneck mapping relative to an optimal sliced structural correspondence. Building on SGMG, we develop SGWIB, an information-bottleneck framework for single-modal video highlight detection that learns compact bottleneck representations while preserving inter-segment temporal structure. We further introduce Home-Away-Related Contextual Pseudo-Labels and a contextual disentanglement module that reduce sports-specific contextual bias by separating highlight oriented information from contextual patterns. Experiments on MrHiSum and MoSu show that SGWIB attains the best Kendall's tau, Spearman's rho, mAP@50, and mAP@30 among the compared single-modal methods on both datasets. On MrHiSum, the visual model improves the strongest previous results by 0.031, 0.031, 0.87, and 0.75 on these four metrics, respectively. These results show that structure-aware information-bottleneck regularization combined with contextual disentanglement improves segment-level highlight prediction.

We present BLInD (Blind Learned Intent Distribution), a compact network that maps recent vehicle-state history (e.g. speed, curvature, indicator, and vehicle type) to a top-k distribution of future ego trajectories, with no camera, LiDAR, map, or object-track inputs. We find that vehiclestate history alone is sufficient to learn a useful multimodal distribution over near-term ego trajectories, and its low-latency nature makes it well-suited for safety-critical deployment. We investigate two distribution architectures, autoregressive (AR) and flow-matching, and train on both mixed-platform opensource and Wayve datasets. Both generalize without datasetspecific adaptation; the flow-matching model achieves best topk ADE/FDE of 0.15/0.37 m on Wayve, 0.15/0.36 m on Waymo, and 0.28/0.59 m on nuScenes, with the AR model reaching comparable coverage. Integrating the distributions into an AEB trigger task, a strict all-candidates policy reduces false positives from 1.51% to 0.11% with AR (13.7x reduction, 94.9% TP) and to 0.06% with flow-matching (25.1x reduction, 98.7% TP) compared to a 1-CTRV policy with 100% true positive score. BLInD runs in 0.87 ms with the AR head and 2.9 ms with the flow-matching head on an NVIDIA DRIVE Orin ECU making it compatible with real-time deployment on automotive ECUs. While existing learned distribution models rely on scene context and blind vehicle-state models typically collapse to a single path, BLInD is learned, blind, and cross-domain simultaneously, a combination not demonstrated by prior work. These results show that such a distribution provides a controllable and plausible intent sampling interface for downstream systems, with AEB as one instantiation.

We study how to update a pre-built aerial scene with a newly captured, unposed ground-view sequence. The aerial scene already contains a reliable metric Structure-from-Motion (SfM) reconstruction and a pre-trained 3D Gaussian Splatting (3DGS) model, whereas the ground-view sequence is collected later to add street-level appearance but has unknown camera poses and global scale. Registering this sequence to the aerial SfM reconstruction is challenging because single-image cross-view localization is brittle and long trajectories are prone to drift. To address these challenges, we present SkyAnchor, which treats the existing aerial scene as a fixed scaffold for ground-view registration and scene update instead of jointly reconstructing aerial and ground imagery from scratch. It first localizes short groups of consecutive ground frames against geometrically verified aerial support, producing sparse anchor poses. It then recovers the full ground trajectory with anchor-constrained submaps, fixing the front and rear anchor poses during incremental registration and bundle adjustment. Finally, it inserts filtered ground Gaussians while preserving the aerial view, followed by lightweight joint refinement. Experiments on seven real aerial–ground scenes show accurate metric ground trajectories and updated 3D Gaussian scenes with strong aerial- and ground-view rendering quality.

Earth-observation (EO) foundation models have become exceptionally effective at learning se mantic, high-dimensional geospatial embeddings, while modern weather and climate models have demonstrated that Earth-specific geometry, spherical operators, meshes, and hybrid physical solvers can materially improve prediction. Yet these two advances are not equivalent. A conventional latent embedding has no inherent physical transformation law, whereas scalar fields, tangent polar-vector fields, axial/pseudovector quantities, covectors, and higher-order tensors transform differently under rotations, reflections, and changes of local coordinate frame. This proposal asks whether a general purpose Earth foundation model should preserve those distinctions explicitly, or whether standard embeddings plus augmentation already learn everything that matters. The central contribution is therefore not a more complicated architecture by assumption, but a staged falsification program. A compute-conscious ERA5 dry run first compares conventional, augmentation-matched, typed equivariant, and Hodge/Helmholtz variants under spatial, temporal, orientation, and low-data shifts. Only if explicit geometric typing yields reproducible improvements does the program advance toward a multimodal Earth foundation model in which semantic embeddings coexist with physically typed fields. The proposed gap is narrower and more defensible than claiming that current models ignore geometry entirely: several systems already respect spherical domain geometry, and emerging work explicitly learns scalar/vector fields on spheres. The unresolved question is whether foundation-scale, multimodal, parity-aware field typing produces practical gains beyond those existing approaches.

Facial emotion recognition (FER) in real-world environments remains challenging due to unconstrained imaging conditions, including multiple faces, occlusions, pose variations, and complex lighting. Most existing studies focus on individual facial emotion classification and do not address the analysis of collective emotional states at the event level. This paper proposes an end-to-end pipeline for event-level emotion recognition from photographs. The approach detects faces in each image, classifies facial expressions using a deep convolutional neural network, and aggregates face-level emotion probabilities to estimate the overall emotional distribution of a public event. A comparative evaluation of several CNN architectures on the FER- 2013 and RAF-DB datasets demonstrates that transfer learning with EfficientNet-B2 trained on RAF-DB is more suitable for real-world RGB data. The proposed method is evaluated on a real-world event dataset containing 1658 images. Experimental results show stable emotion distributions across event subsets, confirming the effectiveness of event-level aggregation for emotion analysis in the wild.

Purpose: To develop and evaluate TotalSynth, a reusable pretrained framework for whole-body synthetic CT (sCT) generation from MRI and cone-beam CT (CBCT) images. Materials and Methods: In this retrospective technical study, the dataset was assembled between 2020 and 2026 from SynthRAD challenge data, four prostate cohorts, and BIC-MAC. After registration quality control, 1450 of 1800 public challenge pairs were retained; 350 were excluded for insufficient registration quality or major source/CT mismatch. The corpus also included 84 additional prostate MRI/CT and 60 external BIC-MAC MRI/CT cases. Three 5-fold model families were evaluated with image-domain, anatomy-aware, registration-based, and uncertainty metrics. Age and sex were not consistently available across public datasets. Results: The released MRI-to-CT model achieved an overall MAE of 67.49 HU, SSIM of 0.920, and PSNR of 29.28 dB. The CBCT-to-CT model achieved an overall MAE of 53.55 HU, SSIM of 0.939, and PSNR of 32.09 dB. The unified model maintained similar performance on MRI inputs (MAE, 67.68 HU) and CBCT inputs (MAE, 54.22 HU). On external BIC-MAC data, MRI-to-CT MAE was 100.91 HU without fine-tuning and 62.21 HU after fine-tuning. Conclusion: TotalSynth provides reusable MRI- and CBCT-based CT synthesis models with broad anatomical coverage, while external evaluation highlights the need for local validation and optional fine-tuning under domain shift.

We present DiVA, a deeply interactive digital life simulator pioneering a new paradigm for long-term, open-ended interactive experiences within digital character worlds. DiVA's architecture pairs a Multimodal Large Language Model (MLLM) as a router with a meticulously designed stacked video pipeline for seamless, multi-turn interactions with action and audio response. To maintain continuity and avoid degradation, we model generation as a three-part coupled system: waiting video, action video, and the transitions between them. These transitions are critically handled by our Anchored Video Continuation (AVC) module, which returns the character to stable states to prevent degradation. By encoding information from the preceding action video segment, AVC ensures smooth transitions, significantly reducing camera jitter and inconsistencies common in current video transition methods. This design also enables complex pose changes (e.g., sitting to standing) typically difficult for audio-driven models. These system designs together ensure high-fidelity identity, coherence, and dynamics for extended experiences. To validate our pipeline design, we comprehensively compare our system against alternatives by replacing our core generation module with mainstream long-video, continuation, and interpolation methods. We further analyze the necessity of the three-stage design, anchor-state selection, transition naturalness, spatial grounding, and the quality-latency trade-off, and we expand the comparison to additional long-form audio-driven avatar models. Results confirm DiVA is markedly superior in maintaining long-term visual quality and realism, validating its effectiveness as a sustainable, interactive simulation.

Point cloud data face serious semantic privacy risks during acquisition, transmission, and cross-institutional sharing. Existing methods mostly rely on geometric perturbation or destructive encryption, which can reduce the recognizability of the original class but often impair downstream usability. This paper proposes a class-transfer-based semantic encryption framework for point clouds, aiming to conceal original class information while preserving task utility and supporting authorized recovery. Specifically, we construct a unified latent space with a shared-backbone Normalizing Flow, and combine LoRA and FiLM to achieve parameter-efficient class-conditional adaptation. We further introduce diffusion-guided flow alignment to regularize the latent distribution, construct an energy-based category transition graph, and obtain an optimal class-transfer table through global matching. Then, a latent-space Neural ODE continuously evolves source-class latents into target-class latents, which are decoded into target-class point clouds through the inverse flow. We adopt attacker-oriented metrics, including New-Class Recognition Rate (NCRR), Original-Class Leakage Rate (OCLR), and Original Label Recovery Rate (OLRR), to evaluate privacy and utility. Experiments on classification and segmentation benchmarks show that the proposed method achieves controllable semantic transformation, effectively reduces original-class semantic leakage, preserves downstream learnability in the protected domain, and supports reliable authorized reconstruction.

Text-centric Visual Question Answering (VQA) requires reading and reasoning over text embedded in images, a task made substantially harder when images suffer from real-world degradation such as motion blur, low resolution, or compression artifacts. While modular OCR-based pipelines and end-to-end vision-language models are both widely used for this task, their comparative robustness under degraded conditions remains underexplored. We present an empirical study comparing two modular pipelines with SA-DBNet, a custom detector architecture combining ResNet-18 with self-attention spatial modeling and deformable convolutions against an end-to-end vision-language baseline, evaluated on 4013 degraded images with 7000 question-answer pairs. Fine-tuned modular pipelines achieve up to 57.50% exact-match accuracy versus 38.00% for the end-to-end baseline, with domain-specific fine-tuning yielding a gain of up to 29.50 percentage points. Critically, we find that conventional OCR error metrics like Character Error Rate and Word Error Rate are unreliable predictors of downstream VQA performance, as semantic reasoning can compensate for recognition failures when contextual cues are present. These findings highlight the importance of task-aware evaluation for text-centric VQA systems under realistic visual conditions. Codes are available here

Natural interaction in digital and physical environments requires continuous perception and timely responses. Spoken dialogue relies on acoustic and linguistic cues, while video interaction also requires grounding the conversation in evolving visual context. We present Realtime-Venus, a proactive full-duplex interaction system with two separately trained 9B models: Realtime-Venus-Omni for audio-visual interaction and Realtime-Venus-Audio for spoken interaction. Each model serves as a complete conversational frontend, integrating continuous perception, conversational control, and native speech generation through a shared causal timeline for user inputs, model outputs, and delegation events. A dual-loop runtime coordinates live interaction with background reasoning and tool execution. Foreground interaction continues while Realtime-Venus-Harness executes tasks asynchronously and returns results for integration into the ongoing dialogue. Both models follow a common post-training recipe combining offline understanding, proactive full-duplex trajectories, and delegation workflows. Among the evaluated online models, Realtime-Venus-Omni achieves the highest scores on six of eight video benchmarks, including StreamingBench (70.2%), OVO-Bench (64.7%), and Daily-Omni (81.3%). Across eight audio understanding and spoken question answering benchmarks, Realtime-Venus-Audio leads the compared models on MMAU (78.0%), MMAU-Pro (63.2%), Llama Questions (83.8%), and Speech CMMLU (67.8%), while matching the best VoiceBench AlpacaEval score of 4.81. On Full-Duplex-Bench v1.5, Realtime-Venus-Audio responds to 75% of user interruptions and achieves continuation rates of 97%, 88%, and 86% under backchannels, other-directed speech, and background speech, respectively, exceeding Gemini 3.1 Live and GPT-4o on all three continuation metrics.

Visual prefixes account for a major portion of the per-layer computation in multimodal large language models (MLLMs), making visual-token pruning a direct approach to accelerating inference. Existing top-K methods typically evaluate tokens independently and apply a uniform budget to all inputs, overlooking both selection-dependent interactions and variations in visual complexity across samples. In contrast, we propose StepPrune, which formulates visual-token pruning as an adaptive sequential decision process. Conditioned on previously selected tokens and textual context, StepPrune progressively constructs the retained subset and automatically determines its size through a learned STOP action. During training, a variance-preserving noise gate provides a differentiable surrogate for the discrete selection process, whereas during inference, unselected tokens are physically removed before language-model prefill. A grouped selection mechanism further extends StepPrune to high-resolution inputs. Experiments across LLaVA-1.5, LLaVA-NeXT, Qwen2.5-VL, and InternVL3 show that StepPrune achieves the best average normalized performance retention across all evaluated pruning rates on LLaVA-1.5, Qwen2.5-VL, and InternVL3, while remaining competitive on the substantially longer AnyRes prefixes of LLaVA-NeXT. On LLaVA-1.5, StepPrune retains 94.6% of the full-prefix normalized performance while pruning 88.9% of the visual tokens. At a mean retained count of 64, StepPrune reduces prefill latency from 59.95 ms to 40.05 ms, corresponding to a 1.50x prefill speed-up.

Recognition pipelines typically adopt a restore-then-recognize workflow, yet decades of experience show that generating visually pleasing images seldom translates to improved recognition. We propose a Joint Restoration-for-Recognition (JR$^2$) paradigm: restore only what downstream tasks truly require, with task signals dictating where, how much, and whether restoration is necessary. JR$^2$ rests on three pillars: (i) Physics, employing optics-accurate turbulence simulation, extensible to blur and noise, to ground restoration in real image formation; (ii) Neuroscience, drawing on selective attention and neuroplasticity to direct model capacity toward identity-critical regions and frames while bypassing already-clean inputs; and (iii) Vision & Learning, coupling recognition loss end-to-end through restoration and alignment so that low-level edits maximize high-level identity stability. Evaluations on IARPA-BRIAR show consistent improvements (e.g., TAR@0.01% FAR +0.6; FNIR@1% FPIR -2.5), while a quality gate skips ~70% of clean frames, reducing cost. Ablations confirm physics priors enhance realism, joint training prevents catastrophic forgetting, and selective restoration suffices in many cases. We conclude that better-looking images are neither necessary nor sufficient; restoration modules must be task-driven, selective, and physically aware. Code, pretrained models, and recipes are provided for integration.

Human trajectory prediction requires modeling both individual motion patterns and social interactions among agents. Existing methods have made substantial progress by using attention mechanisms, graph structures, and temporal encoders to capture dynamic social context. However, most of them primarily focus on how social information is encoded, while paying less explicit attention to how the encoded social context should take effect during future trajectory generation. In this paper, we argue that dynamic social encoding does not necessarily imply dynamic social activation. The same interaction context may require different activation strengths across future horizons and scene densities: social cues should be strengthened when interaction evidence is strong, but suppressed when they are weak or noisy. To address this issue, we propose GEAR, a generation-aware bias activation model for human trajectory prediction. Built upon a bias-decomposed trajectory generation formulation, GEAR dynamically activates the individual-motion and social-resonance bias terms at each future step before final trajectory composition. This allows the model to explicitly control when and how strongly individual and social bias components participate in generation. Experiments on ETH-UCY, SDD, and NBA show that GEAR consistently improves the resonance-based baseline and achieves competitive state-of-the-art performance. Further analyses of activation patterns and density-grouped errors validate the importance of calibrating encoded social context during trajectory generation. Our code is available at https://github.com/11isnotavailable/GEAR.git.

Learned components are increasingly integrated into geometric visual–inertial estimators to provide motion, depth, bias, uncertainty, or confidence cues. Yet it remains unclear whether gains arise from useful learned priors or from changes in the backend, calibration, initialization, temporal association, or evaluation gauge. We present a controlled framework for learning-augmented visual–inertial estimation that separates fusion gain from the incremental value of a learned prior and evaluates four evidence layers: local motion consistency, global trajectory accuracy, physical-state correctness, and numerical consistency. We instantiate the framework with a MonoViT-based monocular motion prior added as a local relative-motion factor to an unchanged VINS backend. We compare Original VINS and learned-prior VINS under matched sensor streams, timestamps, initialization, frontend/backend settings, and camera–IMU extrinsics, while probing calibration, initialization, state coupling, scale, bundle adjustment, and loop closure. On KITTI, with fixed reference extrinsics, translation APE RMSE is 31.4 m for Original VINS and 31.8 m with the learned prior. Across four recordings, the prior changes mean APE by only -0.2%, while a five-times-higher weight worsens it by 8.2%. Online extrinsic updates increase mean APE by 45.7% and 52.1%, respectively, while mean RPE changes by less than 2%. These results show that fusion performance alone cannot establish the value of learned priors. Reliable evaluation requires same-backend controls and joint analysis of prior compatibility, calibration, initialization, global drift, physical-state error, and backend consistency.

One in every five vehicle accidents on the road today is caused simply due to driver fatigue. Fatigue or otherwise drowsiness, significantly reduces the concentration and vigilance of the driver thereby increasing the risk of inherent human error leading to injuries and fatalities. Hence, our primary motive being - to reduce road accidents using a non-intrusive image processing based alert system. In this regard, we have built a system that detects driver drowsiness by real time tracking and monitoring the pattern of the driver's eyes. The stand alone system consists of 3 interconnected components - a processor, a camera and an alarm. After initial facial detection, the eyes are located, extracted and continuously monitored to check whether they are open or closed on the basis of a pixel-by-pixel method. When the eyes are seen to be closed for a certain amount of time, drowsiness is said to be detected and an alarm is issued accordingly to alert the driver and hence, prevent a casualty.

Multimodal Continual Instruction Tuning (MCIT) aims to exploit the incrementally accumulated knowledge to process multimodal inputs of diverse tasks, where parameter routing plays an important role. State-of-the-art methods rely on sample-to-task center similarity and cross-modal fusion with equal weight during routing. However, such solutions face two fundamental flaws: (1) Within each modality, the sample-to-task center distance is sub-optimal for routing since the abundant intra-task diversity information is underleveraged. (2) Different modalities exhibit varying reliability across tasks, where the modality with inter-task ambiguity can easily misguide the routing result. To address these problems, we propose Hyperbolic Uncertainty-aware Modality-Balanced Routing (Hyper-LLaVA) to improve parameter routing capacity based on cross-modality task feature uncertainty modeling. Specifically, to improve intra-modality task matching, Hyper-LLaVA accesses the sample-to-task distribution similarity in the Hyperbolic space. Besides, to alleviate the degradation brought by unreliable modalities, Hyper-LLaVA quantifies the task matching ambiguity within each modality to achieve adaptive balancing between task matching across modalities. Based on the complementary intra- and inter-modality task matching enhancement, our Hyper-LLaVA outperforms state-of-the-art approaches by large margins. Our source code is available at https://github.com/zhoujiahuan1991/ICML2026-Hyper-LLaVA

Stable and reliable 4D spatial understanding is fundamental for autonomous driving systems. While feedforward reconstruction networks can estimate camera motion and 3D structure in one pass, pose estimation over long videos remains challenged by computational cost, long-context ambiguity, and temporal instability. To address these challenges, we propose Feedforward Visual Odometry (FFVO), a pose-specialized adaptation of joint reconstruction architectures for efficient and temporally stable camera-pose estimation. FFVO uses (i) a compact camera-token representation for computationally efficient temporal aggregation, (ii) a hierarchical local-to-global temporal decoder that mitigates geometric ambiguity by separating short-range motion aggregation from sequence-level integration, and (iii) intermediate trajectory supervision that promotes temporal stability. Extensive evaluation on the Waymo Open Dataset (WOD), KITTI, and a large-scale proprietary benchmark demonstrates that our method performs favorably against existing feedforward approaches, and greatly reduces jitter and drift. These results support FFVO as an effective feedforward camera-pose decoder in long-horizon visual odometry settings.

Co-salient object detection (Co-SOD) requires a model to find foreground regions that are salient in individual images and supported by the image group. We present Rank-Consistent Set Reasoning (RCSR), a supervised dense-prediction framework that models a group as an unordered set rather than as a sequence of images or a semantic label. The core idea is to rank how strongly each spatial region agrees with a small collection of learned group slots at every image scale, and to aggregate these ranks with a robust trimmed statistic. This suppresses accidental pairwise matches and prevents one atypical group member from dominating the shared representation. A set encoder builds group slots directly from multi-scale visual features, while a rank-consistency gate measures whether the ordering of candidate regions is stable across group members. The gated slots are decoded jointly with per-image features to produce co-saliency maps. The model contains no natural-language branch, no open-vocabulary detector, and no external segmentation model. We further introduce a group permutation objective and hard-distractor augmentation so that the model learns the properties of a set-level target rather than memorizing image order or isolated visual saliency. We formulate an evaluation protocol for CoCA, CoSal2015, and CoSOD3k, together with tests of group-size robustness, distractor rejection, order invariance, and cross-dataset transfer.