Papers

As AI models grow in size and usage, their energy demands increase dramatically, raising sustainability and economic concerns. Neuromorphic hardware, inspired by the energy efficiency of the brain, seeks to address this challenge by offering low-power, fast-processing alternatives to conventional computing. Such hardware is particularly well-suited to control systems deployed in resource-constrained environments, which are best trained via reinforcement learning (RL). This contribution presents the design and implementation of a fully on-chip, closed-loop Loihi 2 RL agent. Our neuromorphic circuit consists of a fully embedded Q-learning algorithm and an on-chip simulation of the CartPole-v0 environment on Loihi 2. Our Q-learning algorithm trained the same number of successful agents as the CPU implementation in only half the execution time and with two orders of magnitude less dynamic power. These findings demonstrate the viability of RL on neuromorphic hardware and highlight its promise for building energy-efficient, real-time, embedded AI systems.

Communication-centric integrated sensing and communication~(ISAC) directly employs random data-bearing communication signals for both information transmission and environmental sensing. Existing studies generally describe pulse-shaped transmission in continuous time or directly adopt periodic discrete-time sensing models, leaving the digital signal processing~(DSP) operations connecting the physical transceiver and the periodic models unclear. This paper develops a DSP-oriented transceiver framework for single-antenna communication-centric ISAC. A common transmitter is constructed through modulation, cyclic-prefix~(CP) insertion, upsampling, and pulse shaping. For communication reception, the physical transceiver chain is reduced to a symbol-rate equivalent linear channel, which becomes circulant after CP removal and thereby supports frequency-domain equalization. For sensing reception, a high-rate reference waveform is extracted from the transmitted waveform and equivalently represented by circular pulse shaping. CP removal at the receiver then converts physical target delays into circular shifts of this reference waveform, leading to a periodic matched-filtering model and the corresponding range profile. Numerical results validate the developed framework through target range estimation and the symbol error rate performance of pulse-shaped CP-OFDM over frequency-selective Rayleigh fading channels.

Neural ODEs learn dynamics from trajectory losses, but their adjoint gradients lack the regressor-times-parameter-error structure on which Lyapunov analyses of online adaptation rest, so training on streaming data comes without stability guarantees. We show that this structure is in fact present: the adjoint gradient decomposes exactly into a positive semi-definite trajectory operator acting on the parameter error plus a nonlinear perturbation with explicit, horizon-dependent bounds. A quadratic Lyapunov function then certifies online Neural ODE training over sliding windows under computable gain and horizon conditions, and the same certificate extends to stored data: its drift branch recovers concurrent learning, and its trajectory branch yields NODE-CL, a stored-segment Gauss-Newton method built on batched forward sensitivities that needs no state-derivative estimates. On four DeepMind Control Suite domains, NODE-CL attains the lowest median prediction error on three under velocity measurement noise, where observer-based concurrent learning degrades by up to 8x; with clean measurements it is best on the pendulum and within a factor of 1.6 of the best stored-data baseline on the cartpole and reacher.

Audio preprocessing can affect how well a system detects stuttering. We study a simulated chain of denoising,loudness normalisation, Opus coding, and voice activity detection on SEP-28k. We use frozen WavLM Base+ features and report pointwise confidence intervals from episode-level bootstrap resampling. At a fixed threshold of 0.5, the chain reduces block F1 from 0.638 to 0.465, with smaller decreases for the other four classes. ROC-AUC decreases for all five classes. Blocks show the largest F1 and ROC-AUC losses, while sound repetitions show the largest average precision loss. Tuning the threshold on processed validation audio raises block F1 to 0.630. Retraining on processed audio with threshold tuning gives 0.628. Threshold adjustment therefore accounts for most of the observed block F1 recovery. It does not change ROC-AUC, which retraining raises only from 0.620 to 0.633, compared with 0.724 on clean audio.

Acquiring data is costly: higher measurement fidelity costs power and storage and risks collecting irrelevant content, while aggressive cost reduction can discard information that later analysis needs. We address this trade-off with an information-theoretic framework that acquires data relevant to a broad set of tasks rather than to one model. A mask policy, conditioned on side information, chooses which pixels to measure so as to maximize the mutual information between a discrete image and its partial observation under a budget; since the image entropy does not depend on the mask, this is equivalent to minimizing the conditional entropy. A frozen discrete denoising diffusion model (D3PM) supplies the posterior, and we use it in two ways: as an entropy surrogate for training a one-shot mask generator, and as the criterion for sequential greedy acquisition. The one-shot generator outperforms random masks only with care, including an unbiased gradient estimator for binary masks. With sequential acquisition, on MNIST the prior makes $8\times$ fewer errors than random at a $10\%$ budget, and on CIFAR-10 it gains $0.9$–$3.4$~dB. On fastMRI, our proposed technique using a static mask outperforms the well-known methods such as variable density and LOUPE.

Long-tailed encrypted traffic classification requires accurate recognition of infrequent classes under limited computational budgets. We propose Curv-Tail, a lightweight, end-to-end packet–byte framework trained without a separate pretraining stage. Mixed-resolution tokenization preserves exact packet-length identities within a bounded range and coarsens larger values to limit the vocabulary. An auxiliary objective predicts observed length tokens from contextual packet features before pooling, encouraging length-token retention beyond flow-level supervision. Compact temporal encoders process packet sequences and directional byte patches, and Lorentz prototypes with a shared learnable curvature magnitude classify their fused representation. On NUDT-Mobile and DataCon-Website under natural class frequencies, Curv-Tail achieves three-seed mean Tail-F1 scores of 85.70% and 46.30%, exceeding the strongest evaluated baselines by 2.91 and 1.89 percentage points, respectively. In 300-class profiling on an RTX 4090 with FP32 and batch size 256, Curv-Tail uses 98.48% fewer parameters and achieves 10.2 times the batch inference throughput of MM4Flow.

Feedback linearization is a powerful tool in nonlinear control, but finding the linearizing coordinate transform remains challenging. Feedback linearizing transforms are governed by well-established partial differential equations (PDEs) whose well-posedness is based on the Lie-algebraic Frobenius theorem, but solving the PDEs becomes computationally intractable for large system dimensions. To address this challenge, we propose a cascaded physics-informed neural network (PINN) framework to approximately solve these PDEs. By separately parameterizing terms of different Lie derivative orders, we mitigate the compounding errors inherent in fitting high-order Lie derivatives of neural networks. Furthermore, we use the learned transform to design a tracking controller and establish theoretical bounds on its error with respect to inaccuracies in the learned feedback-linearizing representation. We validate our method on a broad class of feedback linearizable systems, including a multi-input, multi-output planar quadrotor and synthetic examples for which analytical approaches are impractical, and demonstrate that our approach can computationally discover effective feedback-linearizing representations of nonlinear systems for control tasks.

This letter proposes a novel form of energy function (EF), called neural-analytic EF, for the stability analysis of power systems. The neural-analytic EF is designed as the sum of neural networks (NNs)-based components and an analytic EF, with the structure informed by the existing EFs derived analytically. It thus integrates NN's expressivity with the generalizability and scalability of analytic EFs. The satisfactory neural-analytic EF is further derived by iterative NN training with a tailored loss function. Numerical results finally demonstrate its superiority.

This letter proposes a novel method to establish decentralized stability conditions (DSCs) of power systems. This method formulates DSCs with meticulously crafted neural networks, based on the insight into the structure and properties of the existing analytically derived DSCs. A tailored loss function is further devised to iteratively train the neural networks, ensuring validity and reducing conservatism of the DSCs. Numerical results demonstrate that the resultant neural DSCs are less conservative than the conventional analytical DSCs, and become generalizable if a relatively large-scale system is used for training.

Foundation-model agents are often modeled as policies over an observed state. In deployed systems, however, a runtime may intervene only after the model has emitted a semantic proposal, making the proposal both an action candidate and a decision-time observation generated by a history-conditioned process. We show that collapsing this structure into a state-only proposal envelope can preserve proposal coverage while destroying certifiability. In a finite robust interface, the viability kernel of the collapsed model is contained in the physical projection of the history-augmented kernel, and the collapse is lossless exactly when every proposal-conditioned collapsed fiber retains a common robust-safe intervention. This gap can be maximal even with constant-size proposal and history alphabets. The same common-action condition yields a dual result: observing the current proposal can restore robust feasibility when it separates latent modes requiring incompatible interventions. We extend these one-step results over time using exact finite beliefs and standard safety and reachability fixed points, separating indefinite operational viability from finite worst-case verified progress. Controlled model-in-the-loop tests reproduce the predicted obstructions when telemetry or effect verification is removed or intervention authority is restricted. Thus, our contribution is not a new fixed-point calculus, but a characterization of when proposal–history correlation at the model–tool boundary is necessary for certification.

High-resolution plant imaging enables detailed characterization of plant morphology, but dense scientific analysis remains limited by costly pixel-level annotations, large image pixel dimensions, and substantial variation in imaging conditions. This work proposes a scalable in-domain self-supervised pretrained foundation model for high-resolution, high-pixel-dimension multi-species plant imagery. A masked autoencoder with a ViT backbone is pretrained on more than 10 million multi-view plant image tiles using distributed training. Following scalable pretraining, the learned foundation-model representations are comprehensively benchmarked across fine-grained dense prediction and coarse global feature recognition, with particular emphasis on limited supervision and realistic downstream imaging conditions. This work focuses on the domain gap of existing foundation models in dense feature representation and transfer. Through extensive experiments involving limited annotations, cross-view variation, and resolution degradation, the in-domain FM achieves a Mean Dice of 0.8686 and a Pooled Dice of 0.8959, outperforming an MAE counterpart pretrained on large-scale natural-image data by 0.0694 and 0.0613, respectively. The results further indicate that increasing pretraining scale produces consistent improvements in dense feature transfer. Overall, these findings suggest that scaling in-domain self-supervised pretraining can reduce the domain gap and improve transferable dense representations for high-pixel-dimension scientific imaging.

Audio-visual segmentation (AVS) aims to segment sounding objects at the pixel level by integrating auditory and visual cues. However, existing methods are predominantly developed under the monaural setting and primarily rely on cross-modal semantic correspondence, which limits their ability to distinguish visually similar instances of the same semantic class. In contrast, humans naturally exploit binaural hearing, where interaural differences and direction-dependent acoustic filtering introduced by the head and pinnae provide physically grounded spatial cues for accurate sound source localization. Motivated by this observation, we introduce binaural audio-visual instance segmentation (BiAVIS), a new task that leverages synchronized binaural audio and video frames to segment sounding instances. To advance research on this task, we establish two benchmarks by manually annotating an existing binaural audio-visual dataset and collecting a new real-world dataset, BiAVIS-Bench, in more challenging and diverse scenarios. We further propose a BiAVIS model, which leverages an audio-only sound source localization network to learn spatial and semantic priors for sounding instances from binaural audio. A query-level audio-visual fusion strategy is subsequently introduced to inject these informative priors into the instance segmentation decoder. Extensive experiments conducted on the two proposed benchmarks demonstrate the superior performance of the BiAVIS model over previous monaural AVS methods, especially in resolving instance-level intra-class ambiguity. On the more challenging BiAVIS-Bench, the proposed BiAVIS model outperforms the best-performing monaural baselines by 17.22% in mAP and 7.71% in FSLA, respectively.

With the deep integration of 5G and IoT in urban rail transit, service traffic grows explosively and accurate classification of heterogeneous service flows is essential for safe and efficient railway operations. Urban rail communication systems are subject not only to operational disturbances such as equipment failures and maintenance interference, but also to complex transmission patterns characterized by the interleaving of multi-service traffic flows. Under these operating conditions, the widespread deployment of proprietary protocols and the high prevalence of encrypted traffic further diminish the applicability of conventional port-based and Deep Packet Inspection (DPI) classification methods. To address these challenges, we propose a priority-aware dual-channel feature fusion framework. Raw traffic bytes and statistical features are mapped into grayscale images and processed by a dual-branch architecture: a transfer learning-enhanced ResNet extracts fine-grained byte-level textures, while a lightweight CNN captures macroscopic statistical patterns. A channel attention mechanism dynamically recalibrates cross-modal features, and a novel Priority-Sensitive Loss (PSL) that integrates business-criticality awareness with class-balance weighting to maximize recall for safety-critical services. Evaluated on a real-world urban rail dataset using priority-weighted metrics, the method achieves 98.74% accuracy and 99.24% weighted recall, with recall of 99.94% and 99.41% on the two safety-critical services:Communication-Based Train Control(CBTC) and Emergency Radio Dispatch(ERD), providing a reliable classification foundation for priority-aware resource scheduling in urban rail communications. With only 0.18M parameters and 0.4–0.7 ms end-to-end latency, offering an excellent balance between high-precision classification, low inference latency, and edge-deployment feasibility.

Molten pool flow significantly influences bead geometry, defect formation, and solidification microstructure in wire-arc directed energy deposition (DED). However, direct flow sensing remains an open challenge due to extreme temperatures, intense arc illumination, and complex flow patterns. Traditional research has relied either on computationally intensive fluid dynamics simulations, which require assumed boundary conditions and lack empirical anchoring, or on vision-based tracer particle tracking, which yields sparse, noisy, and physically unconstrained velocity estimates. This work bridges these two paradigms by reconstructing physics-constrained 2D surface flow fields from high-speed imaging of a GTAW-based wire-arc DED process under distinct flow regimes: quasi-steady convection and periodic droplet impact. The pipeline begins with homography correction to calibrate the oblique camera view, followed by dense optical flow extraction to provide raw velocity observations. For quasi-steady pools, a physics-informed neural network (PINN) reconstructs the surface velocity field by jointly enforcing data fidelity, incompressible Navier-Stokes momentum equations, no-penetration boundary conditions, and an arc-forcing prior through composite loss minimization. For pools subject to periodic droplet impact, a phase-conditioned time-dependent PINN is further developed to resolve the cyclic flow evolution throughout the impact period. This work provides a unified framework that bridges high-speed visual sensing and fluid mechanics, enabling physics-grounded flow diagnostics for wire-arc DED processes.

This work presents a solution to nonconvex, game-theoretic motion planning problems subject to disturbances over long time horizons. The problem is posed as a partially-decoupled generalized Nash equilibrium problem, in which each agent's dynamics depend only on its own state and control, admitting fast solution methods for competitive multi-agent motion planning. An algorithm, WOLF, is developed that applies receding-horizon model predictive control to an open-loop differential games solver based on sequential convexification. In contrast to robust formulations that fix the uncertainty description offline, the robustness tube here is itself a dynamic state, co-optimized with the trajectory, and its thickness directly sets the tightening of the shared coupling constraints. A sufficient condition is derived under which a nominal trajectory satisfying constraints tightened against all agents' error bounds remains feasible for every admissible disturbance realization. The method is demonstrated on two adversarial on-orbit games with coupled translational-attitude dynamics: a stealthy co-orbital jamming game under an active detection-probability bound, and a sun-blocking game in which an adversary disables an evader by decreasing solar power

Endoscopy is widely used in medicine to visualize anatomical structures for diagnosis and treatment evaluation. However, it does not currently allow rapid, quantitative measurement of airway cross-sectional area (CSA), a key metric for diagnosing and managing respiratory pathologies. There is growing interest in AI-enabled quantitative endoscopy using depth estimation, but a fundamental challenge is the need for ground-truth depth data to train algorithms capable of quantifying depth in physical units (e.g., meters). In this study, 60,000 paired virtual endoscopy images and depth maps were created in Blender from 3D airway models built from computed tomography. A U-Net convolutional neural network was trained to predict the depth maps from the endoscopy images. Point clouds were reconstructed from the AI-generated depth maps using the Equisolid fisheye lens model, and airway perimeters were identified by intersecting a plane with the point cloud to estimate CSA. Nasopharyngeal CSA, a key metric in obstructive sleep apnea, was quantified using this AI-enabled approach with a median error of 5.3 % relative to the 3D model ground-truth. This pilot study using synthetic data provides strong evidence for the feasibility of developing AI-enabled quantitative endoscopy for airway CSA measurement.

Automatic Speech Recognition (ASR) systems often underperform for children and non-native speakers, while adapting adult ASR models to child speech can cause adult-speech forgetting. We study child ASR adaptation with adult retention across Arabic and English. We compare full fine-tuning, LoRA, and post-hoc weight-space merging across encoder–decoder, encoder–CTC, and AudioLLM-based ASR systems. Experiments use Arabic native and non-native child speech, English MyST child speech, and adult benchmarks from MGB-2 and LibriSpeech test-clean. We evaluate recognition quality with WER and quantify the adaptation–retention trade-off using Retention Index, Child Adaptation Gain, and Adaptation Recovery. Results show that child adaptation is necessary, especially for non-native Arabic and English child speech, but direct adaptation often reduces adult ASR performance. Bilingual adaptation is more stable than language-specific adaptation. Weight-space merging often improves the trade-off, especially for encoder–CTC, Whisper, and AudioLLM-based ASR, with LERP favoring adult retention and TIES recovering stronger child gains. For the encoder–decoder model, direct bilingual fine-tuning remains strongest in raw WER.\footnote{Code, and models are available at https://github.com/qcri/Child-ASR-Adaptation.

Runtime assurance pairs a verified fallback with an untrusted controller and a switching monitor, and is the leading route to admitting learned policies into safety-relevant network control. Its guarantee rests on a condition it assumes rather than requires: that the monitor's estimation error is zero, or at worst stochastic with a characterisable rate. In mobile networks many measurements originate at untrusted endpoints, where the error is neither. We state this assurance precondition and prove it: at zero measurement noise the guarantee survives an adversary controlling $q$ channels if and only if the plant is $2q$-sparse observable with respect to the safety-relevant output. This is functional observability over attack supports, strictly weaker than full-state observability: on our topology a per-cell outage condition doubles the budget and one stated over total offered load triples it. The converse is a constructive attack making a safe and an unsafe trajectory observationally identical; it defeats every monitor on those channels, not one detector, at any noise level. As the trust split is static and known before deployment, confining the adversary to one family yields a confinement threshold, above which trusted channels alone resolve the state and all untrusted channels may be corrupt at once. Three uninfluenceable counters cross it here, lifting the budget from two to six and making placement the dominant lever. Below the precondition the monitor faces a frontier, not a dichotomy; a set-valued monitor is optimal and carries sufficiency into non-zero noise at the residual test's false-rejection rate. An adversary reaching the trigger radius owns the switch regardless. The same argument bounds any monitor reading agent messages rather than measurements. A rank test decides the budget and identification does not preserve it, so a published budget must name its margin floor.

Modern bioacoustic foundation models like Perch and BirdNET can identify species with high discriminative accuracy, yet their confidence scores are often uncalibrated and difficult to interpret as probabilities of real-world occurrence. This limits their use for ecological inference beyond threshold-based detection. We leverage a global annotated acoustic dataset (WABAD) to produce calibration priors for an acoustic model, optionally incorporating species-level information. We introduce new methods of fusing the acoustic predictions with geopriors, which empirically improves calibration while preserving discrimination. Together, these results suggest a path to simpler and more reliable acoustic monitoring for broad biodiversity.

On-board compression of synthetic aperture radar (SAR) phase history is bandwidth-critical, and block-adaptive quantization (BAQ) remains the operational standard. We test whether a small convolutional autoencoder, with its encoder on the sensor, can compete with BAQ on complex phase-history patches from the AFRL GOTCHA collection. Every method is charged for all transmitted bits, rates are reported in bits per complex sample (b/cs), and detection is scored by one-to-one matching of CA-CFAR detections. The autoencoder (28,656 encoder parameters) loses at every rate. At 16 b/cs it reaches -2.87 dB NMSE, against -35.5 dB for 8-bit BAQ with $\pm 3\sigma$ clipping and -41.0 dB with a tuned clipping range. It also loses to a $16 \times 16$ block Karhunen-Loève transform (KLT), a local linear coder with a tenth of its encoder cost (-5.39 dB). Running the network in a companded Fourier domain helps, but its detection F1 remains bounded at 33%. The evidence points to this model, its normalization, and its objective, not to a fundamental limit of learned coding. Per patch, the data have modest lag-1 coherence ($|\rho| \approx 0.3$) and patch-specific spectral concentration. Two findings concern evaluation itself. First, 97% of CFAR crossings on raw $64 \times 64$ patches are border artifacts of the zero-padded detector. Second, on interior cells BAQ's clipping range decides detection: 8-bit BAQ keeps 69% F1 with tuned clipping but 17% at $\pm 3\sigma$, and at 8 b/cs or less adaptive FFT thresholding preserves more detections than BAQ. We close with an evaluation protocol for learned radar compression.

Recent advances in speech synthesis and voice cloning have increased the need for reliable synthetic-speech detection, yet high-accuracy detectors increasingly rely on large pretrained models that are costly to invoke on every recording. Rather than replacing such detectors, we investigate a compact front-end screen that processes all inputs cheaply and forwards only suspicious recordings for more expensive analysis. To enable lightweight screening without a large learned encoder, we exploit spectral traces introduced by speech-generation operations. We analyze how learned upsampling and inverse short-time Fourier transform synthesis can produce predictable spectral artifacts and measure their presence directly in generated waveforms. Because the strength of these artifacts varies across generators, we combine decoder-guided spectral measurements with complementary descriptors of short-time spectral shape and temporal variation in a compact gradient-boosted tree. Across seven speech generators and two human-speech sources, the proposed screen achieves an equal error rate of 0.021% with an estimated model storage of 151 KiB. When used as the first stage of a simulated cascade with a 1.15-billion-parameter detector, it reduces estimated detection energy by 84.4% while operating at a 0.050% synthetic-speech miss rate, demonstrating the potential of decoder-guided acoustic evidence for low-cost front-end screening.

A ranked-prefix feasible-set restriction is proposed for real-time mixed-integer model predictive control of series-connected reconfigurable battery packs with per-cell bypass switching. Cells are ordered at each sampling instant by a one-step objective-informed suitability score, and the engagement variables are constrained to prefixes of that ordering. The restriction reduces the number of admissible per-step engagement patterns from combinatorial to linear in the number of cells while leaving the model, objective, and physical constraints unchanged. For a 20-cell pack driven over a worldwide harmonized light vehicles test cycle, the proposed controller certifies every active step, solves every instance without branching, and reduces the 95th-percentile solver time by a factor of 6.8 relative to the full subset-selection controller. The cycle-mean cell-to-cell state-of-charge standard deviation decreases by a factor of 3.4, and no power curtailment occurs. Same-state reference solves show a mean restriction gap below 0.11% on jointly certified steps, and the computational gains persist across the tested initial conditions, cell heterogeneity levels, pack sizes, and prediction horizons without retuning.

When mixed-integer model predictive control must select which of several similar units to activate, many same-size selections can yield closely spaced predicted outcomes, and branch-and-bound may fail to certify optimality within one sampling period. We propose a ranking-based feasible-set restriction for this class of decisions: the units are ordered by a suitability score computed at each sampling instant, and a monotone constraint permits only the top-ranked units to be active. The optimizer then chooses only how many units to activate, so the admissible selections per prediction stage grow linearly instead of combinatorially, while the model, cost function, and remaining constraints are retained. We instantiate the method for reconfigurable battery packs in which each series-connected cell can be bypassed, scoring cells with fixed weights on normalized state of health, state of charge, and voltage headroom. In a 20-cell simulation over a dynamic drive cycle with a 1 s limit per online solve, all active online restricted solves terminate at the root node with certified optimality, compared with 77.83% certification for unrestricted subset selection, and the 95th-percentile solver time is lower by a factor of 13.4. The cycle-mean cell-to-cell standard deviation in state of charge falls 8.7-fold without power curtailment, at the expense of 3.45 times as much cell switching. Solving both problems from identical states along two trajectories shows that the restriction raises the optimal cost by 0.26% and 0.14% on average at states where both solves are certified and uncurtailed.

Expressive speech synthesis has outpaced expressive speech perception: systems now render fine-grained vocal performances that no public benchmark can score. Most benchmarks for this inverse problem stop at six to nine basic emotion categories, largely on acted speech. This paper introduces VoiceNet, a human-annotated representation-level benchmark for voice performance understanding on permissively-licensed in-the-wild speech. VoiceNet has two subsets: VoiceNet-Emo applies a 40-emotion taxonomy with three expert ratings per item, and VoiceNet-Ext, a preliminary subset, scores 57 talking-style attributes including speaking rate, vocal tension, breathiness, and register. The paper also releases Emolia, an emotion-annotated version of the Emilia corpus, with a curated rebalanced subset enriched by dense MOSS-Audio Thinking annotations. Two voice-text contrastive baselines train on this data: a 110M-parameter VoiceCLAP-Small for fast large-scale data filtering and a 7B VoiceCLAP-Large for state-of-the-art performance. Both outperform existing CLAP baselines, which sit near chance on VoiceNet-Emo. On VoiceNet-Emo, VoiceCLAP-Large aligns more closely with the aggregate expert consensus than individual experts agree with one another: a comparison against the majority label rather than evidence of surpassing human emotion perception. All systems evaluated here are voice-text embedding models: VoiceNet scores representation-level attribute recognition and retrieval, not end-to-end spoken-dialogue behaviour. Clustering and filtering uncurated speech corpora into subsets that span diverse talking styles and emotions remains an open challenge; VoiceCLAP embeddings offer a promising tool for this task. VoiceNet, Emolia, and VoiceCLAP are publicly available for research use.

Vision-language pretraining objectives differ in the spatial granularity of their supervision, yet the implications of this distribution for chest X-ray interpretation remain underexplored. We present a controlled study that isolates the pretraining objective: holding the encoder and pretraining data fixed, we train nine objectives spanning global and local contrastive learning, captioning, and their combinations, and evaluate across five chest X-ray tasks of increasing spatial granularity. We find that (i) pretraining granularity aligns with task granularity at the extremes, with local objectives leading on abnormality detection and global objectives on classification; (ii) local objectives are surprisingly competitive on global-level generation and question answering tasks; (iii) the merits of captioning and contrastive learning reverse across granularity levels; and (iv) among combinations, mixing captioning and contrastive supervision is strongest on classification and in distribution generation, while pairing two captioning objectives generalizes best on zero-shot report generation. These results show that no single objective is universally optimal, and that the interaction of objective type, granularity, and task governs downstream performance.

When an oilfield digital twin disagrees with its instruments, the operator must decide whether the cause is hardware degradation, harsh-weather effects, or malicious data manipulation. Existing digital-twin work appears to treat these causes separately: sensor-validation architectures target faults, and attack-focused twins have been evaluated on water-sector testbeds. We study joint cause attribution on a simulated four-well wellpad with 15 coupled instruments, legitimate operating transients, and weather. A physics-informed twin, identified from normal data only, produces analytical-redundancy residuals; windowed features feed a gradient-boosted classifier whose posterior drives an alarm gate with a fixed false-alarm rate and a cost-based decision rule that may defer to an analyst. On three independently generated sites, the classifier reaches macro-F1 of 0.818 on windows where the injected deviation is observable (0.723 when latent post-onset windows are included at the primary site). The physics twin accounts for essentially all of this: removing the data-driven twin changes macro-F1 by less than 0.01, whereas removing all twins drops it to about 0.58. Attacks are detected quickly (median 1.7 h) but attributed correctly at alarm time only 42% of the time, rising to 73% six hours later. A cost-aware policy that defers ambiguous cases had the lowest expected cost among all policies in every one of 144 cost and prior settings tested, sometimes by a small margin; the result depends on illustrative costs and on analysts resolving deferred cases. A twin-aware attacker was detected in 45% of episodes yet almost never attributed to attack. These results are conditional on the simulator's generative assumptions and have not been validated on field data.

We propose a framework for the alignment and interpolation of pitch-aligned time-frequency representations. Building on the tonal interval vector, we introduce a series of extensions that reformulate it as an invertible operator, culminating in a new feature extractor that embeds perceptual consonance priors within a pitch-aligned representation. Accordingly, we develop methods for aligning and interpolating between musical structures. For alignment, we cast the problem as a permutation search under perceptually weighted distances, enabling robust matching without explicit key detection, which is inherently ambiguous in music information retrieval. For interpolation, we employ optimal transport to generate musically meaningful transitions between pitch distributions, proposing a circular formulation over the Circle of Fourths/Fifths to respect harmonic structure. Finally, we learn a low-dimensional geometric representation of scale structure aimed to factorize scale color and density. Experimental results demonstrate the effectiveness and musicality of the proposed methods. Together, our contributions provide a principled, purely signal processing approach to modeling pitch structure in audio and symbolic music signals.

Audiovisual Speech Recognition (AVSR) is a multimodal approach to speech recognition that incorporates visual information from lip movements to enhance model performance. Despite its advantages, its development remains constrained by the limited availability of labeled audiovisual (AV) datasets. This work explores the use of synthetic visual data as a solution, using an audio-driven talking-head pipeline to generate lip-synchronized visual content from existing audio data. We evaluate the effectiveness of synthetic visual data both as an augmentation strategy and as a standalone training resource, applying our approach to Spanish and Catalan. Our results show that augmenting real AV data with synthetic samples yields relative Word Error Rate (WER) reductions of up to 16.2%, demonstrating the potential of this approach. Moreover, we demonstrate that synthetic data alone can serve as a baseline for AVSR training in languages lacking AV datasets. These findings provide evidence that synthetic visual data can serve as a scalable solution to AVSR data scarcity, enabling broader language coverage.

Real-time full-duplex speech models can listen while speaking, enabling natural interaction without rigid turn boundaries. Existing benchmarks evaluate turn-taking, interruption handling and multi-round dialogue, but largely centre on a designated user rather than an assistant participating in a shared conversation among several people. We introduce Duplex-MPE to evaluate when such an assistant should answer, remain silent or stop speaking. The benchmark contains 2,000 scenarios with three or four human speakers and one assistant, each paired across explicit and implicit addressing of the same request. Models receive continuous conversation audio without transcripts or supplied turn boundaries. Four scores measure fresh response initiation, answer accuracy, silence preservation and stopping when a human resolves a request. We evaluate five open-weight speech systems: MiniCPM-o 4.5, Moshi, FLM-Audio, Voila and Freeze-Omni. MiniCPM-o 4.5 leads on three scored capabilities, while frequent speech from other systems can coexist with inaccurate answers or failures to remain silent. A transcript-based Gemini 3.1 Pro reference responds 64.3 percentage points more often to explicit than implicit requests; paired tests detect no significant response-rate difference for the speech systems.

Native-latent guidance is a recent paradigm for solving inverse problems with latent diffusion models. It replaces repeated evaluations of the image-space forward model, each requiring a decoder pass, with efficient guidance computed using a learned latent-space surrogate. However, existing methods apply guidance uniformly across latent dimensions, ignoring that measurements are informative only along certain directions and that the reliability of model predictions varies across inputs and timesteps. We propose ANaLOG, a framework for efficient uncertainty-aware guidance with pretrained latent diffusion models. ANaLOG models uncertainty by learning an anisotropic, input- and time-dependent covariance that is integrated into the guidance mechanism to emphasize reliable directions and downweight uncertain ones. We theoretically analyze this framework in a linear model setting and prove that anisotropic, uncertainty-aware weighting is necessary for correct sampling, whereas isotropic guidance induces sampling errors. Experiments across five challenging inverse problems show that ANaLOG improves perceptual reconstruction quality over existing methods while preserving efficiency.