Signal Processing
Intrinsic MIMO Particle Communication Channel with Random Advection
Fatih Merdan and Ozgur B. Akan
In this work, receiver diversity in advection-dominated
diffusion-advection channels is investigated. Strong directed flow fundamentally alters the communication-theoretic properties of molecular communication systems (MC). Specifically, advection preserves the temporal ordering and shape of transmitted pulses, enabling pulse-based and higher-order modulation schemes that are typically infeasible in purely diffusive environments. Focusing on a single transmitter and a single type of information molecule, it is demonstrated that spatially distributed receivers can observe distinct realizations of the same transmitted signal, giving rise to diversity gain. Several receiver combining strategies are evaluated and shown to improve detection performance compared to single-receiver operation, particularly in low-to-moderate signal-to-noise ratio (SNR) regimes. The results provide a structured framework for understanding receiver-side diversity in molecular communication, highlighting the role of advection as a key enabler for reliable pulse-based signaling. This perspective establishes a foundation for future studies on advanced modulation, joint equalization and detection, and multi-molecule
MIMO extensions that can further enhance the performance and physical applicability of MC systems.
Read the full abstract ▾
Systems and Control
Reinforcement Learning-Based Co-Design and Operation of Chiller and Thermal Energy Storage for Cost-Optimal HVAC Systems
Tanay Raghunandan Srinivasa, Vivek Deulkar, Aviruch Bhatia et al. (+1 more)
We study the joint operation and sizing of cooling infrastructure for commercial HVAC systems using reinforcement learning, with the objective of minimizing life-cycle cost over a 30-year horizon. The cooling system consists of a fixed-capacity electric chiller and a thermal energy storage (TES) unit, jointly operated to meet
stochastic hourly cooling demands under time-varying electricity prices. The life-cycle cost accounts for both capital expenditure and discounted operating cost, including electricity consumption and maintenance. A key challenge arises from the strong asymmetry in capital costs: increasing chiller capacity by one unit is far more expensive than an equivalent increase in TES capacity. As a result, identifying the right combination of chiller and TES sizes, while ensuring zero loss-of-cooling-load under optimal operation, is a non-trivial co-design problem. To address this, we formulate the chiller operation problem for a fixed infrastructure configuration as a finite-horizon Markov Decision Process (MDP), in which the control action is the chiller part-load ratio (PLR). The MDP is solved using a Deep Q Network (DQN) with a constrained action space. The learned DQN RL policy minimizes electricity cost over historical traces of cooling demand and electricity prices. For each candidate chiller-TES sizing configuration, the trained policy is evaluated. We then restrict
attention to configurations that fully satisfy the cooling demand and perform a life-cycle cost minimization over this feasible set to identify the cost-optimal infrastructure design. Using this approach, we determine the optimal chiller and thermal energy storage capacities to be 700 and 1500, respectively.
Read the full abstract ▾
All authors Tanay Raghunandan Srinivasa, Vivek Deulkar, Aviruch Bhatia, Vishal Garg
Image and Video Processing
Development of Domain-Invariant Visual Enhancement and Restoration (DIVER) Approach for Underwater Images
Rajini Makam, Sharanya Patil, Dhatri Shankari T M et al. (+2 more)
Underwater images suffer severe degradation due to wavelength-dependent attenuation, scattering, and illumination non-uniformity that vary across water types and depths. We propose an unsupervised Domain-Invariant Visual Enhancement and Restoration (DIVER) framework that integrates empirical correction with physics-guided modeling for robust
underwater image enhancement. DIVER first applies either IlluminateNet for adaptive luminance enhancement or a Spectral Equalization Filter for spectral normalization. An Adaptive Optical Correction Module then refines hue and contrast using channel-adaptive filtering, while Hydro-OpticNet employs physics-constrained learning to compensate for backscatter and wavelength-dependent attenuation. The parameters of IlluminateNet and Hydro-OpticNet are optimized via unsupervised learning using a composite loss function. DIVER is evaluated on eight diverse
datasets covering shallow, deep, and highly turbid environments, including both naturally low-light and artificially illuminated scenes, using reference and non-reference metrics. While state-of-the-art methods such as WaterNet, UDNet, and Phaseformer perform reasonably in shallow water, their performance degrades in deep, unevenly illuminated, or artificially lit conditions. In contrast, DIVER consistently achieves best or near-best performance across all
datasets, demonstrating strong domain-invariant capability. DIVER yields at least a 9% improvement over SOTA methods in UCIQE. On the low-light SeaThru
dataset, where color-palette references enable direct evaluation of color restoration, DIVER achieves at least a 4.9% reduction in GPMAE compared to existing methods. Beyond visual quality, DIVER also improves robotic perception by enhancing ORB-based keypoint repeatability and matching performance, confirming its robustness across diverse
underwater environments.
Read the full abstract ▾
All authors Rajini Makam, Sharanya Patil, Dhatri Shankari T M, Suresh Sundaram, Narasimhan Sundararajan
Audio and Speech Processing
EmoShift: Lightweight Activation Steering for Enhanced Emotion-Aware Speech Synthesis
Li Zhou, Hao Jiang, Junjie Li et al. (+2 more)
Achieving precise and controllable emotional expression is crucial for producing natural and context-appropriate
speech in text-to-
speech (TTS) synthesis. However, many
emotion-aware TTS systems, including large
language model (LLM)-based designs, rely on scaling fixed
emotion embeddings or external guidance, limiting their ability to model
emotion-specific latent characteristics. To address this gap, we present EmoShift, a lightweight activation-steering framework incorporating a EmoSteer layer, which learns a steering vector for each target
emotion in the output
embedding space to capture its latent offset and maintain stable, appropriate expression across utterances and categories. With only 10M trainable parameters,less than 1/30 of full fine-tuning, EmoShift outperforms zero-shot and fully fine-tuned baselines in objective and subjective evaluations, enhancing emotional expressiveness while preserving naturalness and speaker similarity. Further analysis confirms the proposed EmoSteer layer's effectiveness and reveals its potential for controllable emotional intensity in
speech synthesis.
Read the full abstract ▾
All authors Li Zhou, Hao Jiang, Junjie Li, Tianrui Wang, Haizhou Li
Systems and Control
Degradation-Aware Frequency Regulation of a Heterogeneous Battery Fleet via Reinforcement Learning
Tanay Raghunandan Srinivasa, Vivek Deulkar, Jia Bhargava et al. (+2 more)
Battery energy storage systems are increasingly deployed as fast-responding resources for grid balancing services such as frequency regulation and for mitigating renewable generation uncertainty. However, repeated charging and discharging induces cycling degradation and reduces battery lifetime. This paper studies the real-time scheduling of a heterogeneous battery fleet that collectively tracks a
stochastic balancing signal subject to per-battery ramp-rate and capacity constraints, while minimizing long-term cycling degradation. Cycling degradation is fundamentally path-dependent: it is determined by charge-discharge cycles formed by the state-of-charge (SoC) trajectory and is commonly quantified via rainflow cycle counting. This non-Markovian structure makes it difficult to express degradation as an additive per-time-step cost, complicating classical dynamic programming approaches. We address this challenge by formulating the fleet scheduling problem as a Markov decision process (MDP) with constrained action space and designing a dense proxy reward that provides informative feedback at each time step while remaining aligned with long-term cycle-depth reduction. To scale learning to large state-action spaces induced by fine-grained SoC discretization and asymmetric per-battery constraints, we develop a function-approximation reinforcement learning method using an Extreme Learning Machine (ELM) as a random nonlinear feature map combined with linear temporal-difference learning. We evaluate the proposed approach on a toy Markovian signal model and on a Markovian model trained from real-world regulation signal traces obtained from the University of Delaware, and demonstrate consistent reductions in cycle-depth occurrence and degradation metrics compared to baseline scheduling policies.
Read the full abstract ▾
All authors Tanay Raghunandan Srinivasa, Vivek Deulkar, Jia Bhargava, Mohammad Hajiesmaili, Prashant Shenoy
Audio and Speech Processing
CALM: Joint Contextual Acoustic-Linguistic Modeling for Personalization of Multi-Speaker ASR
Muhammad Shakeel, Yosuke Fukumoto, Chikara Maeda et al. (+2 more)
We present CALM, a joint Contextual Acoustic-Linguistic Modeling framework for multi-speaker automatic
speech recognition (ASR). In personalized AI scenarios, the joint availability of acoustic and linguistic cues naturally motivates the integration of target-speaker conditioning with contextual biasing in overlapping conversations. CALM implements this integration in an end-to-end framework through speaker
embedding-driven target-speaker extraction and dynamic vocabulary-based contextual biasing. We evaluate CALM on simulated English (LibriSpeechMix) and Japanese (Corpus of Spontaneous Japanese mixtures, CSJMix). On two-speaker mixtures, CALM reduces biased word error rate (B-WER) from 12.7 to 4.7 on LibriSpeech2Mix and biased character error rate (B-CER) from 16.6 to 8.4 on CSJMix2 (eval3), demonstrating the effectiveness of joint acoustic-linguistic modeling across
languages. We additionally report results on the AMI corpus (IHM-mix condition) to validate performance on standardized
speech mixtures.
Read the full abstract ▾
All authors Muhammad Shakeel, Yosuke Fukumoto, Chikara Maeda, Chyi-Jiunn Lin, Shinji Watanabe
Audio and Speech Processing
Streaming Speech Recognition with Decoder-Only Large Language Models and Latency Optimization
Genshun Wan, Wenhui Zhang, Jing-Xuan Zhang et al. (+3 more)
Recent advances have demonstrated the potential of decoderonly large
language models (LLMs) for automatic
speech recognition (ASR). However, enabling streaming recognition within this framework remains a challenge. In this work, we propose a novel streaming ASR approach that integrates a read/write policy network with monotonic chunkwise
attention (MoChA) to dynamically segment
speech embeddings. These segments are interleaved with label sequences during training, enabling seamless integration with the LLM. During inference, the audio stream is buffered until the MoChA module triggers a read signal, at which point the buffered segment together with the previous
token is fed into the LLM for the next
token prediction. We also introduce a minimal-latency training objective to guide the policy network toward accurate
segmentation boundaries. Furthermore, we adopt a joint training strategy in which a non-streaming LLM-ASR model and our streaming model share parameters. Experiments on the AISHELL-1 and AISHELL-2 Mandarin
benchmarks demonstrate that our method consistently outperforms recent streaming ASR baselines, achieving character error rates of 5.1% and 5.5%, respectively. The latency optimization results in a 62.5% reduction in average
token generation delay with negligible impact on recognition accuracy
Read the full abstract ▾
All authors Genshun Wan, Wenhui Zhang, Jing-Xuan Zhang, Shifu Xiong, Jianqing Gao, Zhongfu Ye
Signal Processing
Rohit Varma Chiluvuri and Santosh Nannuru
The completion of a Euclidean distance
matrix (EDM) from sparse and noisy observations is a fundamental challenge in signal processing, with applications in sensor network localization, acoustic room reconstruction, molecular conformation, and
manifold learning. Traditional approaches, such as rank-constrained optimization and semidefinite programming, enforce
geometric constraints but often struggle under sparse or noisy conditions. This paper introduces a hierarchical
Bayesian framework that places structured priors directly on the latent point set generating the EDM, naturally
embedding geometric constraints. By incorporating a hierarchical prior on latent point set, the model enables automatic regularization and robust noise handling. Posterior inference is performed using a Metropolis-Hastings within Gibbs sampler to handle coupled latent point posterior. Experiments on synthetic data demonstrate improved reconstruction accuracy compared to deterministic baselines in sparse regimes.
Read the full abstract ▾
Image and Video Processing
Synthetic Abundance Maps for Unsupervised Super-Resolution of Hyperspectral Remote Sensing Images
Xinxin Xu, Yann Gousseau, Christophe Kervazo et al. (+1 more)
Hyperspectral single image super-resolution (HS-SISR) aims to enhance the spatial resolution of hyperspectral images to fully exploit their spectral information. While considerable progress has been made in this field, most existing methods are supervised and require ground truth data for training-data that is often unavailable in practice. To overcome this limitation, we propose a novel unsupervised training framework for HS-SISR, based on synthetic abundance data, where no high-resolution ground-truth reference is required for training. The approach begins by unmixing the hyperspectral image into endmembers and abundances. A neural network is then trained to perform abundance super-resolution using synthetic abundances only. These synthetic abundance maps are generated from a dead leaves model whose characteristics are inherited from the low-resolution image to be super-resolved and from the known point spread function (PSF) of the hyperspectral sensor. This trained network is subsequently used to enhance the spatial resolution of the original image's abundances, and the final super-resolution hyperspectral image is reconstructed by combining them with the endmembers. Experimental results demonstrate both the training value of the synthetic data and the effectiveness of the proposed method across 3
datasets, 3 scaling factors, and several evaluation metrics. The code is available at https://github.com/xinxinxu99/SISR-DL.git
Read the full abstract ▾
All authors Xinxin Xu, Yann Gousseau, Christophe Kervazo, Saïd Ladjal
Image and Video Processing
Active Learning-Driven Lightweight YOLOv9: Enhancing Efficiency in Smart Agriculture
Hung-Chih Tu, Bo-Syun Chen and Yun-Chien Cheng
This study addresses the demand for real-time detection of tomatoes and tomato flowers by agricultural
robots deployed on edge devices in greenhouse environments. Under practical imaging conditions, object detection systems often face challenges such as large scale variations caused by varying camera distances, severe occlusion from plant structures, and highly imbalanced class distributions. These factors make conventional object detection approaches that rely on fully annotated
datasets difficult to simultaneously achieve high detection accuracy and deployment efficiency. To overcome these limitations, this research proposes an active learning driven lightweight object detection framework, integrating data analysis, model design, and training strategy. First, the size distribution of objects in raw agricultural images is analyzed to redefine an operational target range, thereby improving learning stability under real-world conditions. Second, an efficient feature extraction module is incorporated to reduce computational cost, while a lightweight
attention mechanism is introduced to enhance feature representation under multi-scale and occluded scenarios. Finally, an active learning strategy is employed to iteratively select high-information samples for annotation and training under a limited labeling budget, effectively improving the recognition performance of minority and small-object categories. Experimental results demonstrate that, while maintaining a low parameter count and inference cost suitable for edge-device deployment, the proposed method effectively improves the detection performance of tomatoes and tomato flowers in raw images. Under limited annotation conditions, the framework achieves an overall detection accuracy of 67.8% mAP, validating its practicality and feasibility for intelligent agricultural applications.
Read the full abstract ▾
Signal Processing
SORIS: A Self-Organized Reconfigurable Intelligent Surface Architecture for Wireless Communications
Evangelos Koutsonas, Alexandros-Apostolos A. Boulogeorgos, Stylianos E. Trevlakis et al. (+3 more)
In this paper, a new reconfigurable intelligent surface (RIS) hardware architecture, called self-organized RIS (SORIS), is proposed. The architecture incorporates a microcontroller connected to a single-
antenna receiver operating at the same frequency as the RIS unit elements, operating either in transmission or reflection mode. The transmitting RIS elements enable the low latency estimation of both the incoming and outcoming channels at the microcontroller's side. In addition, a machine learning approach for estimating the incoming and outcoming channels involving the remaining RIS elements operating in reflection mode is devised. Specifically, by appropriately selecting a small number of elements in transmission mode, and based on the channel reciprocity principle, the respective channel coefficients are first estimated, which are then fed to a low-complexity neural network that, leveraging spatial channel correlation over RIS elements, returns predictions of the channel coefficients referring to the rest of elements. In this way, the SORIS microcontroller acquires channel state information, and accordingly reconfigures the panel's metamaterials to assist data communication between a transmitter and a receiver, without the need for separate connections with them. Moreover, the impact of channel estimation on the proposed solution, and a detailed complexity analysis for the used model, as well as a wiring density and control signaling analysis, is performed. The feasibility and efficacy of the proposed self-organized RIS design and operation are verified by Monte Carlo simulations, providing useful guidelines on the selection of the RIS elements for operating in transmission mode for initial channel estimation.
Read the full abstract ▾
All authors Evangelos Koutsonas, Alexandros-Apostolos A. Boulogeorgos, Stylianos E. Trevlakis, George C. Alexandropoulos, Theodoros A. Tsiftsis, Rui Zhang
Image and Video Processing · Cross-listed from cs.LG
Deep Learning-Based Early-Stage IR-Drop Estimation via CNN Surrogate Modeling
Ritesh Bhadana
IR-drop is a critical power integrity challenge in modern VLSI designs that can cause timing degradation, reliability issues, and functional failures if not detected early in the design flow. Conventional IR-drop analysis relies on physics-based signoff tools, which provide high accuracy but incur significant computational cost and require near-final layout information, making them unsuitable for rapid early-stage design exploration. In this work, we propose a deep learning-based surrogate modeling approach for early-stage IR-drop estimation using a CNN. The task is formulated as a dense pixel-wise regression problem, where spatial physical layout features are mapped directly to IR-drop heatmaps. A U-Net-based encoder-decoder architecture with skip connections is employed to effectively capture both local and global spatial dependencies within the layout. The model is trained on a physics-inspired synthetic
dataset generated by us, which incorporates key physical factors including power grid structure,
cell density distribution, and switching activity. Model performance is evaluated using standard regression metrics such as Mean Squared Error (MSE) and Peak Signal-to-Noise Ratio (PSNR). Experimental results demonstrate that the proposed approach can accurately predict IR-drop distributions with millisecond-level inference time, enabling fast pre-signoff screening and iterative design optimization. The proposed framework is intended as a complementary early-stage analysis tool, providing designers with rapid IR-drop insight prior to expensive signoff analysis. The implementation,
dataset generation scripts, and the interactive inference application are publicly available at: https://github.com/riteshbhadana/IR-Drop-Predictor. The live application can be accessed at: https://ir-drop-predictor.streamlit.app/.
Read the full abstract ▾
Image and Video Processing
Training Beyond Convergence: Grokking nnU-Net for Glioma Segmentation in Sub-Saharan MRI
Mohtady Barakat, Omar Salah, Ahmed Yasser et al. (+8 more)
Gliomas are placing an increasingly clinical burden on Sub-Saharan Africa (SSA). In the region, the median survival for patients remains under two years, and access to diagnostic imaging is extremely limited. These constraints highlight an urgent need for automated tools that can extract the maximum possible information from each available scan, tools that are specifically trained on local data, rather than adapted from high-income settings where conditions are vastly different. We utilize the
Brain Tumor
Segmentation (BraTS) Africa 2025 Challenge
dataset, an expert annotated collection of glioma MRIs. Our objectives are: (i) establish a strong baseline with nnUNet on this
dataset, and (ii) explore whether the celebrated "grokking" phenomenon an abrupt, late training jump from memorization to superior generalization can be triggered to push performance without extra labels. We evaluate two training regimes. The first is a fast, budget-conscious approach that limits optimization to just a few epochs, reflecting the constrained GPU resources typically available in African institutions. Despite this limitation, nnUNet achieves strong Dice scores: 92.3% for whole tumor (WH), 86.6% for tumor core (TC), and 86.3% for enhancing tumor (ET). The second regime extends training well beyond the point of convergence, aiming to trigger a grokking-driven performance leap. With this approach, we were able to achieve grokking and enhanced our results to higher Dice scores: 92.2% for whole tumor (WH), 90.1% for tumor core (TC), and 90.2% for enhancing tumor (ET).
Read the full abstract ▾
All authors Mohtady Barakat, Omar Salah, Ahmed Yasser, Mostafa Ahmed, Zahirul Arief, Waleed Khan, Dong Zhang, Aondona Iorumbur, Confidence Raymond, Mohannad Barakat, Noha Magdy
Image and Video Processing
Bonnet: Ultra-fast whole-body bone segmentation from CT scans
Hanjiang Zhu, Pedro Martelleto Rezende, Zhang Yang et al. (+5 more)
This work proposes Bonnet, an ultra-fast sparse-volume pipeline for whole-body bone
segmentation from CT scans. Accurate bone
segmentation is important for
surgical planning and anatomical analysis, but existing 3D voxel-based models such as nnU-Net and STU-Net require heavy computation and often take several minutes per scan, which limits time-critical use. The proposed Bonnet addresses this by integrating a series of novel framework components including HU-based bone thresholding, patch-wise inference with a sparse spconv-based U-Net, and multi-window fusion into a full-volume prediction. Trained on TotalSegmentator and evaluated without additional tuning on RibSeg, CT-Pelvic1K, and CT-Spine1K, Bonnet achieves high Dice across ribs, pelvis, and spine while running in only 2.69 seconds per scan on an RTX A6000. Compared to strong voxel baselines, Bonnet attains a similar accuracy but reduces inference time by roughly 25x on the same hardware and tiling setup. The toolkit and pre-trained models will be released at https://github.com/HINTLab/Bonnet.
Read the full abstract ▾
All authors Hanjiang Zhu, Pedro Martelleto Rezende, Zhang Yang, Tong Ye, Bruce Z. Gao, Feng Luo, Siyu Huang, Jiancheng Yang
Systems and Control
Approximately Optimal Multi-Stream Quickest Change Detection
Joshua Kartzman, Calvin Hawkins and Matthew Hale
This paper considers the constrained sampling multi-stream quickest change detection problem, also known as the bandit quickest change detection problem. One stream contains a change-point that shifts its mean by an unknown amount. The goal is to quickly detect this change while controlling for false alarms, while being only able to sample one stream at each time. We propose an
algorithm that combines a decaying-$\epsilon$-greedy stream switching rule with a Generalized Likelihood Ratio detection procedure for unknown post-change means. We provide performance bounds for our
algorithm and show it achieves approximate asymptotic first-order optimality with respect to a commonly used surrogate. We are the first to provide guarantees in this setting without assumptions such as a discretized post-change parameter set or a lower bound on the magnitude of change. We provide guarantees for a wide range of light-tailed distributions, including sub-Gaussian and bounded support distributions.
Read the full abstract ▾
Image and Video Processing
EndoCaver: Handling Fog, Blur and Glare in Endoscopic Images via Joint Deblurring-Segmentation
Zhuoyu Wu, Wenhui Ou, Pei-Sze Tan et al. (+4 more)
Endoscopic image analysis is vital for colorectal cancer screening, yet real-world conditions often suffer from lens fogging, motion blur, and specular highlights, which severely compromise automated polyp detection. We propose EndoCaver, a lightweight
transformer with a unidirectional-guided dual-decoder architecture, enabling joint multi-task capability for image deblurring and
segmentation while significantly reducing computational complexity and model parameters. Specifically, it integrates a Global
Attention Module (GAM) for cross-scale aggregation, a Deblurring-
Segmentation Aligner (DSA) to transfer restoration cues, and a cosine-based scheduler (LoCoS) for stable multi-task optimisation. Experiments on the Kvasir-SEG
dataset show that EndoCaver achieves 0.922 Dice on clean data and 0.889 under severe image degradation, surpassing state-of-the-art methods while reducing model parameters by 90%. These results demonstrate its efficiency and robustness, making it well-suited for on-device clinical deployment. Code is available at https://github.com/ReaganWu/EndoCaver.
Read the full abstract ▾
All authors Zhuoyu Wu, Wenhui Ou, Pei-Sze Tan, Jiayan Yang, Wenqi Fang, Zheng Wang, Raphaël C.-W. Phan
Signal Processing
Superimposed-Pilot OTFS Under Fractional Doppler: Modular End-to-End Learning
Yushi Lei, Yusha Liu, Guanghui Liu et al. (+2 more)
Orthogonal time frequency space (OTFS) modulation has emerged as a promising candidate to overcome the performance degradation of orthogonal frequency division multiplexing (OFDM), which are commonly encountered in high-mobility wireless communication scenarios. However, conventional OTFS transceivers rely on multiple separately designed signal-processing modules, whose isolated optimization often limits global optimal performance. To overcome limitations, this paper proposes a modular deep learning (DL) based end-to-end OTFS transceiver framework that consists of trainable and interchangeable neural network (NN) modules, including constellation mapping/demapping, superimposed pilot placement, inverse Zak (IZak)/Zak transforms, and a U-Net-enhanced NN tailored for joint channel estimation and detection (JCED), while explicitly accounting for the impact of the cyclic prefix. This physics-informed modular architecture provides flexibility for integration with conventional OTFS systems and adaptability to different communication configurations. Simulations demonstrate that the proposed design significantly outperforms baseline methods in terms of both normalized mean squared error (NMSE) and detection reliability, maintaining robustness under integer and fractional Doppler conditions. The results highlight the potential of DL-based end-to-end optimization to enable practical and high-performance OTFS transceivers for next-generation high-mobility networks.
Read the full abstract ▾
All authors Yushi Lei, Yusha Liu, Guanghui Liu, Lei Wan, Kun Yang
Audio and Speech Processing
Class-Aware Permutation-Invariant Signal-to-Distortion Ratio for Semantic Segmentation of Sound Scene with Same-Class Sources
Binh Thien Nguyen, Masahiro Yasuda, Daiki Takeuchi et al. (+2 more)
To advance immersive communication, the Detection and Classification of Acoustic Scenes and Events (DCASE) 2025 Challenge recently introduced Task 4 on Spatial Semantic
Segmentation of Sound Scenes (S5). An S5 system takes a multi-channel audio mixture as input and outputs single-channel dry sources along with their corresponding class labels. Although the DCASE 2025 Challenge simplifies the task by constraining class labels in each mixture to be mutually exclusive, real-world mixtures frequently contain multiple sources from the same class. The presence of duplicated labels can significantly degrade the performance of the label-queried source separation (LQSS) model, which is the key component of many existing S5 systems, and can also limit the validity of the official evaluation metric of DCASE 2025 Task 4. To address these issues, we propose a class-aware permutation-invariant loss function that enables the LQSS model to handle queries involving duplicated labels. In addition, we redesign the S5 evaluation metric to eliminate ambiguities caused by these same-class sources. To evaluate the proposed method within the S5 system, we extend the label prediction model to support same-class labels. Experimental results demonstrate the effectiveness of the proposed methods and the robustness of the new metric on mixtures both with and without same-class sources.
Read the full abstract ▾
All authors Binh Thien Nguyen, Masahiro Yasuda, Daiki Takeuchi, Daisuke Niizumi, Noboru Harada
Audio and Speech Processing · Cross-listed from cs.SD
Rethinking Speech Representation Aggregation in Speech Enhancement: A Phonetic Mutual Information Perspective
Seungu Han, Sungho Lee and Kyogu Lee
Recent
speech enhancement (SE) models increasingly leverage self-supervised learning (SSL) representations for their rich semantic information. Typically, intermediate features are aggregated into a single representation via a lightweight adaptation module. However, most SSL models are not trained for noise robustness, which can lead to corrupted semantic representations. Moreover, the adaptation module is trained jointly with the SE model, potentially prioritizing acoustic details over semantic information, contradicting the original purpose. To address this issue, we first analyze the behavior of SSL models on noisy
speech from an information-theoretic perspective. Specifically, we measure the mutual information (MI) between the corrupted SSL representations and the corresponding phoneme labels, focusing on preservation of linguistic contents. Building upon this analysis, we introduce the linguistic aggregation layer, which is pre-trained to maximize MI with phoneme labels (with optional dynamic aggregation) and then frozen during SE training. Experiments show that this decoupled approach improves Word Error Rate (WER) over jointly optimized baselines, demonstrating the benefit of explicitly aligning the adaptation module with linguistic contents.
Read the full abstract ▾