Peer Revenue Publish or profit

Papers

Fresh from arXiv, typeset properly. Highlighted words trade on the market.

Computer Science Hardware Architecture

cs.AR Watch Hardware Architecture

Thursday 6 August

Hardware Architecture

Automated Synthesis of Heterogeneous, Hierarchical, Scoped Coherence Protocols

Fletch Rydell, An Qi Zhang, Nicolai Oswald et al. (+3 more)

Processor design is converging on a new model of cache-coherent shared memory characterized by heterogeneity, hierarchy, and scopes. Protocols like CXL or AMBA CHI are used as global protocols to combine multiple clusters, each with its own cluster-level coherence protocols. Manually designing shims to interface between these cluster and global protocols is subtle and error-prone. Automatic synthesis, on the other hand, makes simplifying assumptions like the single-writer multiple-reader (SWMR) invariant, that sacrifice performance for simplicity and guaranteed correctness. We present a shim API \textemdash a generic abstraction that enables us to classify protocol transactions by their semantic coherence guarantees. Our automated synthesis engine, ShimGen, uses this shim API to automatically compose protocols with both SWMR and relaxed accesses, taking advantage of modern architectural optimizations like scoped memory accesses and lazy invalidation. We demonstrate ShimGen's efficacy on two case studies. First, we compare its output to a manually-designed hierarchical protocol for the AMD APU released in gem5. ShimGen's output is similar to the existing protocol. However, we identify one scenario where the manually-designed protocol fails to uphold compound consistency, while ShimGen's does. Second, we compare the performance of a hierarchical protocol with a global protocol that exploits non-SWMR accesses to a hierarchical protocol with a strictly SWMR global protocol. The result shows the performance benefits of accommodating global protocols with non-SWMR behavior.
Read the full abstract

All authors Fletch Rydell, An Qi Zhang, Nicolai Oswald, Andres Goens, Vijay Nagarajan, Daniel Sorin

Hardware Architecture

An Open-Source Power Measurement Platform for System-Level Semiconductor Testing

Linus Bantel, Sarah Rottacker and Dirk Pflüger

Accurate power measurement is not only essential for evaluating the energy efficiency of modern embedded and semiconductor systems, but power draw is an important proxy during stress testing. Industrial semiconductor test equipment, however, is often expensive and difficult to integrate into flexible experimental workflows. This paper presents a compact and extensible hardware platform for automated system-level power measurement of embedded devices. The proposed setup integrates a Raspberry Pi controller, a precision current measurement device, and a microcontroller-based Device-under-Test (DUT). The system supports automated firmware deployment, synchronized device execution, and high-resolution current acquisition. A lightweight HTTP-based interface allows remote clients to upload firmware binaries and retrieve measurement results. This design enables automated benchmarking and reproducible experiments across multiple firmware variants. The platform is implemented using accessible hardware components and open-source software tools, making it suitable for research, prototyping, and educational environments. We describe the architecture of the platform, the automated testing workflow, and potential application scenarios, including firmware energy profiling and regression testing.
Read the full abstract

Hardware Architecture

Zero-Instruction Sensor Reads: Register-Mapped Peripherals and Hardware PWM on a Five-Stage Soft Processor

Nathanael Ren

We present a case study in application-driven specialization of a five-stage soft processor, evaluated on the inner control loop of a reaction-wheel self-balancing bicycle. Starting from a custom 32-bit RISC core in the MIPS tradition, we specialize the design in two ways. First, two frequently accessed peripheral inputs are mapped directly into architectural register state, written every cycle by hardware and owned exclusively through the register file's write-port structure rather than by arbitration. Second, four periodic PWM channels are offloaded to hardware and driven continuously from four exported registers, removing periodic actuation from software entirely. Because peripheral values are addressable as ordinary register operands, all ten sensor reads in the control loop cost no dedicated instruction and no dedicated cycle, folding into arithmetic that executes anyway; the memory-mapped equivalent requires an explicit load per snapshot and costs five extra instructions and cycles. The actuation path likewise removes waveform maintenance from software entirely. We report two configurations, because the extensions and the single-cycle array multiplier they were deployed alongside are not present together in a single archived build: an archived configuration, whose worst-case loop is 91 cycles, and the integrated configuration matching the deployed system, at 43 cycles. Against a 20 ms actuation frame these are margins of roughly 7,300x and 15,000x. The deadline is met by so wide a margin in either case that the specialization was not necessary for real-time compliance; its value lies in instruction count and software simplicity, not in determinism, which an on-chip single-cycle I/O region already provides. The zero-instruction sensor read is independent of that choice: the multiplier cannot affect whether a peripheral read needs an instruction of its own.
Read the full abstract

Hardware Architecture

PLoRA: An NDP-Enhanced Pooled-Memory System for Cost-Efficient Multi-LoRA Serving

Zhongkai Yu, Ohm Rishabh Venkatachalam, Zheng Wang et al. (+9 more)

Multi-LoRA serving is how one base model becomes thousands of specialized variants, one adapter per user, task, or agent, and the deployments can hold 1000-plus adapters. Serving them is hard because the workload inverts what GPUs provide: terabytes of memory against only tens of TFLOPS, and because every published system stages its adapters from CPU DRAM over PCIe, where each access pays a kernel stop and a host-run copy and capacity ends at the motherboard's DIMM slots. Meanwhile, memory-semantic fabrics such as CXL and NVLink are converging on pooled memory that an accelerator addresses with its own loads and stores, and near-data processing (NDP) can place compute beside the pooled data. How to serve multi-LoRA workloads on such hardware remains unexplored. This paper introduces PLoRA, an NDP-enhanced pooled-memory system for cost-efficient multi-LoRA serving. PLoRA keeps adapters and KV cache in the pool and returns only reduced results over the link, through a read-compute interface the GPU drives with its own loads and stores. Above this architecture, a GPU memory management system picks among four LoRA and two attention execution strategies for each adapter and caches the most performance-critical bytes in GPU memory, guided by a link-parameterized cost model. On one H100 serving 1000 adapters, PLoRA attains the lowest decode latency on every model and workload we measure, averaging 6.6x below a real-machine S-LoRA at under 3.4% added device area. The link itself stops mattering: throughput saturates at 32 GB/s on short contexts, a quarter of CXL 3.1, and the verdict survives scale: per-GPU demand falls from 7B to a modeled 1.2T deployment once adapter traffic shards with the tensor parallelism. The design runs unchanged from CXL-class to NVLink-class fabrics, and surplus bandwidth buys pooled capacity rather than speed.
Read the full abstract

All authors Zhongkai Yu, Ohm Rishabh Venkatachalam, Zheng Wang, Yikai Li, Yichen Lin, Zihao Yu, Yuke Wang, Liu Liu, Xulong Tang, Shuyi Pei, Yangwook Kang, Yufei Ding

Wednesday 5 August

Hardware Architecture

PowerScope: ML-based Intra-Cycle Power Estimation

Jayanth Balasubramanian, Sujay Pandit, Radha Vaidya et al. (+1 more)

Power estimation at sub-clock-cycle temporal resolutions is critical for tasks such as power delivery network (PDN) design, dynamic voltage droop analysis, and pre-silicon power side-channel security evaluation. Designers commonly rely on commercial post-layout gate-level power analysis tools for these tasks, but these flows are computationally expensive and scale poorly with design size and workload length. Machine learning (ML)-based power estimation frameworks have shown promise in accelerating power estimation, but prior efforts only address average power or per-cycle power estimation. We propose PowerScope, the first ML-based intra-cycle power estimation framework. PowerScope operates purely on RTL simulation traces at inference time, eliminating the need for post-layout gate-level simulation and power analysis per workload. Across a diverse benchmark suite, PowerScope achieves 5.88% median and 9% mean absolute percentage error compared to commercial post-layout gate-level power estimates while running ~80x faster. We further demonstrate that PowerScope's predictions can be reliably used for the downstream task of pre-silicon power side-channel leakage assessment.
Read the full abstract

All authors Jayanth Balasubramanian, Sujay Pandit, Radha Vaidya, Anand Raghunathan

Hardware Architecture

EdgeXpert: An Edge Device for Memory-Efficient LLM Inference with Mixture-of-Experts and Speculative Decoding

Sangwoo Ha, Hyunwoo Seo, Yurim Jo et al. (+2 more)

On-device deployment of Large Language Models (LLMs) has become essential for personalized edge applications. A primary bottleneck is external memory access (EMA) in feed-forward network (FFN) layers. Speculative decoding and mixture-of-experts (MoE) are promising solutions. Speculative decoding reduces the number of decoding stages by generating multiple tokens per stage, and MoE minimizes per-stage cost through sparse expert activation. However, there is an incompatibility when combining these two techniques. We propose EdgeXpert, a software-hardware co-designed LLM accelerator that resolves this incompatibility. In the prefill stage, the prompt-wise expert reuse reformulates routing as prompt-level expert reuse rather than independent per-token expert selection. It identifies important tokens using a lightweight encoder, constructs a shared expert set from them, and routes less important tokens with a reduced expert budget to lower expert EMA. In the decode stage, depth-aware expert coalescing exploits the contextual similarity and mutual exclusivity of same-depth candidate tokens. Rather than loading the union of all required channels, EdgeXpert loads only salient channels and applies computational calibration to recover accuracy without additional memory access. Synthesized in Samsung 28nm technology at 800 MHz, EdgeXpert achieves up to 56.3% latency reduction and 44.1% energy reduction compared to prior works, while maintaining near-baseline accuracy.
Read the full abstract

All authors Sangwoo Ha, Hyunwoo Seo, Yurim Jo, Youngjin Moon, Hoi-Jun Yoo

Hardware Architecture · Cross-listed from cs.CR

Hardware Design and Security in the Era of Chiplets and LLMs

Johann Knechtel, Ozgur Sinanoglu, Paul V. Gratz et al. (+1 more)

The semiconductor industry is undergoing a dual revolution: the shift toward heterogeneous 2.5D chiplet systems and the integration of Large Language Models (LLMs) into Electronic Design Automation (EDA) flows. While these paradigms offer unprecedented benefits in yield, modularity, design productivity, etc., they radically expand the hardware attack surface. This paper provides a unified analysis of these frontiers, ranging from attacks on chiplet systems (including hardware stacks for LLM acceleration) across architectural, logical, and physical levels, to various exploits against LLM-driven EDA pipelines. To secure chiplet systems, we review a powerful defense approach that leverages 2.5D split manufacturing and active interposers for physically isolated Root of Trust (RoT) architectures. To secure LLM-driven EDA pipelines, we first identify native threats and then review state-of-the-art defense techniques. Finally, we discuss how LLM systems can advance hardware security efforts for modern systems, including chiplets.
Read the full abstract

All authors Johann Knechtel, Ozgur Sinanoglu, Paul V. Gratz, Ramesh Karri

Hardware Architecture · Cross-listed from cs.CR

Kerckhoffs-Compliant Watermarking for Physical Design IP Protection: From Placement to Routing

Andrew B. Kahng and Yiting Liu

Physical design (PD) intellectual property (IP) is a valuable artifact of modern VLSI implementation. It includes optimized cell placement, clock distribution, and routing decisions produced by carefully tuned PD flows. As access to PD tools expands, unauthorized reuse of placed-and-routed databases becomes an increasing concern. Existing PD watermarking methods either protect only one PD stage or rely on hidden construction details, leaving them vulnerable to a white-box adversary. In this work, we develop PDMarks, a Kerckhoffs-compliant watermarking framework whose security depends only on a secret key. PDMarks embeds ownership evidence across multiple stages of the PD flow, including placement, clock tree synthesis (CTS), and routing. All watermark instances and target values are deterministically derived from a 32-byte secret key using HMAC-SHA256, enabling consistent embedding and verification. PDMarks has been integrated into OpenROAD-flow-scripts. Experiments on NanGate45 and ASAP7 designs show that PDMarks outperforms prior physical design watermarking methods by providing much stronger ownership evidence with comparable or smaller PPA overhead. The approximate joint all-stage coincidence probability is below 10^{-32} for every evaluated design. Wrong-key and attack evaluations further show that incorrect keys do not reproduce the complete ownership proof and that weakening the watermark requires broad perturbation of the protected implementation.
Read the full abstract

Hardware Architecture · Cross-listed from cs.CR

LLM-Assisted Detection and Repair of Hardware Security Vulnerabilities in Verilog Designs

Ethen Santana, Gabriel Gyaase and Hao Zheng

Hardware designs, like software, are susceptible to bugs that can introduce security vulnerabilities and create opportunities for malicious exploitation. Unlike software vulnerabilities, however, hardware flaws become permanently embedded in silicon after fabrication, making them difficult or impossible to patch. Many of these weaknesses are categorized under the Common Weakness Enumeration (CWE) framework and include improper access control, exposure of sensitive information, and unintended privilege escalation. To improve the detection of such vulnerabilities, we propose a methodology that leverages a Large Language Model (LLM) to identify potential hardware CWEs directly from hardware designs in Verilog. The proposed approach is evaluated iteratively on a dataset of single-module Verilog designs to assess its effectiveness in detecting hardware security weaknesses. Our results demonstrate the potential of LLMs to augment traditional hardware security analysis by providing automated, scalable assistance for identifying security vulnerabilities during the hardware design process.
Read the full abstract

Hardware Architecture

A Systolic Array Architecture for Nonlinear Activation Functions and Softmax Computation using Chebyshev Polynomials

Benedikt Schaible, Anirudh Suresh Bharadwaj, Ulf Schlichtmann et al. (+1 more)

Neural Network Accelerators have gained popularity in recent years due to their greater efficiency than CPU-based platforms. Often, these accelerators utilize different hardware units for univariate activation functions, such as tanh, and the multivariate softmax, thereby missing opportunities for resource sharing between them. In this paper, we describe a novel systolic array-based activation unit architecture that supports multiple univariate activation functions as well as the softmax function. By utilizing Chebyshev polynomial approximations, our activation function unit achieves up to 71% lower mean absolute error for tanh compared to a CORDIC baseline, while using 4.6% less area and 5.1% less power. Our softmax approximation enables a 44.6% and 79.0% lower KL divergence compared to CORDIC and a piecewise-linear approximation, respectively.
Read the full abstract

All authors Benedikt Schaible, Anirudh Suresh Bharadwaj, Ulf Schlichtmann, Jiang Hu

Hardware Architecture · Cross-listed from cs.AI

Architectural Implications of Agentic AI Workflows

Jirong Yang, Peizhe Liu, Chaojie Zhang et al. (+1 more)

Agentic AI is emerging in datacenters, but its architectural implications remain unexplored. We organize agentic workflows in a taxonomy and present its first architectural characterization with a production study at Microsoft Azure and a controlled study of open-source frameworks. We show that agentic execution is fragmented and heterogeneous. Requests expand into a workflow of LLM inferences, tool invocations, and orchestration decisions that repeatedly cross the CPU-GPU boundary. Our taxonomy explains how this fragmentation turns into resource demand. As orchestration and tools run on the host, the CPU sits on the critical path. Execution structure sets the load over time, which stays low with sudden spikes. Model composition sets how evenly the workflow uses the GPUs. Diversity in tasks and tools widens this range even further. These characteristics expose architectural mismatches of conventional uniform servers. Fragmented execution strands CPU and GPU capacity despite bursty demand. Different software roles make homogeneous CPU provisioning inefficient. Finally, multiplexing many agents onto shared cores degrades microarchitectural locality. Guided by our findings, we derive implications for agentic servers and examine them through Agora, our prototype for commodity servers. Agora dynamically harvests idle CPU cores for co-located throughput work, while protecting agentic tail latency against tool spikes. It oversubscribes GPU memory by placing more agents on each GPU, prefetching the next agent's state to hide swap latency. To match the machine to the heterogeneous roles, Agora pools cores by role and applies affinity-aware scheduling to restore locality. It automatically tunes mechanisms to the workload. Agora improves utilization and server throughput while preserving agent tail latency. Our insights also identify key directions for future server architectures for agentic AI.
Read the full abstract

All authors Jirong Yang, Peizhe Liu, Chaojie Zhang, Jovan Stojkovic

Hardware Architecture

MCHA: A Memory-Centric Hierarchical Architecture for Parallel-Sequential Computing

Daijing Shi, Hongxiao Zhao, Yihan Fu et al. (+7 more)

Emerging workloads, such as Multi-Agent Reinforcement Learning (MARL), large-scale neuromorphic computing, and probabilistic graphical models, intrinsically exhibit parallel-sequential computing patterns. While these tasks demand massive parallelism to achieve high throughput, they are severely bottlenecked by irregular data access patterns centralized to main memory. Consequently, conventional architectures face fundamental limitations when executing these workloads, primarily manifesting as global buffer saturation and memory-bound bottlenecks. To address these challenges, we propose the Memory-Centric Hierarchical Architecture (MCHA), a reconfigurable hardware solution tailored for parallel-sequential execution. MCHA leverages a hierarchical communication strategy that facilitates distributed, inter-core data routing, thereby significantly reducing the bandwidth burden on the global memory. Complementing the hardware, MCHA introduces a novel parallel-sequential programming model that utilizes event-driven conditional triggers to effectively hide data transmission latency within the execution pipeline. We benchmark MCHA against a diverse suite of parallel-sequential tasks, including MARL, motor variable control, and Markov random fields. Validated through our open-source, cycle-accurate simulator, MCHA demonstrates performance speedups ranging from 153.06$\times$ to 2456.96$\times$ over NVIDIA A100 GPUs on MARL workloads, while maintaining robust programming flexibility across other application domains. Furthermore, the architecture successfully reduces main memory access from 96% to 5.44%. When synthesized in a 28 nm process, the MCHA implementation occupies an area footprint of 2.92mm$^2$ and consumes 115.36 mW of power at 200 MHz. MCHA is open-sourced at https://github.com/carabdis/MCHA.
Read the full abstract

All authors Daijing Shi, Hongxiao Zhao, Yihan Fu, Zhan Chen, Jiayi Li, Yihang Zhu, Anjunyi Fan, Yaoyu Tao, Yuchao Yang, Bonan Yan

Hardware Architecture

Deltoris: Enabling Real-time VLA Inference in Embodied AI via Bit-level Sparsity and Speculative Inference

Zheng Liu, Zeyu Guo, Zihan Liu et al. (+9 more)

Vision-language-action (VLA) models have emerged as a key component in embodied AI. Among existing approaches, diffusion-based VLA models achieve superior motion quality and generalization. However, diffusion-based VLA models are compute-intensive and must run at high control frequency, e.g., 50-200 Hz. Thus, it imposes strict latency and energy constraints on edge devices. In this work, we present Deltoris, an algorithm-hardware co-design framework for efficient diffusion-based VLA inference. First, we exploit the temporal similarity of consecutive inputs and propose a temporal-aware bit-sparsity algorithm that computes only the differences between consecutive inputs, eliminating redundant bit-level operations. To further address the extra off-chip traffic introduced by our algorithm, we propose a speculative inference technique, which amortizes data loading across multiple control steps. Lastly, to support these techniques, we co-design a dedicated accelerator with customized 1D systolic bit-serial PE arrays that eliminate PE workload imbalance. Our evaluation shows that Deltoris achieves up to 34.2$\times$ speedup over mobile GPUs and 6.1$\times$ over prior accelerators, while maintaining comparable accuracy.
Read the full abstract

All authors Zheng Liu, Zeyu Guo, Zihan Liu, Anbang Wu, Han Zhao, Fangxin Liu, Zhezhi He, Yinhe Han, Jingwen Leng, Minyi Guo, Yiming Gan, Yu Feng

Tuesday 4 August

Hardware Architecture

A Centralized Performance Monitoring Architecture for Heterogeneous Multicore SoCs

Mohammed Sajjad Jafri, Abdur Rahman, Emon Sarkar et al. (+4 more)

Hardware Performance Counters (HPCs) are widely used to enable event-driven software mechanisms such as profile guided optimization, performance analysis, and dynamic resource management in real-time systems. However, in modern embedded System-on-a-Chip (SoCs), different components - including processors, accelerators, interconnects, and memory controllers - typically implement separate and heterogeneous HPC modules. This distributed monitoring infrastructure requires multiple software interfaces and complicates the collection, synchronization, and correlation of performance data across the system. In this work, we propose a centralized performance monitoring architecture to efficiently collect, correlate, and process architectural events across multiple hardware components. Our design introduces Event Monitoring Units (EVUs) that capture and forward microarchitectural events to an Advanced Performance Monitoring Unit (APMU). The APMU integrates programmable counters and a specialized processing element to support flexible, event-driven software mechanisms. We implement our design for an AXI4-based system and integrate it into a RISC-V based SoC platform, which lacks advanced cross-component performance monitoring support. We demonstrate the effectiveness of our approach through case studies on real-time resource regulation, application profiling, and counter attribution.
Read the full abstract

All authors Mohammed Sajjad Jafri, Abdur Rahman, Emon Sarkar, Mahdi Hassen, Gopishankar Thayyil, Ashwin Krishna Mani, Rodolfo Pellizzoni

Hardware Architecture

On Design Principles for Efficient Heterogeneous DRAM-PIM-GPU Systems

Corey Lammie, Hadjer Benmeziane, William Andrew Simon et al. (+1 more)

Heterogeneous DRAM-based processing-in-memory (PIM)-GPU systems promise significant efficiency gains for decode-phase large language model (LLM) inference, particularly in long-output generation, yet current design practices overlook critical factors that determine real-world performance. Through systematic evaluation of diverse architectures and workloads (OPT-7B/70B, Mamba2-2.7B/70B), we reveal three fundamental design principles: (i) static power consumption (DRAM leakage, refresh, and GPU idle power) can dominate the efficiency calculus, causing dynamic-only models to overestimate tokens/s/W by up to 3.85X for realistic deployments (Mamba2-2.7B, batch size 1, 128 input tokens, and 2,048 output tokens); (ii) decoding performance is monotonically non-decreasing with channel count across all evaluated models and workloads, generally plateauing at high channel counts for low-batch workloads; under a fixed-capacity sweep, all models instead share a common near-optimal hierarchy configuration, with substantially larger misconfiguration penalties for attention-based models; (iii) workload mapping strategies provide bounded improvements (up to 14.0%/17.4% kernel-level latency/energy reduction, up to 5.6% end-to-end gain) and are not primary bottlenecks. Significant efficiency gains require system-wide co-optimization. These principles provide design-space guidance for architects designing the next generation of memory-accelerated LLM systems.
Read the full abstract

All authors Corey Lammie, Hadjer Benmeziane, William Andrew Simon, Irem Boybat

Hardware Architecture

Heterogeneity-Aware Microscaling for Efficient Low-Bit LLM Inference

Junyi Luo, Xinting Jiang, Tai-Hao Wen et al. (+9 more)

Microscaling (MX) is now the standard for low-bit large language model (LLM) inference. Its 4-bit form MXFP4 still loses substantial accuracy, because existing MX formats fix either the element format or the precision-recovery scheme across blocks, and thus capture only limited quantization heterogeneity. Quantization heterogeneity appears at two levels: 1) across blocks, the preferred element format and precision-recovery scheme vary; 2) across operands, weights and activations require different encoding. We introduce AdaMX (Adaptive Microscaling), a heterogeneity-aware format and accelerator. It selects the precision-recovery scheme per block and the representation per operand, at no increase in equivalent bit width (EBW). One design covers two block sizes, giving a higher-accuracy operating point and a lower-EBW operating point that saves storage. We implement a 22nm FD-SOI AI accelerator prototype with the proposed decoder, computing unit, and quantization logic. Against an otherwise identical MXFP4 accelerator with FP4-only multipliers, AdaMX adds about 1% system energy. At the lower-EBW point, AdaMX stays more accurate than the baseline while lowering both memory footprint and energy. Across LLMs from 3B to 70B, AdaMX removes 83% of the MXFP4 accuracy loss on commonsense and 82% on MMLU, and 43% and 27% of the NVFP4 loss. AdaMX also generalizes to multimodal models. On Gemma-4 12B, it leads MXFP4 on all four vision-language benchmarks and keeps up to 96% of FP16 accuracy.
Read the full abstract

All authors Junyi Luo, Xinting Jiang, Tai-Hao Wen, Ruichen Qi, Minxing Chu, Hongyi Wu, Gregory Kielian, Ben Laurie, Qirui Zhang, Quan Cheng, Dennis Sylvester, Mehdi Saligane

Hardware Architecture

DiffPower: GPU-Accelerated Differentiable Switching Power Analysis and Optimization

Isaac Jacobson, Zheng Zhao, Rashmi Mehrotra et al. (+3 more)

Accurate and scalable switching power analysis remains a critical bottleneck in modern physical design, often forcing a trade-off between computational speed and modeling fidelity. We present DiffPower, a GPU-accelerated framework for differentiable power analysis and optimization. DiffPower translates design netlists into a PDK-agnostic bytecode representation, enabling analytical gradient computation via reverse-mode automatic differentiation, achieving up to a $1{,}002\times$ speedup over single-threaded CPU propagation on the largest evaluated design, with the GPU advantage growing with design scale. A hybrid propagation methodology fusing analytical modeling with parallel simulation achieves a median toggle-rate correlation of $r{=}0.96$ across ten industrial and benchmark designs. The resulting power gradients, computed up to $904\times$ faster than CPU finite-difference methods with near-perfect rank agreement, enable two downstream applications: (1) gradient-weighted cell sizing, which achieves up to $2.98\times$ improvement over local-power heuristics on industrial designs, with even stronger advantages at the 117K-cell scale where competing methods plateau; and (2) power virus generation via gradient ascent, which yields up to $2.13\times$ higher transition-weighted power, replacing a search process that traditionally requires hours.
Read the full abstract

All authors Isaac Jacobson, Zheng Zhao, Rashmi Mehrotra, Guanglei Zhou, Vineet Rashingkar, Yiran Chen

Hardware Architecture

Heterogeneous LLM Serving with General-Purpose Processing-Near-Memory for Retrieval-Based Sparse Attention

Hyungkyu Ham, Junhyeong Bae, Seungheon Lee et al. (+2 more)

This paper presents a heterogeneous decode-phase serving system that relocates the KV cache out of GPU memory, motivated by the retrieval-based sparse attention that recent frontier LLMs adopt to serve million-token contexts. It partitions a decode step by operation type: GPU nodes hold the model weights and execute the projections and MoE layers, while processing-near-memory (PNM) nodes hold the KV cache and index keys and execute every operation that reads them. We first show that the assumptions behind prior PIM and PNM designs no longer hold for these operations, and derive four design requirements for such a node. From these requirements, we propose KARAT (KV-cache-resident Accelerator for Retrieval-based ATtention), a general-purpose PNM design that is the design point meeting all four. A KARAT device combines large LPDDR capacity with general-purpose compute sized for the retrieval indexer, serving an operational intensity beyond what PIM/PNM designs built for low-intensity GEMV target while accommodating diverse sparse attention algorithms that fixed-function units cannot support as they evolve. To reduce pipeline bubbles as the two device types alternate between micro-batches, we further propose opportunistic, fine-grained micro-batch scheduling (OFMS), which hides expert all-to-all behind the other micro-batch's GEMMs, and context-length-aware micro-batch rebalancing (CMR), which equalizes their token counts despite the variance in context length. Across three state-of-the-art models and real agentic traces, our proposed system improves throughput per TDP under a service-level objective by 2.09-6.13x over a GPU-only baseline and runs training-free sparse attention methods with 1.36-3.21x improvements.
Read the full abstract

All authors Hyungkyu Ham, Junhyeong Bae, Seungheon Lee, Myeongjae Jeon, Gwangsun Kim

Hardware Architecture

ComFuse: Fusing Complex Memory-Intensive Subgraphs with Compute-Intensive Kernels For Modern GPU Architectures

Di Mu, Tengyuan Jin, Zhenkun Wang et al. (+6 more)

Modern deep learning workloads increasingly comprise heterogeneous computation graphs that combine compute-intensive operators with memory-intensive subgraphs. Existing deep learning compilers typically optimize these operator classes separately, creating rigid fusion boundaries that limit cross-operator optimization and on-chip data reuse. We observe that downstream memory-intensive operations can execute concurrently with compute-intensive operators, allowing their execution to be hidden behind computation; however, automatically exploiting this opportunity poses new compilation challenges. In this paper, we present ComFuse, an automated GPU compilation system that employs a novel operator fusion strategy to generate high-performance kernels for complex graph structures comprising compute-intensive operators and dependency-rich, memory-intensive elementwise-reduction subgraphs. ComFuse further supports the fusion of back-to-back GEMM (B2BGEMM) patterns, extending its applicability to more complex compute-memory interaction patterns. Additionally, it automatically lowers high-level tensor subprograms into optimized fused kernels, reducing the need for manual kernel engineering. Experimental results show that the fused kernels generated by ComFuse outperform those produced by TorchInductor across post-norm workloads and various complex computation scenarios, while supporting more flexible fusion patterns.
Read the full abstract

All authors Di Mu, Tengyuan Jin, Zhenkun Wang, Jialin Yang, Yusen Li, Mian Huo, Shusong Guo, Gang Wang, Xiaoguang Liu

Hardware Architecture

Beyond Peak TOPS/W: A System-Level Perspective on Hybrid Digital, Analogue and Neuromorphic Computing

Eiman Kanjo and Varuna De Silva

The digital revolution, which progressively replaced analogue methods with digital circuits, has entered a new phase as AI expands across cloud infrastructure, mobile networks, wearables and physical systems, including drones and robots. Digital computing remains the general-purpose foundation of this expansion: it supports heterogeneous, on-device and decentralised AI through programmable control, mature software and decades of accumulated engineering infrastructure. Yet as energy and data-movement constraints become more significant, that same foundation is increasingly being extended rather than replaced by selected analogue and physical principles that it can host, configure and verify. Photonic, in-memory and neuromorphic architectures offer routes to reducing data movement and accelerating matrix-intensive and event-driven processing, not as alternatives to digital infrastructure but as specialised engines operating within it. This paper argues that hybrid digital–analogue computing represents a credible pathway towards more energy-efficient AI systems: one in which physical substrates earn an expanding role only where they deliver a measurable system-level advantage, under digital orchestration that manages integration, uncertainty and fallback. It examines the architectural principles, workload suitability, energy accounting, software requirements, limitations and open challenges associated with this transition, and argues that future progress should be evaluated through deployed-system metrics rather than isolated peak tera operations per second per watt (TOPS/W) claims.
Read the full abstract

Hardware Architecture

Fovea: Physical-Implication-Aware Wafer-Scale DSE with Decision-Domain-Guided Cross-Fidelity Refinement

Jinxi Li, Huizheng Wang, Jinyi Deng et al. (+2 more)

Modern pre-silicon design-space exploration (DSE) follows a coarse-to-fine workflow: low-cost evaluators screen candidate spaces, while detailed evaluation is reserved for a shortlist. Wafer-scale systems strain both stages. Architecture choices induce coupled physical implications for reticle compliance, wafer tiling, die area, D2D capability, boundary access, and placement, so the design space cannot be treated as an unconstrained Cartesian product. Meanwhile, detailed evaluation is too expensive to cover the resulting space, whereas analytical-to-reference ranking inversions make a fixed shortlist unreliable. We present Fovea, a reusable methodology for workload-specific wafer architecture selection rather than a fixed wafer template. Fovea first performs physical-implication-aware design-space formulation to construct a distinct modeled-feasible space while preserving cross-dimensional trade-offs and applying only evaluator-preserving local reductions. It then performs Decision-Domain-guided cross-fidelity refinement. Paired in-domain calibration estimates workload- and space-specific analytical-to-reference disagreement, which parameterizes reference-consistent performance intervals and induces a Decision Domain for selective designated-reference evaluation. Under a valid domain-wide disagreement bound, this domain contains the designated-reference optimum; the sampling-based implementation is evaluated empirically on exhaustive-reference design spaces. Across ten reference-verifiable design spaces and seven LLM-training workloads, Fovea with 10% paired calibration recovers the exhaustive designated-reference optimum in all 70 evaluated pairs while achieving 4.13x average and 7.80x maximum end-to-end speedup.
Read the full abstract

All authors Jinxi Li, Huizheng Wang, Jinyi Deng, Yang Hu, Shouyi Yin

Hardware Architecture

Unified Lookup-Table Inference with Signed-Digit K/V Caches for Ternary LLMs

Ziang Duan, Jiajun Wu, Zetian Chen et al. (+8 more)

Ternary LLMs make their weight-dominated projections compact and efficient, but attention remains a mismatch: its K/V cache is created online and is typically processed by a separate higher-precision engine. Compressing this cache alone does not resolve the mismatch. To execute attention with the same lookup-table machinery as ternary projections, values accumulated in one reduction must retain a compatible representation and scale. This requirement also differs for keys and values during causal decoding, because newly generated values may belong to an unfinished cache block. This work develops a unified lookup-table inference approach for ternary LLMs. It stores runtime K/V states as scaled multi-plane signed digits organized around the reduction structure of attention. The resulting digit planes are consumed directly by activation-derived tables, avoiding dense K/V materialization between cache storage and attention computation. The design combines online K/V formation, bounded handling of incomplete value blocks, and a shared multi-stream datapath for Linear projections and attention. A constraint-guided search selects the representation and execution policy for a target quality–efficiency trade-off. Experiments on native and post-training ternary models validate the approach across cache capacity, model quality, and hardware efficiency.
Read the full abstract

All authors Ziang Duan, Jiajun Wu, Zetian Chen, Hao Song, Yanwen Deng, Zixuan Shen, Nuobei Xie, Simo Wu, Bolun Wang, Peng Zhou, Chao Wang

Hardware Architecture

Interpolation of Non-Linear Functions for LLMs using Partial Reconfiguration in FPGAs

Roger Morales-Monge, Nazareth Jimenez-Chacon, Jose Gabriel Villalobos-Alvarado et al. (+2 more)

Non-linear functions such as exponential and sigmoid are essential in AI and LLM acceleration, although implementing them efficiently on FPGAs is still costly. This paper proposes a PWL interpolation framework based on partial reconfiguration to reduce hardware cost while preserving flexibility. The architecture separates the design into a static region for communication and control, and a reconfigurable region where different interpolation modules can be dynamically loaded. Uniform and non-uniform segmentation strategies are evaluated for exponential and sigmoid functions using FP16 and FP32 arithmetic. Results show that non-uniform segmentation can improve accuracy in high-curvature regions, while uniform segmentation offers lower hardware overhead. At the system level, the reconfigurable implementation achieved significant area savings, reaching up to 43% less LUTs, 50% less flip-flops, BRAMs and DSPs cells, compared against a static design containing both operators; all this with predictable reconfiguration latency. These results show that partial reconfiguration is a practical approach for exploring area-latency trade-offs in FPGA-based acceleration of non-linear functions for LLM workloads.
Read the full abstract

All authors Roger Morales-Monge, Nazareth Jimenez-Chacon, Jose Gabriel Villalobos-Alvarado, Luis G. Leon-Vega, Jorge Castro-Godinez

Hardware Architecture

CAMTA: A Reconfigurable Multi-Region Activation Unit for Nonlinear Function Approximation

Carlos Soto-Porras, Jose Fonseca-Cruz, Pablo Ramirez-Morera et al. (+3 more)

Nonlinear activation functions are widely used in machine learning workloads, but their direct hardware implementation is often costly, function-specific, or difficult to reuse across different models. This work introduces CAMTA, a 16-bit reconfigurable multi-region activation unit for nonlinear function approximation in FPGA and ASIC accelerators. CAMTA combines independent region thresholds, per-region polynomial degrees, coefficient sets, and execution modes over a shared Horner-based datapath. Unlike conventional polynomial or piecewise approximation units that mainly reconfigure coefficients or segment selection, CAMTA also reconfigures the computational behavior of each region through HORNER, CONST, ZERO, and IDENTITY modes, enabling the same hardware to support functions with different symmetry and tail behavior without resynthesis. FPGA validation on an AMD Alveo platform shows RMSE as low as \(3.60\times10^{-6}\) for CAMTA-assisted Softmax, outperforming the CORDIC-based Softmax baseline considered in this work by nearly one order of magnitude. FPGA HLS synthesis reports 3 DSPs, 802 FFs, 1756 LUTs, and an 11-cycle datapath latency. ASIC synthesis in TSMC 65~nm at 250~MHz reports \(6632.40~\mu\mathrm{m}^2\) total cell area and \(1.3634~\mathrm{mW}\) total power. Compared with a same-node, function-specific PLAC implementation, CAMTA incurs \(2.20\times\) area and \(1.75\times\) power overhead, in exchange for runtime configurability and reuse across multiple nonlinear functions.
Read the full abstract

All authors Carlos Soto-Porras, Jose Fonseca-Cruz, Pablo Ramirez-Morera, Erick Obregon-Fonseca, Luis G. Leon-Vega, Jorge Castro-Godinez

Monday 3 August

Hardware Architecture

A Symbolic Execution Framework for Symbolic Timing Analysis of Digital Integrated Circuits

Dennis Eigner, Arman Ferdowsi and Ulrich Schmid

Simulation-based dynamic timing analysis of digital integrated circuits (DDTA) offers a faster alternative to traditional analog SPICE simulations. To achieve timing predictions that are reasonably competitive in terms of accuracy, however, DDTA mandates gate delay models that go beyond the standard pure or inertial delay models used in state-of-the-art tools. Recent advances in analytic gate delay models, which now also capture effects like drafting and multi-input switching, unlock new possibilities for timing analysis, which go way beyond simulation-based approaches towards an exhaustive exploration. In this paper, we present the cornerstones of a novel symbolic execution framework, which utilizes such analytic delay models for automatically computing symbolic delay expressions for all paths in a digital circuit, for some given ordering of the input transitions. To reduce combinatorial explosion, we introduce symbolic pruning methods that also enable path-sensitive, goal-driven reasoning about timing properties and analytic optimization of specific circuit paths.
Read the full abstract

Hardware Architecture

CheckOne: Lightweight Fault Detection and Mitigation for Vision Transformers

Mohammad Hasan Ahmadilivani, Sven-Markus Loorits and Jaan Raik

The wide adoption of Vision Transformers (ViTs) in safety-critical applications raises reliability concerns related to hardware faults. Algorithm-Based Fault Tolerance (ABFT) methods have emerged as lightweight and symmetric protection mechanisms for DNNs. However, they are particularly challenging for ViTs due to their significant computational requirements. This work comprehensively evaluates the reliability of ViTs, emphasizing the need for symmetric protection in their layers. Furthermore, we present CheckOne, a novel, cost-effective method for fault detection and mitigation in ViTs that significantly reduces the computational cost compared to conventional ABFT. Through extensive experiments with multiple ViTs, CheckOne mitigates critical faults by up to $26\times$ and achieves an average 3.8x higher performance than ABFT in ViTs.
Read the full abstract

Hardware Architecture

LowRank-SSM: Hardware-Software Co-Design for Rank-Reduced Mamba Acceleration on FPGA

Haocheng Xu, Yu-an Chou, Bhardwaj Bhat et al. (+6 more)

State Space Models (SSMs) such as Mamba and Mamba-2 achieve linear-time autoregressive inference, making them attractive for latency-sensitive and resource-constrained deployment. Yet every generated token still reads the full input and output projection weights of every layer from off-chip memory, and on an FPGA these two projections take the majority of a layer's cycles. Existing accelerators lower this cost by quantizing the weights or by speeding up the scan, but none reduce the number of weight elements a token must read, and a low-rank factorization on its own does not help: without a datapath built for it, the two-stage product takes as long as the dense one. We present LowRank-SSM, a hardware-software co-design framework that jointly addresses three constraints of rank-reduced Mamba inference: where rank should be spent, how the resulting factors should execute, and how reduced weight traffic should be delivered by the FPGA memory system. A sensitivity-guided per-projection search allocates rank under a perplexity ceiling while restricting compression to Mamba-2 operands that tolerate approximation. A rank-configurable projection datapath maps both factor stages onto the existing GEMV engines, keeping the intermediate vector on chip and changing rank through loop bounds and weight images rather than bitstream recompilation. A platform-aware NoC/AXI memory subsystem then balances and prefetches weight streams to sustain 42\,GB/s. Together, these software and hardware choices convert structural rank reduction into measured latency reduction: in our ablation, 93% of the projection traffic removed by rank reduction appears as saved token latency. With a rank schedule found under a 5% perplexity ceiling, which costs 1.6 points of average zero-shot accuracy, the complete accelerator decodes Mamba2-2.7B at 16.79 tokens/s and 0.41\,J per token.
Read the full abstract

All authors Haocheng Xu, Yu-an Chou, Bhardwaj Bhat, Zhiheng Chen, Ye Qiao, Yifan Zhang, Leyao Han, Saptarshi Mitra, Sitao Huang

Hardware Architecture

LACE: Large Language Model Aided Multi-Agent Framework for Agile RISC-V Instruction Extension

Pingqing Zheng, Jiayin Qin, Fuqi Zhang et al. (+5 more)

Domain-specific Instruction Set Architecture eXtensions (ISAX) are widely adopted in the RISC-V ecosystem to accelerate emerging workloads, but implementing and validating ISAXes across different cores remains slow and fragmented. Existing frameworks still require per-core interface adaptation, and differential testing often breaks once either the microarchitecture or the ISAX changes. We present LACE, an LLM-aided multi-agent workflow that translates natural-language ISAX intents into a compact two-level IR (operation-level and HDL task-level), performs retrieval-guided localized RTL edits over large repositories, and closes the loop with a compiler-agnostic riscv-formal checking flow (assuming RVFI availability or instrumentation). Across four embedded RISC-V cores, LACE raises pass@1 generation accuracy from near-zero to 72.8% within our evaluation setup, while improving code localization and reducing integration rework. The code of LACE is available at https://github.com/UMN-ZhaoLab/LACE.
Read the full abstract

All authors Pingqing Zheng, Jiayin Qin, Fuqi Zhang, Zishen Wan, Shang Wu, Yu Cao, Caiwen Ding, Yang Katie Zhao

Hardware Architecture · Cross-listed from cs.DB

Oasis: Hiding the Cost of Querying Parquet Files in the Datapath

Jonas Dann, Luca Tagliavini and Gustavo Alonso

Cloud-native database systems disaggregate compute and storage resources to improve cost efficiency over traditional monolithic architectures through elasticity and resource pooling. Studies of production data warehouse workloads show that scans (including round trips to storage) account for roughly half of total query runtime. Data lakes and lakehouses amplify this bottleneck through per-query decoding of storage-optimized, compressed file formats such as Parquet. As storage and network bandwidth continue to outpace CPU cost-performance, the CPU cycles spent on decoding increasingly undermine the cloud's cost-efficiency promise. This has led to a wave of specialization across the stack with custom hardware at cloud-vendor scale at the extreme end. We build on this trend and present Oasis, a data-processing SmartNIC that offloads Parquet decoding into the network datapath as a custom hardware accelerator. Oasis features a hardware decoder architecture, software abstraction layer, and end-to-end integration with DuckDB. Our evaluation shows that Oasis hides the cost of Parquet decoding behind the network datapath with minimal overhead, overlapping the scan with the remainder of the query execution. In the best case, this almost doubles DuckDB query throughput.
Read the full abstract

Hardware Architecture

DeGS: A Scalable 3DGS Architecture via Decoupled Workload Parsing and Reorganization

Minnan Pei, Gang Li, Zeyu Zhu et al. (+7 more)

3D Gaussian Splatting (3DGS) has emerged as a leading technique for real-time novel view synthesis, yet existing 3DGS accelerators suffer from poor architectural scalability: increasing the number of PEs leads to marginal performance improvement during rendering. We identify that the root cause is the tightly coupled “checking-while-blending” dataflow, which exacerbates PE underutilization caused by spatial redundancy from irregular Gaussian coverage and temporal redundancy from asynchronous pixel-wise termination under parallel execution. To address this issue, we propose DeGS, a scalable architecture for efficient 3DGS inference. To systematically eliminate the redundancies inherent in rendering, DeGS exploits a decoupled dataflow, restructuring the coupled $\alpha$-checking, transmittance checking, and $\alpha$-blending of the standard rendering process into consecutive workload parsing, reorganization, and blending stages. This allows the fragmented, length-variable, and temporal-dependent workloads to be reorganized into compact, conflict-free, and dense workloads prior to blending, thereby significantly improving PE utilization during parallel blending. Implemented in 28 nm technology, DeGS achieves 2.36$\times$–7.25$\times$ throughput, 1.82$\times$–6.02$\times$ end-to-end speedup, and 1.59$\times$–4.42$\times$ energy efficiency over state-of-the-art 3DGS accelerators (GSCore, GBU, GCC) across diverse scenes and resolutions (720p to 8K). Moreover, scaling from 16 to 1024 PEs, DeGS maintains over 80% PE utilization at high resolutions, significantly outperforming existing accelerators.
Read the full abstract

All authors Minnan Pei, Gang Li, Zeyu Zhu, Siting Wang, Junwen Si, Zhuoran Song, Yu Feng, Fangxin Liu, Xiaoyao Liang, Jian Cheng