Novel Quantum Interior Point Methods with Iterative Refinement for Linear and Semidefinite Optimization

Tamás Terlaky

Lehigh University

Abstract: Quantum Interior Point Methods (QIPMs) for linear and semi-definite optimization (LO and SDO) problems build on classic polynomial time IPMs. Quantum Computing (QC) inspired to design Inexact Infeasible and Inexact Feasible Primal-Dual, and Inexact Dual IPM variants. These are novel algorithms in both the QC and classic computing environments. Enhancing Quantum Interior Point Methods (QIPMs) with Iterative Refinement (IR) leads to exponential improvements in the worst-case overall running time of QIPMs, compared to previous best-performing QIPMs. We also discuss how the proposed IR scheme can be used in classical inexact IPMs with conjugate gradient methods. Further, the proposed IR scheme exhibits quadratic convergence for LO and SDO towards an optimal solution without any assumption on problem characteristics. On the practical side, IR can be useful to find precise solutions while using inexact LO and SDO solvers.

Deep MIMO Detection via Homotopy Optimization

Wing Kin Ma

The Chinese University of Hong Kong

Abstract: Most recently there has been interest in using deep learning to tackle MIMO detection. In particular, the focus has been on deep unfolding, whose idea is to take structures from model-based signal processing methods to build highly-efficient deep nets. The speaker will present one such method. The idea is to apply deep unfolding to a theory-driven non-convex optimization method, which is easy to realize by algorithms such as the projected gradient algorithm. We will cover how the method makes sense by intuition, how theory provides an explanation of the intuition, and how the method works in practice. If time permits, the speaker will also cover extensions to one-bit MIMO detection and the expectation maximization method.

Surprises in Network Information Theory

Wei Yu

University of Toronto

Abstract: Information theory aims to characterize the fundamental limits of compressing, representation, and transmission of information in communication networks. For instance, to identify one of N distinct objects, log(N) bits of information are needed. Moreover, the maximum rate of information transmission through a noisy channel is approximately the logarithm of the signal-to-noise ratio. In this talk, we explore several unusual scenarios where information theoretical analyses yield surprising results. We ask the following questions: 1) Is it possible to transmit information through a noisy channel at a strictly positive rate using only infinitesimal amount of transmit power? 2) For transmitting independent messages to a subset of K active devices among a large pool of N devices, is it possible to avoid using an address field of size log(N) bits to identify the intended recipient of each message? The answers to both questions are surprisingly, yes! We discuss the implications of these results to cooperative communications and to massive random access in wireless networking.

Distributed baseband signal processing for large-scale antenna array systems

Tsung-Hui Chang

The Chinese University of Hong Kong, Shenzhen
Shenzhen Research Institute of Big Data

Abstract: Traditional baseband signal processing algorithms in conventional massive multiple-input multiple-output (MIMO) systems typically rely on a centralized baseband processing (CBP) architecture, where raw baseband signals from all antennas are aggregated at a central computing unit for further processing. However, as the number of base station (BS) antennas increases, the CBP architecture faces two major bottlenecks: 1) excessive fronthaul communication bandwidth between antennas and the computing unit, and 2) high computational complexity at the central processing unit.

To overcome these challenges, distributed baseband processing (DBP) has been proposed, which divides BS antennas into multiple independent antenna clusters, each connected to a baseband unit (BBU). These BBUs are interconnected via star or daisy-chain networks. Under the DBP architecture, both inter-BBU communication bandwidth and the computational capacity of individual BBUs are significantly constrained. Consequently, traditional signal processing tasks—such as channel estimation, downlink precoding, and uplink equalization—require redesign and the development of new distributed signal processing techniques. In this presentation, we will introduce the latest advancements in these areas and outline promising directions for future research.

How good is selfish routing in highly congested transportation networks?

Rolf H. Möhring

Berlin University of Technology

Abstract: The price of anarchy (PoA) is a standard measure to quantify the inefficiency of equilibria in congestion games. This is particularly important in transportation networks, where the PoA indicates how well the available network capacity is used. We investigate the PoA under varying transportation demands in arbitrary (atomic and non-atomic) congestion games and show that it converges to 1 very fast with growing total demand for a large class of cost functions, and regardless of which strategies (pure or mixed) a user chooses. This implies that the selfish choice of routes is the best one can do in highly congested transportation networks.

The lecture will be elementary and explain all concepts.

The material presented is based on joint work with Zijun Wu, Yanyan Chen, Dachuan Xu, and Chunying Ren

Scientific Understanding of Neural Networks: From Representation to Learning Dynamics and From Shallow to Deep

Hongkai Zhao

Duke University

Abstract: In this talk I will present both mathematical and numerical analysis as well as experiments to understand a few basic computational issues in using neural networks, as a particular form of nonlinear representation, and show how the network structure, activation function, and parameter initialization can affect its approximation properties and the learning process. In particular, we propose a structured and balanced approximation using multi-component and multi-layer neural network (MMNN) structure. Using sine as the activation function and an initial scaling strategy, we show that scaled Fourier MMNNs (SFMMNN) have a distinct adaptive property as a nonlinear approximation. Computational examples will be presented to verify our analysis and demonstrate the efficacy of our method.

At the end, I will raise a few issues and challenges when using neural networks in scientific computing.

Adam-family Methods with Decoupled Weight Decay in Deep Learning

Kim-Chuan Toh

National University of Singapore

Abstract: We investigate the convergence properties of a general class of Adam-family methods for minimizing quadratically regularized nonsmooth nonconvex optimization problems, especially in the context of training nonsmooth neural networks with weight decay. Motivated by AdamW, we propose a novel framework for Adam-family methods with decoupled weight decay. Within our framework, the estimators for the first-order and second-order moments of stochastic subgradients are updated independently of the weight decay term. Under mild assumptions and with non-diminishing stepsizes for updating the primary optimization variables, we establish the convergence properties of our proposed framework. In addition, we show that our framework encompasses a wide variety of well-known Adam-family methods, hence offering convergence guarantees for these methods in the training of nonsmooth neural networks. As a practical application of our framework, we propose a method named AdamD (Adam with Decoupled Weight Decay). Numerical experiments demonstrate that AdamD outperforms Adam and is comparable to the popular AdamW, in both the aspects of generalization performance and efficiency.

[Based on joint work with Kuangyu Ding and Nachuan Xiao]

Inverse Problems for Stochastic Diffusion Equations

Peijun Li

Chinese Academy of Sciences

Abstract: Stochastic diffusion equations are essential in modeling complex systems across a variety of fields, including physics, geophysics, finance, biology, and engineering. In this talk, I will present our recent mathematical and computational studies on inverse problems for stochastic diffusion equations driven by temporal, spatial, and space-time random noise. These noise types introduce significant challenges, influencing both the theoretical foundations and computational strategies used to solve these problems. The talk will focus on theoretical aspects such as uniqueness and stability of solutions, and the development of computational methods for recovering key parameters from noisy observations. Finally, I will outline some future directions for research in stochastic inverse problems, exploring emerging challenges and opportunities for innovation.

Sidecar: A structure-preserving framework for solving PDEs with neural networks

Zhonghua Qiao

The Hong Kong Polytechnic University

Abstract: Neural network (NN) solvers for partial differential equations (PDE) have been widely used in simulating complex systems in various scientific and engineering fields. However, most existing NN solvers mainly focus on satisfying the given PDEs, without explicitly considering intrinsic physical properties such as mass conservation or energy dissipation. This limitation can result in unstable or nonphysical solutions, particularly in long-term simulations. To address this issue, we propose Sidecar, a novel framework that enhances the accuracy and physical consistency of existing NN solvers by incorporating structure-preserving knowledge. This framework builds upon our previously proposed TDSR-ETD method for solving gradient flow problems, which satisfies discrete analogues of the energy-dissipation laws by introducing a time-dependent spectral renormalization (TDSR) factor. Inspired by this approach, our Sidecar framework parameterizes the TDSR factor using a small copilot network, which is trained to guide the existing NN solver in preserving physical structure. This design allows flexible integration of the structure-preserving knowledge into various NN solvers and can be easily extended to different types of PDEs. Our experimental results on a set of benchmark PDEs demonstrate that it improves the existing neural network solvers in terms of accuracy and consistency with structure-preserving properties.

Biosignals as the key to individualizing AI

Tanja Schultz

Universität Bremen

Abstract: I will describe AI technologies and systems that automatically adapt to users’ needs by interpreting their biosignals: Human behavior includes physical, mental, and social actions that emit a range of biosignals which can be captured by a variety of sensors. The processing and interpretation of such biosignals provides an inside perspective on human physical and mental activities, complementing the traditional approach of merely observing human behavior. As great strides have been made in recent years in integrating sensor technologies into ubiquitous devices and in machine learning methods for processing and learning from data, I argue that the time has come to harness the full spectrum of biosignals to understand individual user needs. I will present illustrative cases of our work ranging from silent and neural communication interfaces that convert muscle and brain signals directly into audible speech, to interpretation of human attention and decision making in human-robot interaction from multimodal biosignals.

AV-CrossNet: An Audiovisual Complex Spectral Mapping Network for Speech Separation

DeLiang Wang

The Chinese University of Hong Kong, Shenzhen

Abstract: Audition and vision complement each other for perception. Adding visual cues to audio-based speech separation can improve separation performance. This presentation introduces AV-CrossNet, an audiovisual (AV) system for speech enhancement, target speaker extraction, and multi-talker speaker separation. AV-CrossNet is extended from the TF-CrossNet architecture, which is a recently proposed deep neural network (DNN) that performs complex spectral mapping for speech separation by leveraging global attention and positional encoding. Complex spectral mapping trains a DNN to directly estimate the real and imaginary spectrograms of the target signal from those of a noisy mixture. To effectively utilize visual cues, the proposed system incorporates pre-extracted visual embeddings and employs a visual encoder comprising temporal convolutional layers. Audio and visual features are fused in an early fusion layer before feeding to AV-CrossNet blocks. We evaluate AV-CrossNet on multiple open datasets, including LRS, VoxCeleb, TCD-TIMIT, and COG-MHEAR challenge. Evaluation results demonstrate that AV-CrossNet advances the state-of-the-art performance in all audiovisual tasks, even on untrained and mismatched datasets.