The 2022 Â鶹´«Ã½AV East Asian School of Information Theory (EASIT) took place as a hybrid event fromÌýAugust 2 to 5, 2022 atÌýShenzhen Institute for Talents Development, co-organized by Â鶹´«Ã½AV Information Theory Society Guangzhou, Tsinghua Shenzhen International Graduate School (SIGS), Sun Yat-sen University and the Chinese University of Hong Kong (Shenzhen). Â鶹´«Ã½AV Information Theory SocietyÌýandÌýHuawei Technology Co., Ltd.Ìýare sponsoring this event. More than 100 students and researchers from about 26 Universities and Institutes registered for the school.
Â鶹´«Ã½AVÌýEASIT 2022 consisted of eight outstanding tutorialsÌýdelivered by distinguished lecturers and coveredÌýthe most cutting-edge research in information theory, including information theory and statistics, information theory and machine learning, coding for communications, and coding for storage.ÌýThe school started with a tutorial onÌýNeural Compression: Algorithms and Fundamental LimitsÌýpresented by theÌýÂ鶹´«Ã½AV Information Theory Society 2022 Goldsmith LecturerÌýProf.ÌýShirin Saeedi BidokhtiÌýfrom theÌýUniversity of Pennsylvania. Prof.ÌýBidokhti discussed the algorithm and theoretical advances of neural networkÌýcompression. The first part of her presentation was an introduction to lossless and lossy compression and their theoretical limits. Then, she discussed neural compression based on deep generative models from two perspectives. There are techniques to make the compression rate close to the entropy of the source for lossless compression. For lossy compression, a simulation of a rate-distortion function was presented. Later, she discussed theoretically how it ensures rate and distortion in one-shot lossy compression.
In the afternoon,ÌýProf. Erdal Arikan from Bilkent University gave the tutorial on the application of polar coding for high speed communication at Tb/s. In the next decade, such techniques are likely to have a positive impact on the next generation of communication. This is due to the fact that polar code is the first scheme that can demonstrate channel capacity. During this tutorial,ÌýProf. Arikan introduced the basic structure of the polar code and explained how this family of codes achieves the capacity of Binary Memoryless Symmetric channel. Such techniques can be future-embracing and reaching the channel capacity.
The next day opened with a tutorial byÌýProf. Alexander Barg from the University of Maryland.ÌýProf. Barg’s tutorial topic was Information Coding under Communication Constraints.ÌýProf. Barg overviewed the main problems in coding for distributed storage systems. His tutorial introduced some related methods and the associated results, including discussions on basic algebraic constructions of locally recoverable codes (LRC) and regenerating codes, problems of node recovery in systems where the connections between the nodes are constrained by a graph, and some new research directions.
In the afternoon of August 3rd, Prof. Xiaohu Tang from Southwest Jiaotong University presented his recent research on repairing maximum distance separable (MDS) code at a high rate. In actual big data storage systems, node failure is common. Therefore, it is very critical to design a reliable storage encoding scheme to recover data loss caused by node failure. As a result of their good properties, MDS codes are widely used in data storage systems under the backdrop of big data.ÌýProfessor Tang introduces different schemes - aiming to cope with different node failure cases. These efficient designs can be applied to data recovery in different situations and provide rich coding schemes to improve the reliability of storage systems.
The morning of August 4th,ÌýProf. Lizhong Zheng from Massachusetts Institute of Technology, gave us a tutorial on the machinery of simplified information geometry analysis and understanding deep learning with an information geometric method. When applying information theoretic analysis to machine learning problems, we oftenÌýfaceÌýthe difficulty of describing the relation between a number of different distributions. It is a very effective way to describe this complex situation in a geometric approach. At the beginning of the tutorial,ÌýProf. Zheng introduced several fundamental concepts in information theory including Kullback–Leibler divergence, Fisher information metrics, i-projection, information vector, and canonical dependence matrix. Later, Prof. Zheng discussed some learning theory applications of information geometric method in the analysis of strong data processing inequality, generalization error, and model selection. He also discussed more applied problems such as understanding deep neural networks, transfer learning, and multimodal learning.
On the afternoon of August 4th,ÌýProf. Yao Xie from Georgia Institute of Technology, gave the tutorial on some recent advances in modern hypothesis tests. Hypothesis testing is an essential building block for machine learning and signal processing problems. In this tutorial,ÌýProf. Yao Xie introduced: i) robust hypothesis tests utilizing modern optimization, as well as its application in classification and domain adaptation; ii) sequential hypothesis test and change point detection in multi-sensor sparse/subspace/robust setting, as well as methods utilizing deep learning that exploit low-dimensional structure; iii) continuous-time Hawkes network estimation/change point detection and discrete-time Hawkes network convex estimation. Such techniques enable us to leverage deep learning by developing efficient testing tools for modern data, a principled validation tool and a theoretical foundation for a deep learning model.
On theÌýmorningÌýof August 5th,ÌýProf. Tie Liu from Texas A&M University gave a tutorial about how to interpret the two fundamentalÌýproblems in statistical data analysis and learning theory, i.e., concentration and generalization. In the first part,ÌýProf. Liu focused on the concentration of a general function of independent variables. He presented the entropy method for leveraging various notions of functional stability into exponential tail bounds. In the second part,ÌýProf. Liu focused on the generalization of a data-dependent query and proposes a systematic approach for relating it to various information-theoretic notions of algorithm stability.
On the afternoon of August 5th,ÌýProf. Jinhong Yuan from The University of New South Wales presented a class of spatially coupled codes, namely partially information coupled (PIC) and partially parity coupled (PPC). There are two main characteristics of this class of codes. First, the code rate can be flexibly adjusted by varying the coupling ratio. Secondly, component encoders and decoders can adopt those off-the-shelf.ÌýProfessor Yuan first introduced the background and concept of spatially coupled codes, and then introduced the main contents of this report in four parts: (1) Construction methods for PIC turbo codes and PPC turbo codes; (2) Generalized spatially coupled parallel concatenated codes (GSC-PCCs); (3) A new family of spatially coupled product codes called sub-block rearranged staircase codes; (4) Construction methods for PIC LDPC codes and PIC polar codes. These spatially coupled codes are compatible with current standards such that the underlying component code encoding and decoding can be kept uncharged. This improves the transmit spectrum and power efficiency.
The school also included poster sessions on days oneÌýthrough threeÌýof the event.ÌýA total of 60 student posters were received, and 20 were presented on-site. Poster authors are encouraged to communicate freely with other participants during the poster presentation. The organizing committee selected five posters from the 60 received posters for The Best Poster Awards on the last day before the summer school ended. Guodong Li et al. (Shandong University), Yuan Li et al. (Institute of Mathematics and Systems Science, Chinese Academy of Sciences), Chenhao Jin et al. (Sun Yat-sen University),ÌýLijia YangÌýet al. (Sun Yat-sen University),ÌýCheng Du et al. (Fudan University), were awarded student best poster awards for their poster content and presentation.
Our sincere thanks are extended to all lecturers who contributed to the development of science and education. We are also grateful to theÌý Â鶹´«Ã½AV Information Theory Society and Huawei Technology Inc. for the sponsorship of the whole event.Ìý
We would also like to thank all participants who attended despite the severe conditions of the epidemic. The epidemic forces many students to attend the conference online this year, but we hope to see you all at future events.
Ìý