25128: Information and Coding Theory
Course Name: Information and Coding Theory
Course Number: 25128
Prerequisite(s): 25111 (Communication Systems 1)
Co-requisite(s): -
Units: 3
Level: Postgraduate
Last Revision: Fall 2012
Description:
Syllabus:
References:
Course Number: 25128
Prerequisite(s): 25111 (Communication Systems 1)
Co-requisite(s): -
Units: 3
Level: Postgraduate
Last Revision: Fall 2012
Description:
This courses is an introduction to Shannon and information theory concepts, such as entropy, AEP, mutual information, Stationary and ergodic sources and optimal codes, and Shannon theory and information theory applications, including source coding, channel capacity, Data networks.
Syllabus:
- Introduction, Information Measurement (Mutual Information, Entropy)
- Asymptalic Equiporthies Property (AEP), Stationary and Ergodic Sources, Stationary Entropy and Entropy of Markov Sources
- Data Compression (Source Coding), Uniquely Decodable Codes, Instantaneous Codes, Optimal Codes (Huffman Codes), Shannon's First Theorem, Sub-Optimal Codes
- Discrete Memoryless Channels (DMC), Channel Capacity, Special Channels, Shannon's Second Theorem (The fundamental Theorem of Information Theory), Fano’s Inequality and the Converse to the Coding Theorem
- Gaussian Channels, Capacity, Shannon's Second Theorem, Parallel Channels, Feedback Channels
- Network Information Theory
- Typical Sequences
- Two-Way Channel (TWC), Interference Channel (IFC)
- Multiple-Access Channels (MAC)
- Encoding of Correlated Sources and Duality with Slepian–Wolf
- Broadcast Channel (BC)
- Relay Channel
- Source Coding with Side Information
- General Information Networks
References:
- T. M. Cover, J. Thomas, Elements of Information Theory, John Wiley, 2006
- R. Ash, Information Theory, John Wiley, 1965
- R. Gallager, Information Theory and Reliable Communication, John Wiley, 1968
- I. Csiszar, J. Korner, Information Theory: Coding Theorems for Discrete Memoryless Systems, Academic Press, 1981
- R. Yeung, A First Course in Information Theory, Kluwer Academic, 2002
- C. E. Shannon, A Mathematical Theory of Communication, Bell Tech. J., 1948
- D. Slepian, Key Papers in the Development of Information Theory, IEEE, 1974
- E. Van Der Meulen, A Survey of Multiway channels in Information Theory: 1961-1976, IEEE Trans. Inform. Theory, 1977
- S. Verdu, Fifty Years of Shannon Theory, IEEE Trans. Inform. Theory, 1998
- T. M. Cover, Comments on Broadcast Channels, IEEE Trans. Inform. Theory, 1998
- G. Kramer, Topics in Multi-user Information Theory, Foundation and Trends in Communications and Information Theory, 2008
- A. El Gamal and Y.-H. Kim, Lecture Notes on Network Information Theory, Cambridge Press, 2012
Last Update: 2024-07-07