ELL714: Difference between revisions
| [checked revision] | [checked revision] |
Prashantt492 (talk | contribs) Creating course page via bot |
Bot: wrap bare course codes in wikilinks |
||
| Line 4: | Line 4: | ||
| credits = 3 | | credits = 3 | ||
| credit_structure = 3-0-0 | | credit_structure = 3-0-0 | ||
| pre_requisites = ELL205 | | pre_requisites = [[ELL205]] | ||
| overlaps = | | overlaps = | ||
}} | }} | ||
Latest revision as of 16:31, 14 April 2026
| ELL714 | |
|---|---|
| Basic Information Theory | |
| Credits | 3 |
| Structure | 3-0-0 |
| Pre-requisites | ELL205 |
| Overlaps | |
ELL714 : Basic Information Theory
Introduction to entropy, relative entropy, mutual information, fundamental inequalities like Jensen's inequality and log sum inequality. Proof of asymptotic equipartition property and its usage in data compression. Study of entropy rates of the stochastic process following Markov chains. Study of data compression: Kraft inequality and optimal source coding. Channel capacity: symmetric channels, channel coding theorem, Fano's inequality, feedback capacity. Differential entropy. The Gaussian channel: bandlimited channels, channels with colored noise, Gaussian channels with feedback. Detailed study of the rate-distortion theory: rate distortion function, strongly typical sequences, computation of channel capacity. Joint source channel coding/separation theorem. There are no laboratory or design activities involved with this course.