TU Delft
Education Type
Education print this page print this page     
2016/2017 Electrical Engineering, Mathematics and Computer Science Master Computer Science
Random Signal Processing
Responsible Instructor
Name E-mail
Prof.dr. A. Hanjalic    A.Hanjalic@tudelft.nl
Dr. D.M.J. Tax    D.M.J.Tax@tudelft.nl
Contact Hours / Week x/x/x/x
Education Period
Start Education
Exam Period
Course Language
Expected prior knowledge
This course builds heavily on Probability and Statistics, Mathematics (especially integration and differentiation), Signal Processing and Transformations (especially linear systems and signals, Fourier analysis).
Course Contents
Signal processing plays a major role in many applications, like consumer electronics (mp3 player, mobile telephony, CD player, (HD)TV), radar and medical applications.

In this course we will discuss fundamental signal processing principles, methods and algorithms. The course consists of two parts: Part I: Stochastic Processes (given in Q1) and Part II: Digital Signal Processing (given in Q2).

The first part of the course introduces the concept of stochastic models and random processes, while the second part of the course focusses on deterministic signal processing. The stochastic models are used for describing systems and signals that are not deterministic. In fact, no single system or signal is deterministic in practice. For that reason the theory of stochastics and random processes should be considered as a useful extension of known approaches for describing and modeling systems and signals encountered in engineering practices.

Stochastic models will be developed on the basis of probability theory. Probability theory describes the behavior of certain phenomena in terms of how likely it is that certain values will occur. Central features of the models will be discussed are random variables, probability density functions, and the expected value operator. In describing random processes and signals, the correlation function and conditional probabilities play a central role. The part on deterministic signal processing will cover the representations of signals and systems in time-, frequency (Fourier) en Z-domains and the foundations of mathematical models for signal analysis and processing based on these representations.

The course addresses the following subjects:
1. Refresher random variables. Matlab exercise on estimation of PDF, expected value and variance.
2. Refresher correlation. Calculating with correlation functions.
3. Random processes, correlation function, stationarity, wide sense stationarity, estimation of correlation function (Matlab exercise).
4. Random signal processing, power spectral density function, white noise.
5. AR processes, linear prediction: theory and Matlab exercise.
6. Markov chains.
7. Refresher (discrete-time) signals and systems.
8. Z-transform.
9. Fourier transforms.
10. Sampling and reconstruction of signals.

Study Goals
1. Probability Theory
- Conditional) probabilities, the law of total probability, and Bayes’ rule.
- Solve probability problems that require the use of axioms of probability.

2. Definition and Description of Random Variables and Processes
PDF, PMF, CDF, Covariance, Correlation- Determine if a given PDF, PMF, CDF, variance, (auto/cross-)correlation(-function), (auto/cross-)covariance(-function), power spectral density complies with (theoretical and analytical) requirements.
- Convert the description of a probabilistic problem into a probabilistic model using PDF, PMF, or CDF.

3. PDF/PMF and Expected Value
Calculate the various forms of expected value of (combinations of) random variables and random processes
- For a given (amplitude continuous/discrete and time continuous/discrete) probability model calculate the following probabilistic (marginal, joint and conditional) characterizations: PDF, PMF, CDF, probability of an event, expected value, variance, covariance, correlation, correlation coefficient, auto/crosscorrelation function, auto/crosscovariance function, (cross) power spectral density.
- Calculate the PDF, PMF, expected value and variance of a derived random variable.

4. Properties of Random Processes
- Independence, orthogonality, uncorrelated, whiteness, IID- Determine if random variables/processes have the following properties: independent, orthogonal, uncorrelated, white, Poisson, Gaussian, Bernoulli, Markov, IID, stationary, WSS, ergodic.
- Calculate the expected value, variance, auto/crosscorrelation(function), auto/crosscovariance(function), power spectral density of a linear combination of random variables and of a linearly filtered (WSS, amplitude discrete/continuous, time discrete/continuous) random process.

5. Large NumbersCentral limit theorem, law of large numbers
- Solve problems that require the use of the central limit theorem in an engineering context
- Explain the law of the large numbers in an engineering context.

6. Statistical Estimators
- Estimated mean, variance, and correlation function
- Given a set of outcomes, sample functions or realizations, calculate estimators for expected value, variance, and (auto-)correlation function.

7. Application to Engineering Problems and Simulations
- Select and translate a simple electrical engineering or computer science problem into mathematical probability model. The emphasis is on problems in signal and image processing, telecommunication, and media and knowledge technology. The class of probability models encompasses the following random variables/processes: Bernoulli, exponential, binomial, Poisson, Gaussian, uniform.
- Justify and reflect on the approach taken in calculating or simulating (MatLab) the following probabilistic properties: PDF, PMF, expected value, variance, autocorrelation function, autocovariance function.

8. Signals and Systems
- Signal representation, linear time invariant (LTI) systems, impulse response, convolution, causality, difference equations, recursive and non-recursive systems, stability.

9. Z-transform
- Properties of Z-transform, region of convergence, rational transfer functions, inverse Z-transform, system analysis in the Z-domain, poles, zeros, stability.

10. Fourier Transforms
- Fourier series, continuous-time Fourier transform, discrete-time Fourier transform, discrete Fourier transform, Fast Fourier transform (FFT), properties of Fourier transforms, frequency-domain characterization of LTI sytems.

11. Sampling and Reconstruction of Signals
- sampling theorem, sampling frequency, aliasing, folding, interpolation, D/A and A/D conversion.

Education Method
Lectures, working groups (problem solving), laboratory work (Matlab exercises)
For Part I: Workload is around 15 hours for attending lectures, 5 hours of reading study material and preparing lectures, 15 hours for the lab course, 20 hours for preparing the exam, 3 hours for the exam, and 8 hours for a final report (66 hours in total).
Literature and Study Materials
J.G. Proakis/D.G. Manolakis, Digital Signal Processing: Principles, Algorithms and Applications, 4th Edition, Prentice Hall 2007, 1084 pp., ISBN 0-13-187374-1.

R.D. Yates and D.J. Goodman, "Probability and Stochastic Processes: A Friendly Introduction for Electrical and Computer Engineers", ISBN 0-471-17837-3, John Wiley and Sons, New York, 2005, Second Edition.
The two parts will be examined separately, and both need to be passed.
Exams for both parts are closed book, but students are allowed to bring a self made double A4 formula sheet.

Before students can take part in the exam for part I, the students have to hand in pass the final laboratory exercise.