Functions of Random Variables

Probability Density Functions:
Cumulative Distributions:
Probability Spaces
A formal definition of probability involves the specification of:

· a sample space
The sample space, , is the set of all possible outcomes, plus the null outcome. Each element in  is called a sample point.
· a field (or algebra)
The field, or algebra, is a set of subsets of  closed under complementation and union (recall Venn diagrams).
· and a probability measure.
A probability measure obeys these axioms:
1.  (implies probabilities less than one)
2. 
3. For two mutually exclusive events:


Two events are said to be statistically independent if:

The conditional probability of B given A is:

Hence,

Important Probability Density Functions

Uniform (Unix rand function):

Gaussian:

Laplacian (speech signal amplitude, durations):

Gamma (durations):


We can extend these concepts to N-dimensional space. For example:

Two random variables are statistically independent if:

This implies:
     and     
Expectations and Moments

The statistical average of a scalar function, , of a random variable is:
     and     
The central moment is defined as:

The joint central moment between two random variables is defined as:

We can define a correlation coefficient between two random variables as:

We can extend these concepts to a vector of random variables:



What is the difference between a random vector and a random process?

What does wide sense stationary mean? strict sense stationary?

What does it mean to have an ergodic random process?

How does this influence our signal processing algorithms?
Correlation and Covariance of WSS Random Processes

For a signal, , we can compute the following useful quantities:

Autocovariance:

If a random process is wide sense stationary:

Hence, we define a very useful function known as the autocorrelation:

If  is zero mean WSS:

What is the relationship between the autocorrelation and the spectrum:

For a linear time-invariant system, :

The notion of random noise is central to signal processing:
white noise?
Gaussian white noise?
zero-mean Gaussian white noise?
colored noise?

Therefore, we now embark upon one of the last great mysteries of life:
How do we compare two random vectors?



Probability of Events: