>there is nothing *motivating* us to define Rx[k] = E{x[n] x[n+k]} except
that we
>expect that expectation value (which is an average) to be the same as the
other definition
Sure there is. That definition gets you everything you need to work out a
whole list of major results (for example, optimal linear predictors and how
they relate the the properties of the probabilistic model) without any
reference to statistics. You get all the insights into how everything fits
together, and then you move on to the extra wrinkles that arise when
dealing in statistical estimates of the quantities in question.
To relate this back to the OP: Ross gave us a probabilistic description of
a random process, from which we can work out the autocorrelation and psd
without any reference to ergodicity, or any signal realizations to compute
sample autocorrelations on.
>otherwise, given the probabilistic definition, why would we expect the
Fourier Transform of Rx[k] = E{x[n] x[n+k]} to be the power spectrum?
In the modern context, that is the *definition* of the power spectral
density. The question of whether any particular statistic converges to it
is a separate question, considered after setting up the underlying
probabilistic models and relationships. You sort out what the underlying
quantity is, and only then do you consider how well a particular statistic
is able to approach it.
>by definition, **iff** it's ergodic, then the statistical estimate (by
that you mean the average over time)
>converges to the probabilistic expectation value. if it's *not* ergodic,
then you don't know that they are the same.
Right, that's the definition of ergodicity. This seems phrased as a
disagreement or criticism but I'm not seeing the issue?
I certainly agree that autocorrelation and power spectral density are of
limited utility in the context of non-ergodic processes. And even more so
for non-stationary processes. But they're still well-defined (well, not so
much psd for non-WSS processes, but autocorrelation is perfectly general).
>what you call the "statistical estimate" is what i call the "primary
definition".
Right.
>well, it's not just random processes that have autocorrelations.
deterministic signals have them too.
Deterministic signals are a subset of random processes. The probabilistic
treatment is a generalization of the deterministic case. It's overkill if
you only want to deal with deterministic signals, but in the general case
it's all you need.
>your first communications class (the book i had was A.B. Carlson) started
out with probability and stochastic processes???
My first communications class required as a prerequisite an entire course
on random processes. Which in turn required as a prerequisite yet another
entire course on basic probability and statistics. So there were two entire
courses of prob/stat/random processes pre-reqs before you get to day 1 of
communications systems.
Not sure what Carlson looked like in your time, but the modern editions do
a kind of weird parallel-track thing in this area. He does deterministic
signals first, and uses the same definitions as you. Then halfway through
he switches to random signals, and defines autocorrelation and psd directly
in terms of expected values as I describe. So it's "one definition for
deterministic case, another for random case," and then some paragraphs
bringing up the concept of ergodicity and how it bridges the two cases. The
way that the Carlson pedagogy would approach the OP - where we were given
an explicit description of a random signal - is in probabilistic terms
using definitions of acf and psd in terms of expected value.
E
On Wed, Nov 11, 2015 at 5:02 PM, robert bristow-johnson <
r...@audioimagination.com> wrote:
>
>
>
> ---------------------------- Original Message ----------------------------
> Subject: Re: [music-dsp] how to derive spectrum of random sample-and-hold
> noise?
> From: "Ethan Duni" <ethan.d...@gmail.com>
> Date: Wed, November 11, 2015 7:36 pm
> To: "robert bristow-johnson" <r...@audioimagination.com>
> "A discussion list for music-related DSP" <music-dsp@music.columbia.edu>
> --------------------------------------------------------------------------
>
> >>no. we need ergodicity to take a definition of autocorrelation, which we
> > are all familiar with:
> >
> >> Rx[k] = lim_{N->inf} 1/(2N+1) sum_{n=-N}^{+N} x[n] x[n+k]
> >
> >>and turn that into a probabilistic expression
> >
> >> Rx[k] = E{ x[n] x[n-k] }
> >
> >>which we can figger out with the joint p.d.f.
> >
> >
> > That's one way to do it. And if you're working only within the class of
> > stationary signals, it's a convenient way to set everything up. But it's
> > not necessary. There's nothing stopping you from simply defining
> > autocorrelation as r(n,k) = E(x[n]x[n-k]) at the outset.
>
>
>
> well there's nothing stopping us from defining autocorrelation as Rx[k] =
> 5 (for all k). but such a definition is not particularly useful.
>
>
>
> there is nothing *motivating* us to define Rx[k] = E{x[n] x[n+k]} except
> that we expect that expectation value (which is an average) to be the same
> as the other definition, which is what we use in all of this deterministic
> Fourier signal theory we start with in communication systems. otherwise,
> given the probabilistic definition, why would we expect the Fourier
> Transform of Rx[k] = E{x[n] x[n+k]} to be the power spectrum? you get to
> that fact long before any of this statistical communications theory.
>
>
>
> > You then need (WS)
> > stationarity to make that a function of only the lag, and then ergodicity
> > to establish that the statistical estimate of autocorrelation (the sample
> > autocorrelation, as it is commonly known) will converge,
>
>
>
> to *what*? by definition, **iff** it's ergodic, then the statistical
> estimate (by that you mean the average over time) converges to the
> probabilistic expectation value. if it's *not* ergodic, then you don't
> know that they are the same.
>
>
>
> > but you can ignore
> > it if you are just dealing with probabilistic quantities and not worrying
> > about the statistics.
>
>
>
> we got some semantic differences here. by "statistical estimate" i know
> you're referring to the same result that is what we get in our first
> semester communications (long before statistical communications) as the
> **definition** of autocorrelation. what you call the "statistical
> estimate" is what i call the "primary definition".
>
>
>
> >>i totally disagree. i consider this to be fundamental (and it's how i
> > remember doing statistical communication theory back in grad school).
> >
> >
> > That was a common approach in classical signal processing
> > literature/cirricula, since you're typically assuming stationarity at the
> > outset anyway. And this approach matches the historical development of
> the
> > concepts (people were computing sample autocorrelations before they
> squared
> > away the probabilistic interpretation). But this is kind of a historical
> > artifact that has fallen out of favor.
>
>
> in an electrical engineering communications class? are you sure?
>
>
>
> see, they gotta teach these kids things about signals and Fourier and
> spectra and LTI and the like so they have a concept of what that stuff is
> about *without* necessarily bringing into the conversation probability,
> random variables, p.d.f., and random processes. pedagogically, i am quite
> dubious that this *historical artifact* is not how they teach statistical
> communications even now. i know my vanTrees and Wozencraft&Jacobs books
> are old, but this is timeless and classical. i doubt it has fallen out of
> favor.
>
>
> >
> > In modern statistical signal processing contexts (and the wider prob/stat
> > world) it's typically done the other way around: you define all the
> random
> > variables/processes up front, and then define autocorrelation as r(n,k) =
> > E(x[n]x[n-k]).
>
>
>
> well, it's not just random processes that have autocorrelations.
> deterministic signals have them too.
>
>
>
> and pedagocially, i can't imagine teaching statistical signal processing
> before teaching the fundamentals of signal processing.
>
>
>
> > Once you have that all sorted out, you turn to the question
> > of whether the corresponding statistics (the sample autocorrelation
> > function for example) converge, which is where the ergodicity stuff comes
> > in.
>
>
>
> yes.
>
>
>
> > The advantage to doing it this way is that you start with the most
> > general stuff requiring the least assumptions, and then build up more and
> > more specific results as you add assumptions. Assuming ergodicity at the
> > outset and defining everything in terms of the statistics produces the
> same
> > results for that case, but leaves you unable to say anything about
> > non-stationary signals, non-ergodic signals, etc.
> >
> >
> > Leafing through my college books, I can't find a single one that does it
> > the old way.
>
>
>
> you must be a lot younger than me.
>
>
>
> >They all start with definitions in the probability domain, and
> > then tackle the statistics after that's all set up.
> >
>
> your first communications class (the book i had was A.B. Carlson) started
> out with probability and stochastic processes??? i can't imagine teaching
> undergraduates what they need to know (ya know, stuff like what AM and FM
> and SSB and QAM and QPSK is) in one semester doing it that way.
> statistical communications was a subsequent graduate course, for me.
>
>
>
>
> --
>
>
>
>
> r b-j r...@audioimagination.com
>
>
>
>
> "Imagination is more important than knowledge."
>
> _______________________________________________
> dupswapdrop: music-dsp mailing list
> music-dsp@music.columbia.edu
> https://lists.columbia.edu/mailman/listinfo/music-dsp
>
_______________________________________________
dupswapdrop: music-dsp mailing list
music-dsp@music.columbia.edu
https://lists.columbia.edu/mailman/listinfo/music-dsp
