Ian, I really do think we are almost saying the same thing. Let me try to
clarify.
You say that the Gaussian model is not the "correct" data model, and that the
Poisson is correct. I more-or-less agree. If I were being pedantic (me?) I would say that
the Poisson is *more* physically realistic than the Gaussian, and more realistic in a very
important and relevant way --- but in truth the Poisson model does not account for other
physical sources of error that arise from real crystals and real detectors, such as dark
noise and read noise (that's why I would prefer a gamma distribution). I also agree that
for x>10 the Gaussian is a good approximation to the Poisson. I basically agree with
every point you make about the Poisson vs the Gaussian, except for the following.
The Iobs=Ispot-Iback equation cannot be derived from a Poisson assumption, except as
an approximation when Ispot > Iback. It *can* be derived from the Gaussian
assumption (and in fact I think that is probably the *only* justification it has).
It is true that the difference between two Poissons can be negative. It is also true
that for moderate # of counts, the Gaussian is a good approximation to the Poisson.
But we are trying to estimate Itrue, and both of those points are irrelevant to
estimating Itrue when Ispot < Iback. Contrary to your assertion, we are not
concerned with differences of Poissonians, only sums. Here is why:
In the Poisson model you outline, Ispot is the sum of two Poisson variables,
Iback and Iobs. That means Ispot is also Poisson and can never be negative.
Again --- the observed data (Ispot) is a *sum*, so that is what we must deal
with. The likelihood function for this model is:
L(a) = (a+b)^k exp(-a-b)
where 'k' is the # of counts in Ispot, 'a' is the mean of the Iobs Poisson (i.e., a =
Itrue), and 'b' is the mean of the Iback Poisson. Of course k>=0, and both
parameters a>0 and b>0. Our job is to estimate 'a', Itrue. Given the likelihood
function above, there is no valid estimate of 'a' that will give a negative value. For
example, the ML estimate of 'a' is always non-negative. Specifically, if we assume 'b'
is known from background extrapolation, the ML estimate of 'a' is:
a = k-b if k>b
a = 0 if k<=b
You can verify this visually by plotting the likelihood function (vs 'a' as
variable) for any combination of k and b you want. The SD is a bit more
difficult, but it is approximately (a+b)/sqrt(k), where 'a' is now the ML
estimate of 'a'.
Note that the ML estimate of 'a', when k>b (Ispot>Iback), is equivalent to
Ispot-Iback.
Now, to restate: as an estimate of Itrue, Ispot-Iback cannot be derived from the
Poisson model. In contrast, Ispot-Iback *can* be derived from a Gaussian model
(as the ML and LS estimate of Itrue). In fact, I'll wager the Gaussian is the
only reasonable model that gives Ispot-Iback as an estimate of Itrue. This is why
I claim that using Ispot-Iback as an estimate of Itrue, even when Ispot<Iback,
implicitly means you are using a (non-physical) Gaussian model. Feel free to
prove me wrong --- can you derive Ispot-Iback, as an estimate of Itrue, from
anything besides a Gaussian?
Cheers,
Douglas
On Sat, Jun 22, 2013 at 12:06 PM, Ian Tickle <ianj...@gmail.com> wrote:
On 21 June 2013 19:45, Douglas Theobald <dtheob...@brandeis.edu> wrote:
The current way of doing things is summarized by Ed's equation:
Ispot-Iback=Iobs. Here Ispot is the # of counts in the spot (the area
encompassing the predicted reflection), and Iback is # of counts in the
background (usu. some area around the spot). Our job is to estimate the true
intensity Itrue. Ed and others argue that Iobs is a reasonable estimate of
Itrue, but I say it isn't because Itrue can never be negative, whereas Iobs can.
Now where does the Ispot-Iback=Iobs equation come from? It implicitly assumes
that both Iobs and Iback come from a Gaussian distribution, in which Iobs and
Iback can have negative values. Here's the implicit data model:
Ispot = Iobs + Iback
There is an Itrue, to which we add some Gaussian noise and randomly generate an Iobs. To
that is added some background noise, Iback, which is also randomly generated from a
Gaussian with a "true" mean of Ibtrue. This gives us the Ispot, the measured
intensity in our spot. Given this data model, Ispot will also have a Gaussian
distribution, with mean equal to the sum of Itrue + Ibtrue. From the properties of
Gaussians, then, the ML estimate of Itrue will be Ispot-Iback, or Iobs.
Douglas, sorry I still disagree with your model. Please note that I do
actually support your position, that Ispot-Iback is not the best estimate of
Itrue. I stress that I am not arguing against this conclusion, merely (!) with
your data model, i.e. you are arriving at the correct conclusion despite using
the wrong model! So I think it's worth clearing that up.
First off, I can assure you that there is no assumption, either implicit or
explicit, that Ispot and Iback come from a Gaussian distribution. They are both
essentially measured photon counts (perhaps indirectly), so it is logically
impossible that they could ever be negative, even with any experimental error you
can imagine. The concept of a photon counter counting a negative number of photons
is simply a logical impossibility (it would be like counting the coins in your
pocket and coming up with a negative number, even allowing for mistakes in
counting!). This immediately rules out the idea that they are Gaussian. Photon
counting where the photons appear completely randomly in time (essentially as a
consequence of the Heisenberg Uncertainly Principle) obeys a Poisson distribution.
In fact we routinely estimate the standard uncertainties of Ispot & Iback on
the basis that they are Poissonian, i.e. using var(count) = count. That is hardly
a Gaussian assumption for t!
