Re: [Ifeffit] Ifeffit Digest, Vol 170, Issue 21

Matthew Marcus Fri, 21 Apr 2017 16:12:59 -0700

Suppose the data were tabulated locally on a 0.01A^-1 grid, then sampled on the 
default 0.05A grid with cubic-spline interpolation.  In that case, the spline 
interpolated value
depends mostly on the couple of samples nearest the new grid points, which 
means that you're not really using all the data equally.  I suppose it could be 
argued, then, that
smoothing distributes some of the weight of the 'in-between' points so that 
they're accounted for, the way boxcar averaging does.


What my code does (original idea frpm Brian Kincaid) is a linear weighting, 
essentially a triangular convolution kernel rather than a rectangle, as in 
boxcar averaging.
Think of it as A-frame averaging rather than boxcar.  That's sharper, so would 
result in less loss of amplitude in the high shells.

If you have a distance of 5A, then the phase shift between .05A^-1 gridpoints 
is 0.5rad.  At the top of a wiggle, you're essentially averaging cos(phi) over 
the range [-0.25,0.25],
which gives you an amplitude of sin(0.25)/0.25 = 0.98.  Thus, I don't 
understand how boxcar averaging makes a noticeable difference here except for 
really long paths.

For a triangular average over a phase range +-delta, the relevant amplitude 
reduction is

f = Integrate[Cos[phi]*(1 - phi/delta), {phi, 0, delta}]/Integrate[(1 - 
phi/delta), {phi, 0, delta}] (Mathematica notation, using symmetry of the 
functions)

which comes to f = 2(1-Cos(delta))/delta^2, whose leading term is 1-delta^2/12. 
 Thus, to lose 10%
of the amplitude, delta would have to be about 1.12rad, which means a phase 
shift between grid points of 2.24rad, thus a distance of 22A.

A caveat:  This derivation only covers the phase.  Presumably, an amplitude 
varying with k would cause a phase shift of the averaged wave.  I haven't 
worked out
how bad that might be.

The triangular kernel assigns the full weight of each input point to some 
combination of the new grid points on each side, so nobody's left out.
        mam

On 4/21/2017 3:36 PM, Christopher Thomas Chantler wrote:

�� I'm just agreeing with Matt's comment.


Too fine a spacing does not help you due to typical broadening from imfp or 
thermal sources [or structural disorder].


And the difference between operation of ifeffit and larch in this context is 
pretty interesting.


--------------------------------------------------------------------------------------
Christopher Chantler, Professor, FAIP, Fellow American Physical Society
Editor-in-Chief, Radiation Physics and Chemistry
Chair, International IUCr Commission on XAFS
President, International Radiation Physics Society
School of Physics, University of Melbourne
Parkville Victoria 3010 Australia
+61-3-83445437 FAX +61-3-93474783
chant...@unimelb.edu.au 
<https://owa.unimelb.edu.au/owa/redir.aspx?C=c7BoS0kVVkC1_S95-9x9l5cBu6YTjdAITgSrfUpfDAUV5oUH1LFYBcz08w8xvHMJoosZRdagfQM.&URL=mailto%3achantler%40unimelb.edu.au>
 chant...@me.com 
<https://owa.unimelb.edu.au/owa/redir.aspx?C=c7BoS0kVVkC1_S95-9x9l5cBu6YTjdAITgSrfUpfDAUV5oUH1LFYBcz08w8xvHMJoosZRdagfQM.&URL=mailto%3achantler%40me.com>
http://optics.ph.unimelb.edu.au/~chantler/xrayopt/xrayopt.html 
<https://owa.unimelb.edu.au/owa/redir.aspx?C=c7BoS0kVVkC1_S95-9x9l5cBu6YTjdAITgSrfUpfDAUV5oUH1LFYBcz08w8xvHMJoosZRdagfQM.&URL=http%3a%2f%2foptics.ph.unimelb.edu.au%2f%7echantler%2fxrayopt%2fxrayopt.html>
http://optics.ph.unimelb.edu.au/~chantler/home.html 
<https://owa.unimelb.edu.au/owa/redir.aspx?C=c7BoS0kVVkC1_S95-9x9l5cBu6YTjdAITgSrfUpfDAUV5oUH1LFYBcz08w8xvHMJoosZRdagfQM.&URL=http%3a%2f%2foptics.ph.unimelb.edu.au%2f%7echantler%2fhome.html>

------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
*From:* Ifeffit <ifeffit-boun...@millenia.cars.aps.anl.gov> on behalf of 
ifeffit-requ...@millenia.cars.aps.anl.gov <ifeffit-requ...@millenia.cars.aps.anl.gov>
*Sent:* Friday, 21 April 2017 11:01:11 PM
*To:* ifeffit@millenia.cars.aps.anl.gov
*Subject:* Ifeffit Digest, Vol 170, Issue 21

Send Ifeffit mailing list submissions to
        ifeffit@millenia.cars.aps.anl.gov

To subscribe or unsubscribe via the World Wide Web, visit
        
https://protect-au.mimecast.com/s/krZOBlimO6EJCD?domain=millenia.cars.aps.anl.gov 
<http://millenia.cars.aps.anl.gov/mailman/listinfo/ifeffit>
or, via email, send a message with subject or body 'help' to
        ifeffit-requ...@millenia.cars.aps.anl.gov

You can reach the person managing the list at
        ifeffit-ow...@millenia.cars.aps.anl.gov

When replying, please edit your Subject line so it is more specific
than "Re: Contents of Ifeffit digest..."


Today's Topics:

   1. Re: Ifeffit Digest, Vol 170, Issue 19 (Matt Newville)
   2. Fwd:  Ifeffit Digest, Vol 170, Issue 19 (Matt Newville)
   3. Looking for EXAFS data of Ni2+ ion in aqueous solution (Van Vu)


----------------------------------------------------------------------

Message: 1
Date: Thu, 20 Apr 2017 21:18:39 -0500
From: Matt Newville <newvi...@cars.uchicago.edu>
To: XAFS Analysis using Ifeffit <ifeffit@millenia.cars.aps.anl.gov>
Subject: Re: [Ifeffit] Ifeffit Digest, Vol 170, Issue 19
Message-ID:
        <ca+7esbpewenmlpspnyaxt4j2t1lkjaayua59xcgeblptekt...@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"

Hi Chris,

On Thu, Apr 20, 2017 at 5:02 PM, Christopher Thomas Chantler <
chant...@unimelb.edu.au> wrote:

Note that in general any and every smoothing operation reduces the
information content of the data and its ability to reveal structure.


Well, maybe.  If one has mu(E) measured every 0.01 eV over an 600 eV EXAFS
scan to k=12A^-1, one does not really 60,000 independent measures of the
structure.  Yuji's data wasn't that absurd, but it did have close to 4000
measurements for the full EXAFS spectrum out to k=18^-1.  And to be clear,
there is not anything wrong with that, it's just a matter of how you decide
to treat it.

As the plots attached in Yuji's original message and my replies show, the
resulting chi(k) definitely has is amplitude suppressed when doing a simple
boxcar average of data onto the 0.05 Ang^-1 grid (Athena with Ifeffit).
But when using cubic spline interpolation (Larch), the amplitude of the
EXAFS oscillations are not noticeably suppressed, though the high frequency
noise is also much higher.  Applying a Savitzky-Golay filter prior to the
cubic spline interpolation did not appreciably suppress the EXAFS
oscillations though the high frequency noise was reduced.

Very finely-spaced energy data might reveal is at much higher R than we can
hope to model with EXAFS.  A k-grid of 0.05 Ang^-1 can give frequencies to
31Ang, and so is probably accurate to 16Ang without significant signal
loss.   That is, out to k=18Ang^-1, you really only need 360 samples, but
you'd like these as noise-free as possible.  Having 3600 measurements on a
grid of 0.005 Ang^-1  might give you data out to 160Ang in principle, but
the photo-electron tends to not cooperate.

--Matt
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://protect-au.mimecast.com/s/pLGZBoiY2NEkCW?domain=millenia.cars.aps.anl.gov 
<http://millenia.cars.aps.anl.gov/pipermail/ifeffit/attachments/20170420/f835d4df/attachment-0001.html>>

------------------------------

Message: 2
Date: Thu, 20 Apr 2017 22:33:14 -0500
From: Matt Newville <newvi...@cars.uchicago.edu>
To: XAFS Analysis using Ifeffit <ifeffit@millenia.cars.aps.anl.gov>
Subject: [Ifeffit] Fwd:  Ifeffit Digest, Vol 170, Issue 19
Message-ID:
        <ca+7esbopsw1p-9hzyyq+yobhcch++cohocyg2z-gexadn3l...@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"

Hi Matthew,


On Thu, Apr 20, 2017 at 5:34 PM, Matthew Marcus <mamar...@lbl.gov> wrote:

Does Athena use a histogramming method for Fourier filtering?



The autobk() in Ifeffit does.  That is, the background mu0(E) and chi(E)
are found at all input energy points.  This is converted to chi(k) at k for
all energy points, and then a boxcar average is applied to give chi(k) on
the standard 0.05 k-grid.  As shown earlier, this diminishes the noise and
also the amplitude of chi(k), especially at higher frequencies.

In Larch, the cubic spline for the background mu0(E)  is found at all input
energy points, but then evaluated at the values on the uniform k-grid.
.

That's what I use.  The idea is that to grid the data we don't interpolate
but take averages over the data appearing within the bin,



 OK, but that boxcar average will blur the data.    I definitely see this
with QXAFS (well "continuous") where I can scan continuously but bin the
measurements into 0.05Ang-1 bins -- that *does* reduce the amplitude
compared to binning into 0.02 Ang^-1 bins and compared to step scans, where
one measures at fixed energies, not slewing over an energy range.

The data from SPRing8 was much finer spaced than EXAFS needs -- and boxcar
averaging to 0.05 Ang^-1 does blur the data.

with interpolation only when there aren't any points within a bin. For
those, you have to bridge across a gap.  This is the best idea I've come up
with for using data which may be tabulated more finely than the k-grid of
the Fourier filtering process (typically dk=0.05A^-1).


Putting the spline through all the energy points does use all the data to
predict the value on the 0.05 Ang^-1 grid, and avoids blurring the data.
Yes, the result can be noisier, but it does not suppress the oscillations.

Something I've used in a XANES context but never tried for EXAFS is a
convolution with a kernel whose width depends on energy, such that it
matches the sharpest credible feature.  See Manceau, A., Marcus, M. A.,
Lenoir, T. (2014) Estimating the number of pure chemical components in a
mixture by X-ray absorption spectroscopy. J. Synchrotron Radiat.
21,1140-1147,
specifically the SI.



I've read that paper, but I don't think I looked at the SI.  Will do.

The notion is that no real feature can be narrower than the combination of
the core-hole lifetime and instrumental broadening or the EXAFS wiggle
corresponding to the maximum reasonable path length.  This is done
by transforming the data from E-space to a space in which a constant step
in the abscissa corresponds to this energy-dependent minimum credible
feature width.  Of course, in EXAFS this is mostly a constant width in k,
so some kind of smoothing would work if it's a constant kernel in k.



 That's sort of what a Fourier filter does, for example zero all the data
above R>10 and transform back to k.

S-G smoothing assumes uniform tabulation so unless your data were taken on

a uniform k-grid, it doesn't really do the right thing.


The S-G smoothing does appear to help even if the data are not strictly
uniform (see example posted earlier where the data was very finely spaced
in energy, but not uniformly).    I did forget about the uniformity
requirement, but I think that for the purposes here (smooth very fine
slightly noisy data before fitting the cubic spline) it's probably not a
disaster.

I would agree that this filter is designed to reduce high-frequency noise
is largely cosmetic, and is not really going to help reduce the noise in
the frequency range of the data we actually care about.  In fact, for me
the main point would be that the S-G filter does *not* greatly alter the
frequencies we care about, whereas boxcar averaging most certainly does.

A problem with this method is that it drops off the very information you
need to see what your noise floor is.



Yes, that's a good point.  Removing the obvious high-frequency noise gives
the impression that there is no low-frequency noise, which is not correct.

I put this out there only for those who insist on smoothing.  I use my
data un-smoothed for EXAFS analysis, knowing that the treatment of noise
and sampling finer than the k-bin is not really right.

I don't really see the usefulness of smoothing for XANES or EXAFS
analysis, though it might be OK for display if not overdone.


Yes, I agree -- not smoothing chi(k) is the best approach.  But if you do
smooth, use Savitzky-Golay on the uniform chi(k).


--
--Matt
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://protect-au.mimecast.com/s/Ld1ZBmS8pMZwun?domain=millenia.cars.aps.anl.gov 
<http://millenia.cars.aps.anl.gov/pipermail/ifeffit/attachments/20170420/2cafb5d5/attachment-0001.html>>

------------------------------

Message: 3
Date: Fri, 21 Apr 2017 20:01:05 +0700
From: Van Vu <vuxxx...@umn.edu>
To: XAFS Analysis using Ifeffit <ifeffit@millenia.cars.aps.anl.gov>
Subject: [Ifeffit] Looking for EXAFS data of Ni2+ ion in aqueous
        solution
Message-ID:
        <caedz6xa66edg1-badhbnjvzwqsbgd8rs3jmca7wzmhysne-...@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"

Dear everyone,
I am looking for XAS data of Ni2+ ion in aqueous solution that contains
EXAFS data in the k range of 0-12 (Angstrom-1) or so for an undergraduate
student project. I was wondering if anyone would be willing to share the
data with me?
Many thanks in advance.
Best,
Van

On Fri, Apr 21, 2017 at 5:02 AM, Christopher Thomas Chantler <
chant...@unimelb.edu.au> wrote:

Note that in general any and every smoothing operation reduces the
information content of the data and its ability to reveal structure. Also,
many of the smoothing algorithms change the data point values at vertices,
so change the data prior to analysis.


Hence in general avoid unless you know exactly the physical cause
requiring smoothing.


Best wishes


------------------------------------------------------------
--------------------------
Christopher Chantler, Professor, FAIP, Fellow American Physical Society
Editor-in-Chief, Radiation Physics and Chemistry
Chair, International IUCr Commission on XAFS
President, International Radiation Physics Society
School of Physics, University of Melbourne
Parkville Victoria 3010 Australia
+61-3-83445437 FAX +61-3-93474783
chant...@unimelb.edu.au
<https://owa.unimelb.edu.au/owa/redir.aspx?C=c7BoS0kVVkC1_S95-9x9l5cBu6YTjdAITgSrfUpfDAUV5oUH1LFYBcz08w8xvHMJoosZRdagfQM.&URL=mailto%3achantler%40unimelb.edu.au>
chant...@me.com
<https://owa.unimelb.edu.au/owa/redir.aspx?C=c7BoS0kVVkC1_S95-9x9l5cBu6YTjdAITgSrfUpfDAUV5oUH1LFYBcz08w8xvHMJoosZRdagfQM.&URL=mailto%3achantler%40me.com>
http://optics.ph.unimelb.edu.au/~chantler/xrayopt/xrayopt.html
<https://owa.unimelb.edu.au/owa/redir.aspx?C=c7BoS0kVVkC1_S95-9x9l5cBu6YTjdAITgSrfUpfDAUV5oUH1LFYBcz08w8xvHMJoosZRdagfQM.&URL=http%3a%2f%2foptics.ph.unimelb.edu.au%2f%7echantler%2fxrayopt%2fxrayopt.html>
http://optics.ph.unimelb.edu.au/~chantler/home.html
<https://owa.unimelb.edu.au/owa/redir.aspx?C=c7BoS0kVVkC1_S95-9x9l5cBu6YTjdAITgSrfUpfDAUV5oUH1LFYBcz08w8xvHMJoosZRdagfQM.&URL=http%3a%2f%2foptics.ph.unimelb.edu.au%2f%7echantler%2fhome.html>

------------------------------
*From:* Ifeffit <ifeffit-boun...@millenia.cars.aps.anl.gov> on behalf of
ifeffit-requ...@millenia.cars.aps.anl.gov <ifeffit-request@millenia.
https://protect-au.mimecast.com/s/GN1VBvt6aeV8I1?domain=cars.aps.anl.gov 
<http://cars.aps.anl.gov>>
*Sent:* Friday, 21 April 2017 3:00:01 AM
*To:* ifeffit@millenia.cars.aps.anl.gov
*Subject:* Ifeffit Digest, Vol 170, Issue 19

Send Ifeffit mailing list submissions to
        ifeffit@millenia.cars.aps.anl.gov

To subscribe or unsubscribe via the World Wide Web, visit
        
https://protect-au.mimecast.com/s/krZOBlimO6EJCD?domain=millenia.cars.aps.anl.gov 
<http://millenia.cars.aps.anl.gov/mailman/listinfo/ifeffit>
https://protect-au.mimecast.com/s/YZ52BmizXox7tA?domain=millenia.cars.aps.anl.gov 
<http://millenia.cars.aps.anl.gov>
<https://protect-au.mimecast.com/s/krZOBlimO6EJCD?domain=millenia.cars.aps.anl.gov 
<http://millenia.cars.aps.anl.gov/mailman/listinfo/ifeffit>>
or, via email, send a message with subject or body 'help' to
        ifeffit-requ...@millenia.cars.aps.anl.gov

You can reach the person managing the list at
        ifeffit-ow...@millenia.cars.aps.anl.gov

When replying, please edit your Subject line so it is more specific
than "Re: Contents of Ifeffit digest..."


Today's Topics:

   1. Re: The difference of EXAFS oscillation when using Athena and
      when using Larch (Yuji Mahara)


----------------------------------------------------------------------

Message: 1
Date: Thu, 20 Apr 2017 19:47:20 +0900
From: "Yuji Mahara" <mahara.y...@e.mbox.nagoya-u.ac.jp>
To: "'XAFS Analysis using Ifeffit'"
        <ifeffit@millenia.cars.aps.anl.gov>
Subject: Re: [Ifeffit] The difference of EXAFS oscillation when using
        Athena  and when using Larch
Message-ID: <003501d2b9c3$7c75c560$75615020$@e.mbox.nagoya-u.ac.jp>
Content-Type: text/plain; charset="utf-8"

Dear Matt



Thank you for your reply.

I confirmed that chi (k) becomes smooth by the Savitzky-Goley method by
doing the attached lar file.



I will try on various conditions from now.

I may also ask if there is something I do not understand.

Thank you for your help again!



Bests,

Yuji Mahara





From: Ifeffit [mailto:ifeffit-boun...@millenia.cars.aps.anl.gov
<ifeffit-boun...@millenia.cars.aps.anl.gov>] On Behalf Of Matt Newville
Sent: Thursday, April 20, 2017 5:26 AM
To: XAFS Analysis using Ifeffit <ifeffit@millenia.cars.aps.anl.gov>
Subject: Re: [Ifeffit] The difference of EXAFS oscillation when using
Athena and when using Larch



Hi Yuji,

The attached larch script (a modification of your script but note: not
python, so you may have to add a bunch of '_larch=mylarch' arguments) to
show the effects of smoothing with Savitzky-Golay and of interpolating onto
a "classic XAFS energy grid".   You might want to play around with this
different options.  It definitely seems that none of these methods are
systematically over-smoothing the data and reducing the amplitudes of the
oscillations -- one more reason to use Larch instead of Ifeffit.  But it
also seems like smoothing this data with a Savitzky-Golay fitler is helpful.

Also attached are plots showing k- and R-space generated from the attached
script.



--Matt

-------------- next part --------------
An HTML attachment was scrubbed...
URL: 
<http://millenia.cars.aps.anl.gov/pipermail/ifeffit/attachments/20170420/c04024db/attachment-0001.html
https://protect-au.mimecast.com/s/YZ52BmizXox7tA?domain=millenia.cars.aps.anl.gov 
<http://millenia.cars.aps.anl.gov>
<https://protect-au.mimecast.com/s/K41rBvSE4XrOHl?domain=millenia.cars.aps.anl.gov 
<http://millenia.cars.aps.anl.gov/pipermail/ifeffit/attachments/20170420/c04024db/attachment-0001.html>>
>

------------------------------

Subject: Digest Footer

_______________________________________________
Ifeffit mailing list
Ifeffit@millenia.cars.aps.anl.gov
https://protect-au.mimecast.com/s/krZOBlimO6EJCD?domain=millenia.cars.aps.anl.gov 
<http://millenia.cars.aps.anl.gov/mailman/listinfo/ifeffit>
https://protect-au.mimecast.com/s/YZ52BmizXox7tA?domain=millenia.cars.aps.anl.gov 
<http://millenia.cars.aps.anl.gov>
<https://protect-au.mimecast.com/s/krZOBlimO6EJCD?domain=millenia.cars.aps.anl.gov 
<http://millenia.cars.aps.anl.gov/mailman/listinfo/ifeffit>>
Unsubscribe: 
https://protect-au.mimecast.com/s/e4MdB8SG64E7tY?domain=millenia.cars.aps.anl.gov 
<http://millenia.cars.aps.anl.gov/mailman/options/ifeffit>
https://protect-au.mimecast.com/s/YZ52BmizXox7tA?domain=millenia.cars.aps.anl.gov 
<http://millenia.cars.aps.anl.gov>
<https://protect-au.mimecast.com/s/e4MdB8SG64E7tY?domain=millenia.cars.aps.anl.gov 
<http://millenia.cars.aps.anl.gov/mailman/options/ifeffit>>


------------------------------

End of Ifeffit Digest, Vol 170, Issue 19
****************************************


_______________________________________________
Ifeffit mailing list
Ifeffit@millenia.cars.aps.anl.gov
https://protect-au.mimecast.com/s/krZOBlimO6EJCD?domain=millenia.cars.aps.anl.gov 
<http://millenia.cars.aps.anl.gov/mailman/listinfo/ifeffit>
Unsubscribe: 
https://protect-au.mimecast.com/s/e4MdB8SG64E7tY?domain=millenia.cars.aps.anl.gov 
<http://millenia.cars.aps.anl.gov/mailman/options/ifeffit>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://protect-au.mimecast.com/s/W91wBaiw968bcp?domain=millenia.cars.aps.anl.gov 
<http://millenia.cars.aps.anl.gov/pipermail/ifeffit/attachments/20170421/d9fbaf12/attachment.html>>

------------------------------

Subject: Digest Footer

_______________________________________________
Ifeffit mailing list
Ifeffit@millenia.cars.aps.anl.gov
https://protect-au.mimecast.com/s/krZOBlimO6EJCD?domain=millenia.cars.aps.anl.gov 
<http://millenia.cars.aps.anl.gov/mailman/listinfo/ifeffit>
Unsubscribe: 
https://protect-au.mimecast.com/s/e4MdB8SG64E7tY?domain=millenia.cars.aps.anl.gov 
<http://millenia.cars.aps.anl.gov/mailman/options/ifeffit>


------------------------------

End of Ifeffit Digest, Vol 170, Issue 21
****************************************



_______________________________________________
Ifeffit mailing list
Ifeffit@millenia.cars.aps.anl.gov
http://millenia.cars.aps.anl.gov/mailman/listinfo/ifeffit
Unsubscribe: http://millenia.cars.aps.anl.gov/mailman/options/ifeffit


_______________________________________________
Ifeffit mailing list
Ifeffit@millenia.cars.aps.anl.gov
http://millenia.cars.aps.anl.gov/mailman/listinfo/ifeffit
Unsubscribe: http://millenia.cars.aps.anl.gov/mailman/options/ifeffit

Re: [Ifeffit] Ifeffit Digest, Vol 170, Issue 21

Reply via email to