@Jaidev Deshpande The IPython notebook explains it well. But to summarize, the idea is to use SVR prediction to expand the signal such that it is actually symmetric at the edges. This enables you to mirror the signal and spline fit the extrema such that there will be no edge effects, as you would cut the mirrored signal back to normal size (as described in this paper http://link.springer.com/article/10.1007%2FBF02839453)
@Kyle Kastner I will have a look at using a window function but currently I've only been trying to use ideas found in published papers for improving the EMD process as I lack the necessary background to come up with my own ideas. There are several "hacks" for getting around the edge effects as they come from the spline fitting extrema points of the signal, but this method seems the most elegant but the authors SVR code seems to work very well and that alone I've had issues replicating. I guess this is my main issue. Also, is the kernel in the paper the same one in scikit-learn? The data I created was just a simple case study. But you can not assume it will be periodic but the idea was to make the edges of the signal mirror-able. I can change it to use the data I am looking at, which is the summation of the cross-sectional area of magnetic objects on the Sun, it does not look as pretty however! On 27 March 2014 14:57, Kyle Kastner <[email protected]> wrote: > Also, to answer your question about searching for C and gamma, look at > GridSearchCV and friends. > http://scikit-learn.org/stable/modules/generated/sklearn.grid_search.GridSearchCV.html > > The whole grid_search module may be worth looking at for your needs - I am > assuming your cost to score will be something like RMSE? > > > On Thu, Mar 27, 2014 at 9:52 AM, Kyle Kastner <[email protected]>wrote: > >> This may be an obvious question - but did you try applying a simple >> Hamming, Blackman-Harris, etc. window to the data? Before trying EMD? >> >> Pretty much every transform (FFT included) has edge effect problems if >> the signal is not exactly at a periodic boundary, and it sounds like the >> SVR prediction would be used to create a kind of "custom" window function >> for very strange data, but the mirroring process is still assuming it is >> periodic in some way (by basically wrapping the function, the predicting >> that) >> >> I don't know enough about EMD to know whether you are supposed to window >> or not, but the slides I just glanced through definitely had tapers at the >> edges. You may also try moving the "black region" forward until it reaches >> 0 again - this looks like the natural periodic point of your data, and may >> greatly improve your prediction even though it is kind of cheating... >> unless it is always possible to find good "periodic points" and use those >> (maybe by measuring cyclostationarity/autocorrelation?) >> >> Also, is this testing data a good representation of your real dataset? It >> looks EKG-ish to my eyes. >> >> This is cool stuff - thanks for sharing. EMD seems worth investigating... >> >> Kyle >> >> >> >> On Thu, Mar 27, 2014 at 8:53 AM, Jaidev Deshpande < >> [email protected]> wrote: >> >>> >>> >>> >>> On Thu, Mar 27, 2014 at 7:16 PM, Nabil Freij <[email protected]>wrote: >>> >>>> Hey, >>>> >>>> I've been attempting to create an Empirical Mode Decomposition (EMD) >>>> code and I came across a paper that removed the edge effects by using SVR >>>> to predict the signal and then mirror that signal. >>>> >>>> I've created an IPython Notebook with background and my example code >>>> trying to reproduce the SVR prediction. I've also linked the paper but it >>>> might be behind a paywall, so I can provide the PDF >>>> as needed. >>>> >>>> See: >>>> >>>> >>>> http://nbviewer.ipython.org/urls/raw.githubusercontent.com/nabobalis/pyhht/master/Ipython%20Examples/SVM%20Regression%20Fitting.ipynb?create=1 >>>> >>>> Thanks, >>>> Nabil >>>> >>>> >>>> ------------------------------------------------------------------------------ >>>> >>>> _______________________________________________ >>>> Scikit-learn-general mailing list >>>> [email protected] >>>> https://lists.sourceforge.net/lists/listinfo/scikit-learn-general >>>> >>>> >>> Hi Nabil, >>> >>> This is very interesting. Can you also show how the SVR prediction fits >>> into the EMD process? I mean, can you show how to go through the entire EMD >>> pipeline while using this method to remove the edge effects? >>> >>> Thanks. >>> >>> -- >>> JD >>> >>> >>> ------------------------------------------------------------------------------ >>> >>> _______________________________________________ >>> Scikit-learn-general mailing list >>> [email protected] >>> https://lists.sourceforge.net/lists/listinfo/scikit-learn-general >>> >>> >> > > > ------------------------------------------------------------------------------ > > _______________________________________________ > Scikit-learn-general mailing list > [email protected] > https://lists.sourceforge.net/lists/listinfo/scikit-learn-general > >
------------------------------------------------------------------------------
_______________________________________________ Scikit-learn-general mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
