Hi everybody,
I sincerely appreciate all your replies. Now I have gotten some ideas to
challenge with my large data, although, as a researcher am looking for more and
more.
To handle large data set for Variography, then Kriging, then Simulations, the
following researchers kindly shared their experiences with me (and the
community) so far:
Please do gently with my summaries read the original message if you need.
Roderik said (suggested):
1- to identify the distance expected for estimation search area.
2- to assess how local the variogram analysis should be.
3- to find whether the spatial continuity is the same anywhere in the region
or not.
4- the estimation will be a more local affair.
/// summary: to think primarily about locality and globality of spatial analysis
/// got it and will try.
Ulrich said (suggested):
1- to take a (stratified) random sample of your data to reduce the
computational time for the variography.
2- to use a local search neighborhood.
/// summary: random selection strategy
am interested in this idea. Since any decreasing in data has dramatically
positive effects on computation cost (time).
Is anyone aware of this methodology has been published somewhere?
/// liked it and will try.
Dansaid (suggested):
1- working with 200,000 samples is a good challenge.
2- to investigate if there is a single underlying Gaussian random field model
3- to find if the data densely sampled with respect to the length scales
4- the projected process approach (Bayesian / ML) seems as a useful way
5- it depends on the underlying complexity of the actual spatial variable
6- clearly any model based method will struggle with 200,000 points!
7- to do exploratory analysis
8- to split the data set
9- to explore whether stationarity was present
10- to get a feel for the length scales
11- all likelihood based approaches have significant approximations dealing
with large data!
/// more complete list of previous suggestions
/// am interested in applying PPA. Thanks a lot for papers' links
Isobel said (suggested):
1- try moving windows.
2- to choose a sub-region size
3- to calculate and graph from the samples in the selection area
4- to shift half-a-window in one direction
5- to repeat.
6- to display all of resulted graphs as a 'map' for each level.
7- to be able to visually assess stationarity.
8- to consider modeling.
/// an interesting method.
/// not very familiar with the method, may need to some reading
Anatoly said (suggested):
1- to subsample data (either source locations or pairs)
2- to use bootstrap to estimate the uncertainty
3- to pre-processing before estimation
4- to cluster data using the samples density estimation.
///summary: try applying bootstrap
I would appreciate it if anybody would send me a link of related article/source
to his/her suggestions/ideas.
Kind Regards,
Younes
+
+ To post a message to the list, send it to ai-geost...@jrc.ec.europa.eu
+ To unsubscribe, send email to majordomo@ jrc.ec.europa.eu with no subject and
"unsubscribe ai-geostats" in the message body. DO NOT SEND
Subscribe/Unsubscribe requests to the list
+ As a general service to list users, please remember to post a summary of any
useful responses to your questions.
+ Support to the forum can be found at http://www.ai-geostats.org/
