[R] delicate problem with permutation test and raster images

[dattel_palme [via R]]

Hello!

I want to make a permutation test between two variables, whereas the
dependent variable is a set of raster images (remote sensing index). In
total, I only have 7 observation dates. As this is a very small number, a
permutation test enables to simulate a high number of observations and
obtain a more robust statistic. - I am interested in getting the p-value
from a correlation between the two variables. The important thing: The
output should be a raster image with every pixel holding a prober value. 

Data Overview: 
independent variable (single value) dependent variable (raster
image) 
x1y1  -
y1<-raster(raster image), raster func. in raster package
x2y2
x3y3
x4y4
x5y5
x6y6
x7y7

One raster image contains i rows and j columns. The permutation hast to be
executed for every single pixel during all obervation dates, in other words
seven values from every single pixel (one per observation date) have to be
permuted by the seven values of the independent variable x. This process has
to be repeated for every single pixel. It is important to keep the pixels,
as the output should be a map, where every pixel is assigned the p-value
(from the statistic). 

So far I used the 'lmp' function from the lmPerm package. Therefore, all
data have to be in one table. I ordered every pixel side by side to the one
pixel before and obtained the entire number of pixels in one row (pixel
number= number of columns). For every observation date I made one row
resulting in 7 rows. The first column was reserved for the independent
variable x. 

 V1  V2  V3  V4...
date1: x1 px1 px2 px3...
date2: x2 px1 px2 px3...
...
date7: x7 px1 px2 px3...

I then did the permutation test using 'lmp':
permutationTest <- lmp (V243606~V1, data=tableOfData, sep="", header=TRUE) #
here for the pixel in row243606
summary(permutationTest)

the Problem: the test has to be done for every singel pixel and i have more
than 1 million of it! that is not possible. 


I tried another method using the raster package: 
Raster package has the calc function, which allows to apply any formula to
every single pixel. E.g. I introduced the formula for the Pearson
correlation coefficient and the density function for a student
t-distribution to get the p-value into the 'calc' function. I was thinking
to introduce the 'lmp' formula into the 'calc' function, but was not
successful as I have to use 7 raster images and 7 numeric values, that this
function does not take. Does anybody know how to solve this problem?

I will very much appreciate any help for this problem!

Thanks! 







__
If you reply to this email, your message will be added to the discussion below:
http://r.789695.n4.nabble.com/delicate-problem-with-permutation-test-and-raster-images-tp4683123.html
This email was sent by dattel_palme (via Nabble)
To receive all replies by email, subscribe to this discussion: 
http://r.789695.n4.nabble.com/template/NamlServlet.jtp?macro=subscribe_by_code&node=4683123&code=ci1oZWxwQHItcHJvamVjdC5vcmd8NDY4MzEyM3wtNzg0MjM1NTA4
[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] some help

[dattel_palme]

Hey

The code need some corrections and I would kindly ask for help.

Say:
I have a table with two columns:
col1=LST and col2=NDVI
i would like to sort all data by NDVI. 
in reality the NDVI ranges between 0 and 1 (although some values might be
minus also).
I want to sort by NDVI values and then make 100 intervals of 0.01 ndvi.
therefore the first inerval is 0.01-0.02, the second 0.02-0.03 and so on..
for each interval I would like to get the highest (max) and lowest (min)
value of LST. 
It would be very helpful if this values can be written in a seperate
table/file. 

Thanks you very much for help!
Stefan



--
View this message in context: 
http://r.789695.n4.nabble.com/some-help-tp4648316p4650844.html
Sent from the R help mailing list archive at Nabble.com.

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] some help

[dattel_palme]

Hey again!

I finally, after some work done before, had time to apply the code. 
The sorting of the table did not work well or maybe something was
misunderstood.

I have a table with 973 rows and 1329 col (ascii/text file). I want to sort
the table that all columns are one under each other so that at the end I
have 973*1329 rows and 1 col. The col should be sorted in a way that col 2
is under col 1, col 3 under col 2, col 4 under col 3 etc. 

I applied this code:
dat <- read.table(filename, sep=, header=TRUE)
stacked <- do.call(rbind, dat) 
unlist(dat) 

..but putting dim(dat), the number of rows and col was still 973 and 1329.
So seemingly it did not work as i wanted.

Thanks very much for more help.

Stefan







--
View this message in context: 
http://r.789695.n4.nabble.com/some-help-tp4648316p4650828.html
Sent from the R help mailing list archive at Nabble.com.

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] some help

[dattel_palme]

HI David,

Thanks for your answers,
I can't provide data here, but I can explain more:

It's satellite images as a text file (ascii). Two different images with two
different variables, land surface temperature (LST -lstascii =filename) and
an vegetation index (NDVI - ndviascii = filename).

The aim was to do a scatter plot of NDVI against LST, whereas the NDVI
should be on the x-axis. - The scatter plot itself is not needed as you will
see.

As I cannot compare each value (pixel value) in one image to the respective
value in the other image, i had the idea to sort all columns one under each
other. In the end I would have one column of one variable. I wanted to put
the two variables LST and NDVI together in one table, so that I have two
columns with two variables, and the value of one variable (NDVI) in each row
can be compared directly to the value of the other variable (LST) in the
same row. Why that? I wanted to sort the columns by the NDVI variable and
make 0,01 NDVI intervals. From each of these intervals I wanted to know the
max and min LST. These values I need for the regression - see the graph in
the file attached.

I this more clear now?

Attached you can see the regression line I need to find. For this I
developed this process (because I cannot compare pixel value by pixel value
in a multi column and row table). 

 

Best Regards
Stefan



--
View this message in context: 
http://r.789695.n4.nabble.com/some-help-tp4648316p4648410.html
Sent from the R help mailing list archive at Nabble.com.

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] some help

[dattel_palme]

HI everybody,

Thanks for your answers,
I can't provide data here, but I can explain more: 

It's satellite images as a text file (ascii). Two different images with two
different variables, land surface temperature (LST -lstascii =filename) and
an vegetation index (NDVI - ndviascii = filename).

The aim was to do a scatter plot of NDVI against LST, whereas the NDVI
should be on the x-axis. - The scatter plot itself is not needed as you will
see.

As I cannot compare each value (pixel value) in one image to the respective
value in the other image, i had the idea to sort all columns one under each
other. In the end I would have one column of one variable. I wanted to put
the two variables LST and NDVI together in one table, so that I have two
columns with two variables, and the value of one variable (NDVI) in each row
can be compared directly to the value of the other variable (LST) in the
same row. Why that? I wanted to sort the columns by the NDVI variable and
make 0,01 NDVI intervals. From each of these intervals I wanted to know the
max and min LST. These values I need for the regression - see the graph in
the file attached.

I this more clear now?

Attached you can see the regression line I need to find. For this I
developed this process (because I cannot compare pixel value by pixel value
in a multi column and row table). 
 

Best Regards



--
View this message in context: 
http://r.789695.n4.nabble.com/some-help-tp4648316p4648382.html
Sent from the R help mailing list archive at Nabble.com.

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

[R] some help

[dattel_palme]

Hi People!

I have following concern consisting of some steps to do in R: 

I have an ascii file (table) consisting of many columns and rows. 
1. I would like to order all values of the columns one under each other. It
will begin with column 1, then column 2 under column 1, column 3 under
column 2 etc. until at the end there is only 1 column. How do I do it?

2. Second problem is to make a scatterplot of two variables (I think after
further explanation scatter plot itself will not be needed). I have two
columns of two different variables (that I produces before), column 1 with
variable 1 and column 2 with variable 2. I would like to order them by one
variable and 0,01 interval (the varibale values will range between 0 and 1).
>From each 0,01 interval (100 intervals) i want to pick the maximum and
minimum value of variable 2. 

3. From the obtained max and min of values of each interval i would like to
make a linear least square regression. 

I hope someone can help me out!
Thanks
Stefan



--
View this message in context: 
http://r.789695.n4.nabble.com/some-help-tp4648316.html
Sent from the R help mailing list archive at Nabble.com.

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.