date:20200518

Re: [R] iterators : checkFunc with ireadLines

2020-05-18 Thread Laurent Rhelp





GREAT ! It is exactly in the idea of my request !
I like the nextElem call in the skip argument.
Thank you very much William
Best Regards
Laurent


Le 18/05/2020 à 20:37, William Michels a écrit :

Hi Laurent,

Thank you for explaining your size limitations. Below is an example
using the read.fwf() function to grab the first column of your input
file (in 2000 row chunks). This column is converted to an index, and
the index is used to create an iterator useful for skipping lines when
reading input with scan(). (You could try processing your large file
in successive 2000 line chunks, or whatever number of lines fits into
memory). Maybe not as elegant as the approach you were going for, but
read.fwf() should be pretty efficient:


sensors <-  c("N053", "N163")
read.fwf("test2.txt", widths=c(4), as.is=TRUE, flush=TRUE, n=2000, skip=0)

 V1
1 Time
2 N023
3 N053
4 N123
5 N163
6 N193

first_col <- read.fwf("test2.txt", widths=c(4), as.is=TRUE, flush=TRUE, n=2000, 
skip=0)
which(first_col$V1 %in% sensors)

[1] 3 5

index1 <- which(first_col$V1 %in% sensors)
iter_index1 <- iter(1:2000, checkFunc= function(n) {n %in% index1})
unlist(scan(file="test2.txt", what=list("","","","","","","","","",""), 
flush=TRUE, multi.line=FALSE, skip=nextElem(iter_index1)-1, nlines=1, quiet=TRUE))

  [1] "N053"  "-0.014083" "-0.004741" "0.001443"  "-0.010152"
"-0.012996" "-0.005337" "-0.008738" "-0.015094" "-0.012104"

unlist(scan(file="test2.txt", what=list("","","","","","","","","",""), 
flush=TRUE, multi.line=FALSE, skip=nextElem(iter_index1)-1, nlines=1, quiet=TRUE))

  [1] "N163"  "-0.054023" "-0.049345" "-0.037158" "-0.04112"
"-0.044612" "-0.036953" "-0.036061" "-0.044516" "-0.046436"
(Note for this email and the previous one, I've deleted the first
"hash" character from each line of your test file for clarity).

HTH, Bill.

W. Michels, Ph.D.





On Mon, May 18, 2020 at 3:35 AM Laurent Rhelp  wrote:

Dear William,
   Thank you for your answer
My file is very large so I cannot read it in my memory (I cannot use
read.table). So I want to put in memory only the line I need to process.
With readLines, as I did, it works but I would like to use an iterator
and a foreach loop to understand this way to do because I thought that
it was a better solution to write a nice code.


Le 18/05/2020 à 04:54, William Michels a écrit :

Apologies, Laurent, for this two-part answer. I misunderstood your
post where you stated you wanted to "filter(ing) some
selected lines according to the line name... ." I thought that meant
you had a separate index (like a series of primes) that you wanted to
use to only read-in selected line numbers from a file (test file below
with numbers 1:1000 each on a separate line):


library(gmp)
library(iterators)
iprime <- iter(1:100, checkFunc = function(n) isprime(n))
scan(file="one_thou_lines.txt", skip=nextElem(iprime)-1, nlines=1)

Read 1 item
[1] 2

scan(file="one_thou_lines.txt", skip=nextElem(iprime)-1, nlines=1)

Read 1 item
[1] 3

scan(file="one_thou_lines.txt", skip=nextElem(iprime)-1, nlines=1)

Read 1 item
[1] 5

scan(file="one_thou_lines.txt", skip=nextElem(iprime)-1, nlines=1)

Read 1 item
[1] 7
However, what it really seems that you want to do is read each line of
a (possibly enormous) file, test each line "string-wise" to keep or
discard, and if you're keeping it, append the line to a list. I can
certainly see the advantage of this strategy for reading in very, very
large files, but it's not clear to me how the "ireadLines" function (
in the "iterators" package) will help you, since it doesn't seem to
generate anything but a sequential index.

Anyway, below is an absolutely standard read-in of your data using
read.table(). Hopefully some of the code I've posted has been useful
to you.


sensors <-  c("N053", "N163")
read.table("test2.txt")

  V1V2V3V4V5V6V7
 V8V9   V10
1 Time  0.00  0.000999  0.001999  0.002998  0.003998  0.004997
0.005997  0.006996  0.007996
2 N023 -0.031323 -0.035026 -0.029759 -0.024886 -0.024464 -0.026816
-0.033690 -0.041067 -0.038747
3 N053 -0.014083 -0.004741  0.001443 -0.010152 -0.012996 -0.005337
-0.008738 -0.015094 -0.012104
4 N123 -0.019008 -0.013494 -0.013180 -0.029208 -0.032748 -0.020243
-0.015089 -0.014439 -0.011681
5 N163 -0.054023 -0.049345 -0.037158 -0.041120 -0.044612 -0.036953
-0.036061 -0.044516 -0.046436
6 N193 -0.022171 -0.022384 -0.022338 -0.023304 -0.022569 -0.021827
-0.021996 -0.021755 -0.021846

Laurent_data <- read.table("test2.txt")
Laurent_data[Laurent_data$V1 %in% sensors, ]

  V1V2V3V4V5V6V7
 V8V9   V10
3 N053 -0.014083 -0.004741  0.001443 -0.010152 -0.012996 -0.005337
-0.008738 -0.015094 -0.012104
5 N163 -0.054023 -0.049345 -0.037158 -0.041120 -0.044612 -0.036953
-0.036061 -0.044516 -0.046436

Best, Bill.

W. Michels, Ph.D.


On Sun, May 17, 2020 at 5:43 PM Laurent Rhelp  wrote:

Dear R-Help List,

Re: [R] iterators : checkFunc with ireadLines

2020-05-18 Thread Laurent Rhelp




Dear William,
 Thank you for your answer
My file is very large so I cannot read it in my memory (I cannot use 
read.table). So I want to put in memory only the line I need to process. 
With readLines, as I did, it works but I would like to use an iterator 
and a foreach loop to understand this way to do because I thought that 
it was a better solution to write a nice code.



Le 18/05/2020 à 04:54, William Michels a écrit :

Apologies, Laurent, for this two-part answer. I misunderstood your
post where you stated you wanted to "filter(ing) some
selected lines according to the line name... ." I thought that meant
you had a separate index (like a series of primes) that you wanted to
use to only read-in selected line numbers from a file (test file below
with numbers 1:1000 each on a separate line):


library(gmp)
library(iterators)
iprime <- iter(1:100, checkFunc = function(n) isprime(n))
scan(file="one_thou_lines.txt", skip=nextElem(iprime)-1, nlines=1)

Read 1 item
[1] 2

scan(file="one_thou_lines.txt", skip=nextElem(iprime)-1, nlines=1)

Read 1 item
[1] 3

scan(file="one_thou_lines.txt", skip=nextElem(iprime)-1, nlines=1)

Read 1 item
[1] 5

scan(file="one_thou_lines.txt", skip=nextElem(iprime)-1, nlines=1)

Read 1 item
[1] 7
However, what it really seems that you want to do is read each line of
a (possibly enormous) file, test each line "string-wise" to keep or
discard, and if you're keeping it, append the line to a list. I can
certainly see the advantage of this strategy for reading in very, very
large files, but it's not clear to me how the "ireadLines" function (
in the "iterators" package) will help you, since it doesn't seem to
generate anything but a sequential index.

Anyway, below is an absolutely standard read-in of your data using
read.table(). Hopefully some of the code I've posted has been useful
to you.


sensors <-  c("N053", "N163")
read.table("test2.txt")

 V1V2V3V4V5V6V7
V8V9   V10
1 Time  0.00  0.000999  0.001999  0.002998  0.003998  0.004997
0.005997  0.006996  0.007996
2 N023 -0.031323 -0.035026 -0.029759 -0.024886 -0.024464 -0.026816
-0.033690 -0.041067 -0.038747
3 N053 -0.014083 -0.004741  0.001443 -0.010152 -0.012996 -0.005337
-0.008738 -0.015094 -0.012104
4 N123 -0.019008 -0.013494 -0.013180 -0.029208 -0.032748 -0.020243
-0.015089 -0.014439 -0.011681
5 N163 -0.054023 -0.049345 -0.037158 -0.041120 -0.044612 -0.036953
-0.036061 -0.044516 -0.046436
6 N193 -0.022171 -0.022384 -0.022338 -0.023304 -0.022569 -0.021827
-0.021996 -0.021755 -0.021846

Laurent_data <- read.table("test2.txt")
Laurent_data[Laurent_data$V1 %in% sensors, ]

 V1V2V3V4V5V6V7
V8V9   V10
3 N053 -0.014083 -0.004741  0.001443 -0.010152 -0.012996 -0.005337
-0.008738 -0.015094 -0.012104
5 N163 -0.054023 -0.049345 -0.037158 -0.041120 -0.044612 -0.036953
-0.036061 -0.044516 -0.046436

Best, Bill.

W. Michels, Ph.D.


On Sun, May 17, 2020 at 5:43 PM Laurent Rhelp  wrote:

Dear R-Help List,

 I would like to use an iterator to read a file filtering some
selected lines according to the line name in order to use after a
foreach loop. I wanted to use the checkFunc argument as the following
example found on internet to select only prime numbers :

|iprime <- ||iter||(1:100, checkFunc =
||function||(n) ||isprime||(n))|

|(https://datawookie.netlify.app/blog/2013/11/iterators-in-r/)
|

but the checkFunc argument seems not to be available with the function
ireadLines (package iterators). So, I did the code below to solve my
problem but I am sure that I miss something to use iterators with files.
Since I found nothing on the web about ireadLines and the checkFunc
argument, could somebody help me to understand how we have to use
iterator (and foreach loop) on files keeping only selected lines ?

Thank you very much
Laurent

Presently here is my code:

##mock file to read: test.txt
##
# Time00.0009990.0019990.0029980.003998 0.004997
0.0059970.0069960.007996
# N023-0.031323-0.035026-0.029759-0.024886 -0.024464
-0.026816-0.03369-0.041067-0.038747
# N053-0.014083-0.0047410.001443-0.010152 -0.012996
-0.005337-0.008738-0.015094-0.012104
# N123-0.019008-0.013494-0.01318-0.029208 -0.032748
-0.020243-0.015089-0.014439-0.011681
# N163-0.054023-0.049345-0.037158-0.04112 -0.044612
-0.036953-0.036061-0.044516-0.046436
# N193-0.022171-0.022384-0.022338-0.023304 -0.022569
-0.021827-0.021996-0.021755-0.021846


# sensors to keep

sensors <-  c("N053", "N163")


library(iterators)

library(rlist)


file_name <- "test.txt"

con_obj <- file( file_name , "r")
ifile <- ireadLines( con_obj , n = 1 )


## I do not do a loop for the example

res <- l

Re: [R] iterators : checkFunc with ireadLines

2020-05-18 Thread William Michels via R-help



Dear Laurent,

I'm going through your code quickly, and the first question I have is
whether you loaded the "gmp" library?

> library(gmp)

Attaching package: ‘gmp’

The following objects are masked from ‘package:base’:

%*%, apply, crossprod, matrix, tcrossprod

> library(iterators)
> iter(1:100, checkFunc = function(n) isprime(n))
$state


$length
[1] 100

$checkFunc
function (n)
isprime(n)

$recycle
[1] FALSE

attr(,"class")
[1] "containeriter" "iter"
>

HTH, Bill.

W. Michels, Ph.D.



On Sun, May 17, 2020 at 5:43 PM Laurent Rhelp  wrote:
>
> Dear R-Help List,
>
> I would like to use an iterator to read a file filtering some
> selected lines according to the line name in order to use after a
> foreach loop. I wanted to use the checkFunc argument as the following
> example found on internet to select only prime numbers :
>
> |iprime <- ||iter||(1:100, checkFunc =
> ||function||(n) ||isprime||(n))|
>
> |(https://datawookie.netlify.app/blog/2013/11/iterators-in-r/)
> |
>
> but the checkFunc argument seems not to be available with the function
> ireadLines (package iterators). So, I did the code below to solve my
> problem but I am sure that I miss something to use iterators with files.
> Since I found nothing on the web about ireadLines and the checkFunc
> argument, could somebody help me to understand how we have to use
> iterator (and foreach loop) on files keeping only selected lines ?
>
> Thank you very much
> Laurent
>
> Presently here is my code:
>
> ##mock file to read: test.txt
> ##
> # Time00.0009990.0019990.0029980.003998 0.004997
> 0.0059970.0069960.007996
> # N023-0.031323-0.035026-0.029759-0.024886 -0.024464
> -0.026816-0.03369-0.041067-0.038747
> # N053-0.014083-0.0047410.001443-0.010152 -0.012996
> -0.005337-0.008738-0.015094-0.012104
> # N123-0.019008-0.013494-0.01318-0.029208 -0.032748
> -0.020243-0.015089-0.014439-0.011681
> # N163-0.054023-0.049345-0.037158-0.04112 -0.044612
> -0.036953-0.036061-0.044516-0.046436
> # N193-0.022171-0.022384-0.022338-0.023304 -0.022569
> -0.021827-0.021996-0.021755-0.021846
>
>
> # sensors to keep
>
> sensors <-  c("N053", "N163")
>
>
> library(iterators)
>
> library(rlist)
>
>
> file_name <- "test.txt"
>
> con_obj <- file( file_name , "r")
> ifile <- ireadLines( con_obj , n = 1 )
>
>
> ## I do not do a loop for the example
>
> res <- list()
>
> r <- get_Lines_iter( ifile , sensors)
> res <- list.append( res , r )
> res
> r <- get_Lines_iter( ifile , sensors)
> res <- list.append( res , r )
> res
> r <- get_Lines_iter( ifile , sensors)
> do.call("cbind",res)
>
> ## the function get_Lines_iter to select and process the line
>
> get_Lines_iter  <-  function( iter , sensors, sep = '\t', quiet = FALSE){
>## read the next record in the iterator
>r = try( nextElem(iter) )
>   while(  TRUE ){
>  if( class(r) == "try-error") {
>return( stop("The iterator is empty") )
> } else {
> ## split the read line according to the separator
>  r_txt <- textConnection(r)
>  fields <- scan(file = r_txt, what = "character", sep = sep, quiet =
> quiet)
>   ## test if we have to keep the line
>   if( fields[1] %in% sensors){
> ## data processing for the selected line (for the example
> transformation in dataframe)
> n <- length(fields)
> x <- data.frame( as.numeric(fields[2:n]) )
> names(x) <- fields[1]
> ## We return the values
> print(paste0("sensor ",fields[1]," ok"))
> return( x )
>   }else{
>print(paste0("Sensor ", fields[1] ," not selected"))
>r = try(nextElem(iter) )}
> }
> }# end while loop
> }
>
>
>
>
>
>
>
> --
> L'absence de virus dans ce courrier électronique a été vérifiée par le 
> logiciel antivirus Avast.
> https://www.avast.com/antivirus
>
> [[alternative HTML version deleted]]
>
> __
> R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] iterators : checkFunc with ireadLines

2020-05-18 Thread William Michels via R-help



Apologies, Laurent, for this two-part answer. I misunderstood your
post where you stated you wanted to "filter(ing) some
selected lines according to the line name... ." I thought that meant
you had a separate index (like a series of primes) that you wanted to
use to only read-in selected line numbers from a file (test file below
with numbers 1:1000 each on a separate line):

> library(gmp)
> library(iterators)
> iprime <- iter(1:100, checkFunc = function(n) isprime(n))
> scan(file="one_thou_lines.txt", skip=nextElem(iprime)-1, nlines=1)
Read 1 item
[1] 2
> scan(file="one_thou_lines.txt", skip=nextElem(iprime)-1, nlines=1)
Read 1 item
[1] 3
> scan(file="one_thou_lines.txt", skip=nextElem(iprime)-1, nlines=1)
Read 1 item
[1] 5
> scan(file="one_thou_lines.txt", skip=nextElem(iprime)-1, nlines=1)
Read 1 item
[1] 7
>

However, what it really seems that you want to do is read each line of
a (possibly enormous) file, test each line "string-wise" to keep or
discard, and if you're keeping it, append the line to a list. I can
certainly see the advantage of this strategy for reading in very, very
large files, but it's not clear to me how the "ireadLines" function (
in the "iterators" package) will help you, since it doesn't seem to
generate anything but a sequential index.

Anyway, below is an absolutely standard read-in of your data using
read.table(). Hopefully some of the code I've posted has been useful
to you.

> sensors <-  c("N053", "N163")
> read.table("test2.txt")
V1V2V3V4V5V6V7
   V8V9   V10
1 Time  0.00  0.000999  0.001999  0.002998  0.003998  0.004997
0.005997  0.006996  0.007996
2 N023 -0.031323 -0.035026 -0.029759 -0.024886 -0.024464 -0.026816
-0.033690 -0.041067 -0.038747
3 N053 -0.014083 -0.004741  0.001443 -0.010152 -0.012996 -0.005337
-0.008738 -0.015094 -0.012104
4 N123 -0.019008 -0.013494 -0.013180 -0.029208 -0.032748 -0.020243
-0.015089 -0.014439 -0.011681
5 N163 -0.054023 -0.049345 -0.037158 -0.041120 -0.044612 -0.036953
-0.036061 -0.044516 -0.046436
6 N193 -0.022171 -0.022384 -0.022338 -0.023304 -0.022569 -0.021827
-0.021996 -0.021755 -0.021846
> Laurent_data <- read.table("test2.txt")
> Laurent_data[Laurent_data$V1 %in% sensors, ]
V1V2V3V4V5V6V7
   V8V9   V10
3 N053 -0.014083 -0.004741  0.001443 -0.010152 -0.012996 -0.005337
-0.008738 -0.015094 -0.012104
5 N163 -0.054023 -0.049345 -0.037158 -0.041120 -0.044612 -0.036953
-0.036061 -0.044516 -0.046436

Best, Bill.

W. Michels, Ph.D.


On Sun, May 17, 2020 at 5:43 PM Laurent Rhelp  wrote:
>
> Dear R-Help List,
>
> I would like to use an iterator to read a file filtering some
> selected lines according to the line name in order to use after a
> foreach loop. I wanted to use the checkFunc argument as the following
> example found on internet to select only prime numbers :
>
> |iprime <- ||iter||(1:100, checkFunc =
> ||function||(n) ||isprime||(n))|
>
> |(https://datawookie.netlify.app/blog/2013/11/iterators-in-r/)
> |
>
> but the checkFunc argument seems not to be available with the function
> ireadLines (package iterators). So, I did the code below to solve my
> problem but I am sure that I miss something to use iterators with files.
> Since I found nothing on the web about ireadLines and the checkFunc
> argument, could somebody help me to understand how we have to use
> iterator (and foreach loop) on files keeping only selected lines ?
>
> Thank you very much
> Laurent
>
> Presently here is my code:
>
> ##mock file to read: test.txt
> ##
> # Time00.0009990.0019990.0029980.003998 0.004997
> 0.0059970.0069960.007996
> # N023-0.031323-0.035026-0.029759-0.024886 -0.024464
> -0.026816-0.03369-0.041067-0.038747
> # N053-0.014083-0.0047410.001443-0.010152 -0.012996
> -0.005337-0.008738-0.015094-0.012104
> # N123-0.019008-0.013494-0.01318-0.029208 -0.032748
> -0.020243-0.015089-0.014439-0.011681
> # N163-0.054023-0.049345-0.037158-0.04112 -0.044612
> -0.036953-0.036061-0.044516-0.046436
> # N193-0.022171-0.022384-0.022338-0.023304 -0.022569
> -0.021827-0.021996-0.021755-0.021846
>
>
> # sensors to keep
>
> sensors <-  c("N053", "N163")
>
>
> library(iterators)
>
> library(rlist)
>
>
> file_name <- "test.txt"
>
> con_obj <- file( file_name , "r")
> ifile <- ireadLines( con_obj , n = 1 )
>
>
> ## I do not do a loop for the example
>
> res <- list()
>
> r <- get_Lines_iter( ifile , sensors)
> res <- list.append( res , r )
> res
> r <- get_Lines_iter( ifile , sensors)
> res <- list.append( res , r )
> res
> r <- get_Lines_iter( ifile , sensors)
> do.call("cbind",res)
>
> ## the function get_Lines_iter to select and process the line
>
> get_Lines_iter  <-  fun

Re: [R] iterators : checkFunc with ireadLines

2020-05-18 Thread William Michels via R-help



Hi Laurent,

Thank you for explaining your size limitations. Below is an example
using the read.fwf() function to grab the first column of your input
file (in 2000 row chunks). This column is converted to an index, and
the index is used to create an iterator useful for skipping lines when
reading input with scan(). (You could try processing your large file
in successive 2000 line chunks, or whatever number of lines fits into
memory). Maybe not as elegant as the approach you were going for, but
read.fwf() should be pretty efficient:

> sensors <-  c("N053", "N163")
> read.fwf("test2.txt", widths=c(4), as.is=TRUE, flush=TRUE, n=2000, skip=0)
V1
1 Time
2 N023
3 N053
4 N123
5 N163
6 N193
> first_col <- read.fwf("test2.txt", widths=c(4), as.is=TRUE, flush=TRUE, 
> n=2000, skip=0)
> which(first_col$V1 %in% sensors)
[1] 3 5
> index1 <- which(first_col$V1 %in% sensors)
> iter_index1 <- iter(1:2000, checkFunc= function(n) {n %in% index1})
> unlist(scan(file="test2.txt", what=list("","","","","","","","","",""), 
> flush=TRUE, multi.line=FALSE, skip=nextElem(iter_index1)-1, nlines=1, 
> quiet=TRUE))
 [1] "N053"  "-0.014083" "-0.004741" "0.001443"  "-0.010152"
"-0.012996" "-0.005337" "-0.008738" "-0.015094" "-0.012104"
> unlist(scan(file="test2.txt", what=list("","","","","","","","","",""), 
> flush=TRUE, multi.line=FALSE, skip=nextElem(iter_index1)-1, nlines=1, 
> quiet=TRUE))
 [1] "N163"  "-0.054023" "-0.049345" "-0.037158" "-0.04112"
"-0.044612" "-0.036953" "-0.036061" "-0.044516" "-0.046436"
>

(Note for this email and the previous one, I've deleted the first
"hash" character from each line of your test file for clarity).

HTH, Bill.

W. Michels, Ph.D.





On Mon, May 18, 2020 at 3:35 AM Laurent Rhelp  wrote:
>
> Dear William,
>   Thank you for your answer
> My file is very large so I cannot read it in my memory (I cannot use
> read.table). So I want to put in memory only the line I need to process.
> With readLines, as I did, it works but I would like to use an iterator
> and a foreach loop to understand this way to do because I thought that
> it was a better solution to write a nice code.
>
>
> Le 18/05/2020 à 04:54, William Michels a écrit :
> > Apologies, Laurent, for this two-part answer. I misunderstood your
> > post where you stated you wanted to "filter(ing) some
> > selected lines according to the line name... ." I thought that meant
> > you had a separate index (like a series of primes) that you wanted to
> > use to only read-in selected line numbers from a file (test file below
> > with numbers 1:1000 each on a separate line):
> >
> >> library(gmp)
> >> library(iterators)
> >> iprime <- iter(1:100, checkFunc = function(n) isprime(n))
> >> scan(file="one_thou_lines.txt", skip=nextElem(iprime)-1, nlines=1)
> > Read 1 item
> > [1] 2
> >> scan(file="one_thou_lines.txt", skip=nextElem(iprime)-1, nlines=1)
> > Read 1 item
> > [1] 3
> >> scan(file="one_thou_lines.txt", skip=nextElem(iprime)-1, nlines=1)
> > Read 1 item
> > [1] 5
> >> scan(file="one_thou_lines.txt", skip=nextElem(iprime)-1, nlines=1)
> > Read 1 item
> > [1] 7
> > However, what it really seems that you want to do is read each line of
> > a (possibly enormous) file, test each line "string-wise" to keep or
> > discard, and if you're keeping it, append the line to a list. I can
> > certainly see the advantage of this strategy for reading in very, very
> > large files, but it's not clear to me how the "ireadLines" function (
> > in the "iterators" package) will help you, since it doesn't seem to
> > generate anything but a sequential index.
> >
> > Anyway, below is an absolutely standard read-in of your data using
> > read.table(). Hopefully some of the code I've posted has been useful
> > to you.
> >
> >> sensors <-  c("N053", "N163")
> >> read.table("test2.txt")
> >  V1V2V3V4V5V6V7
> > V8V9   V10
> > 1 Time  0.00  0.000999  0.001999  0.002998  0.003998  0.004997
> > 0.005997  0.006996  0.007996
> > 2 N023 -0.031323 -0.035026 -0.029759 -0.024886 -0.024464 -0.026816
> > -0.033690 -0.041067 -0.038747
> > 3 N053 -0.014083 -0.004741  0.001443 -0.010152 -0.012996 -0.005337
> > -0.008738 -0.015094 -0.012104
> > 4 N123 -0.019008 -0.013494 -0.013180 -0.029208 -0.032748 -0.020243
> > -0.015089 -0.014439 -0.011681
> > 5 N163 -0.054023 -0.049345 -0.037158 -0.041120 -0.044612 -0.036953
> > -0.036061 -0.044516 -0.046436
> > 6 N193 -0.022171 -0.022384 -0.022338 -0.023304 -0.022569 -0.021827
> > -0.021996 -0.021755 -0.021846
> >> Laurent_data <- read.table("test2.txt")
> >> Laurent_data[Laurent_data$V1 %in% sensors, ]
> >  V1V2V3V4V5V6V7
> > V8V9   V10
> > 3 N053 -0.014083 -0.004741  0.001443 -0.010152 -0.012996 -0.005337
> > -0.008738 -0.015094 -0.012104
> > 5 N163 -0.054023 -0.049345 -0.037158 -0.041120 -0.044612 -0.036953
> > -0.036061 -0.044516 -0.046436
> >
> > Best, Bill.
> >
> > W. Michels, Ph.D.
> >
>

Re: [R] iterators : checkFunc with ireadLines

2020-05-18 Thread Laurent Rhelp




Dear William,
 Thank you for your answer
My file is very large so I cannot read it in my memory (I cannot use 
read.table). So I want to put in memory only the line I need to process. 
With readLines, as I did, it works but I would like to use an iterator 
and a foreach loop to understand this way to do because I thought that 
it was a better solution to write a nice code.



Le 18/05/2020 à 04:54, William Michels a écrit :

Apologies, Laurent, for this two-part answer. I misunderstood your
post where you stated you wanted to "filter(ing) some
selected lines according to the line name... ." I thought that meant
you had a separate index (like a series of primes) that you wanted to
use to only read-in selected line numbers from a file (test file below
with numbers 1:1000 each on a separate line):


library(gmp)
library(iterators)
iprime <- iter(1:100, checkFunc = function(n) isprime(n))
scan(file="one_thou_lines.txt", skip=nextElem(iprime)-1, nlines=1)

Read 1 item
[1] 2

scan(file="one_thou_lines.txt", skip=nextElem(iprime)-1, nlines=1)

Read 1 item
[1] 3

scan(file="one_thou_lines.txt", skip=nextElem(iprime)-1, nlines=1)

Read 1 item
[1] 5

scan(file="one_thou_lines.txt", skip=nextElem(iprime)-1, nlines=1)

Read 1 item
[1] 7
However, what it really seems that you want to do is read each line of
a (possibly enormous) file, test each line "string-wise" to keep or
discard, and if you're keeping it, append the line to a list. I can
certainly see the advantage of this strategy for reading in very, very
large files, but it's not clear to me how the "ireadLines" function (
in the "iterators" package) will help you, since it doesn't seem to
generate anything but a sequential index.

Anyway, below is an absolutely standard read-in of your data using
read.table(). Hopefully some of the code I've posted has been useful
to you.


sensors <-  c("N053", "N163")
read.table("test2.txt")

 V1V2V3V4V5V6V7
V8V9   V10
1 Time  0.00  0.000999  0.001999  0.002998  0.003998  0.004997
0.005997  0.006996  0.007996
2 N023 -0.031323 -0.035026 -0.029759 -0.024886 -0.024464 -0.026816
-0.033690 -0.041067 -0.038747
3 N053 -0.014083 -0.004741  0.001443 -0.010152 -0.012996 -0.005337
-0.008738 -0.015094 -0.012104
4 N123 -0.019008 -0.013494 -0.013180 -0.029208 -0.032748 -0.020243
-0.015089 -0.014439 -0.011681
5 N163 -0.054023 -0.049345 -0.037158 -0.041120 -0.044612 -0.036953
-0.036061 -0.044516 -0.046436
6 N193 -0.022171 -0.022384 -0.022338 -0.023304 -0.022569 -0.021827
-0.021996 -0.021755 -0.021846

Laurent_data <- read.table("test2.txt")
Laurent_data[Laurent_data$V1 %in% sensors, ]

 V1V2V3V4V5V6V7
V8V9   V10
3 N053 -0.014083 -0.004741  0.001443 -0.010152 -0.012996 -0.005337
-0.008738 -0.015094 -0.012104
5 N163 -0.054023 -0.049345 -0.037158 -0.041120 -0.044612 -0.036953
-0.036061 -0.044516 -0.046436

Best, Bill.

W. Michels, Ph.D.


On Sun, May 17, 2020 at 5:43 PM Laurent Rhelp  wrote:

Dear R-Help List,

 I would like to use an iterator to read a file filtering some
selected lines according to the line name in order to use after a
foreach loop. I wanted to use the checkFunc argument as the following
example found on internet to select only prime numbers :

|iprime <- ||iter||(1:100, checkFunc =
||function||(n) ||isprime||(n))|

|(https://datawookie.netlify.app/blog/2013/11/iterators-in-r/)
|

but the checkFunc argument seems not to be available with the function
ireadLines (package iterators). So, I did the code below to solve my
problem but I am sure that I miss something to use iterators with files.
Since I found nothing on the web about ireadLines and the checkFunc
argument, could somebody help me to understand how we have to use
iterator (and foreach loop) on files keeping only selected lines ?

Thank you very much
Laurent

Presently here is my code:

##mock file to read: test.txt
##
# Time00.0009990.0019990.0029980.003998 0.004997
0.0059970.0069960.007996
# N023-0.031323-0.035026-0.029759-0.024886 -0.024464
-0.026816-0.03369-0.041067-0.038747
# N053-0.014083-0.0047410.001443-0.010152 -0.012996
-0.005337-0.008738-0.015094-0.012104
# N123-0.019008-0.013494-0.01318-0.029208 -0.032748
-0.020243-0.015089-0.014439-0.011681
# N163-0.054023-0.049345-0.037158-0.04112 -0.044612
-0.036953-0.036061-0.044516-0.046436
# N193-0.022171-0.022384-0.022338-0.023304 -0.022569
-0.021827-0.021996-0.021755-0.021846


# sensors to keep

sensors <-  c("N053", "N163")


library(iterators)

library(rlist)


file_name <- "test.txt"

con_obj <- file( file_name , "r")
ifile <- ireadLines( con_obj , n = 1 )


## I do not do a loop for the example

res <- l

Re: [R] Classification of wind events

2020-05-18 Thread Jim Lemon

Sorry, I should know better:

rollmean<-function(x,width=2) {
 lenx<-length(x)
 result<-rep(NA,lenx)
 for(i in 1:lenx) {
  chunk<-i:(i+width-1)
  if(i(lenx-width)) chunk<-c(i:lenx,rep(lenx,i-(width-1)))
  result[i]<-mean(x[chunk])
 }
 return(result)
}

I forgot to replace this with:

library(zoo)
rollmean...

Jim

On Tue, May 19, 2020 at 1:26 AM Jeff Newmiller  wrote:
>
> ? source("../rollmean.R") ?
>
> On May 18, 2020 4:11:52 AM PDT, Jim Lemon  wrote:
> >Hi Stefano,
> >If I understand your request, this may also help, Uses the same data
> >transformations as my previous email.
> >
> >png("SS_foehn.png")
> >plot(mydf$data_POSIX,
> > ifelse(mydf$main_dir %in% c("WSW","SW"),mydf$max_speed,NA),
> > type="b",main="Wind speed (WSW or SW) by time",
> > xlab="Time of day",ylab="Wind speed km/h",
> > col=rainbow(16)[as.numeric(mydf$main_dir)])
> >abline(h=8,col="orange",lwd=2)
> >source("../rollmean.R")
> >rmws<-rollmean(mydf$max_speed,4)
> >lines(mydf$data_POSIX,rmws,col="orange",lwd=2)
> >legend("topleft","Rolling mean of 4 for wind speed",
> > lty=1,lwd=2,col="orange")
> >dev.off()
> >
> >Jim
>
> --
> Sent from my phone. Please excuse my brevity.

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

[R] rgl 0.100.54 using rgl.setMouseCallbacks in pan3d with rglwidget() and shiny server

2020-05-18 Thread Kennard McDaniel



I am currently attempting to implement rgl and the pan3d function on a 
shiny server but can't get the pan3d function to work. Using the right 
mouse button (2).  Here is a little sample code I was using to try to 
get it to work. Session info below the signature line.


  options(rgl.useNULL = TRUE)

  library(shiny)
  library(rgl)

app = shinyApp(
     ui =
       rglwidgetOutput("rglPlot")
  ,
   server = function(input, output) {
      output$rglPlot <- renderRglwidget({
        options(rgl.useNULL = TRUE)
        #rgl::open3d()

        # pan3d(2)

        ## setup pan function from right mouse button
        button <- 2
        dev = rgl.cur()
        subscene = currentSubscene3d(dev)

        start <- list()
        begin <- function(x, y) {
          activeSubscene <- rgl::par3d("activeSubscene", dev = dev)
          start$listeners <<- rgl::par3d("listeners", dev = dev, 
subscene = activeSubscene)
          for (sub in start$listeners) {
        init <- rgl::par3d(c("userProjection","viewport"), dev = 
dev, subscene = sub)
        init$pos <- c(x/init$viewport[3], 1 - 
y/init$viewport[4], 0.5)
        start[[as.character(sub)]] <<- init
          }
        }

        update <- function(x, y) {
          for (sub in start$listeners) {
        init <- start[[as.character(sub)]]
        xlat <- 2*(c(x/init$viewport[3], 1 - y/init$viewport[4], 
0.5) - init$pos)
        mouseMatrix <- rgl::translationMatrix(xlat[1], xlat[2], 
xlat[3])
        rgl::par3d(userProjection = mouseMatrix %*% 
init$userProjection, dev = dev, subscene = sub )
          }
        }
        rgl::rgl.setMouseCallbacks(button, begin, update, dev = dev, 
subscene = subscene)
        #cat("Callbacks set on button", button, "of rgl device", 
dev, "in subscene", subscene, "\n")


        spheres3d(rnorm(100), rnorm(100), rnorm(100,sd = 0.1), col = 
"red",
         radius = 0.1)
          axes3d()

             rglwidget()
         })
     })

runApp(app)
-- 

Thank you,

*Kennard J. McDaniel*

sessionInfo() R version 3.5.3 (2019-03-11) Platform: 
x86_64-w64-mingw32/x64 (64-bit) Running under: Windows 10 x64 (build 
18363) Matrix products: default locale: [1] LC_COLLATE=English_United 
States.1252 LC_CTYPE=English_United States.1252 [3] 
LC_MONETARY=English_United States.1252 LC_NUMERIC=C [5] 
LC_TIME=English_United States.1252 attached base packages: [1] stats 
graphics grDevices utils datasets methods base other attached packages: 
[1] NiQMatrixFoamAnalysis_0.2.4 shiny_1.4.0.2 shinyRGL_0.1.0 [4] 
rgl_0.100.54 loaded via a namespace (and not attached): [1] Rcpp_1.0.4.6 
lattice_0.20-38 tidyr_1.0.3 [4] assertthat_0.2.1 digest_0.6.25 
packrat_0.5.0 [7] mime_0.9 R6_2.4.1 plyr_1.8.6 [10] odbc_1.2.2 
evaluate_0.14 ggplot2_3.3.0 [13] pillar_1.4.4 rlang_0.4.6 qcc_2.7 [16] 
misc3d_0.8-4 rstudioapi_0.11 miniUI_0.1.1.1 [19] blob_1.2.1 
rmarkdown_2.1 webshot_0.5.2 [22] stringr_1.4.0 htmlwidgets_1.5.1 
igraph_1.2.5 [25] bit_1.1-15.2 munsell_0.5.0 compiler_3.5.3 [28] 
httpuv_1.5.2 xfun_0.13 pkgconfig_2.0.3 [31] htmltools_0.4.0 
tidyselect_1.0.0 tibble_3.0.1 [34] SixSigma_0.9-52 codetools_0.2-16 
crayon_1.3.4 [37] dplyr_0.8.5 later_1.0.0 MASS_7.3-51.6 [40] 
nat.utils_0.5.1 grid_3.5.3 jsonlite_1.6.1 [43] xtable_1.8-4 gtable_0.3.0 
lifecycle_0.2.0 [46] DBI_1.1.0 magrittr_1.5 scales_1.1.0 [49] nat_1.8.14 
stringi_1.4.6 nabor_0.5.0 [52] reshape2_1.4.4 promises_1.1.0 
testthat_2.3.2 [55] ellipsis_0.3.0 vctrs_0.2.4 plot3D_1.3 [58] 
tools_3.5.3 bit64_0.9-7 manipulateWidget_0.10.1 [61] glue_1.4.0 
purrr_0.3.4 hms_0.5.3 [64] crosstalk_1.1.0.1 fastmap_1.0.1 yaml_2.2.1 
[67] colorspace_1.4-1 filehash_2.4-2 knitr_1.28



[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] Classification of wind events

2020-05-18 Thread Jeff Newmiller

? source("../rollmean.R") ?

On May 18, 2020 4:11:52 AM PDT, Jim Lemon  wrote:
>Hi Stefano,
>If I understand your request, this may also help, Uses the same data
>transformations as my previous email.
>
>png("SS_foehn.png")
>plot(mydf$data_POSIX,
> ifelse(mydf$main_dir %in% c("WSW","SW"),mydf$max_speed,NA),
> type="b",main="Wind speed (WSW or SW) by time",
> xlab="Time of day",ylab="Wind speed km/h",
> col=rainbow(16)[as.numeric(mydf$main_dir)])
>abline(h=8,col="orange",lwd=2)
>source("../rollmean.R")
>rmws<-rollmean(mydf$max_speed,4)
>lines(mydf$data_POSIX,rmws,col="orange",lwd=2)
>legend("topleft","Rolling mean of 4 for wind speed",
> lty=1,lwd=2,col="orange")
>dev.off()
>
>Jim

-- 
Sent from my phone. Please excuse my brevity.

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] Classification of wind events

2020-05-18 Thread Jeff Newmiller

While I can understand that such techniques might not seem obvious at first, 
they are building blocks that you should be able to use to solve similar 
problems in the future. Don't give up because it surprised you this time, and 
do play with modifying it to better understand this time.

Replace code starting with calculation of foehn1d:

# calculate mean values by candidate group
mydf$foehn1c2 <- ave( mydf$max_speed
   , mydf$foehn1b
   , FUN=mean
   )
# find starts of foehns
mydf$foehn1d <- with( mydf
, 0 < diff( c( 0, foehn1c & 8 
wrote:
>Sorry for my fault.
>I am very grateful for such code, which is extremely efficient. I would
>have never been able to reach these results.
>
>In order to preserve the quality of this code, I dare to ask you a
>final question: once identified each single period in the column
>foehn1c, this period can be taken into consideration only if within it
>the mean of max_speed is higher than 8.0 (which is speed in m/s).
>Could you please help me in this final step?
>
>Thank you again for all your help
>Stefano
>
>
> (oo)
>--oOO--( )--OOo
>Stefano Sofia PhD
>Civil Protection - Marche Region
>Meteo Section
>Snow Section
>Via del Colle Ameno 5
>60126 Torrette di Ancona, Ancona
>Uff: 071 806 7743
>E-mail: stefano.so...@regione.marche.it
>---Oo-oO
>
>
>Da: Jeff Newmiller [jdnew...@dcn.davis.ca.us]
>Inviato: sabato 16 maggio 2020 21.04
>A: Stefano Sofia; Jim Lemon; r-help mailing list
>Oggetto: RE: [R] Classification of wind events
>
>Please run your code before posting it... you forgot the quotes in your
>main_dir column.
>
>first_day_POSIX <- as.POSIXct("2020-02-19-00-00",
>format="%Y-%m-%d-%H-%M")
>last_day_POSIX <- as.POSIXct("2020-02-20-00-00",
>format="%Y-%m-%d-%H-%M")
>mydf <- data.frame(data_POSIX=seq(first_day_POSIX, last_day_POSIX,
>by="10 min"))
>
>mydf$main_dir <- c("WSW", "WSW", "SW", "SW", "W", "WSW", "WSW", "WSW",
>"W", "W", "SW", "WSW", "SSW", "S", "SW", "SW", "WSW", "WNW", "W",
>"WSW", "WSW", "SE", "SE", "SE", "NW", "NNE", "ENE", "SE", "NNW", "NW",
>"NW", "NW", "NW", "NW", "NW", "NE", "NW", "NW", "NW", "NW", "NW", "N",
>"WNW", "NW", "NNW", "NNW", "NW", "NW", "NW", "WNW", "ESE", "W", "WSW",
>"SW", "SW", "SW", "WSW", "SW", "S", "S", "SSW", "SW", "WSW", "WSW",
>"WSW", "WSW", "WSW", "WSW", "WSW", "SW", "WSW", "WSW", "WSW", "WSW",
>"SW", "SW", "WSW", "WSW", "WSW", "WSW", "WSW", "SW", "SW", "SW", "SW",
>"SW", "SW", "SW", "SW", "SW", "WSW", "WSW", "WSW", "WSW", "SW", "SW",
>"SW", "SW", "WSW", "SW", "SW", "SW", "SW", "SW", "WSW", "SW", "SW",
>"W", "WSW", "WSW", "SSW", "S", "WNW", "SW", "W", "WSW", "WSW", "SE",
>"SE", "SE", "NW", "NNE", "ENE", "SE", "NNW", "NW", "NW", "NW", "NW",
>"NW", "NW", "NE", "NW", "NW", "NW", "NW", "NW", "N", "WNW", "NW",
>"NNW", "NNW", "NW", "NW", "NW")
>
>mydf$max_speed <- c(4.60, 4.60, 3.40, 3.10, 4.80, 4.20, 4.10, 4.50,
>4.70, 4.30, 2.40, 2.30, 2.20, 2.10, 2.90, 2.80, 1.80, 2.70, 4.30, 3.30,
>2.30, 2.30, 3.20, 3.20, 2.90, 2.30, 1.50, 1.80, 2.90, 2.40, 1.80, 2.40,
>2.30, 2.60, 1.80, 2.30, 1.90, 2.20, 2.80, 2.40, 1.00, 1.10, 1.60, 2.30,
>2.50, 3.30, 3.40, 3.20, 4.50, 3.90, 3.10, 2.40, 6.00, 7.80, 6.30, 7.80,
>8.10, 6.10, 7.40, 9.50, 8.90, 9.10, 10.10, 10.50, 11.10, 10.10, 10.90,
>11.30, 13.40, 13.50, 12.80, 11.50, 13.10, 13.50, 11.10, 10.50, 8.50,
>10.10, 10.70, 13.60, 11.90, 14.90, 10.90, 10.90, 12.80, 12.10, 9.10,
>8.30, 8.80, 7.40, 8.40, 10.30, 10.00, 7.00, 8.50, 8.40, 8.60, 6.70,
>7.30, 6.20, 5.90, 5.90, 5.10, 5.80, 5.60, 6.50, 6.60, 11.70, 11.30,
>8.70, 7.10, 6.90, 4.30, 3.80, 4.30, 3.30, 2.30, 2.30, 3.20, 3.20, 2.90,
>2.30, 1.50, 1.80, 2.90, 2.40, 1.80, 2.40, 2.30, 2.60, 1.80, 2.30, 1.90,
>2.20, 2.80, 2.40, 1.00, 1.10, 1.60, 2.30, 2.50, 3.30, 3.40, 3.20, 4.50)
># mark candidate rows
>mydf$foehn1a <- mydf$main_dir %in% c( "WSW", "SW" )
># mark unstable conditions
>mydf$foehn1b <- with( mydf
>, cumsum( !foehn1a )
>)
># find minimum length of foehn conditions
>mydf$foehn1c <- ave( rep( 1, nrow( mydf ) )
>   , mydf$foehn1b
>   , FUN=function(v) 10 < length( v )
>   )
># find starts of foehns
>mydf$foehn1d <- with( mydf
>, 0 < diff( c( 0, foehn1c ) )
>)
># identify foehns distinctly (multiple days)
>mydf$foehn1e <- with( mydf
>, ifelse( foehn1c
>, cumsum( foehn1d )
>, 0
>)
>)
>mydf[ , c( 1, 2, 8 ) ]
>
>On May 16, 2020 3:21:24 AM PDT, Stefano Sofia
> wrote:
>>Dear Jim and Jeff,
>>thank you for your comments. You are right, it is quite difficult to
>>detect this process through a single observation point, I am awre of
>>it.
>>I need to set up an automatic algorithm to filter 20 years of data,
>and
>>I have to find an easy way to do it.
>>I know quite

Re: [R] Help with spTransform() function and final plot colors

2020-05-18 Thread Poling, William via R-help

Thank you Ege, I appreciate your response.

I have move this to r-sig-geo.

WHP

Proprietary

-Original Message-
From: Ege Rubak  
Sent: Monday, May 18, 2020 6:41 AM
To: Poling, William ; r-help@r-project.org
Subject: [EXTERNAL] Re: [R] Help with spTransform() function and final plot 
colors

 External Email - Use Caution 

You are more likely to get help with specific problems related to
spTransform() on the dedicated list r-sig-geo.

You should provide a minimal reproducible example. Your code refers to e.g. the 
object `tmp1b`, which we don't have. I think the spTransform() part will work 
with this correction:

xy <- SpatialPointsDataFrame(sample[,2:3], sample[,1,drop=FALSE],
 proj4string=CRS("+proj=longlat
+ellps=WGS84 +datum=WGS84"))

xy <- spTransform(xy, CRS("+init=epsg:27700 +datum=WGS84"))

The spatial workflow in R has largely moved to the `sf` package and you are 
probably better off using that in the future if you are new in the game anyway.

Hope this helps,
Ege

On Sun, 2020-05-17 at 09:50 +, Poling, William via R-help wrote:
> #RStudio Version Version 1.2.1335
> sessionInfo()
> # R version 4.0.0 Patched (2020-05-03 r78349)
> #Platform: x86_64-w64-mingw32/x64 (64-bit) #Running under: Windows 10 
> x64 (build 17763)
> 
> Hello. I am running my data through a routine I found that finds 
> clusters of data points based on distance rule.
> 
https://urldefense.proofpoint.com/v2/url?u=https-3A__gis.stackexchange.com_questions_64392_finding-2Dclusters-2Dof-2Dpoints-2Dbased-2Ddistance-2Drule-2Dusing-2Dr&d=DwIGaQ&c=wluqKIiwffOpZ6k5sqMWMBOn0vyYnlulRJmmvOXCFpM&r=j7MrcIQm2xjHa8v-2mTpmTCtKvneM2ExlYvnUWbsByY&m=sFygsSovnYNWjYOpjSzuSLI1MUrJN8Lih39pDQN98xI&s=2FCT-JBdWSB66pNvGLTM7Ec5Q7IbXatXBLsQqYT0gng&e=
 
> 
> 1. I get this error when I get to this point in the routine, see 
> complete routine below?
> xy <- spTransform(xy, CRS("+init=epsg:27700 +datum=WGS84")) non finite 
> transformation detected:
> 
> I tried converting columns from numeric to Integer but did not help.
> 
> However, I continue to run the rest of the routine and the final 
> sequence, the plot itself, seems to work Can this error be corrected 
> somehow, despite the fact that it continues to work, just curious what 
> it is I guess"
> 
> 2. However in the plot at the end of the routine the color key appears 
> with the colors but the clusters in the plot are black, see plot call 
> at the end of the routine below?
> 
> Thank you for any advice.
> 
> WHP
> 
> ERROR:
> non finite transformation detected:
>   coords.x1 coords.x2
>  [1,] -119.7339  39.53939 Inf Inf
>  [2,] -119.7665  39.39630 Inf Inf
>  [3,] -119.7794  39.28768 Inf Inf
>  [4,] -121.0234  39.20503 Inf Inf
>  [5,] -122.0047  47.19262 Inf Inf
>  [6,] -122.0135  47.18883 Inf Inf
>  [7,] -122.0379  47.52190 Inf Inf
>  [8,] -122.0578  47.60975 Inf Inf
>  [9,] -122.1330  47.13669 Inf Inf
> [10,] -122.1509  47.55962 Inf Inf
> [11,] -122.1706  47.15546 Inf Inf
> [12,] -122.1846  47.23485 Inf Inf
> [13,] -122.1846  48.15307 Inf Inf
> [14,] -122.1851  47.44870 Inf Inf
> [15,] -122.1954  47.68485 Inf Inf
> [16,] -122.1990  47.51610 Inf Inf
> [17,] -122.2014  47.44772 Inf Inf
> [18,] -122.2025  47.69815 Inf Inf
> [19,] -122.2037  47.67190 Inf Inf
> [20,] -122.2090  47.40378 Inf Inf
> [21,] -122.2108  47.25336 Inf Inf
> [22,] -122.2291  47.63880 Inf Inf
> [23,] -122.2419  47.76870 Inf Inf
> [24,] -122.2722  48.04803 Inf Inf
> [25,] -122.2732  47.87700 Inf Inf
> [26,] -122.2804  47.77620 Inf Inf
> [27,] -122.2839  47.82103 Inf Inf
> [28,] -122.2890  47.86899 Inf Inf
> [29,] -122.2993  47.67306 Inf Inf
> [30,] -122.3180  47.38217 Inf Inf
> [31,] -122.3270  47.40378 Inf Inf
> [32,] -122.3474  47.43884 Inf Inf
> [33,] -122.3484  47.53083 Inf Inf
> [34,] -122.3581  47.27678 Inf Inf
> [35,] -122.3618  47.76735 Inf Inf
> [36,] -122.3700  47.56567 Inf Inf
> [37,] -122.3908  47.54938 Inf Inf
> [38,] -122.4128  47.64622 Inf Inf
> [39,] -122.4293  47.17660 Inf Inf
> [40,] -122.4621  47.44505 Inf Inf
> [41,] -122.4904  47.27460 Inf Inf
> [42,] -122.5515  46.93979 Inf Inf
> [43,] -122.7348  42.37320 Inf Inf
> [44,] -122.7827  47.31059 Inf Inf
> [45,] -122.7987  47.23475 Inf Inf
> [46,] -122.8385  42.35119 Inf Inf
> [47,] -122.8537  42.34495 Inf Inf
> [48,] -122.8904  42.39555 Inf Inf
> [49,] -122.8927  42.33022 Inf Inf
> [50,] -122.9451  47.37574 Inf Inf
> [51,] -122.9594  42.30376 Inf Inf
> [52,] -123.0641  47.16428 Inf Inf
> [53,] -123.3413  42.44117 Inf Inf
> Error in spTransform(xSP, CRSobj, ...) : 
>   failure in points
> 1:2:3:4:5:6:7:8:9:10:11:12:13:14:15:16:17:18:19:20:21:22:23:24:25:26:
> 27:28:29:30:31:32:33:34:35:36:37:38:39:40:41:42:43:44:45:46:47:48:49:
> 50:51:52:53
> In addition: Warning message:
> In spTransform(xSP, CRSobj, ...) : 53 projected point(s) not finite
> 
> Here is the data:
> dput(sample)
> structure(list(ID = 1:53, Longitude = c(-119.733899, -119.766493, 
> -119.779416, -121.0234, -122.004736, -122.013456, -122.03

Re: [R] Help with spTransform() function and final plot colors

2020-05-18 Thread Ege Rubak

You are more likely to get help with specific problems related to
spTransform() on the dedicated list r-sig-geo.

You should provide a minimal reproducible example. Your code refers to
e.g. the object `tmp1b`, which we don't have. I think the spTransform()
part will work with this correction:

xy <- SpatialPointsDataFrame(sample[,2:3], sample[,1,drop=FALSE],
 proj4string=CRS("+proj=longlat
+ellps=WGS84 +datum=WGS84"))

xy <- spTransform(xy, CRS("+init=epsg:27700 +datum=WGS84"))

The spatial workflow in R has largely moved to the `sf` package and you
are probably better off using that in the future if you are new in the
game anyway.

Hope this helps,
Ege

On Sun, 2020-05-17 at 09:50 +, Poling, William via R-help wrote:
> #RStudio Version Version 1.2.1335 
> sessionInfo() 
> # R version 4.0.0 Patched (2020-05-03 r78349)
> #Platform: x86_64-w64-mingw32/x64 (64-bit)
> #Running under: Windows 10 x64 (build 17763)
> 
> Hello. I am running my data through a routine I found that finds
> clusters of data points based on distance rule.
> 
https://gis.stackexchange.com/questions/64392/finding-clusters-of-points-based-distance-rule-using-r
> 
> 1. I get this error when I get to this point in the routine, see
> complete routine below?
> xy <- spTransform(xy, CRS("+init=epsg:27700 +datum=WGS84"))
> non finite transformation detected:
> 
> I tried converting columns from numeric to Integer but did not help.
> 
> However, I continue to run the rest of the routine and the final
> sequence, the plot itself, seems to work
> Can this error be corrected somehow, despite the fact that it
> continues to work, just curious what it is I guess"
> 
> 2. However in the plot at the end of the routine the color key
> appears with the colors but the clusters in the plot are black, see
> plot call at the end of the routine below?
> 
> Thank you for any advice.
> 
> WHP
> 
> ERROR:
> non finite transformation detected:
>   coords.x1 coords.x2
>  [1,] -119.7339  39.53939 Inf Inf
>  [2,] -119.7665  39.39630 Inf Inf
>  [3,] -119.7794  39.28768 Inf Inf
>  [4,] -121.0234  39.20503 Inf Inf
>  [5,] -122.0047  47.19262 Inf Inf
>  [6,] -122.0135  47.18883 Inf Inf
>  [7,] -122.0379  47.52190 Inf Inf
>  [8,] -122.0578  47.60975 Inf Inf
>  [9,] -122.1330  47.13669 Inf Inf
> [10,] -122.1509  47.55962 Inf Inf
> [11,] -122.1706  47.15546 Inf Inf
> [12,] -122.1846  47.23485 Inf Inf
> [13,] -122.1846  48.15307 Inf Inf
> [14,] -122.1851  47.44870 Inf Inf
> [15,] -122.1954  47.68485 Inf Inf
> [16,] -122.1990  47.51610 Inf Inf
> [17,] -122.2014  47.44772 Inf Inf
> [18,] -122.2025  47.69815 Inf Inf
> [19,] -122.2037  47.67190 Inf Inf
> [20,] -122.2090  47.40378 Inf Inf
> [21,] -122.2108  47.25336 Inf Inf
> [22,] -122.2291  47.63880 Inf Inf
> [23,] -122.2419  47.76870 Inf Inf
> [24,] -122.2722  48.04803 Inf Inf
> [25,] -122.2732  47.87700 Inf Inf
> [26,] -122.2804  47.77620 Inf Inf
> [27,] -122.2839  47.82103 Inf Inf
> [28,] -122.2890  47.86899 Inf Inf
> [29,] -122.2993  47.67306 Inf Inf
> [30,] -122.3180  47.38217 Inf Inf
> [31,] -122.3270  47.40378 Inf Inf
> [32,] -122.3474  47.43884 Inf Inf
> [33,] -122.3484  47.53083 Inf Inf
> [34,] -122.3581  47.27678 Inf Inf
> [35,] -122.3618  47.76735 Inf Inf
> [36,] -122.3700  47.56567 Inf Inf
> [37,] -122.3908  47.54938 Inf Inf
> [38,] -122.4128  47.64622 Inf Inf
> [39,] -122.4293  47.17660 Inf Inf
> [40,] -122.4621  47.44505 Inf Inf
> [41,] -122.4904  47.27460 Inf Inf
> [42,] -122.5515  46.93979 Inf Inf
> [43,] -122.7348  42.37320 Inf Inf
> [44,] -122.7827  47.31059 Inf Inf
> [45,] -122.7987  47.23475 Inf Inf
> [46,] -122.8385  42.35119 Inf Inf
> [47,] -122.8537  42.34495 Inf Inf
> [48,] -122.8904  42.39555 Inf Inf
> [49,] -122.8927  42.33022 Inf Inf
> [50,] -122.9451  47.37574 Inf Inf
> [51,] -122.9594  42.30376 Inf Inf
> [52,] -123.0641  47.16428 Inf Inf
> [53,] -123.3413  42.44117 Inf Inf
> Error in spTransform(xSP, CRSobj, ...) : 
>   failure in points
> 1:2:3:4:5:6:7:8:9:10:11:12:13:14:15:16:17:18:19:20:21:22:23:24:25:26:
> 27:28:29:30:31:32:33:34:35:36:37:38:39:40:41:42:43:44:45:46:47:48:49:
> 50:51:52:53
> In addition: Warning message:
> In spTransform(xSP, CRSobj, ...) : 53 projected point(s) not finite
> 
> Here is the data:
> dput(sample)
> structure(list(ID = 1:53, Longitude = c(-119.733899, -119.766493, 
> -119.779416, -121.0234, -122.004736, -122.013456, -122.0379, 
> -122.0578, -122.132971, -122.150901, -122.170608, -122.18462, 
> -122.184639, -122.185079, -122.195398, -122.198994, -122.201356, 
> -122.202507, -122.20371, -122.209047, -122.210797, -122.229095, 
> -122.2419, -122.27216, -122.273164, -122.280355, -122.28389, 
> -122.289043, -122.299261, -122.318009, -122.326987, -122.347382, 
> -122.34844, -122.358115, -122.361839, -122.37003, -122.390815, 
> -122.41282, -122.429323, -122.462136, -122.490417, -122.551483, 
> -122.734847, -122.782736, -122.798669, -122.838498, -122.853683, 
> -122.8904, -122.89271, -122.94511, -122.959407, -123.064087, 
> -123.34

Re: [R] Classification of wind events

2020-05-18 Thread Jim Lemon

Hi Stefano,
If I understand your request, this may also help, Uses the same data
transformations as my previous email.

png("SS_foehn.png")
plot(mydf$data_POSIX,
 ifelse(mydf$main_dir %in% c("WSW","SW"),mydf$max_speed,NA),
 type="b",main="Wind speed (WSW or SW) by time",
 xlab="Time of day",ylab="Wind speed km/h",
 col=rainbow(16)[as.numeric(mydf$main_dir)])
abline(h=8,col="orange",lwd=2)
source("../rollmean.R")
rmws<-rollmean(mydf$max_speed,4)
lines(mydf$data_POSIX,rmws,col="orange",lwd=2)
legend("topleft","Rolling mean of 4 for wind speed",
 lty=1,lwd=2,col="orange")
dev.off()

Jim
__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] Classification of wind events

2020-05-18 Thread Stefano Sofia

Sorry for my fault.
I am very grateful for such code, which is extremely efficient. I would have 
never been able to reach these results.

In order to preserve the quality of this code, I dare to ask you a final 
question: once identified each single period in the column foehn1c, this period 
can be taken into consideration only if within it the mean of max_speed is 
higher than 8.0 (which is speed in m/s).
Could you please help me in this final step?

Thank you again for all your help
Stefano


 (oo)
--oOO--( )--OOo
Stefano Sofia PhD
Civil Protection - Marche Region
Meteo Section
Snow Section
Via del Colle Ameno 5
60126 Torrette di Ancona, Ancona
Uff: 071 806 7743
E-mail: stefano.so...@regione.marche.it
---Oo-oO


Da: Jeff Newmiller [jdnew...@dcn.davis.ca.us]
Inviato: sabato 16 maggio 2020 21.04
A: Stefano Sofia; Jim Lemon; r-help mailing list
Oggetto: RE: [R] Classification of wind events

Please run your code before posting it... you forgot the quotes in your 
main_dir column.

first_day_POSIX <- as.POSIXct("2020-02-19-00-00", format="%Y-%m-%d-%H-%M")
last_day_POSIX <- as.POSIXct("2020-02-20-00-00", format="%Y-%m-%d-%H-%M")
mydf <- data.frame(data_POSIX=seq(first_day_POSIX, last_day_POSIX, by="10 min"))

mydf$main_dir <- c("WSW", "WSW", "SW", "SW", "W", "WSW", "WSW", "WSW", "W", 
"W", "SW", "WSW", "SSW", "S", "SW", "SW", "WSW", "WNW", "W", "WSW", "WSW", 
"SE", "SE", "SE", "NW", "NNE", "ENE", "SE", "NNW", "NW", "NW", "NW", "NW", 
"NW", "NW", "NE", "NW", "NW", "NW", "NW", "NW", "N", "WNW", "NW", "NNW", "NNW", 
"NW", "NW", "NW", "WNW", "ESE", "W", "WSW", "SW", "SW", "SW", "WSW", "SW", "S", 
"S", "SSW", "SW", "WSW", "WSW", "WSW", "WSW", "WSW", "WSW", "WSW", "SW", "WSW", 
"WSW", "WSW", "WSW", "SW", "SW", "WSW", "WSW", "WSW", "WSW", "WSW", "SW", "SW", 
"SW", "SW", "SW", "SW", "SW", "SW", "SW", "WSW", "WSW", "WSW", "WSW", "SW", 
"SW", "SW", "SW", "WSW", "SW", "SW", "SW", "SW", "SW", "WSW", "SW", "SW", "W", 
"WSW", "WSW", "SSW", "S", "WNW", "SW", "W", "WSW", "WSW", "SE", "SE", "SE", 
"NW", "NNE", "ENE", "SE", "NNW", "NW", "NW", "NW", "NW", "NW", "NW", "NE", 
"NW", "NW", "NW", "NW", "NW", "N", "WNW", "NW", "NNW", "NNW", "NW", "NW", "NW")

mydf$max_speed <- c(4.60, 4.60, 3.40, 3.10, 4.80, 4.20, 4.10, 4.50, 4.70, 4.30, 
2.40, 2.30, 2.20, 2.10, 2.90, 2.80, 1.80, 2.70, 4.30, 3.30, 2.30, 2.30, 3.20, 
3.20, 2.90, 2.30, 1.50, 1.80, 2.90, 2.40, 1.80, 2.40, 2.30, 2.60, 1.80, 2.30, 
1.90, 2.20, 2.80, 2.40, 1.00, 1.10, 1.60, 2.30, 2.50, 3.30, 3.40, 3.20, 4.50, 
3.90, 3.10, 2.40, 6.00, 7.80, 6.30, 7.80, 8.10, 6.10, 7.40, 9.50, 8.90, 9.10, 
10.10, 10.50, 11.10, 10.10, 10.90, 11.30, 13.40, 13.50, 12.80, 11.50, 13.10, 
13.50, 11.10, 10.50, 8.50, 10.10, 10.70, 13.60, 11.90, 14.90, 10.90, 10.90, 
12.80, 12.10, 9.10, 8.30, 8.80, 7.40, 8.40, 10.30, 10.00, 7.00, 8.50, 8.40, 
8.60, 6.70, 7.30, 6.20, 5.90, 5.90, 5.10, 5.80, 5.60, 6.50, 6.60, 11.70, 11.30, 
8.70, 7.10, 6.90, 4.30, 3.80, 4.30, 3.30, 2.30, 2.30, 3.20, 3.20, 2.90, 2.30, 
1.50, 1.80, 2.90, 2.40, 1.80, 2.40, 2.30, 2.60, 1.80, 2.30, 1.90, 2.20, 2.80, 
2.40, 1.00, 1.10, 1.60, 2.30, 2.50, 3.30, 3.40, 3.20, 4.50)
# mark candidate rows
mydf$foehn1a <- mydf$main_dir %in% c( "WSW", "SW" )
# mark unstable conditions
mydf$foehn1b <- with( mydf
, cumsum( !foehn1a )
)
# find minimum length of foehn conditions
mydf$foehn1c <- ave( rep( 1, nrow( mydf ) )
   , mydf$foehn1b
   , FUN=function(v) 10 < length( v )
   )
# find starts of foehns
mydf$foehn1d <- with( mydf
, 0 < diff( c( 0, foehn1c ) )
)
# identify foehns distinctly (multiple days)
mydf$foehn1e <- with( mydf
, ifelse( foehn1c
, cumsum( foehn1d )
, 0
)
)
mydf[ , c( 1, 2, 8 ) ]

On May 16, 2020 3:21:24 AM PDT, Stefano Sofia  
wrote:
>Dear Jim and Jeff,
>thank you for your comments. You are right, it is quite difficult to
>detect this process through a single observation point, I am awre of
>it.
>I need to set up an automatic algorithm to filter 20 years of data, and
>I have to find an easy way to do it.
>I know quite well my automatic stations, the wind direction is very
>stable during these situations, and therefore I would like to start
>from it. (I should use also wind speed, relative humidity and
>temperature, but I will introduce them only once I will be able to
>manage the direction).
>In the case of the example below reported, I know that the directions
>of this particular automatic station must be only SW or WSW.
>
>My biggest problem, obviously, is to find the beginning and the end of
>each event, when there is a change in the main direction.
>Thinking about categorical data in general, is there a way to detect
>periods when one particular category is more frequent?
>
>Here I reproduce a

Re: [R] iterators : checkFunc with ireadLines

Re: [R] iterators : checkFunc with ireadLines

Re: [R] iterators : checkFunc with ireadLines

Re: [R] iterators : checkFunc with ireadLines

Re: [R] iterators : checkFunc with ireadLines

Re: [R] iterators : checkFunc with ireadLines

Re: [R] Classification of wind events

[R] rgl 0.100.54 using rgl.setMouseCallbacks in pan3d with rglwidget() and shiny server

Re: [R] Classification of wind events

Re: [R] Classification of wind events

Re: [R] Help with spTransform() function and final plot colors

Re: [R] Help with spTransform() function and final plot colors

Re: [R] Classification of wind events

Re: [R] Classification of wind events

14 matches

Site Navigation

Mail list logo

Footer information