Hi, Harold--
The short answer is "Yes"--in your example, the nextElem will always be a list
with the i component equal to the next element of itx1 and the j component
equal to the next element of itx2.
I posted a more detailed explanation in response to a query from David on the
Microsoft TechNet forum:
https://social.technet.microsoft.com/Forums/en-US/724b1dde-03e3-4fff-b061-363bc8ba1652/how-are-multiple-iterobjects-handled-by-foreach?forum=ropen
Cheers,
Rich
Rich Calaway
Release Manager
Microsoft R Product Team
24/1341
+1 (425) 4219919 X19919
Message: 2
Date: Fri, 9 Dec 2016 17:15:38 +
From: "Doran, Harold"
To: "r-help@r-project.org"
Subject: [R] "Safe" use of iterator (package iterators)
Message-ID:
Content-Type: text/plain; charset="iso-8859-1"
I believe I now see the light vis-?-vis iterators when combined with foreach()
calls in R. I have now been able to reduce computational workload to minutes
instead of hours. I want to verify that the way I am using them is "safe". By
safe I mean does the iterator traverse elements in the same way as I have below
in my toy example to illustrate what I mean.
In the first "traditional" example, I have only one index variable for the loop
and so I know that the same list in r1 and r2 are always being grabbed. That
is, in iteration 1 it is guaranteed to use r1[[1]] + r2[[1]].
In the example that uses the iterators, is this also guaranteed even though I
now have two iterator objects? That is, will the index for element i always be
the same as the index for element j when using this across many different cores?
It seems to be true and in all my test cases so far I am seeing it to be true.
But, that could be just luck, so I wonder if there is a condition under which
that would NOT be true.
Thank you
Harold
library(foreach)
library(doParallel)
cl <- makeCluster(2)
registerDoParallel(cl)
### Create random data
r1 <- vector("list", 20)
for(i in 1:20){
r1[[i]] <- rnorm(10)
}
### Create random data
r2 <- vector("list", 20)
for(i in 1:20){
r2[[i]] <- rnorm(10)
}
### Use a for loop traditionally
result1 <- vector("list", 20)
for(i in 1:20){
result1[[i]] <- r1[[i]] + r2[[i]]
}
### Use iterators
itx1 <- iter(r1)
itx2 <- iter(r2)
result2 <- foreach(i = itx1, j = itx2) %dopar% {
i + j
}
all.equal(result1, result2)
Message: 4
Date: Fri, 9 Dec 2016 10:26:55 -0800
From: David Winsemius
To: "Doran, Harold"
Cc: "r-help@r-project.org"
Subject: Re: [R] "Safe" use of iterator (package iterators)
Message-ID:
Content-Type: text/plain; charset=iso-8859-1
> On Dec 9, 2016, at 9:15 AM, Doran, Harold wrote:
>
> I believe I now see the light vis-?-vis iterators when combined with
> foreach() calls in R. I have now been able to reduce computational workload
> to minutes instead of hours. I want to verify that the way I am using them is
> "safe". By safe I mean does the iterator traverse elements in the same way as
> I have below in my toy example to illustrate what I mean.
>
> In the first "traditional" example, I have only one index variable for the
> loop and so I know that the same list in r1 and r2 are always being grabbed.
> That is, in iteration 1 it is guaranteed to use r1[[1]] + r2[[1]].
>
> In the example that uses the iterators, is this also guaranteed even though I
> now have two iterator objects? That is, will the index for element i always
> be the same as the index for element j when using this across many different
> cores?
>
> It seems to be true and in all my test cases so far I am seeing it to be
> true. But, that could be just luck, so I wonder if there is a condition under
> which that would NOT be true.
>
> Thank you
> Harold
>
>
> library(foreach)
> library(doParallel)
> cl <- makeCluster(2)
> registerDoParallel(cl)
>
> ### Create random data
> r1 <- vector("list", 20)
> for(i in 1:20){
> r1[[i]] <- rnorm(10)
> }
>
> ### Create random data
> r2 <- vector("list", 20)
> for(i in 1:20){
> r2[[i]] <- rnorm(10)
> }
>
> ### Use a for loop traditionally
> result1 <- vector("list", 20)
> for(i in 1:20){
> result1[[i]] <- r1[[i]] + r2[[i]]
> }
>
> ### Use iterators
> itx1 <- iter(r1)
> itx2 <- iter(r2)
>
> result2 <- foreach(i = itx1, j = itx2) %dopar% {
> i + j
> }
>
> all.equal(result1, result2)
I wasn't sure how this would or should behave. I'm not an experienced user,
merely a reader of help pages. Neither the help page, not the vignette
references on the help page answered my questions in this case. I expected that
call would behave analogously to the behavior of mapply when given iterators of
unequal length. (The shorter of the objects is recycled to reach the