Re: [R] R parallel / foreach - aggregation of results
You can always just pull the last one off the list. When running things in
parallel, what does the last one mean? Do you want the last from each of
the parallel threads, or just the last one on the list? You might want to
put some flag on the data being returned so you can determine which one you
want to process.
Jim Holtman
Data Munger Guru
What is the problem that you are trying to solve?
Tell me what you want to do, not how you want to do it.
On Sat, Aug 1, 2015 at 3:19 PM, Martin Spindler martin.spind...@gmx.de
wrote:
Dear Jim,
Thank you very much for your response. It seems to work now, but the
return value is not the required matrix but a list of matrices (one for
each repition j).
Any idea how it is possible to return only the last matrix and not all?
Thanks and best,
Martin
Gesendet: Freitag, 31. Juli 2015 um 18:22 Uhr
Von: jim holtman jholt...@gmail.com
An: Martin Spindler martin.spind...@gmx.de
Cc: r-help@r-project.org r-help@r-project.org
Betreff: Re: [R] R parallel / foreach - aggregation of results
Try this chance to actually return values:
library(doParallel)
Simpar3 - function(n1) {
L2distance - matrix(NA, ncol=n1, nrow=n1)
data - rnorm(n1)
diag(L2distance)=0
cl - makeCluster(4)
registerDoParallel(cl)
x - foreach(j=1:n1) %dopar% {
library(np)
datj - data[j]
for(k in j:n1) {
L2distance[j,k] - k*datj
}
L2distance # return the value
}
stopCluster(cl)
return(x)
}
Res - Simpar3(100)
Jim Holtman
Data Munger Guru
What is the problem that you are trying to solve?
Tell me what you want to do, not how you want to do it.
On Fri, Jul 31, 2015 at 8:39 AM, Martin Spindler martin.spind...@gmx.de
wrote:Dear all,
when I am running the code attached below, it seems that no results are
returned, only the predefined NAs. What mistake do I make?
Any comments and help is highly appreciated.
Thanks and best,
Martin
Simpar3 - function(n1) {
L2distance - matrix(NA, ncol=n1, nrow=n1)
data - rnorm(n1)
diag(L2distance)=0
cl - makeCluster(4)
registerDoParallel(cl)
foreach(j=1:n1) %dopar% {
library(np)
datj - data[j]
for(k in j:n1) {
L2distance[j,k] - k*datj
}
}
stopCluster(cl)
return(L2distance)
}
Res - Simpar3(100)
__
R-help@r-project.org[R-help@r-project.org] mailing list -- To UNSUBSCRIBE
and more, see
https://stat.ethz.ch/mailman/listinfo/r-help[https://stat.ethz.ch/mailman/listinfo/r-help]
PLEASE do read the posting guide
http://www.R-project.org/posting-guide.html[http://www.R-project.org/posting-guide.html]
and provide commented, minimal, self-contained, reproducible code.
[[alternative HTML version deleted]]
__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Re: [R] R parallel / foreach - aggregation of results
Dear Jim,
Thank you very much for your response. It seems to work now, but the return
value is not the required matrix but a list of matrices (one for each repition
j).
Any idea how it is possible to return only the last matrix and not all?
Thanks and best,
Martin
Gesendet: Freitag, 31. Juli 2015 um 18:22 Uhr
Von: jim holtman jholt...@gmail.com
An: Martin Spindler martin.spind...@gmx.de
Cc: r-help@r-project.org r-help@r-project.org
Betreff: Re: [R] R parallel / foreach - aggregation of results
Try this chance to actually return values:
library(doParallel)
Simpar3 - function(n1) {
L2distance - matrix(NA, ncol=n1, nrow=n1)
data - rnorm(n1)
diag(L2distance)=0
cl - makeCluster(4)
registerDoParallel(cl)
x - foreach(j=1:n1) %dopar% {
library(np)
datj - data[j]
for(k in j:n1) {
L2distance[j,k] - k*datj
}
L2distance # return the value
}
stopCluster(cl)
return(x)
}
Res - Simpar3(100)
Jim Holtman
Data Munger Guru
What is the problem that you are trying to solve?
Tell me what you want to do, not how you want to do it.
On Fri, Jul 31, 2015 at 8:39 AM, Martin Spindler martin.spind...@gmx.de
wrote:Dear all,
when I am running the code attached below, it seems that no results are
returned, only the predefined NAs. What mistake do I make?
Any comments and help is highly appreciated.
Thanks and best,
Martin
Simpar3 - function(n1) {
L2distance - matrix(NA, ncol=n1, nrow=n1)
data - rnorm(n1)
diag(L2distance)=0
cl - makeCluster(4)
registerDoParallel(cl)
foreach(j=1:n1) %dopar% {
library(np)
datj - data[j]
for(k in j:n1) {
L2distance[j,k] - k*datj
}
}
stopCluster(cl)
return(L2distance)
}
Res - Simpar3(100)
__
R-help@r-project.org[R-help@r-project.org] mailing list -- To UNSUBSCRIBE and
more, see
https://stat.ethz.ch/mailman/listinfo/r-help[https://stat.ethz.ch/mailman/listinfo/r-help]
PLEASE do read the posting guide
http://www.R-project.org/posting-guide.html[http://www.R-project.org/posting-guide.html]
and provide commented, minimal, self-contained, reproducible code.
__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Re: [R] R parallel / foreach - aggregation of results
If you return just the row that the foreach procedure produces instead of
the entire matrix containing that row and use .combine=rbind then you will
end up with the matrix of interest. E.g.,
Simpar3a - function (n1)
{
L2distance - matrix(NA, ncol = n1, nrow = n1)
data - rnorm(n1)
diag(L2distance) = 0
cl - makeCluster(4)
registerDoParallel(cl)
x - foreach(j = 1:n1, .combine = rbind) %dopar% {
library(np)
datj - data[j]
rowJ - numeric(n1)
for (k in j:n1) {
rowJ[k] - k * datj
}
rowJ
}
stopCluster(cl)
x
}
Bill Dunlap
TIBCO Software
wdunlap tibco.com
On Sat, Aug 1, 2015 at 12:19 PM, Martin Spindler martin.spind...@gmx.de
wrote:
Dear Jim,
Thank you very much for your response. It seems to work now, but the
return value is not the required matrix but a list of matrices (one for
each repition j).
Any idea how it is possible to return only the last matrix and not all?
Thanks and best,
Martin
Gesendet: Freitag, 31. Juli 2015 um 18:22 Uhr
Von: jim holtman jholt...@gmail.com
An: Martin Spindler martin.spind...@gmx.de
Cc: r-help@r-project.org r-help@r-project.org
Betreff: Re: [R] R parallel / foreach - aggregation of results
Try this chance to actually return values:
library(doParallel)
Simpar3 - function(n1) {
L2distance - matrix(NA, ncol=n1, nrow=n1)
data - rnorm(n1)
diag(L2distance)=0
cl - makeCluster(4)
registerDoParallel(cl)
x - foreach(j=1:n1) %dopar% {
library(np)
datj - data[j]
for(k in j:n1) {
L2distance[j,k] - k*datj
}
L2distance # return the value
}
stopCluster(cl)
return(x)
}
Res - Simpar3(100)
Jim Holtman
Data Munger Guru
What is the problem that you are trying to solve?
Tell me what you want to do, not how you want to do it.
On Fri, Jul 31, 2015 at 8:39 AM, Martin Spindler martin.spind...@gmx.de
wrote:Dear all,
when I am running the code attached below, it seems that no results are
returned, only the predefined NAs. What mistake do I make?
Any comments and help is highly appreciated.
Thanks and best,
Martin
Simpar3 - function(n1) {
L2distance - matrix(NA, ncol=n1, nrow=n1)
data - rnorm(n1)
diag(L2distance)=0
cl - makeCluster(4)
registerDoParallel(cl)
foreach(j=1:n1) %dopar% {
library(np)
datj - data[j]
for(k in j:n1) {
L2distance[j,k] - k*datj
}
}
stopCluster(cl)
return(L2distance)
}
Res - Simpar3(100)
__
R-help@r-project.org[R-help@r-project.org] mailing list -- To UNSUBSCRIBE
and more, see
https://stat.ethz.ch/mailman/listinfo/r-help[https://stat.ethz.ch/mailman/listinfo/r-help]
PLEASE do read the posting guide
http://www.R-project.org/posting-guide.html[http://www.R-project.org/posting-guide.html]
and provide commented, minimal, self-contained, reproducible code.
__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide
http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
[[alternative HTML version deleted]]
__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
[R] R parallel / foreach - aggregation of results
Dear all,
when I am running the code attached below, it seems that no results are
returned, only the predefined NAs. What mistake do I make?
Any comments and help is highly appreciated.
Thanks and best,
Martin
Simpar3 - function(n1) {
L2distance - matrix(NA, ncol=n1, nrow=n1)
data - rnorm(n1)
diag(L2distance)=0
cl - makeCluster(4)
registerDoParallel(cl)
foreach(j=1:n1) %dopar% {
library(np)
datj - data[j]
for(k in j:n1) {
L2distance[j,k] - k*datj
}
}
stopCluster(cl)
return(L2distance)
}
Res - Simpar3(100)
__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Re: [R] R parallel / foreach - aggregation of results
Martin,
I think the main problem is that you are trying to assign your results
to the result matrix inside the foreach loop. Parallel functions in R
are generally not good at updating parts of matrices from the different
workers in this way. Instead, using e.g. foreach, each loop of the
foreach-call has to return a vector which can be cbind-ed to a result
matrix. Something like:
L2distance = foreach(j=1:n1, .combine = cbind) %dopar% {
res = rep(NA, 10)
for (k in j:n1) res[k] = k*data[j]
res
}
L2distance
I am not sure what the np-library is, but you should consider putting it
in a clusterExport-call after creating the cluster.
Best wishes,
Jon
On 7/31/2015 2:39 PM, Martin Spindler wrote:
Dear all,
when I am running the code attached below, it seems that no results are
returned, only the predefined NAs. What mistake do I make?
Any comments and help is highly appreciated.
Thanks and best,
Martin
Simpar3 - function(n1) {
L2distance - matrix(NA, ncol=n1, nrow=n1)
data - rnorm(n1)
diag(L2distance)=0
cl - makeCluster(4)
registerDoParallel(cl)
foreach(j=1:n1) %dopar% {
library(np)
datj - data[j]
for(k in j:n1) {
L2distance[j,k] - k*datj
}
}
stopCluster(cl)
return(L2distance)
}
Res - Simpar3(100)
__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
--
Jon Olav Skøien
Joint Research Centre - European Commission
Institute for Environment and Sustainability (IES)
Climate Risk Management Unit
Via Fermi 2749, TP 100-01, I-21027 Ispra (VA), ITALY
jon.sko...@jrc.ec.europa.eu
Tel: +39 0332 789205
Disclaimer: Views expressed in this email are those of the individual
and do not necessarily represent official views of the European Commission.
__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Re: [R] R parallel / foreach - aggregation of results
Try this chance to actually return values:
library(doParallel)
Simpar3 - function(n1) {
L2distance - matrix(NA, ncol=n1, nrow=n1)
data - rnorm(n1)
diag(L2distance)=0
cl - makeCluster(4)
registerDoParallel(cl)
x - foreach(j=1:n1) %dopar% {
library(np)
datj - data[j]
for(k in j:n1) {
L2distance[j,k] - k*datj
}
L2distance # return the value
}
stopCluster(cl)
return(x)
}
Res - Simpar3(100)
Jim Holtman
Data Munger Guru
What is the problem that you are trying to solve?
Tell me what you want to do, not how you want to do it.
On Fri, Jul 31, 2015 at 8:39 AM, Martin Spindler martin.spind...@gmx.de
wrote:
Dear all,
when I am running the code attached below, it seems that no results are
returned, only the predefined NAs. What mistake do I make?
Any comments and help is highly appreciated.
Thanks and best,
Martin
Simpar3 - function(n1) {
L2distance - matrix(NA, ncol=n1, nrow=n1)
data - rnorm(n1)
diag(L2distance)=0
cl - makeCluster(4)
registerDoParallel(cl)
foreach(j=1:n1) %dopar% {
library(np)
datj - data[j]
for(k in j:n1) {
L2distance[j,k] - k*datj
}
}
stopCluster(cl)
return(L2distance)
}
Res - Simpar3(100)
__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide
http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
[[alternative HTML version deleted]]
__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
