Re: [Rd] weird bug with parallel, RSQlite and tcltk
Hello and thank you. Indeed gsubfn is responsible for loading tcltk in my case. On Thu, Jan 3, 2013 at 12:14 PM, Gabor Grothendieck ggrothendi...@gmail.com wrote: options(gsubfn.engine = R) __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] weird bug with parallel, RSQlite and tcltk
Hello,
The point is that I do not use tcltk, it gets loaded probably as a
dependency of a dependency of a package.
When I unload it all work perfectly fine. I just found it because one
of my computer did not have tk8.5 installed, and did not exhibit the
mentioned bug. So I really think something should be done about this.
Maybe the gui loop should not be run a the the loading of the tcltk
package, but
at the first function ran, or something like this.
As you can see in my example code, the in-memory database is opened in
the parallel code...
Best,
Karl
On Mon, Dec 31, 2012 at 10:58 PM, Simon Urbanek
simon.urba...@r-project.org wrote:
On Dec 31, 2012, at 1:08 PM, Karl Forner wrote:
Hello,
I spent a lot of a time on a weird bug, and I just managed to narrow it down.
First, tcltk and multicore don't mix well, see the warning in the
documentation (it mentions GUIs and AFAIR tcltk fires up a GUI event loop
even if you don't actually create GUI elements). Second, using any kind of
descriptors in parallel code is asking for trouble since those will be owned
by multiple processes. If you use databases files, etc. they must be opened
in the parallel code, they cannot be shared by multiple workers. The latter
is ok in your code so you're probably bitten by the former.
Cheers,
Simon
In parallel code (here with parallel::mclappy, but I got it
doMC/multicore too), if the library(tcltk) is loaded, R hangs when
trying to open a DB connection.
I got the same behaviour on two different computers, one dual-core,
and one 2 xeon quad-core.
Here's the code:
library(parallel)
library(RSQLite)
library(tcltk)
#unloadNamespace(tcltk)
res - mclapply(1:2, function(x) {
db - DBI::dbConnect(SQLite, :memory:)
}, mc.cores=2)
print(Done)
When I execute it (R --vanilla test_parallel_db.R), it hangs
forever, and I have to type several times CTRL+C to interrupt it. I
then get this message:
Warning messages:
1: In selectChildren(ac, 1) : error 'Interrupted system call' in select
2: In selectChildren(ac, 1) : error 'Interrupted system call' in select
Then, just remove library(tcltk), or uncomment
unloadNamespace(tcltk), and it works fine again.
I guess there's a bug somewhere, but where exactly ?
Best,
Karl Forner
Further info:
R version 2.15.1 (2012-06-22) -- Roasted Marshmallows
Copyright (C) 2012 The R Foundation for Statistical Computing
ISBN 3-900051-07-0
Platform: x86_64-unknown-linux-gnu (64-bit)
ubuntu 12.04 and 12.10
ubuntu package tk8.5
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] weird bug with parallel, RSQlite and tcltk
On 13-01-03 6:20 AM, Gabor Grothendieck wrote: On Thu, Jan 3, 2013 at 5:59 AM, peter dalgaard pda...@gmail.com wrote: On Jan 3, 2013, at 10:32 , Karl Forner wrote: Hello, The point is that I do not use tcltk, it gets loaded probably as a dependency of a dependency of a package. When I unload it all work perfectly fine. I just found it because one of my computer did not have tk8.5 installed, and did not exhibit the mentioned bug. So I really think something should be done about this. Maybe the gui loop should not be run a the the loading of the tcltk package, but at the first function ran, or something like this. Doesn't sound doable. It would be tricky to do and wouldn't help in the cases where people actually want to use the GUI - plus, it would leave a time bomb if you directly or indirectly fire up a Tk window (say, the CRAN menu from install.packages()). Would it be possible to separate the tk and tcl portions of tcltk into two packages so that the tcltk package would be dependent on a new tcl package ? From the viewpoint of a user of tcltk there would be no change but it would allow packages that only use tcl to declare a dependency only on that. By the way, tcl/tk 8.6 was released about 10 days ago with a ton of attractive new features and it would be nice to have it in the R windows build: http://www.tcl.tk/software/tcltk/8.6.html I think it's unlikely that we will upgrade. The binaries available at that link do not have a license that would allow distribution with R, and we no longer have the facilities (specific compiler, time, etc.) to compile it ourselves. I haven't tried myself, but my understanding is that the sources are not compatible with our compiler suite. Duncan Murdoch __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] weird bug with parallel, RSQlite and tcltk
On 03/01/2013 11:42, Duncan Murdoch wrote: On 13-01-03 6:20 AM, Gabor Grothendieck wrote: On Thu, Jan 3, 2013 at 5:59 AM, peter dalgaard pda...@gmail.com wrote: On Jan 3, 2013, at 10:32 , Karl Forner wrote: Hello, The point is that I do not use tcltk, it gets loaded probably as a dependency of a dependency of a package. When I unload it all work perfectly fine. I just found it because one of my computer did not have tk8.5 installed, and did not exhibit the mentioned bug. So I really think something should be done about this. Maybe the gui loop should not be run a the the loading of the tcltk package, but at the first function ran, or something like this. Doesn't sound doable. It would be tricky to do and wouldn't help in the cases where people actually want to use the GUI - plus, it would leave a time bomb if you directly or indirectly fire up a Tk window (say, the CRAN menu from install.packages()). Would it be possible to separate the tk and tcl portions of tcltk into two packages so that the tcltk package would be dependent on a new tcl package ? From the viewpoint of a user of tcltk there would be no change but it would allow packages that only use tcl to declare a dependency only on that. By the way, tcl/tk 8.6 was released about 10 days ago with a ton of attractive new features and it would be nice to have it in the R windows build: http://www.tcl.tk/software/tcltk/8.6.html I think it's unlikely that we will upgrade. The binaries available at that link do not have a license that would allow distribution with R, and we no longer have the facilities (specific compiler, time, etc.) to compile it ourselves. I haven't tried myself, but my understanding is that the sources are not compatible with our compiler suite. More specifically, when last checked the sources do compile but do not work when loaded into R as memory gets corrupted. Note that the version of Tcl/Tk made available for the CRAN Mac binary is older than that for Windows. Duncan Murdoch __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel -- Brian D. Ripley, rip...@stats.ox.ac.uk Professor of Applied Statistics, http://www.stats.ox.ac.uk/~ripley/ University of Oxford, Tel: +44 1865 272861 (self) 1 South Parks Road, +44 1865 272866 (PA) Oxford OX1 3TG, UKFax: +44 1865 272595 __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] weird bug with parallel, RSQlite and tcltk
On Jan 3, 2013, at 12:20 , Gabor Grothendieck wrote: On Thu, Jan 3, 2013 at 5:59 AM, peter dalgaard pda...@gmail.com wrote: On Jan 3, 2013, at 10:32 , Karl Forner wrote: Hello, The point is that I do not use tcltk, it gets loaded probably as a dependency of a dependency of a package. When I unload it all work perfectly fine. I just found it because one of my computer did not have tk8.5 installed, and did not exhibit the mentioned bug. So I really think something should be done about this. Maybe the gui loop should not be run a the the loading of the tcltk package, but at the first function ran, or something like this. Doesn't sound doable. It would be tricky to do and wouldn't help in the cases where people actually want to use the GUI - plus, it would leave a time bomb if you directly or indirectly fire up a Tk window (say, the CRAN menu from install.packages()). Would it be possible to separate the tk and tcl portions of tcltk into two packages so that the tcltk package would be dependent on a new tcl package ? From the viewpoint of a user of tcltk there would be no change but it would allow packages that only use tcl to declare a dependency only on that. Not sure that actually solves much. We do this already if the DISPLAY is unset for X11, so the logic should not be too hard to extend, but the event loop belongs to Tcl, Tk events are just a special type of events. By the way, tcl/tk 8.6 was released about 10 days ago with a ton of attractive new features and it would be nice to have it in the R windows build: http://www.tcl.tk/software/tcltk/8.6.html -- Peter Dalgaard, Professor Center for Statistics, Copenhagen Business School Solbjerg Plads 3, 2000 Frederiksberg, Denmark Phone: (+45)38153501 Email: pd@cbs.dk Priv: pda...@gmail.com __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] weird bug with parallel, RSQlite and tcltk
On Jan 3, 2013, at 13:13 , Prof Brian Ripley wrote: On 03/01/2013 11:42, Duncan Murdoch wrote: On 13-01-03 6:20 AM, Gabor Grothendieck wrote: [snipped] I think it's unlikely that we will upgrade. The binaries available at that link do not have a license that would allow distribution with R, and we no longer have the facilities (specific compiler, time, etc.) to compile it ourselves. I haven't tried myself, but my understanding is that the sources are not compatible with our compiler suite. More specifically, when last checked the sources do compile but do not work when loaded into R as memory gets corrupted. Note that the version of Tcl/Tk made available for the CRAN Mac binary is older than that for Windows. Of course anyone is welcome to try their hand at getting Tcl/Tk to build properly with the R compiler suite. And/or dig into what might be getting in the way. Or, for that matter, build against the ActiveTcl binaries; it is only redistribution that is disallowed --- it is Free as in Beer, but not as in Speech. (Looks like MacPorts moved to Tcl/Tk 8.6. I'll see if it works with their R port.) -- Peter Dalgaard, Professor Center for Statistics, Copenhagen Business School Solbjerg Plads 3, 2000 Frederiksberg, Denmark Phone: (+45)38153501 Email: pd@cbs.dk Priv: pda...@gmail.com __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] weird bug with parallel, RSQlite and tcltk
On Thu, Jan 3, 2013 at 7:45 AM, peter dalgaard pda...@gmail.com wrote: On Jan 3, 2013, at 12:20 , Gabor Grothendieck wrote: On Thu, Jan 3, 2013 at 5:59 AM, peter dalgaard pda...@gmail.com wrote: On Jan 3, 2013, at 10:32 , Karl Forner wrote: Hello, The point is that I do not use tcltk, it gets loaded probably as a dependency of a dependency of a package. When I unload it all work perfectly fine. I just found it because one of my computer did not have tk8.5 installed, and did not exhibit the mentioned bug. So I really think something should be done about this. Maybe the gui loop should not be run a the the loading of the tcltk package, but at the first function ran, or something like this. Doesn't sound doable. It would be tricky to do and wouldn't help in the cases where people actually want to use the GUI - plus, it would leave a time bomb if you directly or indirectly fire up a Tk window (say, the CRAN menu from install.packages()). Would it be possible to separate the tk and tcl portions of tcltk into two packages so that the tcltk package would be dependent on a new tcl package ? From the viewpoint of a user of tcltk there would be no change but it would allow packages that only use tcl to declare a dependency only on that. Not sure that actually solves much. We do this already if the DISPLAY is unset for X11, so the logic should not be too hard to extend, but the event loop belongs to Tcl, Tk events are just a special type of events. OK. I had assumed the event loop was to process the GUI events only and therefore would be in tk. Another possibility might be just to have a standard option that users or other packages could set to indicate that they don't have or don't want to use tcltk so packages that do use tcltk could examine that option and act accordingly (e.g. use alternate code or issue a meaningful error). I already use options(gsubfn.engine = R) for that purpose but that only works for my packages and if something were standardized in tcltk then it could be more broadly applicable and more visible. As far as tcltk this is basically just a matter of documentation to set the standard but for the parallel package, say, the option could be automatically set in the workers. -- Statistics Software Consulting GKX Group, GKX Associates Inc. tel: 1-877-GKX-GROUP email: ggrothendieck at gmail.com __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] weird bug with parallel, RSQlite and tcltk
On Jan 3, 2013, at 5:59 AM, peter dalgaard wrote:
On Jan 3, 2013, at 10:32 , Karl Forner wrote:
Hello,
The point is that I do not use tcltk, it gets loaded probably as a
dependency of a dependency of a package.
When I unload it all work perfectly fine. I just found it because one
of my computer did not have tk8.5 installed, and did not exhibit the
mentioned bug. So I really think something should be done about this.
Maybe the gui loop should not be run a the the loading of the tcltk
package, but
at the first function ran, or something like this.
Doesn't sound doable. It would be tricky to do and wouldn't help in the cases
where people actually want to use the GUI - plus, it would leave a time bomb
if you directly or indirectly fire up a Tk window (say, the CRAN menu from
install.packages()).
More likely, the wisdom of calling R_ProcessEvents and R_PolledEvents in
parallel processes should be questioned. I tend to think that they should
both be disabled completely conditionally on R_isForkedChild. At least in the
Tk loop, some of the events are generated as responses to specific queries,
and having one process ask for something and another one handling the reply,
leaving the first one waiting indefinitely, is just Not Right.
On the other hand, I'm unsure whether the parent process has its hands free
to handle events while a set of parallel processes are in operation. If not,
then I suppose that the GUI could get stopped dead in its tracks.
But it would anyway - children cannot process the events for the parent, so
event processing should be disabled (they are all separate processes after
all). The parent may be tied up in a select loop but that's ok as we know how
to deal with it and it has hooks as needed (and it would be trivial to add
explicit event processing if we think it's a good idea ...).
We already disable it for Cocoa .. the question it whether we should also
remove all input handlers (they are likely to break as the FDs will be shared
by the parent *and* children) or avoid only R_PolledEvents() ...
Cheers,
Simon
As you can see in my example code, the in-memory database is opened in
the parallel code...
Best,
Karl
On Mon, Dec 31, 2012 at 10:58 PM, Simon Urbanek
simon.urba...@r-project.org wrote:
On Dec 31, 2012, at 1:08 PM, Karl Forner wrote:
Hello,
I spent a lot of a time on a weird bug, and I just managed to narrow it
down.
First, tcltk and multicore don't mix well, see the warning in the
documentation (it mentions GUIs and AFAIR tcltk fires up a GUI event loop
even if you don't actually create GUI elements). Second, using any kind of
descriptors in parallel code is asking for trouble since those will be
owned by multiple processes. If you use databases files, etc. they must be
opened in the parallel code, they cannot be shared by multiple workers. The
latter is ok in your code so you're probably bitten by the former.
Cheers,
Simon
In parallel code (here with parallel::mclappy, but I got it
doMC/multicore too), if the library(tcltk) is loaded, R hangs when
trying to open a DB connection.
I got the same behaviour on two different computers, one dual-core,
and one 2 xeon quad-core.
Here's the code:
library(parallel)
library(RSQLite)
library(tcltk)
#unloadNamespace(tcltk)
res - mclapply(1:2, function(x) {
db - DBI::dbConnect(SQLite, :memory:)
}, mc.cores=2)
print(Done)
When I execute it (R --vanilla test_parallel_db.R), it hangs
forever, and I have to type several times CTRL+C to interrupt it. I
then get this message:
Warning messages:
1: In selectChildren(ac, 1) : error 'Interrupted system call' in select
2: In selectChildren(ac, 1) : error 'Interrupted system call' in select
Then, just remove library(tcltk), or uncomment
unloadNamespace(tcltk), and it works fine again.
I guess there's a bug somewhere, but where exactly ?
Best,
Karl Forner
Further info:
R version 2.15.1 (2012-06-22) -- Roasted Marshmallows
Copyright (C) 2012 The R Foundation for Statistical Computing
ISBN 3-900051-07-0
Platform: x86_64-unknown-linux-gnu (64-bit)
ubuntu 12.04 and 12.10
ubuntu package tk8.5
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
--
Peter Dalgaard, Professor
Center for Statistics, Copenhagen Business School
Solbjerg Plads 3, 2000 Frederiksberg, Denmark
Phone: (+45)38153501
Email: pd@cbs.dk Priv: pda...@gmail.com
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
[Rd] weird bug with parallel, RSQlite and tcltk
Hello,
I spent a lot of a time on a weird bug, and I just managed to narrow it down.
In parallel code (here with parallel::mclappy, but I got it
doMC/multicore too), if the library(tcltk) is loaded, R hangs when
trying to open a DB connection.
I got the same behaviour on two different computers, one dual-core,
and one 2 xeon quad-core.
Here's the code:
library(parallel)
library(RSQLite)
library(tcltk)
#unloadNamespace(tcltk)
res - mclapply(1:2, function(x) {
db - DBI::dbConnect(SQLite, :memory:)
}, mc.cores=2)
print(Done)
When I execute it (R --vanilla test_parallel_db.R), it hangs
forever, and I have to type several times CTRL+C to interrupt it. I
then get this message:
Warning messages:
1: In selectChildren(ac, 1) : error 'Interrupted system call' in select
2: In selectChildren(ac, 1) : error 'Interrupted system call' in select
Then, just remove library(tcltk), or uncomment
unloadNamespace(tcltk), and it works fine again.
I guess there's a bug somewhere, but where exactly ?
Best,
Karl Forner
Further info:
R version 2.15.1 (2012-06-22) -- Roasted Marshmallows
Copyright (C) 2012 The R Foundation for Statistical Computing
ISBN 3-900051-07-0
Platform: x86_64-unknown-linux-gnu (64-bit)
ubuntu 12.04 and 12.10
ubuntu package tk8.5
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] weird bug with parallel, RSQlite and tcltk
On Dec 31, 2012, at 4:20 PM, Marius Hofert wrote: Dear Karl, I get exactly the same warning messages with an Rmpi minimal example (also took me a while to narrow it down). I posted it on R-sig-hpc (more appropriate for these issues I believe), some days ago, but haven't received an answer yet. AFAIR MPI doesn't support forking. Cheers, Simon Cheers, Marius PS: https://stat.ethz.ch/pipermail/r-sig-hpc/2012-December/001572.html __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
