Ryan, I agree that in some sense it is a different problem, but my point is with a different approach we can easily answer both. The code I posted returns a named character vector of symbol names with package name being the name.
This makes it a trivial lookup to determine both a) what symbols aren't available in any of the packages and b) what packages provide the remaining required symbols. No extra work required. You do have to give it a list of packages to check, but it is easy to write a wrapper that automatically passes it all currently attached packages if desired (a combination of search() and gsub() would be a quick and dirty way to do this). All that said, I'm simply trying to help. If you guys don't want to use my code/approach that is your perogative as I'm not currently working on BiocParallel myself. ~G On Mon, Nov 4, 2013 at 11:54 AM, Ryan Thompson <r...@thompsonclan.org> wrote: > The code that I wrote intentionally avoids checking for package variables, > since I consider that a separate problem. Package variables can be provided > to the child by leading the package, whereas user-defined variables must be > serialized in the parent and sent to the child. > > I think I could fairly easily adapt the same code to return a list of all > packages that a function depends on. > > -Ryan > On Nov 4, 2013 11:35 AM, "Michael Lawrence" <lawrence.mich...@gene.com> > wrote: > >> The dynamic nature of R limits the extent of these checks. But as Ryan has >> noted, a simple sanity check goes a long way. If what he has done could be >> extended to the rest of the search path (people always forget to attach >> packages), I think we've hit the 80% with 20%. Got a 404 on that URL btw. >> >> Michael >> >> >> On Mon, Nov 4, 2013 at 11:05 AM, Gabriel Becker <gmbec...@ucdavis.edu >> >wrote: >> >> > Hey guys, >> > >> > Here is code that I have written which resolves library names into a >> full >> > list of symbols: >> > >> > >> https://github.com/duncantl/CodeDepends/blob/forCRAN_0.3.5/R/librarySymbols.RNote >> > this does not require that the packages actually be loaded at the time >> > of the check, and does not load them (or rather, it loads them but does >> not >> > attach them, so no searchpath muddying occurs). You do need a list of >> > packages to check though (it adds the base ones automatically). It >> handles >> > dependency and could be easily extended to handle suggests as well I >> think. >> > >> > When CodeDepends gets pushed to cran (not my call and not high on my >> > priority list to push for currently) it will actually do exactly what >> you >> > want. (the forCRAN_0.3.5 branch already does and I believe it is >> > documented, so you could use devtools to install it now). >> > >> > As a side note, I'm not sure that existence of a symbol is sufficient >> (it >> > certainly is necessary). What about situations where the symbol exists >> but >> > is stale compared to the value in the parent? Are we sure that can never >> > happen? >> > >> > ~G >> > >> > >> > On Mon, Nov 4, 2013 at 7:29 AM, Michel Lang <michell...@gmail.com> >> wrote: >> > >> > > You might want to consider using Recall() for recursion which should >> > solve >> > > this. Determining the required variables using heuristics as codetools >> > will >> > > probably lead to some confusion when using functions which include >> calls >> > > to, e.g., with(): >> > > >> > > f = function() { >> > > with(iris, Sepal.Length + Sepal.Width) >> > > } >> > > codetools:::findGlobals(f) >> > > >> > > I would suggest to write up some documentation on what the function's >> > > environment contains and how to to define variables accordingly - or >> why >> > it >> > > can generally be considered a good idea to pass everything essential >> as >> > an >> > > argument. Nevertheless a "bpExport" function would be a good addition >> for >> > > some rare corner cases in my opinion. >> > > >> > > Michel >> > > >> > > >> > > 2013/11/3 Henrik Bengtsson <h...@biostat.ucsf.edu> >> > > >> > > > Hi, >> > > > >> > > > in BiocParallel, is there a suggested (or planned) best standards >> for >> > > > making *locally* assigned variables (e.g. functions) available to >> the >> > > > applied function when it runs in a separate R process (which will be >> > > > the most common use case)? I understand that avoid local variables >> > > > should be avoided and it's preferred to put as mush as possible in >> > > > packages, but that's not always possible or very convenient. >> > > > >> > > > EXAMPLE: >> > > > >> > > > library('BiocParallel') >> > > > library('BatchJobs') >> > > > >> > > > # Here I pick a recursive functions to make the problem a bit >> harder, >> > > i.e. >> > > > # the function needs to call itself ("itself" = see below) >> > > > fib <- function(n=0) { >> > > > if (n < 0) stop("Invalid 'n': ", n) >> > > > if (n == 0 || n == 1) return(1) >> > > > fib(n-2) + fib(n-1) >> > > > } >> > > > >> > > > # Executing in the current R session >> > > > cluster.functions <- makeClusterFunctionsInteractive() >> > > > bpParams <- BatchJobsParam(cluster.functions=cluster.functions) >> > > > register(bpParams) >> > > > values <- bplapply(0:9, FUN=fib) >> > > > ## SubmitJobs |++++++++++++++++++++++++++++++++++| 100% (00:00:00) >> > > > ## Waiting [S:0 R:0 D:10 E:0] |+++++++++++++++++++| 100% (00:00:00) >> > > > >> > > > >> > > > # Executing in a separate R process, where fib() is not defined >> > > > # (not specific to BiocParallel) >> > > > cluster.functions <- makeClusterFunctionsLocal() >> > > > bpParams <- BatchJobsParam(cluster.functions=cluster.functions) >> > > > register(bpParams) >> > > > values <- bplapply(0:9, FUN=fib) >> > > > ## SubmitJobs |++++++++++++++++++++++++++++++++++| 100% (00:00:00) >> > > > ## Waiting [S:0 R:0 D:10 E:0] |+++++++++++++++++++| 100% (00:00:00) >> > > > Error in LastError$store(results = results, is.error = !ok, >> > throw.error = >> > > > TRUE) >> > > > : >> > > > Errors occurred during execution. First error message: >> > > > Error in FUN(...): could not find function "fib" >> > > > [...] >> > > > >> > > > >> > > > # The following illustrates that the solution is not always >> > > > straightforward. >> > > > # (not specific to BiocParallel; must have been discussed >> previously) >> > > > values <- bplapply(0:9, FUN=function(n, fib) { >> > > > fib(n) >> > > > }, fib=fib) >> > > > Error in LastError$store(results = results, is.error = !ok, >> > > > throw.error = TRUE) : >> > > > Errors occurred during execution. First error message: >> > > > Error in fib(n): could not find function "fib" >> > > > [...] >> > > > >> > > > # Workaround; make fib() aware of itself >> > > > # (this is something the user need to do, and would be very >> > > > # hard for BiocParallel et al. to automate. BTW, should all >> > > > # recursive functions be implemented this way?). >> > > > fib <- function(n=0) { >> > > > if (n < 0) stop("Invalid 'n': ", n) >> > > > if (n == 0 || n == 1) return(1) >> > > > fib <- sys.function() # Make function aware of itself >> > > > fib(n-2) + fib(n-1) >> > > > } >> > > > values <- bplapply(0:9, FUN=function(n, fib) { >> > > > fib(n) >> > > > }, fib=fib) >> > > > >> > > > >> > > > WISHLIST: >> > > > Considering the above recursive issue solved, a slightly more >> explicit >> > > > and standardized solution is then: >> > > > >> > > > values <- bplapply(0:9, FUN=function(n, BPGLOBALS=NULL) { >> > > > for (name in names(BPGLOBALS)) assign(name, BPGLOBALS[[name]]) >> > > > fib(n) >> > > > }, BPGLOBALS=list(fib=fib)) >> > > > >> > > > Could the above be generalized into something as neat as: >> > > > >> > > > bpExport("fib") >> > > > values <- bplapply(0:9, FUN=function(n) { >> > > > BiocParallel::bpImport("fib") >> > > > fib(n) >> > > > }) >> > > > >> > > > or ideally just (analogously to parallel::clusterExport()): >> > > > >> > > > bpExport("fib") >> > > > values <- bplapply(0:9, FUN=fib) >> > > > >> > > > /Henrik >> > > > >> > > > _______________________________________________ >> > > > Bioc-devel@r-project.org mailing list >> > > > https://stat.ethz.ch/mailman/listinfo/bioc-devel >> > > > >> > > >> > > [[alternative HTML version deleted]] >> > > >> > > _______________________________________________ >> > > Bioc-devel@r-project.org mailing list >> > > https://stat.ethz.ch/mailman/listinfo/bioc-devel >> > > >> > >> > >> > >> > -- >> > Gabriel Becker >> > Graduate Student >> > Statistics Department >> > University of California, Davis >> > >> > [[alternative HTML version deleted]] >> > >> > _______________________________________________ >> > Bioc-devel@r-project.org mailing list >> > https://stat.ethz.ch/mailman/listinfo/bioc-devel >> > >> >> [[alternative HTML version deleted]] >> >> _______________________________________________ >> Bioc-devel@r-project.org mailing list >> https://stat.ethz.ch/mailman/listinfo/bioc-devel >> > -- Gabriel Becker Graduate Student Statistics Department University of California, Davis [[alternative HTML version deleted]] _______________________________________________ Bioc-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/bioc-devel
