i've worried about something similar, but on a different scale - some kind of pipeline for processing (image) data, where you have a series of commands that produce intermediate images and you might go back over the commands and edit some parameter, then request that necessary work is re-computed.
i don't have any answers, but would be interested in anything you find. andrew On Tuesday, 30 June 2015 01:42:40 UTC-3, Kevin Owens wrote: > > My question is basically: what is the best way to say "here's what I've > got, solve it as much as you can so it will be fast when I get the rest"? > > The kind of problem I'm trying to solve is "we have this fancy model ... > hey, wouldn't it be cool to look at how it changes if we vary X?" "I'd love > to try that, but it takes a day to run each time." > > Say there's a piece of code that has many inputs and takes a long time to > run. Some inputs may be known ahead of time and others only on an ad-hoc > basis. Or, given a set of inputs, you may want to vary other arbitrary > inputs, but you don't know what values to test yet. Sometimes you have all > of the inputs and can run all of the code at once, and sometimes you want > to pre-compile/lazy/cache to finish later. Maybe saving the object in HDF5. > > The documentation show how to use expression interpolation but the > drawbacks are that you have to edit code for each combination of inputs > that are known, and the code won't run interactively once you've edited it. > You could invent a new syntax or do find/replace, but that can be risky. > > Here's an example of what I'd like to do: > > > versioninfo() > > const n = 4 > > ex1 = :(x .* exp(y .* z)) > > y = rand(n) > z = rand(n) > data = [:y=>y,:z=>z] > > function plug_in!(expr,data) > for i in 1:length(expr.args) > if isa(expr.args[i],Expr) > expr2 = plug_in!(expr.args[i],data) > expr.args[i] = try eval(expr.args[i]) catch expr.args[i] end > elseif isa(expr.args[i],Symbol) > for k in keys(data) > if expr.args[i] == k > expr.args[i] = data[k] > end > end > else > expr.args[i] = try eval(expr.args[i]) catch expr.args[i] end > end > end > return expr > end > > plug_in!(ex1,data) > > x = rand(n) > > eval(ex1) > > > It's not completely "arbitrary" in that there's a question of how to > organize the initial ex1 to be able to pre-process as much as possible, but > that's a separate issue. So, if y was unknown and x and z were known it > would be more optimal to re-factor ex1, but that could be another function > called within plug_in!. Also, I know there are needless assignments, and > there are probably other things that could be optimized in the function, > but I just wanted to give a working example. > > So, has someone smarter than me already figured this out? Is this already > in a package somewhere? Is there a much more elegant way to do it? 99% of > my motivation for asking this is to learn more about Julia. > > Thanks, > > Kevin >
