As a first prototype, you can (using env.execute): - execute the bulk iteration as one job - grab the number of elapsed iterations using the static variable that Stephan mentioned - execute the delta iteration as a second job if needed
On Fri, Jul 11, 2014 at 5:06 PM, Stephan Ewen <[email protected]> wrote: > Hi! > > That is exactly the incremental rollout variant. > > You can do this with intermediate file writing, currently. Would that be a > way to start prototyping? For benchmarking, we can even hack a caching > output and input format that keeps the data in the task manager's memory. > > Stephan >
