> > why don't we revisit the problem from the other direction and see if we > can remove checkpoints?
Simplicity, again :-) If it turns out we don't need the observer anyhow, it saves a lot of time. I'm just poking at different parts to make sure we can still justify their weight. On Mon, Apr 4, 2016 at 1:54 PM, Zameer Manji <zma...@apache.org> wrote: > On Mon, Apr 4, 2016 at 1:47 PM, Bill Farner <wfar...@apache.org> wrote: > > > We clearly have different experiences - i've never really benefited from > > viewing the process graph, as most jobs have very simple sequences that > > could be easily explained by a text file in the sandbox. On the > contrary, > > i've encountered people confused by the process graph, the observer, and > > sandbox browsing...so i must respectfully disagree that it is universally > > appreciated. > > > > What i'm trying to achieve is simplicity. The observer is an extra > moving > > part, and another thing for operators to understand and maintain. It > also > > couples Aurora to one relatively specific way of running tasks, which > makes > > it difficult to open new use cases like Docker tasks. Removing the > > observer starts to pull on a thread of complexity that i don't think > Aurora > > benefits much from, for example state checkpointing by the executor. > > > > My goal is not to apply pressure, but to perform a gut check. If the > > answer is "No", that's fine. > > > > > Bill, > > I think you are pulling on the right thread here but I think revisiting the > observer is the wrong way of approaching the problem. I also agree that > Aurora doesn't benefit much from state checkpointing by the executor and > the observer is an extension of that since it provides a read only human > friendly view of the data in the checkpoints. However, instead of removing > the observer (and degrading the UX around accessing the data in the > checkpoints), why don't we revisit the problem from the other direction and > see if we can remove checkpoints? > > > -- > Zameer Manji >
