On Mon, Apr 4, 2016 at 1:47 PM, Bill Farner <wfar...@apache.org> wrote:
> We clearly have different experiences - i've never really benefited from > viewing the process graph, as most jobs have very simple sequences that > could be easily explained by a text file in the sandbox. On the contrary, > i've encountered people confused by the process graph, the observer, and > sandbox browsing...so i must respectfully disagree that it is universally > appreciated. > > What i'm trying to achieve is simplicity. The observer is an extra moving > part, and another thing for operators to understand and maintain. It also > couples Aurora to one relatively specific way of running tasks, which makes > it difficult to open new use cases like Docker tasks. Removing the > observer starts to pull on a thread of complexity that i don't think Aurora > benefits much from, for example state checkpointing by the executor. > > My goal is not to apply pressure, but to perform a gut check. If the > answer is "No", that's fine. > Bill, I think you are pulling on the right thread here but I think revisiting the observer is the wrong way of approaching the problem. I also agree that Aurora doesn't benefit much from state checkpointing by the executor and the observer is an extension of that since it provides a read only human friendly view of the data in the checkpoints. However, instead of removing the observer (and degrading the UX around accessing the data in the checkpoints), why don't we revisit the problem from the other direction and see if we can remove checkpoints? -- Zameer Manji
