Yes, offering a way to get a pipeline from the job service directly would be a completely reasonable thing to do (and likely not hard at all). We welcome pull requests.
Alternative UIs built on top of this abstraction would be an interesting project to explore. On Wed, Jun 26, 2019 at 8:44 AM Chad Dombrova <chad...@gmail.com> wrote: > > Hi all, > I've been poking around the beam source code trying to determine whether it's > possible to get the definition of a pipeline via beam's gPRC-based services. > It looks like the message types are there for describing a Pipeline but as > far as I can tell, they're only used by JobService.Prepare() for submitting a > new job. > > If I were to create a PR to add support for a JobService.GetPipeline() > method, would that be interesting to others? Is it technically feasible? > i.e. is the pipeline definition readily available to the job service after > the job has been prepared and sent to the runner? > > Bigger picture, what I'm thinking about is writing a UI that's designed to > view and monitor Beam pipelines via the portability abstraction, rather than > using the (rather clunky) UIs that come with runners like Flink and Dataflow. > My thinking is that using beam's abstractions would future proof the UI by > allowing it to work with any portable runner. Right now it's just an idea, > so I'd love to know what others think of this. > > thanks! > -chad >
