Hi Shen, Responding just to one part of your message - "remember the line at which the PTransform was added": take a look at https://github.com/apache/beam/pull/2247 which does this for PAssert.
On Mon, Aug 14, 2017 at 7:32 PM Shen Li <cs.she...@gmail.com> wrote: > In 0.5.0 or earlier releases, PipelineRunner provides an > apply(PTransform<InputT, OutputT>, InputT) method which allows runner > implementation to perform actions when the PTransform is added to the > pipeline. In later releases, that apply(...) method has been replaced by a > Pipeline#replaceAll() method, where the runner can only get involved after > the pipeline has been fully constructed. In terms of override PTransforms, > these two APIs are identical. But, the early API could still be helpful. > For example, the runner could remember the line at which the PTransform was > added, and provide that info to users to assist debugging. Is it possible > to add that API back? Or is there any other way to involve the runner when > a PTransform is added? > > Thanks, > Shen >
