Re: What is the Impulse and why do we need it?

2018-05-18 Thread Kenneth Knowles
I think it makes a lot of sense to move it to the Beam web site. There's already a good landing point: https://beam.apache.org/contribute/runner-guide/ That page is a collection of advice for legacy-style runners on how to use runners-core, etc, and just general stuff about how to write one,

Re: What is the Impulse and why do we need it?

2018-05-18 Thread Lukasz Cwik
The Beam Runner API doc needs a lot of updating to discuss impulse and SDF, (and deprecate / remove Read): https://s.apache.org/beam-runner-api It could also use examples from Go/Python code base. Alternatively we could start to codify this information on the Apache Beam website as the

Re: What is the Impulse and why do we need it?

2018-05-18 Thread Eugene Kirpichov
Hi Ismael, Impulse is a primitive necessary for the Portability world, where sources do not exist. Impulse is the only possible root of the pipeline, it emits a single empty byte array, and it's all DoFn's and SDF's from there. E.g. when using Fn API, Read.from(BoundedSource) is translated into:

Re: What is the Impulse and why do we need it?

2018-05-18 Thread Jean-Baptiste Onofré
Fully agree. I already started to take a look. Regards JB On 18/05/2018 16:12, Ismaël Mejía wrote: I have seen multiple mentions of 'Impulse' in JIRAs and some on other discussions, but have not seen any document or concrete explanation on what's Impulse and why we need it. This seems like an

What is the Impulse and why do we need it?

2018-05-18 Thread Ismaël Mejía
I have seen multiple mentions of 'Impulse' in JIRAs and some on other discussions, but have not seen any document or concrete explanation on what's Impulse and why we need it. This seems like an internal implementation detail but it is probably a good idea to explain it somewhere (my excuses if