The common pattern for this sort of thing is to have the pipeline write its results to somewhere, eg to files, and have the client read them back. This also makes sense for pipelines that never finish.
On Sun, Sep 10, 2017, 3:56 AM Chaim Turkel <[email protected]> wrote: > thanks for the reply. > I think it would have been nicer if i could get this information as a > combine at the end. > The current solution means that the client that deploys to job, needs > to wait for the results (block) and if the client machine dies, then i > loose the information. > If I could add it to the collection pipeline as a combine at then end > it would be better > > chaim > > On Sun, Sep 10, 2017 at 9:03 AM, Jean-Baptiste Onofré <[email protected]> > wrote: > > Hi Chaim, > > > > The pipeline result provides the metrics to you. You can periodically > poll > > (with a thread) the pipeline result object to get the updated data. > > > > Regards > > JB > > > > > > On 09/10/2017 07:45 AM, Chaim Turkel wrote: > >> > >> Hi, > >> I am having trouble figuring out what to do with the results. I have > >> multiple collections running on the pipeline, and since the sink does > >> not give me the option to get the result, i need to wait for the > >> pipeline to finish and then poll the results. > >> From what i can see my only option is to use the metrics, is there > >> another way to pass information from the collections to the results? > >> > >> chaim > >> > > > > -- > > Jean-Baptiste Onofré > > [email protected] > > http://blog.nanthrax.net > > Talend - http://www.talend.com >
