FinishBundle() does the job. Should I keep using Setup()? What is the difference between Setup() and StartBundle()?
Thanks again. On 2018/07/03 20:10:21, Henning Rohde <[email protected]> wrote: > Teardown has very loose guarantees on when it's called and you essentially > can't rely on it. Currently, for Go on non-direct runners, we hang on to > the bundle descriptors forever and never destroy them (and in turn never > call Teardown). Even if we didn't, failures/restarts could cause Teardown > to not be called. > > If something _must_ happen, FinishBundle is the right method. > > Thanks, > Henning > > On Tue, Jul 3, 2018 at 10:37 AM [email protected] < > [email protected]> wrote: > > > Essentially I have the following code: > > > > type Writer struct { > > Pool WriterPool > > } > > > > func (w *Writer) Setup() { > > w.Pool = Init() > > } > > > > func (w* Writer) ProcessElement(ctx, elem Elem) { > > w.Pool.Add(elem) > > } > > > > func (w* Writer) Teardown() { > > w.Pool.Write() > > w.Pool.Close() > > } > > > > beam.ParDo0(scope, &Writer{}, elemCollection) > > > > The above code runs fine with the direct runner but not with dataflow. > > > > I added log lines to the above methods, and the ones in Teardown() never > > appear in the logs. > > If I change my code as follows: > > > > func (w* Writer) ProcessElement(ctx, elem Elem) { > > w.Pool.Add(elem) > > w.Pool.Write() > > } > > > > Then I see the data being written, but I lose the ability to pool, plus I > > am leaking connections. > > > > Is this a known issue, or I am going something wrong? > > > > Thanks again for the help. > > >
