I agree, not a lot of value, it is just a little utility function to
avoid users to code it themselves and to avoid missCode (forget to flush
batch in finishBundle for ex as a user was saying on stackoverflow). I
guess Ben created the ticket because some users were searching for that
kind of function in the API.
But if people think that it is too little value to deserve a verb in the
API, no pb, let's close the ticket.
Etienne
Le 17/01/2017 à 17:52, Jean-Baptiste Onofré a écrit :
OK, but I'm afraid we would be too specific. If the batching is just a
List<T> or Set<T> that we populate in @ProcessElement and flush in
@FinishBundle (or when we raise a limit), I don't know if it brings
lot of value.
People might wants a more fine-grained logic (based on the type in the
list for instance).
I'm not against, just try to think about a generic way to do that.
Regards
JB
On 01/17/2017 08:48 AM, Etienne Chauchot wrote:
Hi JB,
I meant jira vote but discussion on the ML works also :)
As I understand the need (see stackoverflow links in jira ticket) the
aim is to avoid the user having to code the batching logic in his own
DoFn.processElement() and DoFn.finishBundle() regardless of the bundles.
For example, possible use case is to batch a call to an external service
(for performance).
I was thinking about providing a PTransform that implements the batching
in its own DoFn and that takes user defined functions for customization.
Etienne
Le 17/01/2017 à 17:30, Jean-Baptiste Onofré a écrit :
Hi
I guess you mean discussion on the mailing list about that, right ?
AFAIR the idea is to provide a utility class to deal with
pooling/batching. However not sure it's required as with @StartBundle
etc in DoFn and batching depends of the end user "logic".
Regards
JB
On Jan 17, 2017, 08:26, at 08:26, Etienne Chauchot
<echauc...@gmail.com> wrote:
Hi all,
I have started to work on this ticket
https://issues.apache.org/jira/browse/BEAM-135
As there where no vote since March 18th, is the issue still
relevant/needed?
Regards,
Etienne
