Hi Arjun, Thanks for your feedback.
I agree with moving to Future<Void>, those are good points. I believe an earlier point made for asynchronous functionality were that modern APIs tend to be asynchronous as they result in more expressive and better defined APIs. Additionally, because a lot of Kafka Connect functionality is already asynchronous, I am inclined to believe that customers will want an asynchronous solution for this as well. And if is relatively simple to block with future.get() to make it synchronous, would you not say that having an opt-in synchronous functionality rather than synchronous only functionality allows for customer control while maintaining that not too much burden of implementation is placed on the customer? WDYT? Thanks, Aakash On Sun, May 17, 2020 at 2:51 PM Arjun Satish <[email protected]> wrote: > Thanks for all the feedback, folks. > > re: having a callback as a parameter, I agree that at this point, it might > not add much value to the proposal. > > re: synchronous vs asynchronous, is the motivation performance/higher > throughput? Taking a step back, calling report(..) in the new interface > does a couple of things: > > 1. at a fundamental level, it is a signal to the framework that a failure > occurred when processing records, specifically due to the given record. > 2. depending on whether errors.log and errors.deadletterqueue has been set, > some messages are written to zero or more destinations. > 3. depending on the value of errors.tolerance (none or all), the task is > failed after reporters have completed. > > for kip-610, the asynchronous method has the advantage of working with the > internal dead letter queue (which has been transparent to the developer so > far). but, how does async method help if the DLQ is not enabled? in this > case RecordMetadata is not very useful, AFAICT? also, if we add more error > reporters in the future (say, for example, a new reporter in a future that > writes to a RDBMS), would the async version return success on all or > nothing, and what about partial successes? > > overall, if we really need async behavior, I'd prefer to just use > Future<Void>. but if we can keep it simple, then let's go with a > synchronous function with the parameters Randall proposed above (with > return type as void, and if any of the reporters fail, the task is failed > if error.tolerance is none, and kept alive if tolerance is all), and maybe > add asynchronous methods in a future KIP? > > Best, >
