In that case please have a look at: https://github.com/apache/apex-malhar/blob/master/kafka/src/main/java/org/apache/apex/malhar/kafka/KafkaSinglePortExactlyOnceOutputOperator.java
The operator will ensure that messages are not duplicated, under the stated assumptions. On Thu, Oct 6, 2016 at 3:37 PM, Jaspal Singh <jaspal.singh1...@gmail.com> wrote: > Hi Thomas, > > In our case we are writing the results back to maprstreams topic based on > some validations. > > > Thanks > Jaspal > > > On Thursday, October 6, 2016, Thomas Weise <t...@apache.org> wrote: > >> Hi, >> >> which operators in your application are writing to external systems? >> >> When you look at the example from the blog (https://github.com/DataTorren >> t/examples/tree/master/tutorials/exactly-once), there is Kafka input, >> which is configured to be idempotent. The results are written to JDBC. That >> operator by itself supports exactly-once through transactions (in >> conjunction with idempotent input), hence there is no need to configure the >> processing mode at all. >> >> Thomas >> >>
