Hi Eric, I would recommend to use the latest version of Apex Core (3.4.0) as well as the latest version of Malhar (3.4.0 as well).
My responses inline based on above versions: On Wed, Jun 29, 2016 at 5:54 AM, Martin, Eric <[email protected]> wrote: > Hi, > > > > I have a few quick questions around the Kafka Input Operators, to confirm > that I am implementing them correctly. > > > > 1) We are using kafka version 0.8.x, and I want to confirm that the > 0.8.x input operators for kafka are located in > https://github.com/apache/apex-malhar/tree/master/contrib/src/main/java/com/datatorrent/contrib/kafka > Yes, this is the location for the Kafka 0.8.x compatible consumer. > > 2) For setting zookeeper and topic in the input operator, I see that > setZookeeper and setTopic methods are both deprecated. I just want to > confirm that I should not be creating a consumer instance and setting these > properties in the consumer vs the operator level I am currently setting > them on the operator via the properties file as shown below and I want to > confirm that this is correct: > > > > <*property*> > <*name*>dt.operator.kafkaInputOperator.prop.topic</*name*> > <*value*></*value*> > </*property*> > > > Please use the setters of the consumer instead. Path would be dt.operator. kafkaInputOperator.prop.consumer.topic > 3) For exactly once processing, we in examples we see that we need to > setIdempotentStorageManager on the on the input operator. It appears though > that IdempotentStorageManager is deprecated. Is there an alternative that > we can use if we are running 3.3.0 of apex engine and 3.3.1 of Malhar? > The replacement is WindowDataManager and the operator was updated in v3.4.0 to reflect this. > > > ------------------------------ > > The information contained in this e-mail is confidential and/or > proprietary to Capital One and/or its affiliates and may only be used > solely in performance of work or services for Capital One. The information > transmitted herewith is intended only for use by the individual or entity > to which it is addressed. If the reader of this message is not the intended > recipient, you are hereby notified that any review, retransmission, > dissemination, distribution, copying or other use of, or taking of any > action in reliance upon this information is strictly prohibited. If you > have received this communication in error, please contact the sender and > delete the material from your computer. >
