Re: Datatorrent fault tolerance

Oh I see, you want to send to different topics. Well, then you have to give some dummy value to the topic property on the operator. Regards, Siyuan On Fri, Oct 7, 2016 at 11:38 AM, Jaspal Singh wrote: > Siyuan, > > So for the output operator, we have specified it as

Re: Datatorrent fault tolerance

Siyuan, So for the output operator, we have specified it as a part of our logic itself. public class KafkaSinglePortExactlyOnceOutputOperator extends AbstractKafkaOutputOperator { private static final Logger LOG = LoggerFactory.getLogger(KafkaSinglePortExactlyOnceOutputOperator.class);

Re: Datatorrent fault tolerance

Jaspal, I think you miss the kafkaOut :) Regards, Siyuan On Fri, Oct 7, 2016 at 11:32 AM, Jaspal Singh wrote: > Siyuan, > > That's how we have given it in properties file: > > [image: Inline image 1] > > > Thanks!! > > On Fri, Oct 7, 2016 at 1:27 PM,

Re: Datatorrent fault tolerance

After making the change, we are getting the below error while application launch: *An error occurred trying to launch the application. Server message: javax.validation.ConstraintViolationException: Operator kafkaOut violates constraints

Re: Datatorrent fault tolerance

Should we use malhar-library version 3.5 then ? Thanks!! On Fri, Oct 7, 2016 at 1:04 PM, Thomas Weise wrote: > Please make sure to depend on version 3.5 of malhar-kafka in pom.xml. This > operator is not in malhar-library, it's a separate module. > > > On Fri, Oct 7, 2016 at

Re: Datatorrent fault tolerance

Hey Jaspal, Did you add any code to existing KafkaSinglePortExactlyOnceOutputOperator from malhar? If so please make sure the producer you use here is org.apache.kafka.clients.producer.KafkaProducer instead of kafka.javaapi.producer.Producer. That is old api and that is not supported by MapR

Re: Datatorrent fault tolerance

Thomas, We have added the dependency in pom.xml for lafka client API and also for malhar kafka. Please highlight if you are specifying some other dependency that we need to add. org.apache.apex malhar-kafka ${malhar.version} org.apache.kafka kafka-clients

Re: Datatorrent fault tolerance

Thomas, Below is the operator implementation we are trying to run. This operator is getting an object of tenant class from updtream operator. public class KafkaSinglePortExactlyOnceOutputOperator extends AbstractKafkaOutputOperator { private static final Logger LOG =

Re: Datatorrent fault tolerance

The KafkaSinglePortExactlyOnceOutputOperator takes whatever output from previous operator and writes to Kafka. Sent from my iPhone > On Oct 7, 2016, at 07:59, Jaspal Singh wrote: > > Hi Thomas, > > I have a question, so when we are using >

Re: Datatorrent fault tolerance

Hi Thomas, I have a question, so when we are using *KafkaSinglePortExactlyOnceOutputOperator* to write results into maprstream topic will it be able to read messgaes from the previous operator ? Thanks Jaspal On Thu, Oct 6, 2016 at 6:28 PM, Thomas Weise wrote: > For recovery