Re: [EXTERNAL] kafka

Hey Raja, The setup for secure kafka input operator is not easy. You can follow these steps. 1. Follow kafka document to setup your brokers properly ( https://kafka.apache.org/090/documentation.html#security_overview) 2. You have to manually create the client JAAS file (

Re: Datatorrent fault tolerance

String, > String>("/datalake/corporate/ses_dlpoc/stream_fabbddev/jsingh73/sample-stream:error", > tenant.getVolumeName(), gson.toJson(tenant))); > > } > producer.flush(); > } > } > > > &g

Re: Datatorrent fault tolerance

On Fri, Oct 7, 2016 at 1:27 PM, hsy...@gmail.com <hsy...@gmail.com> wrote: > >> Jaspal, >> >> Topic is a mandatory property you have to set. In mapr, the value should >> be set to the full stream path example: /your/stream/path:streamname >> >> Regards, >>

Re: Datatorrent fault tolerance

Hey Jaspal, Did you add any code to existing KafkaSinglePortExactlyOnceOutputOperator from malhar? If so please make sure the producer you use here is org.apache.kafka.clients.producer.KafkaProducer instead of kafka.javaapi.producer.Producer. That is old api and that is not supported by MapR

Re: Apex and Malhar Java 8 Certified

I think the problem is what people expect when we say "certified". To me, If I see something is certified with java 8, I would assume that I can use java 8 api(new features stream, lambda etc.) to write the operator code, not only just run the code with jre 8 or compile existing code with jdk 8

Re: Emit values from an array

Is you pojo a shared object? I think you need to create new pojo every time. Regards, Siyuan On Thu, Sep 29, 2016 at 3:03 PM, Jaikit Jilka wrote: > Hello, > > I am trying to emit values from an array. I am emitting using an for loop. > Number of records emitted is

Re: Malhar 0.8.1.1 Kafka Operator and chroot

Hey Alex, Does the workaround work? I just want to follow up to see my hypothesis for the root cause is correct. Thanks! Regards, Siyuan On Wed, Aug 24, 2016 at 10:56 AM, hsy...@gmail.com <hsy...@gmail.com> wrote: > Hey Alex, > > Yeah, I think there is a bug for multitenan

Re: Malhar 0.8.1.1 Kafka Operator and chroot

Hey Alex, Do you use ONE_TO_ONE or ONE_TO_MANY partition? Regards, Siyuan On Wed, Aug 24, 2016 at 10:27 AM, McCullough, Alex < alex.mccullo...@capitalone.com> wrote: > Hey Siyuan, > > > > We are using 3.4.0 > > > > Thanks, > > Alex > > *From: *&

Re: can operators emit on a different from the operator itself thread?

Hey Vlad, Thanks for bringing this up. Is there an easy way to detect unexpected use of emit method without hurt the performance. Or at least if we can detect this in debug mode. Regards, Siyuan On Wed, Aug 10, 2016 at 11:27 AM, Vlad Rozov wrote: > The short answer is

Re: Force Fail Application

He Michael, You can throw a ShutdownException. Siyuan On Mon, Jul 18, 2016 at 10:06 AM, Silver, Michael < michael.sil...@capitalone.com> wrote: > > > > > Hello, > > > > I am looking for a solution to force shutdown or fail my application. I > have an operator that checks that a file (which is

Re: Regarding using Scala to develop Apex app.

BTW Akshay, if you are using anonymous function as a field in the operator, it's very likely that your function is stateless? If that's the case, you can try to mark it transient ( http://www.scala-lang.org/api/rc2/scala/transient.html) On Thu, Jul 14, 2016 at 11:06 PM, Akshay S Harale <

Re: A proposal for Malhar

<a...@datatorrent.com> wrote: > > My vote is to do 2&3 > > Thks > Amol > > > On Tue, Jul 12, 2016 at 12:14 PM, Kottapalli, Venkatesh < > vkottapa...@directv.com> wrote: > >> +1 for deprecating the packages listed below. >> >> -Original

Re: A proposal for Malhar

+1 On Tue, Jul 12, 2016 at 11:53 AM, David Yan wrote: > Hi all, > > I would like to renew the discussion of retiring operators in Malhar. > > As stated before, the reason why we would like to retire operators in > Malhar is because some of them were written a long time

Re: DataTorrent with SBT: .apa file not created

I've never used SBT to build Apex application. But I guess you can try 2 things here Use the sbt maven plugin https://github.com/shivawu/sbt-maven-plugin or use sbt assembly plugin https://github.com/sbt/sbt-assembly In the 2nd way, you need to translate the plugin configuration part in pom.xml

Re: kafka offset commit

hanks Siyuan. > > So, to confirm, to apex is not storing offsets status at any location ? > Like how Storm stores in Zookeeper ? > > > Regards, > Raja. > > From: "hsy...@gmail.com" <hsy...@gmail.com> > Reply-To: "users@apex.apache.org" <users@