Re: State flush & recovery during failures

Thank you Matthias, That cleared things up, have one follow up question on punctuations. Does punctuations get restored after failure? If not what is the right way to do so? Is it by storing the info currently active punctuations in some state and by scheduling them again in the init method? Tha

Re: Subcribe kafka

Follow instructions here: https://kafka.apache.org/contact On 9/21/18 5:01 AM, ?? wrote: > Hi: > I am new a user??I want to subscribe Kafka > signature.asc Description: OpenPGP digital signature

Subcribe kafka

Hi: I am new a user??I want to subscribe Kafka

Re: Problems trying to make kafka 'rack-aware'

Hi Eno, many thanks for trying that. That is very helpful for me. That basic check didn't work for me but I have since discovered what my issue was. Despite using a version of kafka that supports rack-awareness we have been deliberately setting 'inter.broker.protocol.version' to an older vers

Re: Terminate Streams application from within Transformer?

Hi Tim, I wouldn't recommend System.exit(), as it won't give streams a chance to go through a shutdown process. You can throw a RuntimeException from inside the Transformer.transform() method and I will bubble up and the Steams application will shut down. -Bill On Fri, Sep 21, 2018 at 6:59 AM Ti

Re: Problems trying to make kafka 'rack-aware'

Hi Bryan, I did a simple check with starting a broker with no rack id and then restarting with a rack id and I can confirm I could get the rack id from zookeeper after the restart. This was on trunk. Does that basic check work for you (i.e., without reassigning partitions)? Thanks Eno On Fri, Se

Re: Problems trying to make kafka 'rack-aware'

I didn't get a response to this, but I've been investigating more and can now frame the problem slightly differently (hopefully, more accurately). According to this document https://cwiki.apache.org/confluence/display/KAFKA/Kafka+data+structures+in+Zookeeper Which defines broker data struct

Re: Kafka consumer offset topic deletion

Hello, Please find the server.properties info. The Kafka server is running on a Ubuntu 14.XX instance. I have removed all the commented section. So the data I post below would be the only properties we are using for our server. All the other properties are commented(so I guess the default values a

Terminate Streams application from within Transformer?

>From within a Transformer I detect an error, which is a temporary error, such >as failure to get a connection to an external microservice. A typical scenario is that Kuberbetes is starting up a bunch of pods, of which mine is one, and they come up in the wrong order. This is perfectly normal, a

Re: Kafka producer huge memory usage (leak?)

Hi, In that case, it is not related to KAFKA-4493. I see StackOverFlowException in the uploaded heapdump. I am pretty sure this is related to StackOverFlowException. Some objects are dangling after this exception. You need to check/find out when this exception is occuring. check your application s

Re: Kafka_2.12-1.1.0 config

bin/kafka-console-producer.sh add --producer.config your_producer.properties file. you don't add --producer.config , you still use the default value of max.request.size. sarath reddy 于2018年9月21日周五 上午12:27写道： > ++pushkar > > On Thu 20 Sep, 2018, 16:20 sarath reddy, wrote: > > > Hi Team, > > > >

Re: Kafka producer huge memory usage (leak?)

Hi Manikumar, I am using correct protocol. SSL and truststore is also correctly configured and there is a single port which is for SSL. Otherwise it simply wouldn't work. Data is getting produced all fine, it's just that producer object is consuming massive amount of memory. On Fri, Sep 21, 2018 a

Re: Kafka producer huge memory usage (leak?)

Hi, Instead trying the PR, make sure you are setting valid security protocol and connecting to valid broker port. also looks for any errors in producer logs. Thanks, On Fri, Sep 21, 2018 at 12:35 PM Shantanu Deshmukh wrote: > Hi Manikumar, > > I checked this issue. There is this patch avai

Re: Kafka_2.12-1.1.0 config

You can refer this closed issue in github . On Thu, Sep 20, 2018 at 9:57 PM sarath reddy wrote: > ++pushkar > > On Thu 20 Sep, 2018, 16:20 sarath reddy, wrote: > > > Hi Team, > > > > We are trying to configure Kafka to produce larger mess

Re: Kafka producer huge memory usage (leak?)

Hi Manikumar, I checked this issue. There is this patch available. https://github.com/apache/kafka/pull/2408.patch I pulled Kafka 0.10.1.0 from github. Then tried applying this patch. But several places I am getting error that patch doesn't apply. I am new to git and patching process. Can you gui