Re: [DISCUSS] Java 8 as a minimum requirement

Hi Jan, That's interesting. Do you have some references you can share on this? It would be good to know which Java 8 versions have been tested and whether it is something that is being worked on. Ismael On Fri, Jun 17, 2016 at 12:02 AM, wrote: > > Hi Ismael, > > Unfortunately Java 8 doesn't pl

Re: [DISCUSS] Java 8 as a minimum requirement

Hi Grant, Comments below. On Thu, Jun 16, 2016 at 11:13 PM, Grant Henke wrote: > Would it make sense to drop Java 1.7 and Scala 2.10 support in Kafka 0.11 > to maintain upgrade compatibility throughout the 0.10.x series? > The issue is that we don't know when Kafka 0.11 will be released. Also,

Re: [DISCUSS] Java 8 as a minimum requirement

On Thu, Jun 16, 2016 at 11:13 PM, Stephen Boesch wrote: > @Jeff Klukas What is the concern about scala 2.11 vs 2.12? 2.11 runs on > both java7 and java8 > Scala 2.10.5 and 2.10.6 also support Java 8 for what it's worth. Ismael

[DISCUSS] Java 8 as a minimum requirement

Hi all, I would like to start a discussion on making Java 8 a minimum requirement for Kafka's next feature release (let's say Kafka 0.10.1.0 for now). This is the first discussion on the topic so the idea is to understand how people feel about it. If people feel it's too soon, then we can pick up

Re: Invalid Version for API key

plied to me! > > Chris > > > > > From: Ismael Juma > To: users@kafka.apache.org > Date: 06/08/2016 11:29 AM > Subject:Re: Invalid Version for API key > Sent by:isma...@gmail.com > > > > Hi Chris, > > The error includes the IP address of

Re: Invalid Version for API key

Hi Chris, The error includes the IP address of the client: 172.20.3.0:53901 Would that help identify the client? We are not aware of issues with the built-in clients. Ismael On Wed, Jun 8, 2016 at 4:13 PM, Chris Barlock wrote: > I think everyone is using the Maven kafka-clients 0.8.2.1 but I

Re: Problem with Kafka TLS configuration "no cipher suites in common"

te: > Hi Ismael, > > Thanks for the reply - i don't have permissions to see that link - is there > any chance you can expand on it in the thread please ? > > Thanks, > Mick > > On Tue, Jun 7, 2016 at 12:39 PM, Ismael Juma wrote: > > > Hi Mick, > >

Re: Problem with Kafka TLS configuration "no cipher suites in common"

Hi Mick, This can happen if you are using a DSS key, but the client only supports RSA-based ciphers, see the following for more details: https://issues.apache.org/jira/secure/EditComment!default.jspa?id=12964371&commentId=15270520 Ismael On Tue, Jun 7, 2016 at 10:24 AM, Mick Mahoney wrote: >

Re: Brokers changing mtime on data files during startup?

Also, it would be good to have a JIRA as it seems to be a 0.9 regression. Ismael On 6 Jun 2016 20:03, "Dustin Cote" wrote: > For those that have seen this issue on 0.9, can you provide some more > insight into your environments? What OS and filesystem are you running? > Do you find that you can

Re: Kafka encryption

On Mon, Jun 6, 2016 at 3:12 PM, Tom Brown wrote: > How would it be possible to encrypt an entire batch? My understanding > is that the Kafka server needs to know the boundaries of each message. > (E.g. The server decompresses compressed message sets and re-compresses > individual messages). > Qu

Re: one question when upgrade from 0.8.2.2 to 0.10.0

Hi Fredo, A comment below: On Mon, Jun 6, 2016 at 11:19 AM, Fredo Lee wrote: > add`inner.broker.protocol.version=0.8.2` > There's a typo here, it should be: inter.broker.protocol.version=0.8.2 Was the typo only in the email or also in your server properties file? Ismael

Re: Dynamic bootstrap.servers with multiple data centers

> > Sent from my iPhone > > > On Jun 2, 2016, at 8:32 AM, Ismael Juma wrote: > > > > Hi Danny, > > > > A KIP has not been drafted for that yet. Would you be interested in > working > > on it? > > > > Ismael > > > >> On Thu, Ju

Re: Dynamic bootstrap.servers with multiple data centers

Hi Danny, A KIP has not been drafted for that yet. Would you be interested in working on it? Ismael On Thu, Jun 2, 2016 at 1:15 PM, Danny Bahir wrote: > Thanks Ben. > > The comments on the Jira mention a pluggable component that will manage > the bootstrap list from a discovery service. > > Th

Re: SSL certificate CN validation against FQDN in v0.9

Martin, What I said is correct, see: The JDK 8 release supports endpoint identification algorithms for TLS 1.2. The algorithm name can be passed to the setEndpointIdentificationAlgorithm() method of javax.net.ssl.SSLParameters. The following table shows the currently recognized names. Endpoint Id

Re: SSL certificate CN validation against FQDN in v0.9

at 2:29 PM, Ismael Juma wrote: > Martin, > > What I said is correct, see: > > The JDK 8 release supports endpoint identification algorithms for TLS 1.2. > The algorithm name can be passed to the > setEndpointIdentificationAlgorithm() method of javax.net.ssl.SSLParameters.

Re: SSL certificate CN validation against FQDN in v0.9

Hi Phil, You are right that the check is not done by default. We have a couple of JIRAs tracking that: https://issues.apache.org/jira/browse/KAFKA-3665 https://issues.apache.org/jira/browse/KAFKA-3667 Enabling the check is a matter of setting `ssl.endpoint.identification.algorithm` to `https`, b

Re: WARN Property [sasl-stuff] is not valid

ew-consumer option) proved right and > I've been able to run my simple Flink job reading from and writing to Kafka > in a secure environment. > > Best, > Stefano > > On Mon, May 30, 2016 at 8:21 PM, Ismael Juma wrote: > > > Hi Stefano, > > > > For the

Re: Compatibility between different versions of Kafka server and kafka-clients library

Ismael On Tue, May 31, 2016 at 8:46 AM, Mikael Ståldal wrote: > Is this documented somewhere? > > On Mon, May 30, 2016 at 8:16 PM, Ismael Juma wrote: > > > Hi Mikael, > > > > This is expected. Older clients work with newer brokers, but newer > clients > >

Re: WARN Property [sasl-stuff] is not valid

Hi Stefano, For the console consumer, you need to pass --new-consumer for it to work. For the producer, it seems like you are using configs for Kafka 0.10 with a 0.9 producer, but the warning is benign. Are you sure the producer is not working? Ismael On 30 May 2016 11:50, "Stefano Baghino" wro

Re: Compatibility between different versions of Kafka server and kafka-clients library

Hi Mikael, This is expected. Older clients work with newer brokers, but newer clients don't work with older brokers. Ismael On 30 May 2016 17:29, "Mikael Ståldal" wrote: > I am experiencing compatibility issues between different versions of Kafka > server and kafka-clients library. > > kafka-cl

Re: kafka-consumer-group.sh failed on 0.10.0 but works on 0.9.0.1

Yes, that sounds good. Would you like to submit a PR to our upgrade page? https://github.com/apache/kafka/blob/trunk/docs/upgrade.html Thanks, Ismael On Wed, May 25, 2016 at 6:11 AM, allen chan wrote: > Thanks Jason for that insight. I will use the 0.9 tools until i upgrade all > the brokers.

Re: Kafka 0.10.0.0: kafka.utils.VerifiableProperties not found - continuing with a stub.

Hi Niko, VerifiableProperties is part of the kafka jar and your build only depends on kafka-clients. Ismael On Wed, May 25, 2016 at 6:09 PM, Niko Davor wrote: > Using a bare-bones build.sbt: http://pastebin.com/CADyngYs > > Results in: > > [warn] Class kafka.utils.VerifiableProperties not foun

Re: [ANNOUCE] Apache Kafka 0.10.0.0 Released

I had an offline chat with Gwen and the 0.10.0.0-rc6 tag is the right one. I've updated the 0.10.0.0 tag to be the same: https://github.com/apache/kafka/tree/0.10.0.0 Ismael On Tue, May 24, 2016 at 6:04 PM, Ismael Juma wrote: > Hmm, sorry. The tag seems wrong. The commit you linked T

Re: [ANNOUCE] Apache Kafka 0.10.0.0 Released

ongjoon Hyun, Drausin Wulsin, Duncan Sands, Dustin Cote, > Eamon Zhang, edoardo, Edward Ribeiro, Eno Thereska, Ewen > Cheslack-Postava, Flavio Junqueira, Francois Visconte, Frank Scholten, > Gabriel Zhang, gaob13, Geoff Anderson, glikson, Grant Henke, Greg > Fodor, Guozhang Wang, Gwen

Re: [ANNOUCE] Apache Kafka 0.10.0.0 Released

Hmm, sorry. The tag seems wrong. The commit you linked Tom seems the correct one: https://github.com/apache/kafka/commit/b8642491e78c5a137f5012e31d347c01f3b02339 Gwen, is this right? Ismael On Tue, May 24, 2016 at 6:03 PM, Ismael Juma wrote: > Hi Tom, > > The official commit can

Re: [ANNOUCE] Apache Kafka 0.10.0.0 Released

> > > A big thank you for the following people who have contributed to the > > > > 0.10.0.0 release. > > > > > > > > Adam Kunicki, Aditya Auradkar, Alex Loddengaard, Alex Sherwin, Allen > > > > Wang, Andrea Cosentino, Anna Povzn

Reporting security issues

Hi all, Since Kafka implements a number of security features, we need a procedure for reporting potential security vulnerabilities privately (as per http://www.apache.org/security/). We have added a simple page to the website that describes the procedure (thanks Flavio): http://kafka.apache.org/p

Re: Configuring client-side timeouts in the Java Producer so that send() doesn't block

Hi Samuel, Your second example should work as well. Can you please include the code you are using to test the scenario and what Kafka version you are using (0.9.0.0 or 0.9.0.1, I guess)? Ismael On Wed, May 18, 2016 at 11:07 AM, Samuel Chase wrote: > Hello Kafka Users, > > We've been using the

Re: Kafka Protocol API versions

Hi Dana, One comment below. On Tue, May 17, 2016 at 2:39 PM, Dana Powers wrote: > (myself). Asish or others can correct me if I'm wrong here, but I > believe the java client is not attempting to do this yet outside of a > single request flow (SASL authentication). > The Java client is not usin

Re: Kafka Protocol API versions

Hi Oleksiy, Ashish has just created a PR to add some information about how to use KIP-35: https://github.com/apache/kafka/pull/1395 Maybe you can have a look and leave a comment with areas that are not covered by the first draft. Ismael On Tue, May 17, 2016 at 1:34 PM, Oleksiy Krivoshey wrote

Re: Peer not authenticated when trying to send message to kafka over SSL with Authentication

Hi Mayuresh, You need to enable client authentication by setting `ssl.client.auth` to `required` or `requested` (I suggest the former). Ismael On Thu, May 12, 2016 at 10:35 PM, Mayuresh Gharat < gharatmayures...@gmail.com> wrote: > HI I am trying to establish an SSL connection from kafkaProduce

Re: [VOTE] 0.10.0.0 RC4

Hi Tom, This is puzzling because, as you said, not much has changed in the TLS code since 0.9.0.1. A JIRA sounds good. I was going to ask if you could test the commit before/after KAFKA-3025, but I see that Gwen has already done that. :) Ismael On Thu, May 12, 2016 at 9:26 PM, Tom Crayford wrot

Re: Can't send message to Plaintext port on Kerberized Kafka Cluster

Hi Michal, There is no authentication in the PLAINTEXT port, but authorization still happens. And the user will always be `KafkaPrincipal.ANONYMOUS` for that case. Ismael On Thu, May 12, 2016 at 11:18 AM, Michał Kabocik wrote: > Dears, > > > > I have a three node cluster of Kafka 0.9 with two

Re: ERROR Processor got uncaught exception. (kafka.network.Processor)

Message- > From: isma...@gmail.com [mailto:isma...@gmail.com] On Behalf Of EXT > Ismael Juma > Sent: Tuesday, May 10, 2016 5:00 PM > To: users@kafka.apache.org > Subject: Re: ERROR Processor got uncaught exception. > (kafka.network.Processor) > > Thanks Buvana. Is this happening

Re: ERROR Processor got uncaught exception. (kafka.network.Processor)

) < buvana.rama...@nokia.com> wrote: > Ismael, > > Created bug: > https://issues.apache.org/jira/browse/KAFKA-3689 > > Hope to get a quick resolution. > > Thanks, > Buvana > > -Original Message- > From: isma...@gmail.com [mailto:isma...@gmail.com] On B

Re: ERROR Processor got uncaught exception. (kafka.network.Processor)

com> wrote: > Ismael, > > Version 0.9.0.1 > > Do you have any idea how to prevent this from happening? Is it a Kafka > issue? > > -Buvana > > -Original Message- > From: isma...@gmail.com [mailto:isma...@gmail.com] On Behalf Of EXT > Ismael Juma > S

Re: ERROR Processor got uncaught exception. (kafka.network.Processor)

Hi Buvana, What is the version of the broker? It looks like the state of connection counts for quotas became inconsistent, that is, a given client ip address has no connections according to the internal state, but a disconnection for that ip happened. Ismael On Tue, May 10, 2016 at 12:14 AM, Ram

Re: KAFKA-3112

Hi Raj and Tao, I just merged the KAFKA-3112 PR, so this issue will be fixed in 0.10.0.0. Thanks, Ismael On Fri, May 6, 2016 at 7:47 PM, tao xiao wrote: > KAFKA-2657 is unresolved so you can safely assume it hasn't been fixed yet. > > On Fri, 6 May 2016 at 07:38 Raj Tanneru wrote: > > > Yeah

Re: Receiving "The session timeout is not within an acceptable range" but AFAIK it is within range

Hi Mario, A warning should be logged by the consumer if you have unused properties in your config (precisely for this reason). In addition, 0.10 will also have improved error messages for cases such as this and we have increased the default `group.max.session.timeout.ms

Re: [ANNOUNCE] New committer: Ismael Juma

Thank you Neha and the PMC for the opportunity. I am both grateful and excited. :) And thanks to everyone else in the Kafka community too. Ismael On Mon, Apr 25, 2016 at 10:52 PM, Neha Narkhede wrote: > The PMC for Apache Kafka has invited Ismael Juma to join as a committer and >

Re: KafkaProducer Retries in .9.0.1

This is being tracked in KAFKA-3594 for others who find this. Ismael On Wed, Apr 20, 2016 at 10:45 PM, Ismael Juma wrote: > Hi Nicolas, > > That seems to be a different issue than the one initially discussed in > this thread. I suggest starting a new mailing list thread wi

Re: KafkaProducer Retries in .9.0.1

> > From my understanding, if the broker (leader) is unavailable, it should > buffer the message in the producer till the broker is available. > > Regards, > Nicolas > > On Thu, Apr 21, 2016 at 7:33 AM, Ismael Juma wrote: > > > Hi, > > > > This was expl

Re: KafkaProducer Retries in .9.0.1

Hi, This was explained earlier, I think. Retries are only attempted for retriable errors. If a message is too large, retrying won't help (it will still be too large). However, if a leader is not available, then a retry will happen as the leader may be available then. Ismael On Wed, Apr 20, 2016

Re: Using SSL with KafkaConsumer w/o client certificates

Hi, That warning is harmless. Personally, I think it may be a good idea to remove as it confuses people in cases such as this. Do you have multiple brokers? Are the brokers configured to use SSL for inter-broker communication (security.inter.broker.protocol)? This is required if the only listener

Re: Producer Bug? Moving form 0.9.0.1 to 0.10.1.0-SNAPSHOT (latest)

mmit is pretty > much just comment changes! > I can't see anything that would change actual behavior. I'm going to play > with these results more next week because it doesn't make any sense. I was > methodical to keep a clean environment but clearly something's not rig

Re: Producer Bug? Moving form 0.9.0.1 to 0.10.1.0-SNAPSHOT (latest)

from 0.9.0.1 to 0.10.1.0 and clean+test. Code > is identical, including the Docker packaging. > OK, so I'm looking at the logs and honestly I'm not 100% sure what I'm > looking at. I didn't see anything that looked like an error but that > doesn't mean its not there.

Re: Producer Bug? Moving form 0.9.0.1 to 0.10.1.0-SNAPSHOT (latest)

0.9 version Sent from my iPhone > On Apr 15, 2016, at 7:17 PM, Ismael Juma wrote: > > Hi Greg, > > What is the broker version? Have you checked that there are no errors > logged in the broker? > > Ismael > > On Fri, Apr 15, 2016 at 11:55 PM, Greg Zoller > wrote:

Re: Producer Bug? Moving form 0.9.0.1 to 0.10.1.0-SNAPSHOT (latest)

Hi Greg, What is the broker version? Have you checked that there are no errors logged in the broker? Ismael On Fri, Apr 15, 2016 at 11:55 PM, Greg Zoller wrote: > ok... one more attempt for a working link... > https://gist.github.com/gzoller/145faef1fefc8acea212e87e06fc86e8 > (If this doesn't

Re: how to config kafka security with plaintext && acl?

Hi Fredo, This may help: http://www.confluent.io/blog/apache-kafka-security-authorization-authentication-encryption Ismael On Fri, Apr 15, 2016 at 4:50 AM, Fredo Lee wrote: > how to config kafka security with plaintext && acl? i just want to deny > some ips. >

Re: [0.10.1.0-SNAPSHOT] KafkaProducer.close() not committing

Hi Greg, It may be worth passing a callback to the `send` to see if there's any error (or calling `get` on the Future returned). Ismael On Wed, Apr 13, 2016 at 2:59 PM, Greg Zoller wrote: > So I rebuilt last night with the latest from master branch. > Unfortunately same problem--producer doesn

Re: KafkaProducer 0.9.0.1 continually sends metadata requests

e result > and then calls this.metadata.requestUpdate() which tirggers the metadata to > be requested again. And of course the same thing happens when checking the > next response and we're suddenly in the loop forever. > > Hope this helps, > > Phil > > > -Or

users@kafka.apache.org

Hi Bharat, It looks authorization is not configured correctly. I suggest taking a look at the following blog post that configures authentication and authorization and includes a vagrant setup that you can test: http://www.confluent.io/blog/apache-kafka-security-authorization-authentication-encryp

Re: KafkaProducer 0.9.0.1 continually sends metadata requests

t up > producers until a message is ready to be sent. > > On Tue, Apr 12, 2016 at 6:59 AM Ismael Juma wrote: > > > Sorry, I should say that KAFKA-3306 will fix the issue where we request > > data for all topics instead of no topics. However, it seems like there is >

Re: KafkaProducer 0.9.0.1 continually sends metadata requests

that we can figure out if more needs to be done? Ismael On Tue, Apr 12, 2016 at 2:52 PM, Ismael Juma wrote: > Note that this should be fixed as part of > https://issues.apache.org/jira/browse/KAFKA-3306 > > Ismael > > On Tue, Apr 12, 2016 at 2:17 PM, Phil Luckhurst < > ph

Re: KafkaProducer 0.9.0.1 continually sends metadata requests

Note that this should be fixed as part of https://issues.apache.org/jira/browse/KAFKA-3306 Ismael On Tue, Apr 12, 2016 at 2:17 PM, Phil Luckhurst wrote: > Thanks Jonathan, I didn't spot that JIRA. > > Phil > > -Original Message- > From: Jonathan Bond [mailto:jb...@netflix.com.INVALID] >

Re: Error happen when kafka 0.9 client try connect with server, with SSL enabled

Hi Ping, The problem is advertised.host.name, which only advertises a PLAINTEXT port. You should use advertised.listeners instead. Ismael On Tue, Apr 5, 2016 at 8:42 PM, westfox wrote: > Hi, > > Got error when client try to talk with kafka server kafka_2.11-0.9.0.1, > with SSL enabled. same e

Re: New consumer API waits indefinitely

Hi Lohith, Are there any errors in your broker logs? I think there may be some issues with compacted topics on Windows and the new consumer uses a compacted topic to store offsets. Ismael On Mon, Apr 4, 2016 at 12:20 PM, Lohith Samaga M wrote: > Dear All, > The error seems to be NOT_CO

Re: Upgrade path to 0.10.x

Hi Tommy, I think it's reasonable to think it should work. However, our upgrade tests don't currently test that scenario and that is why we only mention {0.8.2.x, 0.9.0.x} to 0.10.0.0: https://github.com/apache/kafka/blob/trunk/tests/kafkatest/tests/upgrade_test.py#L62 The best way to verify wou

Re: Wait for 0.10 or go to 0.9 first

Hi Cees, On Fri, Apr 1, 2016 at 4:50 PM, Cees de Groot wrote: > FWIW we're using the new consumer in production. 0.9.0.0 had some glitches, > but we haven't had issues with 0.9.0.1. YMMV, of course. > That's good to hear. We believe 0.9.0.1 to be a solid release indeed. Even so, we are still de

Re: New async producer NetworkClient endlessly trying to fetch metadata for deleted topics

See: https://github.com/apache/kafka/pull/645 https://github.com/apache/kafka/pull/735 Ismael On Fri, Apr 1, 2016 at 5:04 PM, Stevo Slavić wrote: > Correction, it's much more often logging these warnings - every 100ms. > Current workaround is to regularly deploy fresh instance of app using > K

Re: Wait for 0.10 or go to 0.9 first

Hi John, 0.10.0.0 is at the RC stage at the moment (two RCs have been released and we know that there will be a third one at least). There are never guarantees when it comes to dates, but it is very likely that the final release will happen in Q2. If upgrading is a big deal, as you say, then perh

Re: kafka 0.9.0.1: FATAL exception on startup

Hi Anatoly and James, Can one of you please file a JIRA? Please describe the problem in detail and please include logs, if available. Ismael On Wed, Mar 23, 2016 at 5:44 AM, James Cheng wrote: > Hi, we ran into this problem too. The only way we were able to bypass this > was by stopping Kafka

Re: two questions

pointers are fine. > > I was under the impression that I could the 0.9 clients with 0.8 brokers. > > > > thanks > > > > Alexis > > > > On Mon, Mar 21, 2016 at 2:05 AM Ismael Juma wrote: > > > > > Hi Allen, > > > > > > Answers inline. >

Re: Security with SSL and not Kerberos?

Hi Gopal, As you suspected, you have to set the appropriate ACLs for it to work. The following will make the producer work: kafka-acls --authorizer-properties zookeeper.connect=localhost:2181 \ --add --allow-principal "User:CN=kafka.example.com,OU=Client,O=Confluent,L=London,ST=London,C=GB" \ --p

Re: two questions

el, > > > > could you elaborate on "newer clients don't work with older brokers > > though."? doc pointers are fine. > > I was under the impression that I could the 0.9 clients with 0.8 brokers. > > > > thanks > > > > Alexis > > &

Re: Security with SSL and not Kerberos?

Hi Christopher, On Mon, Mar 21, 2016 at 3:53 PM, christopher palm wrote: > Does Kafka support SSL authentication and ACL authorization without > Kerberos? > Yes. The following branch modifies the blog example slightly to only allow SSL authentication. https://github.com/confluentinc/securing-k

Re: two questions

Hi Allen, Answers inline. On Mon, Mar 21, 2016 at 5:56 AM, allen chan wrote: > 1) I am using the upgrade instructions to upgrade from 0.8 to 0.9. Can > someone tell me if i need to continue to bump the > inter.broker.protocol.version after each upgrade? Currently the broker code > is 0.9.0.1 bu

Re: Consumer 0.9 /Broker 0.9.0.1 - for certain partitions, the commit offset in kafka alway gets rewinded to a fixed point

See here for important issues fixed in 0.9.0.1: http://www.confluent.io/blog/announcing-apache-kafka-0.9.0.1-and-confluent-platform-2.0.1 Upgrading is highly recommended. Ismael On Sun, Mar 20, 2016 at 11:20 PM, Saurabh Daftary wrote: > Yes, I do mean 0.9.0.0. I am planning to upgrade to 0.9.

Re: Consumer 0.9 /Broker 0.9.0.1 - for certain partitions, the commit offset in kafka alway gets rewinded to a fixed point

Hi Saurabh, When you say Consumer 0.9, do you mean 0.9.0.0? If so, you should really upgrade to 0.9.0.1 as it contains a number of important fixes. Ismael On Sun, Mar 20, 2016 at 6:31 PM, Saurabh Daftary wrote: > I am seeing a very wired issue with my consumer group when running consumer > 0.9

Re: Kafka 0.9.0.1 broker 0.9 consumer location of consumer group data

Please upgrade indeed, 0.9.0.1 includes a number of important fixes. Ismael On 14 Mar 2016 18:36, "Rajiv Kurian" wrote: > No I haven't. It's still running the 0.9.0 client. I'll try upgrading if it > sounds like an old bug. > > On Mon, Mar 14, 2016 at 11:24 AM, Jason Gustafson > wrote: > > > He

Re: Connect bug in 0.9.0.1 client

Well spotted Larkin. Please file an issue as we definitely want to fix this before the next release. Ismael On Wed, Mar 9, 2016 at 10:46 PM, Christian Posta wrote: > Open a JIRA here: https://issues.apache.org/jira/browse/KAFKA > and open a github.com pull request here: https://github.com/apach

Re: [DISCUSS] KIP-45 Standardize all client sequence interaction on j.u.Collection.

Coming back to this, see below. On Wed, Jan 27, 2016 at 9:01 PM, Jason Gustafson wrote: > > 1. For subscribe() and assign(), change the parameter type to collection as > planned in the KIP. This is at least source-compatible, so as long as users > compile against the updated release, there shoul

Re: Kafka Security

Hi Martin, I suggest reading http://www.confluent.io/blog/apache-kafka-security-authorization-authentication-encryption for an end to end example of how to secure Kafka. Ismael On Fri, Mar 4, 2016 at 12:38 PM, Martin Gainty wrote: > Although authors suggest using existing Cloud security produc

Re: [sdc-user] Re: Having trouble to connect StreamSets to Kafka with Kerberos authentication

Hi Harikiran, One comment: `advertised.host.name` is not used if `advertised.listeners` is set and similarly `host.name` is not used if `listeners` is set. In general, the use of those properties is now discouraged in favour of listeners. There is a PR to make the documentation clearer: https://g

Re: Kafka Security quality level in 0.9.0.1

Hi Anatoliy, We labelled 0.9.0.0 as beta as it's a lot of new code and we want to: 1. Give our users a chance to test it and give us feedback 2. Do additional testing ourselves 0.9.0.1 has fixes for all the security issues we became aware of after the 0.9.0.0 release, but we haven't removed the

Re: Kafka 0.9.0.0 on Solaris 5.10 not running

Hi Marcus, One thing that changed in 0.9.0.0 is that we now use `NetworkClient` for inter-broker communication. `NetworkClient` is also used by the new Java producer and consumer. Does the Java producer in 0.8.2 work fine for you? Ismael On Mon, Feb 22, 2016 at 3:06 PM, Marcus Gründler wrote:

Re: new consumer still classified as beta in 0.9.0.1?

Even though we did not remove the beta label, all significant bugs we are aware of have been fixed (thanks Jason!). I'd say you should try it out. :) Ismael On Fri, Feb 19, 2016 at 9:06 PM, allen chan wrote: > My company is waiting for the new consumer to move out of "beta" mode > before using

Re: 0.9.0.1 RC1

+1 (non-binding). Verified source and binary artifacts, ran ./gradlew testAll with JDK 7u80, quick start on source artifact and Scala 2.11 binary artifact. Ismael On Fri, Feb 12, 2016 at 2:55 AM, Jun Rao wrote: > This is the first candidate for release of Apache Kafka 0.9.0.1. This a bug > fix

Re: Consumption: poll one message at a time

Hi Andre, Please see KIP-41: https://cwiki.apache.org/confluence/display/KAFKA/KIP-41%3A+KafkaConsumer+Max+Records The aim is to include this in the next release of Kafka. Ismael On Mon, Feb 15, 2016 at 12:21 PM, André wrote: > Hi there > > I've just started evaluating Kafka as an additional

Re: SSL Mirror Maker

Hi Lee, Is the CA used to sign the client certificates in the server truststore and the CA used to sign the server certificates in the client truststore? See the following blog post for a working example (including a Vagrant setup): http://www.confluent.io/blog/apache-kafka-security-authorization

Re: Building the 0.9.0 branch

Hi John, The core jar can be built with either Scala 2.10 or Scala 2.11 and the Scala version is included as part of the artifact. The read me page explains this in more detail. Search for "all scala versions" to find out how to run commands for all Scala versions or "particular version of Scala"

Re: Kafka 0.9.0.1 plan

t's still early in 0.9.0.0's life, if KAFKA-3006 has a chance of > making the cut (provided a resolution is attained on the KIP-45) it would > be great to avoid leaving too much time for code relying on Arrays to > become common place. > > On Sat, Feb 6, 2016 at 12:05 AM,

Re: Kafka 0.9.0.1 plan

gt; > > On Fri, Feb 5, 2016 at 1:19 PM, Ismael Juma wrote: > > > Hi Becket, > > > > On Fri, Feb 5, 2016 at 9:15 PM, Becket Qin wrote: > > > > > I am taking KAFKA-3177 off the list because the correct fix might > involve > > > some refactoring of

Re: Kafka 0.9.0.1 plan

Hi Becket, On Fri, Feb 5, 2016 at 9:15 PM, Becket Qin wrote: > I am taking KAFKA-3177 off the list because the correct fix might involve > some refactoring of exception hierarchy in new consumer. That may take some > time and 0.9.0.1 probably does not need to block on it. > Sounds good to me.

Re: FW: 0.9 consumer log spam - Marking the coordinator dead

t; > Thanks, > Rajiv > > On Fri, Feb 5, 2016 at 5:15 AM, Ismael Juma wrote: > > > Hi Simon, > > > > It may be worth trying the 0.9.0 branch as it includes a number of > > important fixes to the new consumer. > > > > Ismael > > > >

Re: Kafka protocol fetch request max wait.

Thanks for getting to the bottom of this Rajiv. Ismael On Fri, Feb 5, 2016 at 5:50 PM, Rajiv Kurian wrote: > I've updated Kafka-3159 with my findings. > > Thanks, > Rajiv > > On Thu, Feb 4, 2016 at 10:25 PM, Rajiv Kurian wrote: > > > I think I found out when the problem happens. When a broker

Re: FW: 0.9 consumer log spam - Marking the coordinator dead

Hi Simon, It may be worth trying the 0.9.0 branch as it includes a number of important fixes to the new consumer. Ismael On Fri, Feb 5, 2016 at 12:33 PM, Simon Cooper < simon.coo...@featurespace.co.uk> wrote: > Actually, this is incorrect - it looks like the consumer does not receive > any mess

Re: [DISCUSS] KIP-45 Standardize all client sequence interaction on j.u.Collection.

Hi Becket, On Wed, Jan 27, 2016 at 10:51 PM, Becket Qin wrote: > 2. For seek(), pause(), resume(), it depends on how easily user can use > them. > If we take current interface, and user have a list of partitions to > pause(), what they can do is something like: > pause(patitionList.toArr

Re: Kafka SSL Configuration Problems

INFO Registered broker 0 at path /brokers/ids/0 > with addresses: PLAINTEXT -> EndPoint(servername,9092,PLAINTEXT) > (kafka.utils.ZkUtils) > > > The only way I have been able to do this, is setting both. > > > > > On Feb 1, 2016, at 2:54 PM, Ismael Juma wrote: >

Re: Kafka SSL Configuration Problems

On Mon, Feb 1, 2016 at 7:15 PM, Nazario Parsacala wrote: > So it looks like you need both listeners and advertised.listeners ..? > No, you always need to set `listeners` (`advertised.listeners` defaults to `listeners`). If you want `advertised.listeners` to be different than `listeners`, then yo

Re: Kafka SSL Configuration Problems

Please use advertised.listeners instead of advertised.host.name. See this comment: https://github.com/apache/kafka/pull/793#issuecomment-174287124 Ismael On Mon, Feb 1, 2016 at 4:44 PM, Nazario Parsacala wrote: > Hi, > > We were using kafka for a while now. We have been using the binary releas

Re: [DISCUSS] KIP-45 Standardize all client sequence interaction on j.u.Collection.

Hi Pierre and Jason, A comment below. On Wed, Jan 27, 2016 at 9:01 PM, Jason Gustafson wrote: > Hi Pierre, > > Thanks for your persistence on this issue. I've gone back and forth on this > a few times. The current API can definitely be annoying in some cases, but > breaking compatibility still

Re: Broker Exception: Attempt to read with a maximum offset less than start offset

Hi Manu and Robert, It would help to know if this still happens in trunk or the 0.9.0 branch. Ismael On 27 Jan 2016 13:05, "Robert Metzger" wrote: > Hi Manu, > > in the streaming-benchmark, are seeing the issue only when reading with > Gearpump, or is it triggered by a different processing fram

Re: [DISCUSS] KIP-45 Standardize all client sequence interaction on j.u.Collection.

Thanks Pierre. Including the dev mailing list. A few comments: 1. It's worth mentioning that the KafkaConsumer has the @InterfaceStability.Unstable annotation. 2. It would be good to show the existing signatures of the methods being changed before we show the changed signatures. 3. The proposed c

Re: Memory records is not writable in MirrorMaker

Thanks. Ismael On Mon, Jan 25, 2016 at 8:28 PM, Meghana Narasimhan < mnarasim...@bandwidth.com> wrote: > I have created a JIRA, https://issues.apache.org/jira/browse/KAFKA-3147 > > Thanks, > Meghana > > On Tue, Jan 19, 2016 at 5:07 AM, Ismael Juma wrote: > > >

Re: Stuck consumer with new consumer API in 0.9

veloper > >>> > >>> Novazone, Edingsesteenweg 302, B-1755 Gooik, Belgium > >>> T: +32(0)54/26.02.03 - M:+32(0)477/39.01.15 > >>> bruno.rassae...@novazone.be -www.novazone.be > >>> > >>> > On 23 Jan 2016, at 17:52, Ismael Juma

Re: Kafka 0.9 -> consumer.poll() occasionally returns 0 elements

Hi, On Sun, Jan 24, 2016 at 7:17 PM, Krzysztof Ciesielski < krzysztof.ciesiel...@softwaremill.com> wrote: > Yes, it's exactly 5 seconds on every machine. Sure, I'll open a JIRA > Jason already did so: https://issues.apache.org/jira/browse/KAFKA-3135 Feel free to add relevant information there.

Re: End to end compression?

Hi Elias, Please see KIP-31: https://cwiki.apache.org/confluence/display/KAFKA/KIP-31+-+Move+to+relative+offsets+in+compressed+message+sets With regards to broker compression performance, the following is also relevant: http://www.confluent.io/blog/compression-in-apache-kafka-is-now-34-percent-

Re: Stuck consumer with new consumer API in 0.9

Hi, Can you please file an issue in JIRA so that we make sure this is investigated? Ismael On Fri, Jan 22, 2016 at 3:13 PM, Han JU wrote: > Hi, > > I'm prototyping with the new consumer API of kafka 0.9 and I'm particularly > interested in the `ConsumerRebalanceListener`. > > My test setup is

Re: trouble upgrading from 0.8.2.1 to 0.9.0.0: invalid message

> > at > > kafka.server.ReplicaManager.appendToLocalLog(ReplicaManager.scala:386) > > at > > kafka.server.ReplicaManager.appendMessages(ReplicaManager.scala:322) > > at > > kafka.server.KafkaApis.handleProducerRequest(KafkaApis.sca

<    1   2   3   4   5   >