Spring Integration Kafka support
Hello, Spring Integration extensions has a new module to support Kafka 0.8 integration. Currently, adapters for producer and the high level consumer are available. It is still in development. Here is the current snapshot of this support: https://github.com/SpringSource/spring-integration-extensions/tree/master/spring-integration-kafka If you are a Spring/Java user and also use Kafka 0.8, please take a look. Any feedback, suggestions, and PR's are welcomed!! Thank you, Soby Chacko
Re: kafka 0.8
Jun, That is great to hear. Looking forward to it. Thanks, Soby On Tue, Jun 11, 2013 at 12:20 PM, Jun Rao jun...@gmail.com wrote: Soby, Sorry for the delay. This week and early next week, all Kafka committers at LinkedIn are busy with upgrading Kafka to 0.8 internally at LinkedIn. We will try to start the release process as soon as we have things under control. Thanks, Jun On Mon, Jun 10, 2013 at 6:54 PM, Soby Chacko sobycha...@gmail.com wrote: Hello, Any updates on the 0.8 beta release? Soby Chacko On Tue, Jun 4, 2013 at 12:24 PM, Neha Narkhede neha.narkh...@gmail.com wrote: I was just about to send an update. We can release beta right away. Joe, I remember you were interested in helping out. Let me know if you are still up for managing the release. Thanks, Neha On Jun 4, 2013 9:22 AM, Soby Chacko sobycha...@gmail.com wrote: Hello, Just wanted to know where we are with the beta release for 0.8? More importantly, is 0.8 going to be publicly available from a maven repository? How about different versions of 0.8 built for different versions of scala? (for example scala 2.8 vs 2.9 etc.) Much appreciated. Soby Chacko
Re: kafka 0.8
Hello, Any updates on the 0.8 beta release? Soby Chacko On Tue, Jun 4, 2013 at 12:24 PM, Neha Narkhede neha.narkh...@gmail.comwrote: I was just about to send an update. We can release beta right away. Joe, I remember you were interested in helping out. Let me know if you are still up for managing the release. Thanks, Neha On Jun 4, 2013 9:22 AM, Soby Chacko sobycha...@gmail.com wrote: Hello, Just wanted to know where we are with the beta release for 0.8? More importantly, is 0.8 going to be publicly available from a maven repository? How about different versions of 0.8 built for different versions of scala? (for example scala 2.8 vs 2.9 etc.) Much appreciated. Soby Chacko
kafka 0.8
Hello, Just wanted to know where we are with the beta release for 0.8? More importantly, is 0.8 going to be publicly available from a maven repository? How about different versions of 0.8 built for different versions of scala? (for example scala 2.8 vs 2.9 etc.) Much appreciated. Soby Chacko
Re: kafka 0.8 beta release status
Any status on the mavenization of 0.8? On Mon, Apr 29, 2013 at 11:04 AM, Jun Rao jun...@gmail.com wrote: Yes. Thanks, Jun On Mon, Apr 29, 2013 at 6:13 AM, Oleg Ruchovets oruchov...@gmail.com wrote: Does Kafka 0.8 become official beta? On Mon, Apr 29, 2013 at 8:52 AM, Jun Rao jun...@gmail.com wrote: We have updated the 0.8 documentation in our website ( http://kafka.apache.org/index.html). Please review the docs. We have the following blockers for the 0.8 beta release: additional docs: * examples of using the 0.8 high level consumer api * description of additional 0.8 tools KAFKA-885 (sbt package builds two kafka jars) It would be great if people can help on the blockers. Thanks, Jun
Re: kafka key serializer
Thanks for the reply. But, when I did some more research, it seems like its using the same encoder for both. For example, if I provide serializer.class explicitly, this serializer is used for both key and value. However, if I don't specify any serializer, then it appears that Kafka defaults to DefaultEncoder. Is that what you ment? Thanks again!! Soby Chacko On Wed, Apr 10, 2013 at 1:59 PM, Neha Narkhede neha.narkh...@gmail.comwrote: It will use DefaultEncoder. Thanks, Neha On Wed, Apr 10, 2013 at 8:27 AM, Soby Chacko sobycha...@gmail.com wrote: If I don't provide an explicit key serializer but a serializer class (for value encoding), and then use a key in KeyedMessage, what will be the encoder used for key? Is it going to default to the same encoder used for value or the DefaultEncoder? Thanks, Soby Chacko
kafka key serializer
If I don't provide an explicit key serializer but a serializer class (for value encoding), and then use a key in KeyedMessage, what will be the encoder used for key? Is it going to default to the same encoder used for value or the DefaultEncoder? Thanks, Soby Chacko
Re: kafka 0.8 release schedule?
Thanks!! On Thu, Apr 4, 2013 at 12:25 AM, Jun Rao jun...@gmail.com wrote: Soby, We plan to release 0.8 beta in Apr. Our build tool supports building on different versions of scala and the default one is 2.8. Thanks, Jun On Wed, Apr 3, 2013 at 4:02 PM, Soby Chacko sobycha...@gmail.com wrote: Hello, I am sure this question has been asked before. But, can someone tell me the tentative release schedule for Kafka 0.8? And also, when its released, is it going to be based on Scala 2.9.2 or default to 2.8? Regards, Soby Chacko.
kafka 0.8 release schedule?
Hello, I am sure this question has been asked before. But, can someone tell me the tentative release schedule for Kafka 0.8? And also, when its released, is it going to be based on Scala 2.9.2 or default to 2.8? Regards, Soby Chacko.
Re: Kafka 0.8 build artifacts
Hello, Whats the likelihood of more changes to these two yammer metrics libraries? Or are they going to stay the same for kafka 0.8? Regards, Soby Chacko On Tue, Mar 26, 2013 at 7:34 PM, Soby Chacko sobycha...@gmail.com wrote: Thanks Dragos!! Is the zkclient 0.2 change is pushed? I will see what I can do with the metrics jars locally. Thanks again!! Soby Chacko. On Tue, Mar 26, 2013 at 7:18 PM, Dragos Manolescu dragos.manole...@servicenow.com wrote: Hi Soby -- As far as I can tell the zkclient dependency is a leftover from 0.7.2. With the exception of the migration tool, which has a comment along these lines but I haven't looked into 0.8 uses the 0.2 version of the library (available from maven). I submitted a patch to address this, it is in the pipeline (http://j.mp/13uX6rs). It is not as simple with the metics library as there are significant API changes between the 3.0.0-c0c8be71 used in the Kafka code and 3.0.0-SNAPSHOT. There's an open item on that (http://j.mp/10ccTnE); meanwhile the path of least resistance is to manually add the 2 metrics JARs to the class path. I have a fork that addresses the above and has a couple of other minor fixes. With the caveat that it hasn't been reviewed by the Kafka team yore welcome to have a look: http://j.mp/ZsSld2 Welcome to the club ;) -Dragos On 3/26/13 3:28 PM, Soby Chacko sobycha...@gmail.com wrote: Hello, I built kafka 0.8 with scala 2.9.2. However, I ran into some issues after publishing the 0.8-SNAPSHOT into my local maven repository. From my application, I simply depended on this new jar thinking that it would pull all the other dependencies transitively from a public maven repository. (I look in a handful of repos in my build script) But, it could not find the following 3 jars from any repository: zkclient-20120522.jar metrics-annotation-3.0.0-c0c8be71.jar metrics-core-3.0.0-c0c8be71.jar I found these in the kafka 0.8 artifact that I built and was able to manually copy them. Are there any plans to make these jars available through any maven repositories, as they seem to contain custom changes? Appreciate any help. Regards, Soby Chacko
Re: Kafka 0.8 build artifacts
That will be awesome. Thanks, Soby Chacko On Wed, Mar 27, 2013 at 11:51 AM, Neha Narkhede neha.narkh...@gmail.comwrote: We are looking into the possibility of changing Kafka 0.8 to depend on metrics 2.2.0 instead. This will allow us to mavenize Kafka. THanks, Neha On Wed, Mar 27, 2013 at 8:45 AM, Soby Chacko sobycha...@gmail.com wrote: Hello, Whats the likelihood of more changes to these two yammer metrics libraries? Or are they going to stay the same for kafka 0.8? Regards, Soby Chacko On Tue, Mar 26, 2013 at 7:34 PM, Soby Chacko sobycha...@gmail.com wrote: Thanks Dragos!! Is the zkclient 0.2 change is pushed? I will see what I can do with the metrics jars locally. Thanks again!! Soby Chacko. On Tue, Mar 26, 2013 at 7:18 PM, Dragos Manolescu dragos.manole...@servicenow.com wrote: Hi Soby -- As far as I can tell the zkclient dependency is a leftover from 0.7.2. With the exception of the migration tool, which has a comment along these lines but I haven't looked into 0.8 uses the 0.2 version of the library (available from maven). I submitted a patch to address this, it is in the pipeline (http://j.mp/13uX6rs). It is not as simple with the metics library as there are significant API changes between the 3.0.0-c0c8be71 used in the Kafka code and 3.0.0-SNAPSHOT. There's an open item on that (http://j.mp/10ccTnE); meanwhile the path of least resistance is to manually add the 2 metrics JARs to the class path. I have a fork that addresses the above and has a couple of other minor fixes. With the caveat that it hasn't been reviewed by the Kafka team yore welcome to have a look: http://j.mp/ZsSld2 Welcome to the club ;) -Dragos On 3/26/13 3:28 PM, Soby Chacko sobycha...@gmail.com wrote: Hello, I built kafka 0.8 with scala 2.9.2. However, I ran into some issues after publishing the 0.8-SNAPSHOT into my local maven repository. From my application, I simply depended on this new jar thinking that it would pull all the other dependencies transitively from a public maven repository. (I look in a handful of repos in my build script) But, it could not find the following 3 jars from any repository: zkclient-20120522.jar metrics-annotation-3.0.0-c0c8be71.jar metrics-core-3.0.0-c0c8be71.jar I found these in the kafka 0.8 artifact that I built and was able to manually copy them. Are there any plans to make these jars available through any maven repositories, as they seem to contain custom changes? Appreciate any help. Regards, Soby Chacko
Kafka 0.8 build artifacts
Hello, I built kafka 0.8 with scala 2.9.2. However, I ran into some issues after publishing the 0.8-SNAPSHOT into my local maven repository. From my application, I simply depended on this new jar thinking that it would pull all the other dependencies transitively from a public maven repository. (I look in a handful of repos in my build script) But, it could not find the following 3 jars from any repository: zkclient-20120522.jar metrics-annotation-3.0.0-c0c8be71.jar metrics-core-3.0.0-c0c8be71.jar I found these in the kafka 0.8 artifact that I built and was able to manually copy them. Are there any plans to make these jars available through any maven repositories, as they seem to contain custom changes? Appreciate any help. Regards, Soby Chacko
Re: Kafka 0.8 build artifacts
Thanks Dragos!! Is the zkclient 0.2 change is pushed? I will see what I can do with the metrics jars locally. Thanks again!! Soby Chacko. On Tue, Mar 26, 2013 at 7:18 PM, Dragos Manolescu dragos.manole...@servicenow.com wrote: Hi Soby -- As far as I can tell the zkclient dependency is a leftover from 0.7.2. With the exception of the migration tool, which has a comment along these lines but I haven't looked into 0.8 uses the 0.2 version of the library (available from maven). I submitted a patch to address this, it is in the pipeline (http://j.mp/13uX6rs). It is not as simple with the metics library as there are significant API changes between the 3.0.0-c0c8be71 used in the Kafka code and 3.0.0-SNAPSHOT. There's an open item on that (http://j.mp/10ccTnE); meanwhile the path of least resistance is to manually add the 2 metrics JARs to the class path. I have a fork that addresses the above and has a couple of other minor fixes. With the caveat that it hasn't been reviewed by the Kafka team yore welcome to have a look: http://j.mp/ZsSld2 Welcome to the club ;) -Dragos On 3/26/13 3:28 PM, Soby Chacko sobycha...@gmail.com wrote: Hello, I built kafka 0.8 with scala 2.9.2. However, I ran into some issues after publishing the 0.8-SNAPSHOT into my local maven repository. From my application, I simply depended on this new jar thinking that it would pull all the other dependencies transitively from a public maven repository. (I look in a handful of repos in my build script) But, it could not find the following 3 jars from any repository: zkclient-20120522.jar metrics-annotation-3.0.0-c0c8be71.jar metrics-core-3.0.0-c0c8be71.jar I found these in the kafka 0.8 artifact that I built and was able to manually copy them. Are there any plans to make these jars available through any maven repositories, as they seem to contain custom changes? Appreciate any help. Regards, Soby Chacko
