[jira] [Commented] (KAFKA-865) Mavenize and separate the client.
[ https://issues.apache.org/jira/browse/KAFKA-865?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14078121#comment-14078121 ] Jay Kreps commented on KAFKA-865: - The new producer has been completed and the consumer is underway. > Mavenize and separate the client. > - > > Key: KAFKA-865 > URL: https://issues.apache.org/jira/browse/KAFKA-865 > Project: Kafka > Issue Type: Improvement > Components: clients >Affects Versions: 0.8.0 >Reporter: Ashwanth Fernando > > It seems that the java client for Kafka is also bundled with the server JAR > file and this is generated using sbt package. This is difficult for java > folks to work with because: > 1) Many java shops use maven and they want to specify the GAV of kafka in > their pom and bang, the client jar and all its dependencies should be added > to the application's classpath. I can't do that right now, because I need to > run ./sbt eclipse, get the .JAR, add that to my classpath, add a whole lot of > dependencies (log4j, slf4j, zkClient and so on) manually, which is a pain. > There are 90 million maven central uploads/downloads in 2012 alone. Almost > all the java shops out there have maven (either central or in house sonatype). > 2) Separation of concerns - keeping the server (core) and the client's > classes together in same jar file, increases the size of the bundle for a > client and also everytime the server's code changes and a release is > performed, the client also needs to update their .JAR file. which is not very > great. We don't want a ton of clients to update their .JAR file, just because > a faster replication strategy for the kafka server cluster changed in a new > release. > Action items are to separate the client and server portions of Kafka, add it > in a pom along with the compile time dependencies and upload it to Maven > Central or if you have a LinkedIn externally exposed Nexus, over there. > This will increase adoption of the Kafka framework. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (KAFKA-865) Mavenize and separate the client.
[ https://issues.apache.org/jira/browse/KAFKA-865?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14078114#comment-14078114 ] Jad Naous commented on KAFKA-865: - Any updates on this? It's a real pain using the client jar right now. Thanks! > Mavenize and separate the client. > - > > Key: KAFKA-865 > URL: https://issues.apache.org/jira/browse/KAFKA-865 > Project: Kafka > Issue Type: Improvement > Components: clients >Affects Versions: 0.8.0 >Reporter: Ashwanth Fernando > > It seems that the java client for Kafka is also bundled with the server JAR > file and this is generated using sbt package. This is difficult for java > folks to work with because: > 1) Many java shops use maven and they want to specify the GAV of kafka in > their pom and bang, the client jar and all its dependencies should be added > to the application's classpath. I can't do that right now, because I need to > run ./sbt eclipse, get the .JAR, add that to my classpath, add a whole lot of > dependencies (log4j, slf4j, zkClient and so on) manually, which is a pain. > There are 90 million maven central uploads/downloads in 2012 alone. Almost > all the java shops out there have maven (either central or in house sonatype). > 2) Separation of concerns - keeping the server (core) and the client's > classes together in same jar file, increases the size of the bundle for a > client and also everytime the server's code changes and a release is > performed, the client also needs to update their .JAR file. which is not very > great. We don't want a ton of clients to update their .JAR file, just because > a faster replication strategy for the kafka server cluster changed in a new > release. > Action items are to separate the client and server portions of Kafka, add it > in a pom along with the compile time dependencies and upload it to Maven > Central or if you have a LinkedIn externally exposed Nexus, over there. > This will increase adoption of the Kafka framework. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (KAFKA-865) Mavenize and separate the client.
[ https://issues.apache.org/jira/browse/KAFKA-865?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13792941#comment-13792941 ] Jay Kreps commented on KAFKA-865: - I think we will fix this in the client rewrite, timeline would be early next year. Separating the existing code into two jars will be pretty hard. Proposal is here: https://cwiki.apache.org/confluence/display/KAFKA/Client+Rewrite > Mavenize and separate the client. > - > > Key: KAFKA-865 > URL: https://issues.apache.org/jira/browse/KAFKA-865 > Project: Kafka > Issue Type: Improvement > Components: clients >Affects Versions: 0.8 >Reporter: Ashwanth Fernando > > It seems that the java client for Kafka is also bundled with the server JAR > file and this is generated using sbt package. This is difficult for java > folks to work with because: > 1) Many java shops use maven and they want to specify the GAV of kafka in > their pom and bang, the client jar and all its dependencies should be added > to the application's classpath. I can't do that right now, because I need to > run ./sbt eclipse, get the .JAR, add that to my classpath, add a whole lot of > dependencies (log4j, slf4j, zkClient and so on) manually, which is a pain. > There are 90 million maven central uploads/downloads in 2012 alone. Almost > all the java shops out there have maven (either central or in house sonatype). > 2) Separation of concerns - keeping the server (core) and the client's > classes together in same jar file, increases the size of the bundle for a > client and also everytime the server's code changes and a release is > performed, the client also needs to update their .JAR file. which is not very > great. We don't want a ton of clients to update their .JAR file, just because > a faster replication strategy for the kafka server cluster changed in a new > release. > Action items are to separate the client and server portions of Kafka, add it > in a pom along with the compile time dependencies and upload it to Maven > Central or if you have a LinkedIn externally exposed Nexus, over there. > This will increase adoption of the Kafka framework. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (KAFKA-865) Mavenize and separate the client.
[ https://issues.apache.org/jira/browse/KAFKA-865?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13791782#comment-13791782 ] Joe Stein commented on KAFKA-865: - Hi, does KAFKA-1018 clear this up or are there other updates/changes to make? > Mavenize and separate the client. > - > > Key: KAFKA-865 > URL: https://issues.apache.org/jira/browse/KAFKA-865 > Project: Kafka > Issue Type: Improvement > Components: clients >Affects Versions: 0.8 >Reporter: Ashwanth Fernando > > It seems that the java client for Kafka is also bundled with the server JAR > file and this is generated using sbt package. This is difficult for java > folks to work with because: > 1) Many java shops use maven and they want to specify the GAV of kafka in > their pom and bang, the client jar and all its dependencies should be added > to the application's classpath. I can't do that right now, because I need to > run ./sbt eclipse, get the .JAR, add that to my classpath, add a whole lot of > dependencies (log4j, slf4j, zkClient and so on) manually, which is a pain. > There are 90 million maven central uploads/downloads in 2012 alone. Almost > all the java shops out there have maven (either central or in house sonatype). > 2) Separation of concerns - keeping the server (core) and the client's > classes together in same jar file, increases the size of the bundle for a > client and also everytime the server's code changes and a release is > performed, the client also needs to update their .JAR file. which is not very > great. We don't want a ton of clients to update their .JAR file, just because > a faster replication strategy for the kafka server cluster changed in a new > release. > Action items are to separate the client and server portions of Kafka, add it > in a pom along with the compile time dependencies and upload it to Maven > Central or if you have a LinkedIn externally exposed Nexus, over there. > This will increase adoption of the Kafka framework. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (KAFKA-865) Mavenize and separate the client.
[ https://issues.apache.org/jira/browse/KAFKA-865?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13636127#comment-13636127 ] Esko Suomi commented on KAFKA-865: -- I guess I should point out that when it comes to dependencies, SBT is a bastardization of Ivy which on its own works really nice with Maven repositories, among other things. I do agree with several points here, especially the point about separating the server and client. As an example in our case we have to treat servers as really slowly evolving creatures and most of the bugs we've hit during our period of running Kafka have been exclusively client bugs. Because the two are tightly coupled, all we can do is to wait for the next release. If these two were separated, we could've remedied those client bugs with a quick update since the actual software we have can be updated as often as we like. > Mavenize and separate the client. > - > > Key: KAFKA-865 > URL: https://issues.apache.org/jira/browse/KAFKA-865 > Project: Kafka > Issue Type: Improvement > Components: clients >Affects Versions: 0.8 >Reporter: Ashwanth Fernando > > It seems that the java client for Kafka is also bundled with the server JAR > file and this is generated using sbt package. This is difficult for java > folks to work with because: > 1) Many java shops use maven (and a lot of them have a Sonatype Nexus > repository in house) for dependency management. They want to specify the GAV > and bang, the client jar and all its dependencies should be added to the > application's classpath. I can't do that right now, because I need to run > ./sbt eclipse, get the .JAR, add that to my classpath, add a whole lot of > dependencies (log4j, slf4j, zkClient and so on) manually, which is a pain. > There are 90 million maven central uploads/downloads in 2012 alone. Almost > all the java shops out there have maven (either central or in house sonatype). > 2) Separation of concerns - keeping the server (core) and the client's > classes increases the size of the bundle for the client and also everytime > the server's code changes and a release is performed, the client also needs > to update their .JAR file. which is not very great. We don't want a ton of > clients to update their .JAR file, just because a faster replication strategy > for my kafka cluster changed in a new release. > Action items are to separate the client portion of Kafka, add it in a pom > along with the compile time dependencies and upload it to Maven Central or if > you have a LinkedIn externally exposed Nexus, over there. > This will increase adoption of the Kafka framework. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (KAFKA-865) Mavenize and separate the client.
[ https://issues.apache.org/jira/browse/KAFKA-865?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13635522#comment-13635522 ] Ashwanth Fernando commented on KAFKA-865: - You are correct. Maven is not necessary to upload to a maven repo. sbt can be used as well. Just stating that kafka is not present on maven central right now and it needs to be pushed, server separate and client separate with the proper dependencies specified in the pom, so that integrating the java client with a java application becomes easy. > Mavenize and separate the client. > - > > Key: KAFKA-865 > URL: https://issues.apache.org/jira/browse/KAFKA-865 > Project: Kafka > Issue Type: Improvement > Components: clients >Affects Versions: 0.8 >Reporter: Ashwanth Fernando > > It seems that the java client for Kafka is also bundled with the server JAR > file and this is generated using sbt package. This is difficult for java > folks to work with because: > 1) Many java shops use maven (and a lot of them have a Sonatype Nexus > repository in house) for dependency management. They want to specify the GAV > and bang, the client jar and all its dependencies should be added to the > application's classpath. I can't do that right now, because I need to run > ./sbt eclipse, get the .JAR, add that to my classpath, add a whole lot of > dependencies (log4j, slf4j, zkClient and so on) manually, which is a pain. > There are 90 million maven central uploads/downloads in 2012 alone. Almost > all the java shops out there have maven (either central or in house sonatype). > 2) Separation of concerns - keeping the server (core) and the client's > classes increases the size of the bundle for the client and also everytime > the server's code changes and a release is performed, the client also needs > to update their .JAR file. which is not very great. We don't want a ton of > clients to update their .JAR file, just because a faster replication strategy > for my kafka cluster changed in a new release. > Action items are to separate the client portion of Kafka, add it in a pom > along with the compile time dependencies and upload it to Maven Central or if > you have a LinkedIn externally exposed Nexus, over there. > This will increase adoption of the Kafka framework. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (KAFKA-865) Mavenize and separate the client.
[ https://issues.apache.org/jira/browse/KAFKA-865?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13635478#comment-13635478 ] Scott Carey commented on KAFKA-865: --- I'm a big fan of Maven (It has some big flaws, but so does SBT and ant), but it is not necessary for this. SBT can publish artifacts to maven central with proper pom files. I'm not sure how much the client dependency tree differs from the server one at the moment -- perhaps only with zookeeper. I do feel that it is important to have dependencies declared in the published pom be accurate and not contain cruft for different use cases (e.g. producer vs consumer may have different transitive dependencies, and thus should have different artifacts). > Mavenize and separate the client. > - > > Key: KAFKA-865 > URL: https://issues.apache.org/jira/browse/KAFKA-865 > Project: Kafka > Issue Type: Improvement > Components: clients >Affects Versions: 0.8 >Reporter: Ashwanth Fernando > > It seems that the java client for Kafka is also bundled with the server JAR > file and this is generated using sbt package. This is difficult for java > folks to work with because: > 1) Many java shops use maven (and a lot of them have a Sonatype Nexus > repository in house) for dependency management. They want to specify the GAV > and bang, the client jar and all its dependencies should be added to the > application's classpath. I can't do that right now, because I need to run > ./sbt eclipse, get the .JAR, add that to my classpath, add a whole lot of > dependencies (log4j, slf4j, zkClient and so on) manually, which is a pain. > There are 90 million maven central uploads/downloads in 2012 alone. Almost > all the java shops out there have maven (either central or in house sonatype). > 2) Separation of concerns - keeping the server (core) and the client's > classes increases the size of the bundle for the client and also everytime > the server's code changes and a release is performed, the client also needs > to update their .JAR file. which is not very great. We don't want a ton of > clients to update their .JAR file, just because a faster replication strategy > for my kafka cluster changed in a new release. > Action items are to separate the client portion of Kafka, add it in a pom > along with the compile time dependencies and upload it to Maven Central or if > you have a LinkedIn externally exposed Nexus, over there. > This will increase adoption of the Kafka framework. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
