[jira] [Commented] (KAFKA-826) Make Kafka 0.8 depend on metrics 2.2.0 instead of 3.x
[ https://issues.apache.org/jira/browse/KAFKA-826?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13625222#comment-13625222 ] Swapnil Ghike commented on KAFKA-826: - Thanks, the unit tests were ok. But while trying to boot up a kafka server, I got the following error. This is what I did, rm -rf ~/.ivy2 git rm -r core/lib ./sbt clean ./sbt update ./sbt package bin/zookeeper-server-start.sh config/zookeeper.properties bin/kafka-server-start.sh config/server.properties [2013-04-08 00:48:29,743] FATAL (kafka.Kafka$) java.lang.NoClassDefFoundError: com/yammer/metrics/reporting/CsvReporter at kafka.metrics.KafkaCSVMetricsReporter.init(KafkaCSVMetricsReporter.scala:53) at kafka.metrics.KafkaMetricsReporter$$anonfun$startReporters$1.apply(KafkaMetricsReporter.scala:60) at kafka.metrics.KafkaMetricsReporter$$anonfun$startReporters$1.apply(KafkaMetricsReporter.scala:58) at scala.collection.IndexedSeqOptimized$class.foreach(IndexedSeqOptimized.scala:34) at scala.collection.mutable.WrappedArray.foreach(WrappedArray.scala:32) at kafka.metrics.KafkaMetricsReporter$.startReporters(KafkaMetricsReporter.scala:58) at kafka.Kafka$.main(Kafka.scala:36) at kafka.Kafka.main(Kafka.scala) Caused by: java.lang.ClassNotFoundException: com.yammer.metrics.reporting.CsvReporter at java.net.URLClassLoader$1.run(URLClassLoader.java:202) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(URLClassLoader.java:190) at java.lang.ClassLoader.loadClass(ClassLoader.java:306) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301) at java.lang.ClassLoader.loadClass(ClassLoader.java:247) ... 8 more I am not currently sure what's causing this issue. > Make Kafka 0.8 depend on metrics 2.2.0 instead of 3.x > - > > Key: KAFKA-826 > URL: https://issues.apache.org/jira/browse/KAFKA-826 > Project: Kafka > Issue Type: Bug > Components: core >Affects Versions: 0.8 >Reporter: Neha Narkhede >Assignee: Dragos Manolescu >Priority: Blocker > Labels: build, kafka-0.8, metrics > Attachments: kafka-fix-for-826-complete.patch, > kafka-fix-for-826.patch, kafka-fix-for-826-take2.patch > > > In order to mavenize Kafka 0.8, we have to depend on metrics 2.2.0 since > metrics 3.x is a huge change as well as not an officially supported release. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Comment Edited] (KAFKA-826) Make Kafka 0.8 depend on metrics 2.2.0 instead of 3.x
[ https://issues.apache.org/jira/browse/KAFKA-826?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13625222#comment-13625222 ] Swapnil Ghike edited comment on KAFKA-826 at 4/8/13 8:04 AM: - Thanks, the unit tests were ok. But while trying to boot up a kafka server, I got the following error. This is what I did, rm -rf ~/.ivy2 patch -p1 < kafka-fix-for-826-complete.patch git rm -r core/lib ./sbt clean ./sbt update ./sbt package bin/zookeeper-server-start.sh config/zookeeper.properties bin/kafka-server-start.sh config/server.properties [2013-04-08 00:48:29,743] FATAL (kafka.Kafka$) java.lang.NoClassDefFoundError: com/yammer/metrics/reporting/CsvReporter at kafka.metrics.KafkaCSVMetricsReporter.init(KafkaCSVMetricsReporter.scala:53) at kafka.metrics.KafkaMetricsReporter$$anonfun$startReporters$1.apply(KafkaMetricsReporter.scala:60) at kafka.metrics.KafkaMetricsReporter$$anonfun$startReporters$1.apply(KafkaMetricsReporter.scala:58) at scala.collection.IndexedSeqOptimized$class.foreach(IndexedSeqOptimized.scala:34) at scala.collection.mutable.WrappedArray.foreach(WrappedArray.scala:32) at kafka.metrics.KafkaMetricsReporter$.startReporters(KafkaMetricsReporter.scala:58) at kafka.Kafka$.main(Kafka.scala:36) at kafka.Kafka.main(Kafka.scala) Caused by: java.lang.ClassNotFoundException: com.yammer.metrics.reporting.CsvReporter at java.net.URLClassLoader$1.run(URLClassLoader.java:202) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(URLClassLoader.java:190) at java.lang.ClassLoader.loadClass(ClassLoader.java:306) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301) at java.lang.ClassLoader.loadClass(ClassLoader.java:247) ... 8 more I am not currently sure what's causing this issue. was (Author: swapnilghike): Thanks, the unit tests were ok. But while trying to boot up a kafka server, I got the following error. This is what I did, rm -rf ~/.ivy2 git rm -r core/lib ./sbt clean ./sbt update ./sbt package bin/zookeeper-server-start.sh config/zookeeper.properties bin/kafka-server-start.sh config/server.properties [2013-04-08 00:48:29,743] FATAL (kafka.Kafka$) java.lang.NoClassDefFoundError: com/yammer/metrics/reporting/CsvReporter at kafka.metrics.KafkaCSVMetricsReporter.init(KafkaCSVMetricsReporter.scala:53) at kafka.metrics.KafkaMetricsReporter$$anonfun$startReporters$1.apply(KafkaMetricsReporter.scala:60) at kafka.metrics.KafkaMetricsReporter$$anonfun$startReporters$1.apply(KafkaMetricsReporter.scala:58) at scala.collection.IndexedSeqOptimized$class.foreach(IndexedSeqOptimized.scala:34) at scala.collection.mutable.WrappedArray.foreach(WrappedArray.scala:32) at kafka.metrics.KafkaMetricsReporter$.startReporters(KafkaMetricsReporter.scala:58) at kafka.Kafka$.main(Kafka.scala:36) at kafka.Kafka.main(Kafka.scala) Caused by: java.lang.ClassNotFoundException: com.yammer.metrics.reporting.CsvReporter at java.net.URLClassLoader$1.run(URLClassLoader.java:202) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(URLClassLoader.java:190) at java.lang.ClassLoader.loadClass(ClassLoader.java:306) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301) at java.lang.ClassLoader.loadClass(ClassLoader.java:247) ... 8 more I am not currently sure what's causing this issue. > Make Kafka 0.8 depend on metrics 2.2.0 instead of 3.x > - > > Key: KAFKA-826 > URL: https://issues.apache.org/jira/browse/KAFKA-826 > Project: Kafka > Issue Type: Bug > Components: core >Affects Versions: 0.8 >Reporter: Neha Narkhede >Assignee: Dragos Manolescu >Priority: Blocker > Labels: build, kafka-0.8, metrics > Attachments: kafka-fix-for-826-complete.patch, > kafka-fix-for-826.patch, kafka-fix-for-826-take2.patch > > > In order to mavenize Kafka 0.8, we have to depend on metrics 2.2.0 since > metrics 3.x is a huge change as well as not an officially supported release. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (KAFKA-855) Ant+Ivy build for Kafka
David Arthur created KAFKA-855: -- Summary: Ant+Ivy build for Kafka Key: KAFKA-855 URL: https://issues.apache.org/jira/browse/KAFKA-855 Project: Kafka Issue Type: Improvement Affects Versions: 0.9 Reporter: David Arthur Fix For: 0.9 Kafka has very simple build requirements and a system like Ant is well suited for a clean and concise build. I have an experimental patch that does just this - replaces SBT with Ant+Ivy. IMO, this approach is cleaner, clearer, and more developer friendly. Dependencies are localized to one directory in the project rather than living in ~/.ivy2 and elsewhere. This makes manual classpath building very simple (just one glob) and also makes packaging the libs very easy. Testing is done through junit rather than scalatest. The Kafka tests use `org.scalatest.junit.JUnitSuite` which allow the tests to be executed through the junit test runner. Management of the Scala version is handled through Ivy. The way I have laid out the Ant script, the Scala version can be changed by setting a different runtime property (-Dscala.version=2.8.2). Cross-compilation of the Kafka artifact would be simple to add. The one downside to this approach is lack of an incremental build. `scalac` is deprecating its incremental build capabilities in coming versions. The suggested solution to this is to use an IDE that supports incremental builds. The main motivation for this approach, to me at least, is that a developer can look at build.xml and immediately understand what is going on (with the exception maybe of the actions which would not be changing). This is largely not true for SBT unless someone is already familiar with SBT. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: Thoughts on using Ant+Ivy for the build?
Jun, I have opened a ticket targeting 0.9 https://issues.apache.org/jira/browse/KAFKA-855 -David On 4/8/13 1:06 AM, Jun Rao wrote: David, We'd be open to a new build tool if SBT is deemed too hard to use. We can revisit this post 0.8. Do you want to open a jira to track this? Thanks, Jun On Fri, Apr 5, 2013 at 12:55 PM, David Arthur wrote: After getting frustrated with SBT, and being unable to figure out seemingly simple problems like KAFKA-843, I decided to try something completely different. I spent some time yesterday adapting some Ant/Ivy boilerplate I use for projects to Kafka. It was actually pretty easy to get working (Kafka is a very simple build), and I think it's _much_ cleaner than the existing SBT stuff. Attached is a patchset of the work I did. N.B., this is totally experimental and only considers the "core" part of the project. At this point I can: * Resolve all deps through Ivy (except yammer.metrics which is checked in) * Resolve different versions of Scala through Ivy (i.e., cross compile-ability) * Compile the source * Run all the unit tests (all passing) * Compile a jar * Package a tarball of the libs, conf, and bin scripts Since all the libraries are localized to the project (and not in ~/.ivy2), the packaging is trivial. Also, the bin scripts could be cleaned up significantly (which they need to be IMO). I would love to hear what everyone thinks of this. Am I crazy? Is SBT really better? Convince me! -David
[jira] [Updated] (KAFKA-855) Ant+Ivy build for Kafka
[ https://issues.apache.org/jira/browse/KAFKA-855?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Arthur updated KAFKA-855: --- Attachment: 0003-Parameterize-scala-version-add-offline-mode.patch 0002-Tests-all-passing.patch 0001-Compile-and-package-Kafka-with-Ant-Ivy.patch Attaching initial patch set > Ant+Ivy build for Kafka > --- > > Key: KAFKA-855 > URL: https://issues.apache.org/jira/browse/KAFKA-855 > Project: Kafka > Issue Type: Improvement >Affects Versions: 0.9 >Reporter: David Arthur > Labels: build, experimental > Fix For: 0.9 > > Attachments: 0001-Compile-and-package-Kafka-with-Ant-Ivy.patch, > 0002-Tests-all-passing.patch, > 0003-Parameterize-scala-version-add-offline-mode.patch > > Original Estimate: 168h > Remaining Estimate: 168h > > Kafka has very simple build requirements and a system like Ant is well suited > for a clean and concise build. I have an experimental patch that does just > this - replaces SBT with Ant+Ivy. IMO, this approach is cleaner, clearer, and > more developer friendly. > Dependencies are localized to one directory in the project rather than living > in ~/.ivy2 and elsewhere. This makes manual classpath building very simple > (just one glob) and also makes packaging the libs very easy. > Testing is done through junit rather than scalatest. The Kafka tests use > `org.scalatest.junit.JUnitSuite` which allow the tests to be executed through > the junit test runner. > Management of the Scala version is handled through Ivy. The way I have laid > out the Ant script, the Scala version can be changed by setting a different > runtime property (-Dscala.version=2.8.2). Cross-compilation of the Kafka > artifact would be simple to add. > The one downside to this approach is lack of an incremental build. `scalac` > is deprecating its incremental build capabilities in coming versions. The > suggested solution to this is to use an IDE that supports incremental builds. > The main motivation for this approach, to me at least, is that a developer > can look at build.xml and immediately understand what is going on (with the > exception maybe of the actions which would not be changing). This > is largely not true for SBT unless someone is already familiar with SBT. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (KAFKA-856) Correlation id for OffsetFetch request (#2) always responds with 0
Milosz Tanski created KAFKA-856: --- Summary: Correlation id for OffsetFetch request (#2) always responds with 0 Key: KAFKA-856 URL: https://issues.apache.org/jira/browse/KAFKA-856 Project: Kafka Issue Type: Bug Components: network Affects Versions: 0.8, 0.8.1 Environment: Linux & Mac Reporter: Milosz Tanski Assignee: Jun Rao The in the new Kafka when making an OffsetFetch request the correlation id always response is always sent as 0. It doesn't matter if the client request specifies a correlation id other then 0. Example wireshark capture: 00 00 00 31 00 07 00 00 00 00 00 2a 00 03 66 6f ...1 ...*..fo 0010 6f 00 0a 74 65 73 74 2d 67 72 6f 75 70 00 00 00 o..test- group... 0020 01 00 0a 74 65 73 74 5f 74 6f 70 69 63 00 00 00 ...test_ topic... 0030 01 00 00 00 00 . Request #1 len:00 00 00 31 api:00 07 ver:00 00 cor:00 00 00 2a cid:2a 00 03 66 6f 6f 00 00 00 2d 00 00 00 2a 00 03 66 6f 6f 00 00 00 ...-...* ..foo... 0010 01 00 0a 74 65 73 74 5f 74 6f 70 69 63 00 00 00 ...test_ topic... 0020 01 00 00 00 00 ff ff ff ff ff ff ff ff 00 00 00 0030 03 Response #1 len:00 00 00 2d cor:00 00 00 2a cid:2a 00 03 66 6f 6f 0035 00 00 00 35 00 02 00 00 00 00 00 2a 00 03 66 6f ...5 ...*..fo 0045 6f ff ff ff ff 00 00 00 01 00 0a 74 65 73 74 5f o... ...test_ 0055 74 6f 70 69 63 00 00 00 01 00 00 00 00 ff ff ff topic... 0065 ff ff ff ff fe 00 00 00 01 . Request #2 - len:00 00 00 35 api:00 02 ver:00 00 cor:00 00 00 2a cid:00 03 66 6f 6f 0031 00 00 00 2a 00 00 00 00 00 00 00 01 00 0a 74 65 ...* ..te 0041 73 74 5f 74 6f 70 69 63 00 00 00 01 00 00 00 00 st_topic 0051 00 00 00 00 00 01 00 00 00 00 00 00 00 00 .. Response #2: -- len:00 00 00 2a cor:00 00 00 00 alen: 00 00 00 01 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
Incubator PMC/Board report for Apr 2013 ([ppmc])
Dear podling, This email was sent by an automated system on behalf of the Apache Incubator PMC. It is an initial reminder to give you plenty of time to prepare your quarterly board report. The board meeting is scheduled for Wed, 17 April 2013, 10:30:00:00 PST. The report for your podling will form a part of the Incubator PMC report. The Incubator PMC requires your report to be submitted 2 weeks before the board meeting, to allow sufficient time for review and submission (Wed, Apr 3rd). Please submit your report with sufficient time to allow the incubator PMC, and subsequently board members to review and digest. Again, the very latest you should submit your report is 2 weeks prior to the board meeting. Thanks, The Apache Incubator PMC Submitting your Report -- Your report should contain the following: * Your project name * A brief description of your project, which assumes no knowledge of the project or necessarily of its field * A list of the three most important issues to address in the move towards graduation. * Any issues that the Incubator PMC or ASF Board might wish/need to be aware of * How has the community developed since the last report * How has the project developed since the last report. This should be appended to the Incubator Wiki page at: http://wiki.apache.org/incubator/April2013 Note: This manually populated. You may need to wait a little before this page is created from a template. Mentors --- Mentors should review reports for their project(s) and sign them off on the Incubator wiki page. Signing off reports shows that you are following the project - projects that are not signed may raise alarms for the Incubator PMC. Incubator PMC
[jira] [Commented] (KAFKA-850) add an option to show under replicated partitions in list topic command
[
https://issues.apache.org/jira/browse/KAFKA-850?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13625512#comment-13625512
]
Neha Narkhede commented on KAFKA-850:
-
Thanks for the patch. Minor comments -
ListTopicCommand-
1.1 Typo in ListTopicCommand -> "partiitons"
1.2 Why do we print the topic separately like this ? It becomes quite
unwieldily to parse the output where one line just has topic, but not the
partitions and replicas. Also, we print the topic anyways with the rest of the
partition and replica info. So I think we can remove this
if (!reportUnderReplicatedPartitions && !reportUnavailablePartitions)
println("topic: " + topic)
> add an option to show under replicated partitions in list topic command
> ---
>
> Key: KAFKA-850
> URL: https://issues.apache.org/jira/browse/KAFKA-850
> Project: Kafka
> Issue Type: Improvement
> Components: core
>Affects Versions: 0.8
>Reporter: Jun Rao
>Assignee: Jun Rao
>Priority: Blocker
> Attachments: kafka-850.patch, kafka-850_v2.patch, kafka-850_v3.patch
>
>
> For debugging purpose, it's very important to be able to find out the under
> replicated partitions quickly. List topic command is a good place to add this
> feature.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (KAFKA-850) add an option to show under replicated partitions in list topic command
[ https://issues.apache.org/jira/browse/KAFKA-850?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13625513#comment-13625513 ] Neha Narkhede commented on KAFKA-850: - Also +1 once those comments are addressed. > add an option to show under replicated partitions in list topic command > --- > > Key: KAFKA-850 > URL: https://issues.apache.org/jira/browse/KAFKA-850 > Project: Kafka > Issue Type: Improvement > Components: core >Affects Versions: 0.8 >Reporter: Jun Rao >Assignee: Jun Rao >Priority: Blocker > Attachments: kafka-850.patch, kafka-850_v2.patch, kafka-850_v3.patch > > > For debugging purpose, it's very important to be able to find out the under > replicated partitions quickly. List topic command is a good place to add this > feature. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (KAFKA-852) Remove clientId from OffsetFetchResponse and OffsetCommitResponse
[ https://issues.apache.org/jira/browse/KAFKA-852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13625515#comment-13625515 ] Neha Narkhede commented on KAFKA-852: - Thanks for the patch, David. However, I'm not able to apply it using the following commands - patch -p1 kafka-852.patch patch -p0 kafka-852.patch Please create a patch using the following command - git diff > kafka-852.patch > Remove clientId from OffsetFetchResponse and OffsetCommitResponse > - > > Key: KAFKA-852 > URL: https://issues.apache.org/jira/browse/KAFKA-852 > Project: Kafka > Issue Type: Bug > Components: core >Affects Versions: 0.8.1 >Reporter: David Arthur >Assignee: David Arthur >Priority: Minor > Fix For: 0.8.1 > > Attachments: > 0001-KAFKA-852-remove-clientId-from-Offset-Fetch-Commit-R.patch > > Original Estimate: 2h > Remaining Estimate: 2h > > These are not needed and conflict with the API documentation. Should be > removed to be consistent with other APIs -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (KAFKA-854) Upgrade dependencies for 0.8
[ https://issues.apache.org/jira/browse/KAFKA-854?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13625520#comment-13625520 ] Neha Narkhede commented on KAFKA-854: - Scott, I think that is a remainder that we forgot to delete when we upgraded to latest sbt version. We want to keep only the dependencies mentioned in project/Build.scala and the individual project build files . For example, core/build.sbt. Feel free to upload a patch to clean up the dependencies, it will be great help to us while releasing 0.8 > Upgrade dependencies for 0.8 > > > Key: KAFKA-854 > URL: https://issues.apache.org/jira/browse/KAFKA-854 > Project: Kafka > Issue Type: Improvement >Affects Versions: 0.8 >Reporter: Scott Carey > Attachments: 0001-KAFKA-854-Upgrade-Deps-to-latest.patch > > > Many of the dependencies that Kafka 0.8 uses are old. It would be a good > idea to upgrade all of these where possible. > log4j is set to 1.2.15, but the latest is 1.2.17 > zookeeper is at 3.3.4, , there is 3.3.6 (August 2012), and 3.4.5 (Nov 2012. > 3.4.x includes several major enhancements and fixes over the 3.3.x line. > org.slf4j is at 1.6.4, there is 1.6.6 (June 2012) and 1.7.5 (March 2013) > net.sf.jopt-simple is at 3.2, there is 3.3 (May 2011) and 4.4 (Jan 2013) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Comment Edited] (KAFKA-852) Remove clientId from OffsetFetchResponse and OffsetCommitResponse
[ https://issues.apache.org/jira/browse/KAFKA-852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13625515#comment-13625515 ] Neha Narkhede edited comment on KAFKA-852 at 4/8/13 4:40 PM: - Thanks for the patch, David. However, I'm not able to apply it using the following commands - patch -p1 kafka-852.patch patch -p0 kafka-852.patch Please create a patch using the following command - git diff > kafka-852.patch OR git remote update git diff origin/0.8 > kafka-852.patch was (Author: nehanarkhede): Thanks for the patch, David. However, I'm not able to apply it using the following commands - patch -p1 kafka-852.patch patch -p0 kafka-852.patch Please create a patch using the following command - git diff > kafka-852.patch > Remove clientId from OffsetFetchResponse and OffsetCommitResponse > - > > Key: KAFKA-852 > URL: https://issues.apache.org/jira/browse/KAFKA-852 > Project: Kafka > Issue Type: Bug > Components: core >Affects Versions: 0.8.1 >Reporter: David Arthur >Assignee: David Arthur >Priority: Minor > Fix For: 0.8.1 > > Attachments: > 0001-KAFKA-852-remove-clientId-from-Offset-Fetch-Commit-R.patch > > Original Estimate: 2h > Remaining Estimate: 2h > > These are not needed and conflict with the API documentation. Should be > removed to be consistent with other APIs -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (KAFKA-850) add an option to show under replicated partitions in list topic command
[ https://issues.apache.org/jira/browse/KAFKA-850?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao updated KAFKA-850: -- Resolution: Fixed Fix Version/s: 0.8 Status: Resolved (was: Patch Available) Thanks for the review. Committed to 0.8 with the suggested changes. > add an option to show under replicated partitions in list topic command > --- > > Key: KAFKA-850 > URL: https://issues.apache.org/jira/browse/KAFKA-850 > Project: Kafka > Issue Type: Improvement > Components: core >Affects Versions: 0.8 >Reporter: Jun Rao >Assignee: Jun Rao >Priority: Blocker > Fix For: 0.8 > > Attachments: kafka-850.patch, kafka-850_v2.patch, kafka-850_v3.patch > > > For debugging purpose, it's very important to be able to find out the under > replicated partitions quickly. List topic command is a good place to add this > feature. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (KAFKA-856) Correlation id for OffsetFetch request (#2) always responds with 0
[ https://issues.apache.org/jira/browse/KAFKA-856?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Milosz Tanski updated KAFKA-856: Status: Patch Available (was: Open) I have a patch to fix this > Correlation id for OffsetFetch request (#2) always responds with 0 > -- > > Key: KAFKA-856 > URL: https://issues.apache.org/jira/browse/KAFKA-856 > Project: Kafka > Issue Type: Bug > Components: network >Affects Versions: 0.8, 0.8.1 > Environment: Linux & Mac >Reporter: Milosz Tanski >Assignee: Jun Rao > > The in the new Kafka when making an OffsetFetch request the correlation id > always response is always sent as 0. It doesn't matter if the client request > specifies a correlation id other then 0. > Example wireshark capture: > 00 00 00 31 00 07 00 00 00 00 00 2a 00 03 66 6f ...1 ...*..fo > 0010 6f 00 0a 74 65 73 74 2d 67 72 6f 75 70 00 00 00 o..test- group... > 0020 01 00 0a 74 65 73 74 5f 74 6f 70 69 63 00 00 00 ...test_ topic... > 0030 01 00 00 00 00 . > > Request #1 > > len:00 00 00 31 > api:00 07 > ver:00 00 > cor:00 00 00 2a > cid:2a 00 03 66 6f 6f > > > 00 00 00 2d 00 00 00 2a 00 03 66 6f 6f 00 00 00 ...-...* ..foo... > 0010 01 00 0a 74 65 73 74 5f 74 6f 70 69 63 00 00 00 ...test_ topic... > 0020 01 00 00 00 00 ff ff ff ff ff ff ff ff 00 00 00 > 0030 03 > > Response #1 > > len:00 00 00 2d > cor:00 00 00 2a > cid:2a 00 03 66 6f 6f > > > 0035 00 00 00 35 00 02 00 00 00 00 00 2a 00 03 66 6f ...5 ...*..fo > 0045 6f ff ff ff ff 00 00 00 01 00 0a 74 65 73 74 5f o... ...test_ > 0055 74 6f 70 69 63 00 00 00 01 00 00 00 00 ff ff ff topic... > 0065 ff ff ff ff fe 00 00 00 01 . > > Request #2 > - > len:00 00 00 35 > api:00 02 > ver:00 00 > cor:00 00 00 2a > cid:00 03 66 6f 6f > > 0031 00 00 00 2a 00 00 00 00 00 00 00 01 00 0a 74 65 ...* ..te > 0041 73 74 5f 74 6f 70 69 63 00 00 00 01 00 00 00 00 st_topic > 0051 00 00 00 00 00 01 00 00 00 00 00 00 00 00 .. > > Response #2: > -- > len:00 00 00 2a > cor:00 00 00 00 > alen: 00 00 00 01 > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (KAFKA-856) Correlation id for OffsetFetch request (#2) always responds with 0
[ https://issues.apache.org/jira/browse/KAFKA-856?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Milosz Tanski updated KAFKA-856: Attachment: KAFKA-856.patch The mentioned patch. > Correlation id for OffsetFetch request (#2) always responds with 0 > -- > > Key: KAFKA-856 > URL: https://issues.apache.org/jira/browse/KAFKA-856 > Project: Kafka > Issue Type: Bug > Components: network >Affects Versions: 0.8, 0.8.1 > Environment: Linux & Mac >Reporter: Milosz Tanski >Assignee: Jun Rao > Attachments: KAFKA-856.patch > > > The in the new Kafka when making an OffsetFetch request the correlation id > always response is always sent as 0. It doesn't matter if the client request > specifies a correlation id other then 0. > Example wireshark capture: > 00 00 00 31 00 07 00 00 00 00 00 2a 00 03 66 6f ...1 ...*..fo > 0010 6f 00 0a 74 65 73 74 2d 67 72 6f 75 70 00 00 00 o..test- group... > 0020 01 00 0a 74 65 73 74 5f 74 6f 70 69 63 00 00 00 ...test_ topic... > 0030 01 00 00 00 00 . > > Request #1 > > len:00 00 00 31 > api:00 07 > ver:00 00 > cor:00 00 00 2a > cid:2a 00 03 66 6f 6f > > > 00 00 00 2d 00 00 00 2a 00 03 66 6f 6f 00 00 00 ...-...* ..foo... > 0010 01 00 0a 74 65 73 74 5f 74 6f 70 69 63 00 00 00 ...test_ topic... > 0020 01 00 00 00 00 ff ff ff ff ff ff ff ff 00 00 00 > 0030 03 > > Response #1 > > len:00 00 00 2d > cor:00 00 00 2a > cid:2a 00 03 66 6f 6f > > > 0035 00 00 00 35 00 02 00 00 00 00 00 2a 00 03 66 6f ...5 ...*..fo > 0045 6f ff ff ff ff 00 00 00 01 00 0a 74 65 73 74 5f o... ...test_ > 0055 74 6f 70 69 63 00 00 00 01 00 00 00 00 ff ff ff topic... > 0065 ff ff ff ff fe 00 00 00 01 . > > Request #2 > - > len:00 00 00 35 > api:00 02 > ver:00 00 > cor:00 00 00 2a > cid:00 03 66 6f 6f > > 0031 00 00 00 2a 00 00 00 00 00 00 00 01 00 0a 74 65 ...* ..te > 0041 73 74 5f 74 6f 70 69 63 00 00 00 01 00 00 00 00 st_topic > 0051 00 00 00 00 00 01 00 00 00 00 00 00 00 00 .. > > Response #2: > -- > len:00 00 00 2a > cor:00 00 00 00 > alen: 00 00 00 01 > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (KAFKA-852) Remove clientId from OffsetFetchResponse and OffsetCommitResponse
[ https://issues.apache.org/jira/browse/KAFKA-852?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Arthur updated KAFKA-852: --- Attachment: KAFKA-852.diff > Remove clientId from OffsetFetchResponse and OffsetCommitResponse > - > > Key: KAFKA-852 > URL: https://issues.apache.org/jira/browse/KAFKA-852 > Project: Kafka > Issue Type: Bug > Components: core >Affects Versions: 0.8.1 >Reporter: David Arthur >Assignee: David Arthur >Priority: Minor > Fix For: 0.8.1 > > Attachments: > 0001-KAFKA-852-remove-clientId-from-Offset-Fetch-Commit-R.patch, > KAFKA-852.diff > > Original Estimate: 2h > Remaining Estimate: 2h > > These are not needed and conflict with the API documentation. Should be > removed to be consistent with other APIs -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (KAFKA-852) Remove clientId from OffsetFetchResponse and OffsetCommitResponse
[ https://issues.apache.org/jira/browse/KAFKA-852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13625576#comment-13625576 ] David Arthur commented on KAFKA-852: @Neha, I attached a diff created using git-diff. The previous patch was generated using git format-patch. Also, this issue is targeting 0.8.1 (which I'm assuming is trunk) not 0.8. The Offset fetch/commit APIs are not slated for the 0.8 release AFAIK > Remove clientId from OffsetFetchResponse and OffsetCommitResponse > - > > Key: KAFKA-852 > URL: https://issues.apache.org/jira/browse/KAFKA-852 > Project: Kafka > Issue Type: Bug > Components: core >Affects Versions: 0.8.1 >Reporter: David Arthur >Assignee: David Arthur >Priority: Minor > Fix For: 0.8.1 > > Attachments: > 0001-KAFKA-852-remove-clientId-from-Offset-Fetch-Commit-R.patch, > KAFKA-852.diff > > Original Estimate: 2h > Remaining Estimate: 2h > > These are not needed and conflict with the API documentation. Should be > removed to be consistent with other APIs -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (KAFKA-826) Make Kafka 0.8 depend on metrics 2.2.0 instead of 3.x
[ https://issues.apache.org/jira/browse/KAFKA-826?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dragos Manolescu updated KAFKA-826: --- Attachment: (was: kafka-fix-for-826-complete.patch) > Make Kafka 0.8 depend on metrics 2.2.0 instead of 3.x > - > > Key: KAFKA-826 > URL: https://issues.apache.org/jira/browse/KAFKA-826 > Project: Kafka > Issue Type: Bug > Components: core >Affects Versions: 0.8 >Reporter: Neha Narkhede >Assignee: Dragos Manolescu >Priority: Blocker > Labels: build, kafka-0.8, metrics > Attachments: kafka-fix-for-826.patch, kafka-fix-for-826-take2.patch > > > In order to mavenize Kafka 0.8, we have to depend on metrics 2.2.0 since > metrics 3.x is a huge change as well as not an officially supported release. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (KAFKA-854) Upgrade dependencies for 0.8
[ https://issues.apache.org/jira/browse/KAFKA-854?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13625601#comment-13625601 ] Scott Carey commented on KAFKA-854: --- The problem is I'm an SBT neophyte, and I keep thinking I could do all this easier with Maven -- aside from the cross-version stuff SBT provides. I don't know what is used and what is cruft, what people care about keeping and what can be trimmed. > Upgrade dependencies for 0.8 > > > Key: KAFKA-854 > URL: https://issues.apache.org/jira/browse/KAFKA-854 > Project: Kafka > Issue Type: Improvement >Affects Versions: 0.8 >Reporter: Scott Carey > Attachments: 0001-KAFKA-854-Upgrade-Deps-to-latest.patch > > > Many of the dependencies that Kafka 0.8 uses are old. It would be a good > idea to upgrade all of these where possible. > log4j is set to 1.2.15, but the latest is 1.2.17 > zookeeper is at 3.3.4, , there is 3.3.6 (August 2012), and 3.4.5 (Nov 2012. > 3.4.x includes several major enhancements and fixes over the 3.3.x line. > org.slf4j is at 1.6.4, there is 1.6.6 (June 2012) and 1.7.5 (March 2013) > net.sf.jopt-simple is at 3.2, there is 3.3 (May 2011) and 4.4 (Jan 2013) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (KAFKA-826) Make Kafka 0.8 depend on metrics 2.2.0 instead of 3.x
[ https://issues.apache.org/jira/browse/KAFKA-826?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dragos Manolescu updated KAFKA-826: --- Attachment: kafka-fix-for-826-complete.patch Hmm, it looks like for some reason the metrics JARs didn't end up on your CLASSPATH. Here's the same patch with one additional update that packages the required libraries into a single jar, kafka-assembly-0.8-SNAPSHOT-deps.jar. This change removes ALL references to ~/.ivy2/cache from the script and requires one additional step: execute the "assembly-package-dependency" sbt task (provided by the sbt-assembly plugin that was already included). Here are the steps: rm -rf ~/.ivy2/cache/ find . -name target | xargs rm -rf bash sbt update bash sbt package bash sbt assembly-package-dependency # NEW STEP bin/zookeeper-server-start.sh config/zookeeper.properties bin/kafka-server-start.sh config/server.properties I verified and the server starts up fine. Please LMK if you run into problems. Here's the output from assembly-package-dependency: % bash sbt assembly-package-dependency [info] Loading global plugins from /Users/dragos.manolescu/.sbt/plugins [info] Loading project definition from /Users/dragos.manolescu/Repos/kafka/project [info] Set current project to Kafka (in build file:/Users/dragos.manolescu/Repos/kafka/) [info] Including metrics-core-2.2.0.jar [info] Including scala-compiler.jar [info] Including zkclient-0.2.jar [info] Including metrics-annotation-2.2.0.jar [info] Including snappy-java-1.0.4.1.jar [info] Including log4j-1.2.15.jar [info] Including slf4j-api-1.7.2.jar [info] Including zookeeper-3.3.4.jar [info] Including jopt-simple-3.2.jar [info] Including slf4j-simple-1.6.4.jar [info] Including scala-library.jar [warn] Merging 'META-INF/NOTICE' with strategy 'rename' [warn] Merging 'org/xerial/snappy/native/README' with strategy 'rename' [warn] Merging 'META-INF/maven/org.xerial.snappy/snappy-java/LICENSE' with strategy 'rename' [warn] Merging 'LICENSE.txt' with strategy 'rename' [warn] Merging 'META-INF/LICENSE' with strategy 'rename' [warn] Merging 'META-INF/MANIFEST.MF' with strategy 'discard' [warn] Strategy 'discard' was applied to a file [warn] Strategy 'rename' was applied to 5 files [info] Packaging /Users/dragos.manolescu/Repos/kafka/core/target/scala-2.8.0/kafka-assembly-0.8-SNAPSHOT-deps.jar ... [info] Done packaging. [success] Total time: 44 s, completed Apr 8, 2013 10:58:33 AM > Make Kafka 0.8 depend on metrics 2.2.0 instead of 3.x > - > > Key: KAFKA-826 > URL: https://issues.apache.org/jira/browse/KAFKA-826 > Project: Kafka > Issue Type: Bug > Components: core >Affects Versions: 0.8 >Reporter: Neha Narkhede >Assignee: Dragos Manolescu >Priority: Blocker > Labels: build, kafka-0.8, metrics > Attachments: kafka-fix-for-826-complete.patch, > kafka-fix-for-826.patch, kafka-fix-for-826-take2.patch > > > In order to mavenize Kafka 0.8, we have to depend on metrics 2.2.0 since > metrics 3.x is a huge change as well as not an officially supported release. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (KAFKA-854) Upgrade dependencies for 0.8
[
https://issues.apache.org/jira/browse/KAFKA-854?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13625652#comment-13625652
]
Scott Carey commented on KAFKA-854:
---
{quote}
While you're at it you could also upgrade Jackson. Currently the build file
references org.codehaus.jackson which is old. The upgrade may require some code
changes (the API differs in a few places), all trivial through.
{quote}
Where is the Jackson dependency specified?
find . -name *.sbt | xargs grep jackson
returns nothing, and in project/Build.scala it is only used in relation to the
hadoop settings. We could upgrade to the latest 1.x but not 2.x, because other
hadoop dependencies use it (like the also ancient avro 1.4).
What is the status of the contrib hadoop stuff?
Matt's patch already upgrades everything, if we want to upgrade only the core
Kafka dependencies, we can trim it down to those simpler, safer updates.
(As an SBT newbie) --
It appears that project/Build.scala is completely unnecessary, and could be
significantly simplified if moved to core/build.sbt. Most of the content of
Build.scala seems to be cruft or have simpler features available to do the same
thing.
> Upgrade dependencies for 0.8
>
>
> Key: KAFKA-854
> URL: https://issues.apache.org/jira/browse/KAFKA-854
> Project: Kafka
> Issue Type: Improvement
>Affects Versions: 0.8
>Reporter: Scott Carey
> Attachments: 0001-KAFKA-854-Upgrade-Deps-to-latest.patch
>
>
> Many of the dependencies that Kafka 0.8 uses are old. It would be a good
> idea to upgrade all of these where possible.
> log4j is set to 1.2.15, but the latest is 1.2.17
> zookeeper is at 3.3.4, , there is 3.3.6 (August 2012), and 3.4.5 (Nov 2012.
> 3.4.x includes several major enhancements and fixes over the 3.3.x line.
> org.slf4j is at 1.6.4, there is 1.6.6 (June 2012) and 1.7.5 (March 2013)
> net.sf.jopt-simple is at 3.2, there is 3.3 (May 2011) and 4.4 (Jan 2013)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Subscription: outstanding kafka patches
Issue Subscription Filter: outstanding kafka patches (59 issues) The list of outstanding kafka patches Subscriber: kafka-mailing-list Key Summary KAFKA-856 Correlation id for OffsetFetch request (#2) always responds with 0 https://issues.apache.org/jira/browse/KAFKA-856 KAFKA-855 Ant+Ivy build for Kafka https://issues.apache.org/jira/browse/KAFKA-855 KAFKA-854 Upgrade dependencies for 0.8 https://issues.apache.org/jira/browse/KAFKA-854 KAFKA-852 Remove clientId from OffsetFetchResponse and OffsetCommitResponse https://issues.apache.org/jira/browse/KAFKA-852 KAFKA-826 Make Kafka 0.8 depend on metrics 2.2.0 instead of 3.x https://issues.apache.org/jira/browse/KAFKA-826 KAFKA-823 merge 0.8 (51421fcc0111031bb77f779a6f6c00520d526a34) to trunk https://issues.apache.org/jira/browse/KAFKA-823 KAFKA-815 Improve SimpleConsumerShell to take in a max messages config option https://issues.apache.org/jira/browse/KAFKA-815 KAFKA-808 Migration tool internal queue between consumer and producer threads should be configurable https://issues.apache.org/jira/browse/KAFKA-808 KAFKA-745 Remove getShutdownReceive() and other kafka specific code from the RequestChannel https://issues.apache.org/jira/browse/KAFKA-745 KAFKA-739 Handle null values in Message payload https://issues.apache.org/jira/browse/KAFKA-739 KAFKA-735 Add looping and JSON output for ConsumerOffsetChecker https://issues.apache.org/jira/browse/KAFKA-735 KAFKA-733 Fat jar option for build, or override for ivy cache location https://issues.apache.org/jira/browse/KAFKA-733 KAFKA-717 scala 2.10 build support https://issues.apache.org/jira/browse/KAFKA-717 KAFKA-705 Controlled shutdown doesn't seem to work on more than one broker in a cluster https://issues.apache.org/jira/browse/KAFKA-705 KAFKA-686 0.8 Kafka broker should give a better error message when running against 0.7 zookeeper https://issues.apache.org/jira/browse/KAFKA-686 KAFKA-682 java.lang.OutOfMemoryError: Java heap space https://issues.apache.org/jira/browse/KAFKA-682 KAFKA-677 Retention process gives exception if an empty segment is chosen for collection https://issues.apache.org/jira/browse/KAFKA-677 KAFKA-674 Clean Shutdown Testing - Log segments checksums mismatch https://issues.apache.org/jira/browse/KAFKA-674 KAFKA-652 Create testcases for clean shut-down https://issues.apache.org/jira/browse/KAFKA-652 KAFKA-645 Create a shell script to run System Test with DEBUG details and "tee" console output to a file https://issues.apache.org/jira/browse/KAFKA-645 KAFKA-637 Separate log4j environment variable from KAFKA_OPTS in kafka-run-class.sh https://issues.apache.org/jira/browse/KAFKA-637 KAFKA-621 System Test 9051 : ConsoleConsumer doesn't receives any data for 20 topics but works for 10 https://issues.apache.org/jira/browse/KAFKA-621 KAFKA-607 System Test Transient Failure (case 4011 Log Retention) - ConsoleConsumer receives less data https://issues.apache.org/jira/browse/KAFKA-607 KAFKA-606 System Test Transient Failure (case 0302 GC Pause) - Log segments mismatched across replicas https://issues.apache.org/jira/browse/KAFKA-606 KAFKA-598 decouple fetch size from max message size https://issues.apache.org/jira/browse/KAFKA-598 KAFKA-583 SimpleConsumerShell may receive less data inconsistently https://issues.apache.org/jira/browse/KAFKA-583 KAFKA-552 No error messages logged for those failing-to-send messages from Producer https://issues.apache.org/jira/browse/KAFKA-552 KAFKA-547 The ConsumerStats MBean name should include the groupid https://issues.apache.org/jira/browse/KAFKA-547 KAFKA-530 kafka.server.KafkaApis: kafka.common.OffsetOutOfRangeException https://issues.apache.org/jira/browse/KAFKA-530 KAFKA-493 High CPU usage on inactive server https://issues.apache.org/jira/browse/KAFKA-493 KAFKA-479 ZK EPoll taking 100% CPU usage with Kafka Client https://issues.apache.org/jira/browse/KAFKA-479 KAFKA-465 Performance test scripts - refactoring leftovers from tools to perf package https://issues.apache.org/jira/browse/KAFKA-465 KAFKA-438 Code cleanup in MessageTest https://issues.apache.org/jira/browse/KAFKA-438 KAFKA-414 Evaluate mmap-based writes for Log implementation https://issues.apache.org/jira/browse/KAFKA-414 KAFKA-411 Message Error in high cocurrent environment https://issues.apache.org/jira/browse/KAFKA-411 KAFKA-404 When using chroot path, create chroot on startup if it doesn't exist https://issues.apache.org/jira/browse/KAFKA-404 KAFKA-399 0.7.1 seems to show less pe
Re: Thoughts on using Ant+Ivy for the build?
My first reaction to looking at the current build was that I could do a lot better with Maven and I'd get a lot of free stuff with that (dependency:tree, versions:display-dependency-updates), except that cross scala versions would be uglier. Modularization is trivial with maven (the stuff outside core). A little more digging, and it seems that the current sbt build could be significantly simplified and has a lot of cruft. On 4/5/13 12:55 PM, "David Arthur" wrote: >After getting frustrated with SBT, and being unable to figure out >seemingly simple problems like KAFKA-843, I decided to try something >completely different. > >I spent some time yesterday adapting some Ant/Ivy boilerplate I use for >projects to Kafka. It was actually pretty easy to get working (Kafka is >a very simple build), and I think it's _much_ cleaner than the existing >SBT stuff. > >Attached is a patchset of the work I did. N.B., this is totally >experimental and only considers the "core" part of the project. > >At this point I can: > >* Resolve all deps through Ivy (except yammer.metrics which is checked in) >* Resolve different versions of Scala through Ivy (i.e., cross >compile-ability) >* Compile the source >* Run all the unit tests (all passing) >* Compile a jar >* Package a tarball of the libs, conf, and bin scripts > >Since all the libraries are localized to the project (and not in >~/.ivy2), the packaging is trivial. Also, the bin scripts could be >cleaned up significantly (which they need to be IMO). > >I would love to hear what everyone thinks of this. Am I crazy? Is SBT >really better? Convince me! > >-David > > >
Re: Thoughts on using Ant+Ivy for the build?
Ant and Ivy also bring in lots of free stuff as well as the flexibility to do non-standard things It seems that SBT is a moving target (as is Scala itself, being such a new tech), and keeping up with SBT changes and people's plugins hosted on GitHub sounds like a pain. Maven and Ant/Ivy are mature and stable and well understood. Without digressing too much into a Maven/Ant+Ivy/SBT debate, it is clear that there are issues with the current build. I'm proposing Ant+Ivy, if enough people buy into this - great we'll use Ant. If not, also great - I'm simply forcing the dialog :) Also, if what I'm proposing is accepted, I would be willing to maintain this piece of the project in the longer term. -David On 4/8/13 3:40 PM, Scott Carey wrote: My first reaction to looking at the current build was that I could do a lot better with Maven and I'd get a lot of free stuff with that (dependency:tree, versions:display-dependency-updates), except that cross scala versions would be uglier. Modularization is trivial with maven (the stuff outside core). A little more digging, and it seems that the current sbt build could be significantly simplified and has a lot of cruft. On 4/5/13 12:55 PM, "David Arthur" wrote: After getting frustrated with SBT, and being unable to figure out seemingly simple problems like KAFKA-843, I decided to try something completely different. I spent some time yesterday adapting some Ant/Ivy boilerplate I use for projects to Kafka. It was actually pretty easy to get working (Kafka is a very simple build), and I think it's _much_ cleaner than the existing SBT stuff. Attached is a patchset of the work I did. N.B., this is totally experimental and only considers the "core" part of the project. At this point I can: * Resolve all deps through Ivy (except yammer.metrics which is checked in) * Resolve different versions of Scala through Ivy (i.e., cross compile-ability) * Compile the source * Run all the unit tests (all passing) * Compile a jar * Package a tarball of the libs, conf, and bin scripts Since all the libraries are localized to the project (and not in ~/.ivy2), the packaging is trivial. Also, the bin scripts could be cleaned up significantly (which they need to be IMO). I would love to hear what everyone thinks of this. Am I crazy? Is SBT really better? Convince me! -David
Re: Thoughts on using Ant+Ivy for the build?
I think the biggest problem is that there isn't clear ownership of the build stuff we have. SBT is probably fine if you know it well. Ant and maven are probably the same. The problem is that none of the committers seem to know SBT well that that is a big barrier for getting basic stuff like a good unit test report, etc. My personal preference would be for Maven followed by Ant followed by anything else (including SBT), but I think the ownership maintenance issue is more important than the framework. The two useful things that SBT bought us where compile targets (i.e. scala 2.8 vs 2.9) and incremental compilation. I think both are available outside sbt now...? -Jay On Mon, Apr 8, 2013 at 12:52 PM, David Arthur wrote: > Ant and Ivy also bring in lots of free stuff as well as the flexibility to > do non-standard things > > It seems that SBT is a moving target (as is Scala itself, being such a new > tech), and keeping up with SBT changes and people's plugins hosted on GitHub > sounds like a pain. Maven and Ant/Ivy are mature and stable and well > understood. > > Without digressing too much into a Maven/Ant+Ivy/SBT debate, it is clear > that there are issues with the current build. I'm proposing Ant+Ivy, if > enough people buy into this - great we'll use Ant. If not, also great - I'm > simply forcing the dialog :) > > Also, if what I'm proposing is accepted, I would be willing to maintain this > piece of the project in the longer term. > > -David > > > On 4/8/13 3:40 PM, Scott Carey wrote: >> >> My first reaction to looking at the current build was that I could do a >> lot better with Maven and I'd get a lot of free stuff with that >> (dependency:tree, versions:display-dependency-updates), except that cross >> scala versions would be uglier. Modularization is trivial with maven (the >> stuff outside core). >> >> A little more digging, and it seems that the current sbt build could be >> significantly simplified and has a lot of cruft. >> >> >> On 4/5/13 12:55 PM, "David Arthur" wrote: >> >>> After getting frustrated with SBT, and being unable to figure out >>> seemingly simple problems like KAFKA-843, I decided to try something >>> completely different. >>> >>> I spent some time yesterday adapting some Ant/Ivy boilerplate I use for >>> projects to Kafka. It was actually pretty easy to get working (Kafka is >>> a very simple build), and I think it's _much_ cleaner than the existing >>> SBT stuff. >>> >>> Attached is a patchset of the work I did. N.B., this is totally >>> experimental and only considers the "core" part of the project. >>> >>> At this point I can: >>> >>> * Resolve all deps through Ivy (except yammer.metrics which is checked >>> in) >>> * Resolve different versions of Scala through Ivy (i.e., cross >>> compile-ability) >>> * Compile the source >>> * Run all the unit tests (all passing) >>> * Compile a jar >>> * Package a tarball of the libs, conf, and bin scripts >>> >>> Since all the libraries are localized to the project (and not in >>> ~/.ivy2), the packaging is trivial. Also, the bin scripts could be >>> cleaned up significantly (which they need to be IMO). >>> >>> I would love to hear what everyone thinks of this. Am I crazy? Is SBT >>> really better? Convince me! >>> >>> -David >>> >>> >>> >
Re: Thoughts on using Ant+Ivy for the build?
Targeting multiple Scala versions is easy with Ant/Ivy and I believe is possible in Maven. Incremental compilation is available through some IDEs: Eclipse has a plugin from Typesafe that does incremental build: http://typesafe.com/technology/scala-ide Intellij supports it by using SBT: http://blog.jetbrains.com/scala/2012/12/28/a-new-way-to-compile/ Lowly vim users like myself would have to suffer longer compile times -David On 4/8/13 4:20 PM, Jay Kreps wrote: I think the biggest problem is that there isn't clear ownership of the build stuff we have. SBT is probably fine if you know it well. Ant and maven are probably the same. The problem is that none of the committers seem to know SBT well that that is a big barrier for getting basic stuff like a good unit test report, etc. My personal preference would be for Maven followed by Ant followed by anything else (including SBT), but I think the ownership maintenance issue is more important than the framework. The two useful things that SBT bought us where compile targets (i.e. scala 2.8 vs 2.9) and incremental compilation. I think both are available outside sbt now...? -Jay On Mon, Apr 8, 2013 at 12:52 PM, David Arthur wrote: Ant and Ivy also bring in lots of free stuff as well as the flexibility to do non-standard things It seems that SBT is a moving target (as is Scala itself, being such a new tech), and keeping up with SBT changes and people's plugins hosted on GitHub sounds like a pain. Maven and Ant/Ivy are mature and stable and well understood. Without digressing too much into a Maven/Ant+Ivy/SBT debate, it is clear that there are issues with the current build. I'm proposing Ant+Ivy, if enough people buy into this - great we'll use Ant. If not, also great - I'm simply forcing the dialog :) Also, if what I'm proposing is accepted, I would be willing to maintain this piece of the project in the longer term. -David On 4/8/13 3:40 PM, Scott Carey wrote: My first reaction to looking at the current build was that I could do a lot better with Maven and I'd get a lot of free stuff with that (dependency:tree, versions:display-dependency-updates), except that cross scala versions would be uglier. Modularization is trivial with maven (the stuff outside core). A little more digging, and it seems that the current sbt build could be significantly simplified and has a lot of cruft. On 4/5/13 12:55 PM, "David Arthur" wrote: After getting frustrated with SBT, and being unable to figure out seemingly simple problems like KAFKA-843, I decided to try something completely different. I spent some time yesterday adapting some Ant/Ivy boilerplate I use for projects to Kafka. It was actually pretty easy to get working (Kafka is a very simple build), and I think it's _much_ cleaner than the existing SBT stuff. Attached is a patchset of the work I did. N.B., this is totally experimental and only considers the "core" part of the project. At this point I can: * Resolve all deps through Ivy (except yammer.metrics which is checked in) * Resolve different versions of Scala through Ivy (i.e., cross compile-ability) * Compile the source * Run all the unit tests (all passing) * Compile a jar * Package a tarball of the libs, conf, and bin scripts Since all the libraries are localized to the project (and not in ~/.ivy2), the packaging is trivial. Also, the bin scripts could be cleaned up significantly (which they need to be IMO). I would love to hear what everyone thinks of this. Am I crazy? Is SBT really better? Convince me! -David
Re: Kafka and Scala versions
We are on 2.10 already. String interpolation (2.10), Value classes (2.10), parallel collections (2.9), are too good to give up. Akka actors, futures, and promises being part of the distribution are nice as well. The compiler is faster, and emits a lot of warnings in Kafka code that could be bugs in Kafka. Upgrading to 2.8.2 significantly simplifies the ability to cross-compile with 2.10 (only one small file needs to have a version that differs, instead of ~1500 lines). See my recent patches in KAFKA-717. On 4/7/13 10:17 PM, "Jun Rao" wrote: >Scott, > >Yes, we are still using scala 2.8.0 at LinkedIn. We do plan to upgrade to >a >later version of scala at some point. However, this may take some time >since we will need to upgrade other scala projects at LinkedIn at the same >time. > >It's unfortunate that code change is needed in order to support scala >2.10. >What are the things that you are looking for in scala 2.10? > >Thanks, > >Jun > > >On Fri, Apr 5, 2013 at 1:10 PM, Scott Carey >wrote: > >> Kafka 0.8 and trunk by default have SBT set up to cross-compile 2.8.0, >> 2.8.2, 2.9.1 and 2.9.2, and it defaults to 2.8.0. >> >> I assume that one of the major contributors requires 2.8.0. Any >> approximation on how long will that last? >> Dropping 2.8.0 would be beneficial because some uses of Java Conversions >> is replaced with Java Converters (in 2.10 +), and Java Converters was >>not >> added until 2.8.1. >> >>http://stackoverflow.com/questions/8301947/what-is-the-difference-between >>-j >> avaconverters-and-javaconversions-in-scala >> >> https://issues.apache.org/jira/browse/KAFKA-717 is attempting to add >>build >> support for Scala 2.10. The required changes to support 2.10 use Java >> Converters, which is not available in 2.8.0. >> >> The other item required to support Scala 2.10 is to move from >> scala.StaticAnnotation to scala.annotation.StaticAnnotation. >> Scala 2.10.x has only scala.annotation.StaticAnnotation. >> Scala 2.9.x has both scala.StaticAnnotation (deprecated) and >> scala.annotation.StaticAnnotation. >> Scala 2.8.x has only scala.StaticAnnotation >> >> Dropping support for 2.8.x entirely and supporting only 2.9.x and 2.10.x >> would make 2.10.x support easy. >> If 2.8.2 is required (but not 2.8.0), then some sort of conditional >> compile or two flavors of the source code where >> scala.annotation.StaticAnnotation is used will be required. Use of >> StaticAnnotation is very limited. >> If 2.8.0 must be kept, then two versions of all code that uses Java >> Conversions APIs will also need to be maintained. >> >> I can't find any discussions about 2.8.0 usage requirements. I also >>don't >> know enough about sbt to configure cross compilation to use different >> source files for different cases, which would solve the problems above >>but >> add some maintenance burden. >> >> >>
Re: Thoughts on using Ant+Ivy for the build?
Gradle is also gaining a lot of popularity, and has a good deal of Scala support. I don't know if it has incremental compilation. The maven scala plugin at one time had incremental compilation, I don't know what state that is in now. I'm giving learning enough SBT a shot right now to clean the current mess up, and will submit a patch as part of Kafka-854. Having KAFKA-826 committed would help. On 4/8/13 1:31 PM, "David Arthur" wrote: >Targeting multiple Scala versions is easy with Ant/Ivy and I believe is >possible in Maven. Incremental compilation is available through some IDEs: > >Eclipse has a plugin from Typesafe that does incremental build: >http://typesafe.com/technology/scala-ide >Intellij supports it by using SBT: >http://blog.jetbrains.com/scala/2012/12/28/a-new-way-to-compile/ > >Lowly vim users like myself would have to suffer longer compile times > >-David > >On 4/8/13 4:20 PM, Jay Kreps wrote: >> I think the biggest problem is that there isn't clear ownership of the >> build stuff we have. SBT is probably fine if you know it well. Ant and >> maven are probably the same. The problem is that none of the >> committers seem to know SBT well that that is a big barrier for >> getting basic stuff like a good unit test report, etc. >> >> My personal preference would be for Maven followed by Ant followed by >> anything else (including SBT), but I think the ownership maintenance >> issue is more important than the framework. >> >> The two useful things that SBT bought us where compile targets (i.e. >> scala 2.8 vs 2.9) and incremental compilation. I think both are >> available outside sbt now...? >> >> -Jay >> >> >> On Mon, Apr 8, 2013 at 12:52 PM, David Arthur wrote: >>> Ant and Ivy also bring in lots of free stuff as well as the >>>flexibility to >>> do non-standard things >>> >>> It seems that SBT is a moving target (as is Scala itself, being such a >>>new >>> tech), and keeping up with SBT changes and people's plugins hosted on >>>GitHub >>> sounds like a pain. Maven and Ant/Ivy are mature and stable and well >>> understood. >>> >>> Without digressing too much into a Maven/Ant+Ivy/SBT debate, it is >>>clear >>> that there are issues with the current build. I'm proposing Ant+Ivy, if >>> enough people buy into this - great we'll use Ant. If not, also great >>>- I'm >>> simply forcing the dialog :) >>> >>> Also, if what I'm proposing is accepted, I would be willing to >>>maintain this >>> piece of the project in the longer term. >>> >>> -David >>> >>> >>> On 4/8/13 3:40 PM, Scott Carey wrote: My first reaction to looking at the current build was that I could do a lot better with Maven and I'd get a lot of free stuff with that (dependency:tree, versions:display-dependency-updates), except that cross scala versions would be uglier. Modularization is trivial with maven (the stuff outside core). A little more digging, and it seems that the current sbt build could be significantly simplified and has a lot of cruft. On 4/5/13 12:55 PM, "David Arthur" wrote: > After getting frustrated with SBT, and being unable to figure out > seemingly simple problems like KAFKA-843, I decided to try something > completely different. > > I spent some time yesterday adapting some Ant/Ivy boilerplate I use >for > projects to Kafka. It was actually pretty easy to get working (Kafka >is > a very simple build), and I think it's _much_ cleaner than the >existing > SBT stuff. > > Attached is a patchset of the work I did. N.B., this is totally > experimental and only considers the "core" part of the project. > > At this point I can: > > * Resolve all deps through Ivy (except yammer.metrics which is >checked > in) > * Resolve different versions of Scala through Ivy (i.e., cross > compile-ability) > * Compile the source > * Run all the unit tests (all passing) > * Compile a jar > * Package a tarball of the libs, conf, and bin scripts > > Since all the libraries are localized to the project (and not in > ~/.ivy2), the packaging is trivial. Also, the bin scripts could be > cleaned up significantly (which they need to be IMO). > > I would love to hear what everyone thinks of this. Am I crazy? Is SBT > really better? Convince me! > > -David > > > >
Re: Thoughts on using Ant+Ivy for the build?
Let me google that for myself: Gradle has incremental scala compilation: http://www.gradle.org/docs/current/userguide/scala_plugin.html Maven does too: http://davidb.github.io/scala-maven-plugin/example_incremental.html So I suspect it could be solved with Ant. Maven and Gradle both support calling ant tasks from within them, if necessary. In general, any tool will work if maintained and used properly, and any tool can be abused and poorly organized for maintenance. On 4/8/13 2:09 PM, "Scott Carey" wrote: >Gradle is also gaining a lot of popularity, and has a good deal of Scala >support. I don't know if it has incremental compilation. The maven scala >plugin at one time had incremental compilation, I don't know what state >that is in now. > >I'm giving learning enough SBT a shot right now to clean the current mess >up, and will submit a patch as part of Kafka-854. Having KAFKA-826 >committed would help. > >On 4/8/13 1:31 PM, "David Arthur" wrote: > >>Targeting multiple Scala versions is easy with Ant/Ivy and I believe is >>possible in Maven. Incremental compilation is available through some >>IDEs: >> >>Eclipse has a plugin from Typesafe that does incremental build: >>http://typesafe.com/technology/scala-ide >>Intellij supports it by using SBT: >>http://blog.jetbrains.com/scala/2012/12/28/a-new-way-to-compile/ >> >>Lowly vim users like myself would have to suffer longer compile times >> >>-David >> >>On 4/8/13 4:20 PM, Jay Kreps wrote: >>> I think the biggest problem is that there isn't clear ownership of the >>> build stuff we have. SBT is probably fine if you know it well. Ant and >>> maven are probably the same. The problem is that none of the >>> committers seem to know SBT well that that is a big barrier for >>> getting basic stuff like a good unit test report, etc. >>> >>> My personal preference would be for Maven followed by Ant followed by >>> anything else (including SBT), but I think the ownership maintenance >>> issue is more important than the framework. >>> >>> The two useful things that SBT bought us where compile targets (i.e. >>> scala 2.8 vs 2.9) and incremental compilation. I think both are >>> available outside sbt now...? >>> >>> -Jay >>> >>> >>> On Mon, Apr 8, 2013 at 12:52 PM, David Arthur wrote: Ant and Ivy also bring in lots of free stuff as well as the flexibility to do non-standard things It seems that SBT is a moving target (as is Scala itself, being such a new tech), and keeping up with SBT changes and people's plugins hosted on GitHub sounds like a pain. Maven and Ant/Ivy are mature and stable and well understood. Without digressing too much into a Maven/Ant+Ivy/SBT debate, it is clear that there are issues with the current build. I'm proposing Ant+Ivy, if enough people buy into this - great we'll use Ant. If not, also great - I'm simply forcing the dialog :) Also, if what I'm proposing is accepted, I would be willing to maintain this piece of the project in the longer term. -David On 4/8/13 3:40 PM, Scott Carey wrote: > My first reaction to looking at the current build was that I could do >a > lot better with Maven and I'd get a lot of free stuff with that > (dependency:tree, versions:display-dependency-updates), except that >cross > scala versions would be uglier. Modularization is trivial with maven >(the > stuff outside core). > > A little more digging, and it seems that the current sbt build could >be > significantly simplified and has a lot of cruft. > > > On 4/5/13 12:55 PM, "David Arthur" wrote: > >> After getting frustrated with SBT, and being unable to figure out >> seemingly simple problems like KAFKA-843, I decided to try something >> completely different. >> >> I spent some time yesterday adapting some Ant/Ivy boilerplate I use >>for >> projects to Kafka. It was actually pretty easy to get working (Kafka >>is >> a very simple build), and I think it's _much_ cleaner than the >>existing >> SBT stuff. >> >> Attached is a patchset of the work I did. N.B., this is totally >> experimental and only considers the "core" part of the project. >> >> At this point I can: >> >> * Resolve all deps through Ivy (except yammer.metrics which is >>checked >> in) >> * Resolve different versions of Scala through Ivy (i.e., cross >> compile-ability) >> * Compile the source >> * Run all the unit tests (all passing) >> * Compile a jar >> * Package a tarball of the libs, conf, and bin scripts >> >> Since all the libraries are localized to the project (and not in >> ~/.ivy2), the packaging is trivial. Also, the bin scripts could be >> cleaned up significantly (which they need to be IMO). >> >> I would love to hear what everyone thinks of this. Am I crazy? Is >>>
Re: Thoughts on using Ant+Ivy for the build?
Yes Ant supports it too since scalac supports it. Most likely Maven and Gradle are using scalac or SBT under the hood (unless they have written their own compilers - unlikely). However, from what I've read when researching the Ant build, these scalac options (-make:transitive -dependencyfile) are going away. On 4/8/13 5:15 PM, Scott Carey wrote: Let me google that for myself: Gradle has incremental scala compilation: http://www.gradle.org/docs/current/userguide/scala_plugin.html Maven does too: http://davidb.github.io/scala-maven-plugin/example_incremental.html So I suspect it could be solved with Ant. Maven and Gradle both support calling ant tasks from within them, if necessary. In general, any tool will work if maintained and used properly, and any tool can be abused and poorly organized for maintenance. On 4/8/13 2:09 PM, "Scott Carey" wrote: Gradle is also gaining a lot of popularity, and has a good deal of Scala support. I don't know if it has incremental compilation. The maven scala plugin at one time had incremental compilation, I don't know what state that is in now. I'm giving learning enough SBT a shot right now to clean the current mess up, and will submit a patch as part of Kafka-854. Having KAFKA-826 committed would help. On 4/8/13 1:31 PM, "David Arthur" wrote: Targeting multiple Scala versions is easy with Ant/Ivy and I believe is possible in Maven. Incremental compilation is available through some IDEs: Eclipse has a plugin from Typesafe that does incremental build: http://typesafe.com/technology/scala-ide Intellij supports it by using SBT: http://blog.jetbrains.com/scala/2012/12/28/a-new-way-to-compile/ Lowly vim users like myself would have to suffer longer compile times -David On 4/8/13 4:20 PM, Jay Kreps wrote: I think the biggest problem is that there isn't clear ownership of the build stuff we have. SBT is probably fine if you know it well. Ant and maven are probably the same. The problem is that none of the committers seem to know SBT well that that is a big barrier for getting basic stuff like a good unit test report, etc. My personal preference would be for Maven followed by Ant followed by anything else (including SBT), but I think the ownership maintenance issue is more important than the framework. The two useful things that SBT bought us where compile targets (i.e. scala 2.8 vs 2.9) and incremental compilation. I think both are available outside sbt now...? -Jay On Mon, Apr 8, 2013 at 12:52 PM, David Arthur wrote: Ant and Ivy also bring in lots of free stuff as well as the flexibility to do non-standard things It seems that SBT is a moving target (as is Scala itself, being such a new tech), and keeping up with SBT changes and people's plugins hosted on GitHub sounds like a pain. Maven and Ant/Ivy are mature and stable and well understood. Without digressing too much into a Maven/Ant+Ivy/SBT debate, it is clear that there are issues with the current build. I'm proposing Ant+Ivy, if enough people buy into this - great we'll use Ant. If not, also great - I'm simply forcing the dialog :) Also, if what I'm proposing is accepted, I would be willing to maintain this piece of the project in the longer term. -David On 4/8/13 3:40 PM, Scott Carey wrote: My first reaction to looking at the current build was that I could do a lot better with Maven and I'd get a lot of free stuff with that (dependency:tree, versions:display-dependency-updates), except that cross scala versions would be uglier. Modularization is trivial with maven (the stuff outside core). A little more digging, and it seems that the current sbt build could be significantly simplified and has a lot of cruft. On 4/5/13 12:55 PM, "David Arthur" wrote: After getting frustrated with SBT, and being unable to figure out seemingly simple problems like KAFKA-843, I decided to try something completely different. I spent some time yesterday adapting some Ant/Ivy boilerplate I use for projects to Kafka. It was actually pretty easy to get working (Kafka is a very simple build), and I think it's _much_ cleaner than the existing SBT stuff. Attached is a patchset of the work I did. N.B., this is totally experimental and only considers the "core" part of the project. At this point I can: * Resolve all deps through Ivy (except yammer.metrics which is checked in) * Resolve different versions of Scala through Ivy (i.e., cross compile-ability) * Compile the source * Run all the unit tests (all passing) * Compile a jar * Package a tarball of the libs, conf, and bin scripts Since all the libraries are localized to the project (and not in ~/.ivy2), the packaging is trivial. Also, the bin scripts could be cleaned up significantly (which they need to be IMO). I would love to hear what everyone thinks of this. Am I crazy? Is SBT really better? Convince me! -David
[jira] [Commented] (KAFKA-826) Make Kafka 0.8 depend on metrics 2.2.0 instead of 3.x
[ https://issues.apache.org/jira/browse/KAFKA-826?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13626135#comment-13626135 ] Scott Carey commented on KAFKA-826: --- Can a committer look at this? This would be great to get in the 0.8 branch soon and looks good to me. I have a cleanup of sbt to significantly simplify it further that I'd like to do as part of KAFKA-854 but it will fail to merge with this change. > Make Kafka 0.8 depend on metrics 2.2.0 instead of 3.x > - > > Key: KAFKA-826 > URL: https://issues.apache.org/jira/browse/KAFKA-826 > Project: Kafka > Issue Type: Bug > Components: core >Affects Versions: 0.8 >Reporter: Neha Narkhede >Assignee: Dragos Manolescu >Priority: Blocker > Labels: build, kafka-0.8, metrics > Attachments: kafka-fix-for-826-complete.patch, > kafka-fix-for-826.patch, kafka-fix-for-826-take2.patch > > > In order to mavenize Kafka 0.8, we have to depend on metrics 2.2.0 since > metrics 3.x is a huge change as well as not an officially supported release. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (KAFKA-854) Upgrade dependencies for 0.8
[ https://issues.apache.org/jira/browse/KAFKA-854?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Scott Carey updated KAFKA-854: -- Attachment: KAFKA-854.patch Cleans up dependencies and SBT build in general (~350 lines fewer than prior). > Upgrade dependencies for 0.8 > > > Key: KAFKA-854 > URL: https://issues.apache.org/jira/browse/KAFKA-854 > Project: Kafka > Issue Type: Improvement >Affects Versions: 0.8 >Reporter: Scott Carey > Attachments: 0001-KAFKA-854-Upgrade-Deps-to-latest.patch, > KAFKA-854.patch > > > Many of the dependencies that Kafka 0.8 uses are old. It would be a good > idea to upgrade all of these where possible. > log4j is set to 1.2.15, but the latest is 1.2.17 > zookeeper is at 3.3.4, , there is 3.3.6 (August 2012), and 3.4.5 (Nov 2012. > 3.4.x includes several major enhancements and fixes over the 3.3.x line. > org.slf4j is at 1.6.4, there is 1.6.6 (June 2012) and 1.7.5 (March 2013) > net.sf.jopt-simple is at 3.2, there is 3.3 (May 2011) and 4.4 (Jan 2013) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (KAFKA-826) Make Kafka 0.8 depend on metrics 2.2.0 instead of 3.x
[ https://issues.apache.org/jira/browse/KAFKA-826?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13626304#comment-13626304 ] Matt Christiansen commented on KAFKA-826: - I made my own branch and applyed this patch, in all my testing it seems like its doing great. I would also love to see it committed. > Make Kafka 0.8 depend on metrics 2.2.0 instead of 3.x > - > > Key: KAFKA-826 > URL: https://issues.apache.org/jira/browse/KAFKA-826 > Project: Kafka > Issue Type: Bug > Components: core >Affects Versions: 0.8 >Reporter: Neha Narkhede >Assignee: Dragos Manolescu >Priority: Blocker > Labels: build, kafka-0.8, metrics > Attachments: kafka-fix-for-826-complete.patch, > kafka-fix-for-826.patch, kafka-fix-for-826-take2.patch > > > In order to mavenize Kafka 0.8, we have to depend on metrics 2.2.0 since > metrics 3.x is a huge change as well as not an officially supported release. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
