Re: Kafka setup in eclipse
i figured out a way to setup the kafka project in eclipse, works for the time being. once the sbt update command is run, and the jars downloaded i had to manually place the jars (though selectively) in the build path of the project. roughly there are 102 jars as of past week. lower version of the jars were not added in the build path of the eclipse project. one more important thing to do is not to place zkClient-0.1.jar in the build path but rather choose the zkclient jar that is present in the lib folder. -MIS On Mon, Mar 18, 2013 at 9:49 PM, MIS misapa...@gmail.com wrote: hi all, i'm just getting started with the apache kafka. and was following the below link for setting up kafka dev env in eclipse : https://cwiki.apache.org/KAFKA/developer-setup.html but been running into unresolved compilation issues. further, the link seems to be outdated too. could any one from the kafka community please update the link so as to setup the project in eclipse without any errors. thanks in advance. -MIS
[jira] [Commented] (KAFKA-185) The files in the kafka\clients\csharp\src\Kafka\Kafka.Client\cfg folder of the csharp client are out of date.
[ https://issues.apache.org/jira/browse/KAFKA-185?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13614063#comment-13614063 ] Jackson Wilson commented on KAFKA-185: -- Has anyone found the missing files? Has anyone instead rolled back to the previous check-in and re-built? Revision: 1185772 Author: junrao Date: Tuesday, October 18, 2011 1:52:13 PM Message: Add compression to C# client; patched by Eric Hauser; KAFKA-153 The files in the kafka\clients\csharp\src\Kafka\Kafka.Client\cfg folder of the csharp client are out of date. - Key: KAFKA-185 URL: https://issues.apache.org/jira/browse/KAFKA-185 Project: Kafka Issue Type: Bug Environment: Dotnet csharp Reporter: Mike Wilkerson When I open the Kafka.sln there are many files missing. When I remove the missing and add the files that are in the cfg folder I get many other errors. I think these files are from a newer version and the latest files have not been added. This is the complete list of missing files. Cfg\AsyncProducerConfiguration.cs Cfg\BrokerConfiguration.cs Cfg\BrokerConfigurationElement.cs Cfg\BrokerConfigurationElementCollection.cs Cfg\ConsumerConfigurationSection.cs Cfg\ConsumerConfiguration.cs Cfg\ProducerConfiguration.cs Cfg\ProducerConfigurationSection.cs Cfg\ZooKeeperConfigurationElement.cs Cfg\ZooKeeperServerConfigurationElement.cs Cfg\ZooKeeperServerConfigurationElementCollection.cs Cfg\SyncProducerConfiguration.cs Cfg\ZooKeeperConfiguration.cs Exceptions\IllegalStateException.cs Exceptions\InvalidMessageSizeException.cs Exceptions\NoSuchElementException.cs Exceptions\UnknownCodecException.cs Messages\CompressionCodec.cs Messages\CompressionCodecs.cs Messages\CompressionUtils.cs Messages\MessageAndOffset.cs KafkaStopWatch.cs -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (KAFKA-827) improve list topic output format
[ https://issues.apache.org/jira/browse/KAFKA-827?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao updated KAFKA-827: -- Status: Patch Available (was: Open) improve list topic output format Key: KAFKA-827 URL: https://issues.apache.org/jira/browse/KAFKA-827 Project: Kafka Issue Type: Bug Components: core Affects Versions: 0.8 Reporter: Jun Rao Assignee: Jun Rao Priority: Blocker Attachments: kafka-827.patch We need to make the output of list topic command more readable. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (KAFKA-827) improve list topic output format
[ https://issues.apache.org/jira/browse/KAFKA-827?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao updated KAFKA-827: -- Attachment: kafka-827.patch Attach a patch. improve list topic output format Key: KAFKA-827 URL: https://issues.apache.org/jira/browse/KAFKA-827 Project: Kafka Issue Type: Bug Components: core Affects Versions: 0.8 Reporter: Jun Rao Assignee: Jun Rao Priority: Blocker Attachments: kafka-827.patch We need to make the output of list topic command more readable. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (KAFKA-809) Dependency on zkclient 0.1 (redundant) prevents building in IntelliJ
[ https://issues.apache.org/jira/browse/KAFKA-809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dragos Manolescu updated KAFKA-809: --- Fix Version/s: 0.8 Status: Patch Available (was: Open) Index: core/build.sbt IDEA additional info: Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP +UTF-8 === --- core/build.sbt (date 1364250148000) +++ core/build.sbt (date 1364252653000) @@ -11,7 +11,6 @@ libraryDependencies ++= Seq( org.apache.zookeeper % zookeeper % 3.3.4, - com.github.sgroschupf % zkclient% 0.1, org.xerial.snappy % snappy-java % 1.0.4.1, org.easymock % easymock% 3.0 % test, junit % junit % 4.1 % test Index: project/Build.scala IDEA additional info: Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP +UTF-8 === --- project/Build.scala (date 1364250148000) +++ project/Build.scala (date 1364252653000) @@ -34,7 +34,8 @@ libraryDependencies ++= Seq( log4j % log4j% 1.2.15, net.sf.jopt-simple% jopt-simple % 3.2, - org.slf4j % slf4j-simple % 1.6.4 + org.slf4j % slf4j-simple % 1.6.4, + com.101tec% zkclient % 0.2 ), // The issue is going from log4j 1.2.14 to 1.2.15, the developers added some features which required // some dependencies on various sun and javax packages. @@ -100,9 +101,9 @@ // POM Tweaking for core: def zkClientDep = dependency - groupIdzkclient/groupId + groupIdcom.101tec/groupId artifactIdzkclient/artifactId - version20120522/version + version0.2/version scopecompile/scope /dependency @@ -116,7 +117,7 @@ /dependency dependency groupIdcom.yammer.metrics/groupId -artifactIdmetrics-annotations/artifactId +artifactIdmetrics-annotation/artifactId version3.0.0-c0c8be71/version scopecompile/scope /dependency Index: project/build/KafkaProject.scala IDEA additional info: Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP +UTF-8 === --- project/build/KafkaProject.scala(date 1364250148000) +++ project/build/KafkaProject.scala(date 1364252653000) @@ -74,7 +74,7 @@ dependency groupIdcom.yammer.metrics/groupId artifactIdmetrics-core/artifactId -version3.0.0-SNAPSHOT/version +version3.0.0-c0c8be71/version scopecompile/scope /dependency @@ -82,7 +82,7 @@ dependency groupIdcom.yammer.metrics/groupId artifactIdmetrics-annotation/artifactId -version3.0.0-SNAPSHOT/version +version3.0.0-c0c8be71/version scopecompile/scope /dependency Dependency on zkclient 0.1 (redundant) prevents building in IntelliJ Key: KAFKA-809 URL: https://issues.apache.org/jira/browse/KAFKA-809 Project: Kafka Issue Type: Bug Affects Versions: 0.8 Reporter: Dragos Manolescu Priority: Minor Labels: build, patch Fix For: 0.8 Per email thread from 3/14/2013; patch with fix included at the end of the description. Thanks for looking into this, Dragos. We should remove com.github.sgroschupf % zkclient% 0.1, from the build.sbt files. Would you like to create a JIRA and/or attach a patch ? -Neha On Thu, Mar 14, 2013 at 5:21 PM, Dragos Manolescu dragos.manole...@servicenow.com wrote: I dug into this and found a problem. The kafka build files show dependencies on two different versions of the zkclient code: In core/build.sbt: libraryDependencies ++= Seq( org.apache.zookeeper % zookeeper % 3.3.4, com.github.sgroschupf % zkclient% 0.1, org.xerial.snappy % snappy-java % 1.0.4.1, org.easymock % easymock% 3.0 % test, junit % junit % 4.1 % test ) And in project/Build.scala: // POM Tweaking for core: def zkClientDep = dependency groupIdzkclient/groupId artifactIdzkclient/artifactId version20120522/version scopecompile/scope /dependency In version 0.1 of com.github.sgroschupf.ZkClient (listed in build.sbt) writeData doesn't return anything (http://j.mp/XBnn5g), which causes the compile error from my initial message (see below); that changed in 0.2 (http://j.mp/WqUx4x). zkclient-20120522 (listed in Build.scala) is checked in as a JAR in the core/lib folder (http://j.mp/10U4XgE). I haven't researched why between sbt and IntelliJ ZkClient resolves differently, and don't know the history behind the POM
[jira] [Commented] (KAFKA-827) improve list topic output format
[ https://issues.apache.org/jira/browse/KAFKA-827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13614434#comment-13614434 ] Joel Koshy commented on KAFKA-827: -- While you are touching this, would it be reasonable to also switch from using the AdminUtil to a blank TopicMetadataRequest? It runs a lot quicker if there are a large number of topics and you run the tool from outside the ZK cluster's DC. Also, the topicOpt description has been misleading for a while. improve list topic output format Key: KAFKA-827 URL: https://issues.apache.org/jira/browse/KAFKA-827 Project: Kafka Issue Type: Bug Components: core Affects Versions: 0.8 Reporter: Jun Rao Assignee: Jun Rao Priority: Blocker Attachments: kafka-827.patch We need to make the output of list topic command more readable. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (KAFKA-828) Preferred Replica Election does not delete the admin on controller failover
Swapnil Ghike created KAFKA-828: --- Summary: Preferred Replica Election does not delete the admin on controller failover Key: KAFKA-828 URL: https://issues.apache.org/jira/browse/KAFKA-828 Project: Kafka Issue Type: Bug Affects Versions: 0.8 Reporter: Swapnil Ghike Assignee: Swapnil Ghike Priority: Blocker Fix For: 0.8 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (KAFKA-828) Preferred Replica Election does not delete the admin path on controller failover
[ https://issues.apache.org/jira/browse/KAFKA-828?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Neha Narkhede updated KAFKA-828: Labels: bugs kafka-0.8 p1 (was: bugs) Preferred Replica Election does not delete the admin path on controller failover Key: KAFKA-828 URL: https://issues.apache.org/jira/browse/KAFKA-828 Project: Kafka Issue Type: Bug Affects Versions: 0.8 Reporter: Swapnil Ghike Assignee: Swapnil Ghike Priority: Blocker Labels: bugs, kafka-0.8, p1 Fix For: 0.8 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (KAFKA-828) Preferred Replica Election does not delete the admin path on controller failover
[ https://issues.apache.org/jira/browse/KAFKA-828?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Swapnil Ghike updated KAFKA-828: Attachment: kafka-828-v2.patch Converted KAFKA-814 to subtask of this as Sriram asked to. Added a one line change to PreferredReplicaLeaderSelector to log instead of throwing an exception if the preferred leader is already the current leader. Preferred Replica Election does not delete the admin path on controller failover Key: KAFKA-828 URL: https://issues.apache.org/jira/browse/KAFKA-828 Project: Kafka Issue Type: Bug Affects Versions: 0.8 Reporter: Swapnil Ghike Assignee: Swapnil Ghike Priority: Blocker Labels: bugs, kafka-0.8, p1 Fix For: 0.8 Attachments: kafka-828-v1.patch, kafka-828-v2.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (KAFKA-830) partition replica assignment map in the controller should be a Set
Neha Narkhede created KAFKA-830: --- Summary: partition replica assignment map in the controller should be a Set Key: KAFKA-830 URL: https://issues.apache.org/jira/browse/KAFKA-830 Project: Kafka Issue Type: Bug Reporter: Neha Narkhede Assignee: Neha Narkhede Priority: Blocker Fix For: 0.8 partitionReplicaAssignment currently stores the list of assigned replicas as a sequence. When a broker comes online, the replica state machine adds the broker to the list of assigned replicas. It should do that only if the replica is already not in the list of assigned replicas. This causes the replication factor to be incorrectly calculated -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (KAFKA-828) Preferred Replica Election does not delete the admin path on controller failover
[ https://issues.apache.org/jira/browse/KAFKA-828?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Swapnil Ghike updated KAFKA-828: Attachment: kafka-828-v3.patch Actually since preferred replica election is not a heavy weight operation like partition reassignment (where we need to wait until the new followers catch up etc), we can move reading of the controllerContext.partitionsUndergoingPreferredReplicaElection to inside the controllerLock. Attached patch v3. Preferred Replica Election does not delete the admin path on controller failover Key: KAFKA-828 URL: https://issues.apache.org/jira/browse/KAFKA-828 Project: Kafka Issue Type: Bug Affects Versions: 0.8 Reporter: Swapnil Ghike Assignee: Swapnil Ghike Priority: Blocker Labels: bugs, kafka-0.8, p1 Fix For: 0.8 Attachments: kafka-828-v1.patch, kafka-828-v2.patch, kafka-828-v3.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
