[jira] [Commented] (KAFKA-949) Integrate kafka into YARN
[ https://issues.apache.org/jira/browse/KAFKA-949?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13748147#comment-13748147 ] Kam Kasravi commented on KAFKA-949: --- This work is available under https://github.com/kkasravi/kafka-yarn, I can provide a patch which adds the project to https://github.com/apache/kafka/tree/0.8/contrib/yarn. Kafka-yarn has a dependency on BIGTOP-989 which installs kafka 0.8 beta1 as a service on linux (deb, rpm). Please advise. Kam > Integrate kafka into YARN > - > > Key: KAFKA-949 > URL: https://issues.apache.org/jira/browse/KAFKA-949 > Project: Kafka > Issue Type: New Feature > Components: contrib >Affects Versions: 0.8 > Environment: hadoop 2-0.X >Reporter: Kam Kasravi > > kafka is being added to bigtop (BIGTOP-989). Having kafka services available > under YARN will enable a number of cluster operations for kafka that YARN > handles. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (KAFKA-949) Integrate kafka into YARN
Kam Kasravi created KAFKA-949: - Summary: Integrate kafka into YARN Key: KAFKA-949 URL: https://issues.apache.org/jira/browse/KAFKA-949 Project: Kafka Issue Type: New Feature Components: contrib Affects Versions: 0.8 Environment: hadoop 2-0.X Reporter: Kam Kasravi kafka is being added to bigtop (BIGTOP-989). Having kafka services available under YARN will enable a number of cluster operations for kafka that YARN handles. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (KAFKA-884) Get java.lang.NoSuchMethodError: com.yammer.metrics.core.TimerContext.stop()J when stopping kafka brokers
[
https://issues.apache.org/jira/browse/KAFKA-884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kam Kasravi resolved KAFKA-884.
---
Resolution: Fixed
That resolved it. Thanks Neha.
> Get java.lang.NoSuchMethodError: com.yammer.metrics.core.TimerContext.stop()J
> when stopping kafka brokers
> -
>
> Key: KAFKA-884
> URL: https://issues.apache.org/jira/browse/KAFKA-884
> Project: Kafka
> Issue Type: Bug
> Components: config
>Affects Versions: 0.8
> Environment: MAC OSX 10.8.3
>Reporter: Kam Kasravi
>Priority: Minor
> Fix For: 0.8
>
>
> When stopping kafka brokers I get the following stack trace:
> 2013-04-26 14:30:16,394] INFO [Replica Manager on Broker 1]: Shutted down
> completely (kafka.server.ReplicaManager)
> [2013-04-26 14:30:16,398] WARN com.yammer.metrics.core.TimerContext.stop()J
> (kafka.utils.Utils$)
> java.lang.NoSuchMethodError: com.yammer.metrics.core.TimerContext.stop()J
> at kafka.metrics.KafkaTimer.time(KafkaTimer.scala:36)
> at kafka.log.FileMessageSet.flush(FileMessageSet.scala:164)
> at kafka.log.FileMessageSet.close(FileMessageSet.scala:173)
> at
> kafka.log.LogSegment$$anonfun$close$2.apply$mcV$sp(LogSegment.scala:161)
> at kafka.utils.Utils$.swallow(Utils.scala:186)
> at kafka.utils.Logging$class.swallowWarn(Logging.scala:91)
> at kafka.utils.Utils$.swallowWarn(Utils.scala:45)
> at kafka.utils.Logging$class.swallow(Logging.scala:93)
> at kafka.utils.Utils$.swallow(Utils.scala:45)
> at kafka.log.LogSegment.close(LogSegment.scala:161)
> at kafka.log.Log$$anonfun$close$2.apply(Log.scala:248)
> at kafka.log.Log$$anonfun$close$2.apply(Log.scala:247)
> at
> scala.collection.IndexedSeqOptimized$class.foreach(IndexedSeqOptimized.scala:34)
> at scala.collection.mutable.ArrayOps.foreach(ArrayOps.scala:34)
> at kafka.log.Log.close(Log.scala:247)
> at kafka.log.LogManager$$anonfun$shutdown$2.apply(LogManager.scala:290)
> at kafka.log.LogManager$$anonfun$shutdown$2.apply(LogManager.scala:290)
> at scala.collection.Iterator$class.foreach(Iterator.scala:631)
> at
> scala.collection.JavaConversions$JIteratorWrapper.foreach(JavaConversions.scala:474)
> at scala.collection.IterableLike$class.foreach(IterableLike.scala:79)
> at
> scala.collection.JavaConversions$JCollectionWrapper.foreach(JavaConversions.scala:495)
> at kafka.log.LogManager.shutdown(LogManager.scala:290)
> at
> kafka.server.KafkaServer$$anonfun$shutdown$7.apply$mcV$sp(KafkaServer.scala:125)
> at kafka.utils.Utils$.swallow(Utils.scala:186)
> at kafka.utils.Logging$class.swallowWarn(Logging.scala:91)
> at kafka.utils.Utils$.swallowWarn(Utils.scala:45)
> at kafka.utils.Logging$class.swallow(Logging.scala:93)
> at kafka.utils.Utils$.swallow(Utils.scala:45)
> at kafka.server.KafkaServer.shutdown(KafkaServer.scala:125)
> at
> kafka.server.KafkaServerStartable.shutdown(KafkaServerStartable.scala:46)
> at kafka.Kafka$$anon$1.run(Kafka.scala:42)
> Steps to reproduce:
> git checkout remotes/origin/0.8
> ./sbt update
> ./sbt +package
> ./sbt assembly-package-dependency
> ./sbt eclipse
> Now create 3 server.properties under config as described in the quick-start
> for 0.8
> Additionally comment out the JMX_PORT in bin/kafka-server-start.sh.
> Now start up servers and create a topic
> bin/zookeeper-server-start.sh config/zookeeper.properties&
> bin/kafka-server-start.sh config/server0.properties&
> bin/kafka-server-start.sh config/server1.properties&
> bin/kafka-server-start.sh config/server2.properties&
> bin/kafka-create-topic.sh --topic mytopic --replica 3 --zookeeper
> localhost:2181
> bin/kafka-console-producer.sh --broker-list
> localhost:9092,localhost:9093,localhost:9094 --sync --topic mytopic
> Confirm that kafka servers are running as well as QuorumPeerMain
> $ jps
> 43668 QuorumPeerMain
> 43669 Jps
> 43666 Kafka
> 43667 Kafka
> 43665 Kafka
> Now stop the brokers and zookeeper
> kill $(ps -eaf|grep Kafka|grep -v grep|awk '{print $2}')
> kill $(ps -eaf|grep QuorumPeerMain|grep -v grep|awk '{print $2}')
> You should see the above stack trace.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (KAFKA-884) Get java.lang.NoSuchMethodError: com.yammer.metrics.core.TimerContext.stop()J when stopping kafka brokers
Kam Kasravi created KAFKA-884:
-
Summary: Get java.lang.NoSuchMethodError:
com.yammer.metrics.core.TimerContext.stop()J when stopping kafka brokers
Key: KAFKA-884
URL: https://issues.apache.org/jira/browse/KAFKA-884
Project: Kafka
Issue Type: Bug
Components: config
Affects Versions: 0.8
Environment: MAC OSX 10.8.3
Reporter: Kam Kasravi
Priority: Minor
Fix For: 0.8
When stopping kafka brokers I get the following stack trace:
2013-04-26 14:30:16,394] INFO [Replica Manager on Broker 1]: Shutted down
completely (kafka.server.ReplicaManager)
[2013-04-26 14:30:16,398] WARN com.yammer.metrics.core.TimerContext.stop()J
(kafka.utils.Utils$)
java.lang.NoSuchMethodError: com.yammer.metrics.core.TimerContext.stop()J
at kafka.metrics.KafkaTimer.time(KafkaTimer.scala:36)
at kafka.log.FileMessageSet.flush(FileMessageSet.scala:164)
at kafka.log.FileMessageSet.close(FileMessageSet.scala:173)
at
kafka.log.LogSegment$$anonfun$close$2.apply$mcV$sp(LogSegment.scala:161)
at kafka.utils.Utils$.swallow(Utils.scala:186)
at kafka.utils.Logging$class.swallowWarn(Logging.scala:91)
at kafka.utils.Utils$.swallowWarn(Utils.scala:45)
at kafka.utils.Logging$class.swallow(Logging.scala:93)
at kafka.utils.Utils$.swallow(Utils.scala:45)
at kafka.log.LogSegment.close(LogSegment.scala:161)
at kafka.log.Log$$anonfun$close$2.apply(Log.scala:248)
at kafka.log.Log$$anonfun$close$2.apply(Log.scala:247)
at
scala.collection.IndexedSeqOptimized$class.foreach(IndexedSeqOptimized.scala:34)
at scala.collection.mutable.ArrayOps.foreach(ArrayOps.scala:34)
at kafka.log.Log.close(Log.scala:247)
at kafka.log.LogManager$$anonfun$shutdown$2.apply(LogManager.scala:290)
at kafka.log.LogManager$$anonfun$shutdown$2.apply(LogManager.scala:290)
at scala.collection.Iterator$class.foreach(Iterator.scala:631)
at
scala.collection.JavaConversions$JIteratorWrapper.foreach(JavaConversions.scala:474)
at scala.collection.IterableLike$class.foreach(IterableLike.scala:79)
at
scala.collection.JavaConversions$JCollectionWrapper.foreach(JavaConversions.scala:495)
at kafka.log.LogManager.shutdown(LogManager.scala:290)
at
kafka.server.KafkaServer$$anonfun$shutdown$7.apply$mcV$sp(KafkaServer.scala:125)
at kafka.utils.Utils$.swallow(Utils.scala:186)
at kafka.utils.Logging$class.swallowWarn(Logging.scala:91)
at kafka.utils.Utils$.swallowWarn(Utils.scala:45)
at kafka.utils.Logging$class.swallow(Logging.scala:93)
at kafka.utils.Utils$.swallow(Utils.scala:45)
at kafka.server.KafkaServer.shutdown(KafkaServer.scala:125)
at
kafka.server.KafkaServerStartable.shutdown(KafkaServerStartable.scala:46)
at kafka.Kafka$$anon$1.run(Kafka.scala:42)
Steps to reproduce:
git checkout remotes/origin/0.8
./sbt update
./sbt +package
./sbt assembly-package-dependency
./sbt eclipse
Now create 3 server.properties under config as described in the quick-start for
0.8
Additionally comment out the JMX_PORT in bin/kafka-server-start.sh.
Now start up servers and create a topic
bin/zookeeper-server-start.sh config/zookeeper.properties&
bin/kafka-server-start.sh config/server0.properties&
bin/kafka-server-start.sh config/server1.properties&
bin/kafka-server-start.sh config/server2.properties&
bin/kafka-create-topic.sh --topic mytopic --replica 3 --zookeeper localhost:2181
bin/kafka-console-producer.sh --broker-list
localhost:9092,localhost:9093,localhost:9094 --sync --topic mytopic
Confirm that kafka servers are running as well as QuorumPeerMain
$ jps
43668 QuorumPeerMain
43669 Jps
43666 Kafka
43667 Kafka
43665 Kafka
Now stop the brokers and zookeeper
kill $(ps -eaf|grep Kafka|grep -v grep|awk '{print $2}')
kill $(ps -eaf|grep QuorumPeerMain|grep -v grep|awk '{print $2}')
You should see the above stack trace.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
