Re: Review Request 28536: Patch for KAFKA-1799

2014-11-29 Thread Jun Rao 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28536/#review63295
---


Thanks for the patch. Some comments below.

Also, could we add a unit test for getConfiguredInstances()?


clients/src/main/java/org/apache/kafka/common/config/AbstractConfig.java


Do we really need to special case this? If we don't handle this, we will 
get a ConfigException anyway.



clients/src/main/java/org/apache/kafka/common/config/AbstractConfig.java


We probably should throw a ConfigException instead.


- Jun Rao


On Nov. 29, 2014, 10:20 a.m., Manikumar Reddy O wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/28536/
> ---
> 
> (Updated Nov. 29, 2014, 10:20 a.m.)
> 
> 
> Review request for kafka.
> 
> 
> Bugs: KAFKA-1799
> https://issues.apache.org/jira/browse/KAFKA-1799
> 
> 
> Repository: kafka
> 
> 
> Description
> ---
> 
> Explicit string to class conversion done in 
> gAbstractConfig.getConfiguredInstances()
> 
> 
> Diffs
> -
> 
>   clients/src/main/java/org/apache/kafka/common/config/AbstractConfig.java 
> 8d886105341555a548ecc7b2901e7fc5d6b1ee8c 
> 
> Diff: https://reviews.apache.org/r/28536/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Manikumar Reddy O
> 
>

[jira] [Commented] (KAFKA-1791) Corrupt index after safe shutdown and restart

2014-11-29 Thread Jun Rao (JIRA) 

[ 
https://issues.apache.org/jira/browse/KAFKA-1791?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14228917#comment-14228917
 ] 

Jun Rao commented on KAFKA-1791:


Thanks for attaching the log file. I couldn't reproduce the problem locally 
though. I copied the .log file to an empty Kafka log dir and then restarted the 
broker. The broker was able to recreate the missing index and I didn't see any 
exception during startup. Could you repeat the above steps yourself locally and 
see you if you can reproduce the issue?

> Corrupt index after safe shutdown and restart
> -
>
> Key: KAFKA-1791
> URL: https://issues.apache.org/jira/browse/KAFKA-1791
> Project: Kafka
>  Issue Type: Bug
>  Components: core
>Affects Versions: 0.8.1
> Environment: Debian6 with Sun-Java6
>Reporter: Vamsi Subhash Achanta
>Priority: Critical
> Attachments: 0233.index, 0233.log
>
>
> We have 3 kafka brokers - all VMs. One of the broker was stopped for around 
> 30 minutes to fix a problem with the bare metal. Upon restart, after some 
> time, the broker went out of file-descriptors (FDs) and started throwing 
> errors. Upon restart, it started throwing this corrupted index exceptions. I 
> followed the other JIRAs related to corrupted indices but the solutions 
> mentioned there (deleting the index and restart) didn't work - the index gets 
> created again. The other JIRAs solution of deleting those indexes which got 
> wrongly compacted (> 10MB) didn't work either.
> What is the error? How can I fix this and bring back the broker? Thanks.
> INFO [2014-11-21 02:57:17,510] [main][] kafka.log.LogManager - Found clean 
> shutdown file. Skipping recovery for all logs in data directory 
> '/var/lib/fk-3p-kafka/logs'
>  INFO [2014-11-21 02:57:17,510] [main][] kafka.log.LogManager - Loading log 
> 'kf.production.b2b.return_order.status-25'
> FATAL [2014-11-21 02:57:17,533] [main][] kafka.server.KafkaServerStartable - 
> Fatal error during KafkaServerStable startup. Prepare to shutdown
> java.lang.IllegalArgumentException: requirement failed: Corrupt index found, 
> index file 
> (/var/lib/fk-3p-kafka/logs/kf.production.b2b.return_order.status-25/0233.index)
>  has non-zero size but the last offset is 233 and the base offset is 233
>   at scala.Predef$.require(Predef.scala:145)
>   at kafka.log.OffsetIndex.sanityCheck(OffsetIndex.scala:352)
>   at kafka.log.Log$$anonfun$loadSegments$5.apply(Log.scala:159)
>   at kafka.log.Log$$anonfun$loadSegments$5.apply(Log.scala:158)
>   at scala.collection.Iterator$class.foreach(Iterator.scala:631)
>   at 
> scala.collection.JavaConversions$JIteratorWrapper.foreach(JavaConversions.scala:474)
>   at scala.collection.IterableLike$class.foreach(IterableLike.scala:79)
>   at 
> scala.collection.JavaConversions$JCollectionWrapper.foreach(JavaConversions.scala:495)
>   at kafka.log.Log.loadSegments(Log.scala:158)
>   at kafka.log.Log.(Log.scala:64)
>   at 
> kafka.log.LogManager$$anonfun$loadLogs$1$$anonfun$apply$4.apply(LogManager.scala:118)
>   at 
> kafka.log.LogManager$$anonfun$loadLogs$1$$anonfun$apply$4.apply(LogManager.scala:113)
>   at 
> scala.collection.IndexedSeqOptimized$class.foreach(IndexedSeqOptimized.scala:34)
>   at scala.collection.mutable.ArrayOps.foreach(ArrayOps.scala:34)
>   at kafka.log.LogManager$$anonfun$loadLogs$1.apply(LogManager.scala:113)
>   at kafka.log.LogManager$$anonfun$loadLogs$1.apply(LogManager.scala:105)
>   at 
> scala.collection.IndexedSeqOptimized$class.foreach(IndexedSeqOptimized.scala:34)
>   at scala.collection.mutable.WrappedArray.foreach(WrappedArray.scala:32)
>   at kafka.log.LogManager.loadLogs(LogManager.scala:105)
>   at kafka.log.LogManager.(LogManager.scala:57)
>   at kafka.server.KafkaServer.createLogManager(KafkaServer.scala:275)
>   at kafka.server.KafkaServer.startup(KafkaServer.scala:72)
>   at 
> kafka.server.KafkaServerStartable.startup(KafkaServerStartable.scala:34)
>   at kafka.Kafka$.main(Kafka.scala:46)
>   at kafka.Kafka.main(Kafka.scala)
>  INFO [2014-11-21 02:57:17,534] [main][] kafka.server.KafkaServer - [Kafka 
> Server 2], shutting down
>  INFO [2014-11-21 02:57:17,538] [main][] kafka.server.KafkaServer - [Kafka 
> Server 2], shut down completed
>  INFO [2014-11-21 02:57:17,539] [Thread-2][] kafka.server.KafkaServer - 
> [Kafka Server 2], shutting down



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (KAFKA-1799) ProducerConfig.METRIC_REPORTER_CLASSES_CONFIG doesn't work

2014-11-29 Thread Manikumar Reddy (JIRA) 

 [ 
https://issues.apache.org/jira/browse/KAFKA-1799?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Manikumar Reddy updated KAFKA-1799:
---
Assignee: Manikumar Reddy
  Status: Patch Available  (was: Open)

> ProducerConfig.METRIC_REPORTER_CLASSES_CONFIG doesn't work
> --
>
> Key: KAFKA-1799
> URL: https://issues.apache.org/jira/browse/KAFKA-1799
> Project: Kafka
>  Issue Type: Bug
>Affects Versions: 0.8.2
>Reporter: Jun Rao
>Assignee: Manikumar Reddy
>Priority: Blocker
>  Labels: newbie++
> Fix For: 0.8.2
>
> Attachments: KAFKA-1799.patch
>
>
> When running the following test, we got an unknown configuration exception.
> @Test
> public void testMetricsReporter() {
> Properties producerProps = new Properties();
> producerProps.put(ProducerConfig.BOOTSTRAP_SERVERS_CONFIG, 
> "host1:123");
> producerProps.put(ProducerConfig.METRIC_REPORTER_CLASSES_CONFIG, 
> "org.apache.kafka.clients.producer.new-metrics-reporter");
> new KafkaProducer(producerProps);
> }
> org.apache.kafka.common.config.ConfigException: Unknown configuration 
> 'org.apache.kafka.clients.producer.new-metrics-reporter'
>   at 
> org.apache.kafka.common.config.AbstractConfig.get(AbstractConfig.java:60)
>   at 
> org.apache.kafka.common.config.AbstractConfig.getClass(AbstractConfig.java:91)
>   at 
> org.apache.kafka.common.config.AbstractConfig.getConfiguredInstances(AbstractConfig.java:147)
>   at 
> org.apache.kafka.clients.producer.KafkaProducer.(KafkaProducer.java:105)
>   at 
> org.apache.kafka.clients.producer.KafkaProducer.(KafkaProducer.java:94)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (KAFKA-1799) ProducerConfig.METRIC_REPORTER_CLASSES_CONFIG doesn't work

2014-11-29 Thread Manikumar Reddy (JIRA) 

 [ 
https://issues.apache.org/jira/browse/KAFKA-1799?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Manikumar Reddy updated KAFKA-1799:
---
Attachment: KAFKA-1799.patch

> ProducerConfig.METRIC_REPORTER_CLASSES_CONFIG doesn't work
> --
>
> Key: KAFKA-1799
> URL: https://issues.apache.org/jira/browse/KAFKA-1799
> Project: Kafka
>  Issue Type: Bug
>Affects Versions: 0.8.2
>Reporter: Jun Rao
>Priority: Blocker
>  Labels: newbie++
> Fix For: 0.8.2
>
> Attachments: KAFKA-1799.patch
>
>
> When running the following test, we got an unknown configuration exception.
> @Test
> public void testMetricsReporter() {
> Properties producerProps = new Properties();
> producerProps.put(ProducerConfig.BOOTSTRAP_SERVERS_CONFIG, 
> "host1:123");
> producerProps.put(ProducerConfig.METRIC_REPORTER_CLASSES_CONFIG, 
> "org.apache.kafka.clients.producer.new-metrics-reporter");
> new KafkaProducer(producerProps);
> }
> org.apache.kafka.common.config.ConfigException: Unknown configuration 
> 'org.apache.kafka.clients.producer.new-metrics-reporter'
>   at 
> org.apache.kafka.common.config.AbstractConfig.get(AbstractConfig.java:60)
>   at 
> org.apache.kafka.common.config.AbstractConfig.getClass(AbstractConfig.java:91)
>   at 
> org.apache.kafka.common.config.AbstractConfig.getConfiguredInstances(AbstractConfig.java:147)
>   at 
> org.apache.kafka.clients.producer.KafkaProducer.(KafkaProducer.java:105)
>   at 
> org.apache.kafka.clients.producer.KafkaProducer.(KafkaProducer.java:94)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (KAFKA-1799) ProducerConfig.METRIC_REPORTER_CLASSES_CONFIG doesn't work

2014-11-29 Thread Manikumar Reddy (JIRA) 

[ 
https://issues.apache.org/jira/browse/KAFKA-1799?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14228703#comment-14228703
 ] 

Manikumar Reddy commented on KAFKA-1799:


Created reviewboard https://reviews.apache.org/r/28536/diff/
 against branch origin/trunk

> ProducerConfig.METRIC_REPORTER_CLASSES_CONFIG doesn't work
> --
>
> Key: KAFKA-1799
> URL: https://issues.apache.org/jira/browse/KAFKA-1799
> Project: Kafka
>  Issue Type: Bug
>Affects Versions: 0.8.2
>Reporter: Jun Rao
>Priority: Blocker
>  Labels: newbie++
> Fix For: 0.8.2
>
> Attachments: KAFKA-1799.patch
>
>
> When running the following test, we got an unknown configuration exception.
> @Test
> public void testMetricsReporter() {
> Properties producerProps = new Properties();
> producerProps.put(ProducerConfig.BOOTSTRAP_SERVERS_CONFIG, 
> "host1:123");
> producerProps.put(ProducerConfig.METRIC_REPORTER_CLASSES_CONFIG, 
> "org.apache.kafka.clients.producer.new-metrics-reporter");
> new KafkaProducer(producerProps);
> }
> org.apache.kafka.common.config.ConfigException: Unknown configuration 
> 'org.apache.kafka.clients.producer.new-metrics-reporter'
>   at 
> org.apache.kafka.common.config.AbstractConfig.get(AbstractConfig.java:60)
>   at 
> org.apache.kafka.common.config.AbstractConfig.getClass(AbstractConfig.java:91)
>   at 
> org.apache.kafka.common.config.AbstractConfig.getConfiguredInstances(AbstractConfig.java:147)
>   at 
> org.apache.kafka.clients.producer.KafkaProducer.(KafkaProducer.java:105)
>   at 
> org.apache.kafka.clients.producer.KafkaProducer.(KafkaProducer.java:94)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Review Request 28536: Patch for KAFKA-1799

2014-11-29 Thread Manikumar Reddy O 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28536/
---

Review request for kafka.


Bugs: KAFKA-1799
https://issues.apache.org/jira/browse/KAFKA-1799


Repository: kafka


Description
---

Explicit string to class conversion done in 
gAbstractConfig.getConfiguredInstances()


Diffs
-

  clients/src/main/java/org/apache/kafka/common/config/AbstractConfig.java 
8d886105341555a548ecc7b2901e7fc5d6b1ee8c 

Diff: https://reviews.apache.org/r/28536/diff/


Testing
---


Thanks,

Manikumar Reddy O