Re: [DISCUSS] KIP-63: Unify store and downstream caching in streams

[Henry Cai]

One more thing for this KIP:

Currently RocksDBWindowStore serialize the key/value before it puts into
the in-memory cache, I think we should delay this
serialization/deserialization unless it needs flush to db.  For a simple
countByKey for 100 records, this would trigger 100
serialization/deserialization even if everything is in-memory.

If we move this internal cache from RocksDBStore to a global place, I hope
we can reduces the time it needs to do the serialization.


On Mon, Jun 6, 2016 at 11:07 AM, Ismael Juma  wrote:

> On Mon, Jun 6, 2016 at 6:48 PM, Guozhang Wang  wrote:
> >
> > About using Instrumentation.getObjectSize, yeah we were worried a lot
> about
> > its efficiency as well as accuracy when discussing internally, but not a
> > better solution was proposed. So if people have better ideas, please
> throw
> > them here, as it is also the purpose for us to call out such KIP
> discussion
> > threads.
> >
>
> Note that this requires a Java agent to be configured. A few links:
>
>
> https://github.com/apache/spark/blob/b0ce0d13127431fa7cd4c11064762eb0b12e3436/core/src/main/scala/org/apache/spark/util/SizeEstimator.scala
>
> https://github.com/apache/cassandra/blob/3dcbe90e02440e6ee534f643c7603d50ca08482b/src/java/org/apache/cassandra/utils/ObjectSizes.java
> https://github.com/jbellis/jamm
> http://openjdk.java.net/projects/code-tools/jol/
> https://github.com/DimitrisAndreou/memory-measurer
>
> OK, maybe that's more than what you wanted. :)
>
> Ismael
>

[jira] [Commented] (KAFKA-3809) Auto-generate documentation for topic-level configuration

[James Cheng (JIRA)]

[ 
https://issues.apache.org/jira/browse/KAFKA-3809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15321787#comment-15321787
 ] 

James Cheng commented on KAFKA-3809:


Ah, I had missed that one. KAFKA-3234 handles auto-generation, as well as 
renaming and deprecation of old config names. Is there any benefit to doing the 
work in two parts, instead of doing it all at once?


> Auto-generate documentation for topic-level configuration
> -
>
> Key: KAFKA-3809
> URL: https://issues.apache.org/jira/browse/KAFKA-3809
> Project: Kafka
>  Issue Type: Bug
>Reporter: James Cheng
>Assignee: James Cheng
> Attachments: configuration.html, topic_config.html
>
>
> The documentation for topic-level configuration is not auto-generated from 
> the code. configuration.html still contains hand-maintained documentation.
> I noticed this because I wanted to set message.timestamp.type on a topic, and 
> didn't see that it was supported, but grepped through the code and it looked 
> like it was.
> The code to auto-generate the docs is quite close, but needs some additional 
> work. In particular, topic-level configuration is different from all the 
> other ConfigDefs in that topic-level configuration docs list the broker-level 
> config that they inherit from. We would need to have a way to show what 
> broker-level config applies to each topic-level config.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (KAFKA-3809) Auto-generate documentation for topic-level configuration

[Ismael Juma (JIRA)]

[ 
https://issues.apache.org/jira/browse/KAFKA-3809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15321781#comment-15321781
 ] 

Ismael Juma commented on KAFKA-3809:


Also see: https://github.com/apache/kafka/pull/907

> Auto-generate documentation for topic-level configuration
> -
>
> Key: KAFKA-3809
> URL: https://issues.apache.org/jira/browse/KAFKA-3809
> Project: Kafka
>  Issue Type: Bug
>Reporter: James Cheng
>Assignee: James Cheng
> Attachments: configuration.html, topic_config.html
>
>
> The documentation for topic-level configuration is not auto-generated from 
> the code. configuration.html still contains hand-maintained documentation.
> I noticed this because I wanted to set message.timestamp.type on a topic, and 
> didn't see that it was supported, but grepped through the code and it looked 
> like it was.
> The code to auto-generate the docs is quite close, but needs some additional 
> work. In particular, topic-level configuration is different from all the 
> other ConfigDefs in that topic-level configuration docs list the broker-level 
> config that they inherit from. We would need to have a way to show what 
> broker-level config applies to each topic-level config.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (KAFKA-3806) Adjust default values of log.retention.hours and offsets.retention.minutes

[James Cheng (JIRA)]

[ 
https://issues.apache.org/jira/browse/KAFKA-3806?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15321771#comment-15321771
 ] 

James Cheng commented on KAFKA-3806:


Thanks Jun.

If it's per topic/partition, then that means that the garbage collection of 
offsets is not only dependent on the consumer, it's also dependent on the 
partitioning strategy of the producer. Within a given topic, a hot partition is 
less likely to be incorrectly garbage collected, and a low-traffic partition is 
more likely to be incorrectly garbage collected.

For now, it looks like the only way to mitigate this is on the consumer, by 
committing offsets even though no data is flowing.

It might be more usable if garbage collection was took into account not just 
time-since-last-committed-offset but also time-since-last-produced-message. 
Although, there are some corner cases where that might not work. For example, 
what's the right thing to do if you have an active consumer to a topic, but 
where the producer has been turned off for a week? Not sure.


> Adjust default values of log.retention.hours and offsets.retention.minutes
> --
>
> Key: KAFKA-3806
> URL: https://issues.apache.org/jira/browse/KAFKA-3806
> Project: Kafka
>  Issue Type: Improvement
>  Components: config
>Affects Versions: 0.9.0.1, 0.10.0.0
>Reporter: Michal Turek
>Priority: Minor
>
> Combination of default values of log.retention.hours (168 hours = 7 days) and 
> offsets.retention.minutes (1440 minutes = 1 day) may be dangerous in special 
> cases. Offset retention should be always greater than log retention.
> We have observed the following scenario and issue:
> - Producing of data to a topic was disabled two days ago by producer update, 
> topic wasn't deleted.
> - Consumer consumed all data and properly committed offsets to Kafka.
> - Consumer made no more offset commits for that topic because there was no 
> more incoming data and there was nothing to confirm. (We have auto-commit 
> disabled, I'm not sure how behaves enabled auto-commit.)
> - After one day: Kafka cleared too old offsets according to 
> offsets.retention.minutes.
> - After two days: Long-term running consumer was restarted after update, it 
> didn't find any committed offsets for that topic since they were deleted by 
> offsets.retention.minutes so it started consuming from the beginning.
> - The messages were still in Kafka due to larger log.retention.hours, about 5 
> days of messages were read again.
> Known workaround to solve this issue:
> - Explicitly configure log.retention.hours and offsets.retention.minutes, 
> don't use defaults.
> Proposals:
> - Prolong default value of offsets.retention.minutes to be at least twice 
> larger than log.retention.hours.
> - Check these values during Kafka startup and log a warning if 
> offsets.retention.minutes is smaller than log.retention.hours.
> - Add a note to migration guide about differences between storing of offsets 
> in ZooKeeper and Kafka (http://kafka.apache.org/documentation.html#upgrade).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (KAFKA-3809) Auto-generate documentation for topic-level configuration

[James Cheng (JIRA)]

 [ 
https://issues.apache.org/jira/browse/KAFKA-3809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

James Cheng updated KAFKA-3809:
---
Attachment: topic_config.html
configuration.html

Attached is the current hand-maintained documentation, and the current 
auto-generated documentation.

I can easily update the descriptions of the auto-generated docs to match the 
hand-maintained ones. The hand-maintained description are the ones that are 
currently visible on the website, and so presumably are the most readable and 
most understandable.

The harder part of the work is getting the "Server Default Property" column in 
the generated docs. Are there any other configs in the Kafka code where config 
values inherit from other config values? 

Is there a need at this point for a generic inheritable ConfigDefs mechanism? 
The inheritance at this point is encapsulated in code. See 
https://github.com/apache/kafka/blob/trunk/core/src/main/scala/kafka/server/KafkaServer.scala#L58-L83

If there is no need for a generic mechanism, I would likely implement this by 
creating a custom LogConfigDefs() class that supports an additional argument, 
which is the name of the Server Default Property that this config inherits 
from. Then I would create a subclass of ConfigDef that would override 
toHtmlTable() and toRst() that would output the additional column.

Not sure who to ask for guidance on this, so starting with [~gwenshap] who has 
done some previous auto-generated-docs work. Gwen, let me know if there is a 
more appropriate person to ask about this.

> Auto-generate documentation for topic-level configuration
> -
>
> Key: KAFKA-3809
> URL: https://issues.apache.org/jira/browse/KAFKA-3809
> Project: Kafka
>  Issue Type: Bug
>Reporter: James Cheng
>Assignee: James Cheng
> Attachments: configuration.html, topic_config.html
>
>
> The documentation for topic-level configuration is not auto-generated from 
> the code. configuration.html still contains hand-maintained documentation.
> I noticed this because I wanted to set message.timestamp.type on a topic, and 
> didn't see that it was supported, but grepped through the code and it looked 
> like it was.
> The code to auto-generate the docs is quite close, but needs some additional 
> work. In particular, topic-level configuration is different from all the 
> other ConfigDefs in that topic-level configuration docs list the broker-level 
> config that they inherit from. We would need to have a way to show what 
> broker-level config applies to each topic-level config.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (KAFKA-3809) Auto-generate documentation for topic-level configuration

[James Cheng (JIRA)]

James Cheng created KAFKA-3809:
--

 Summary: Auto-generate documentation for topic-level configuration
 Key: KAFKA-3809
 URL: https://issues.apache.org/jira/browse/KAFKA-3809
 Project: Kafka
  Issue Type: Bug
Reporter: James Cheng
Assignee: James Cheng


The documentation for topic-level configuration is not auto-generated from the 
code. configuration.html still contains hand-maintained documentation.

I noticed this because I wanted to set message.timestamp.type on a topic, and 
didn't see that it was supported, but grepped through the code and it looked 
like it was.

The code to auto-generate the docs is quite close, but needs some additional 
work. In particular, topic-level configuration is different from all the other 
ConfigDefs in that topic-level configuration docs list the broker-level config 
that they inherit from. We would need to have a way to show what broker-level 
config applies to each topic-level config.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (KAFKA-3806) Adjust default values of log.retention.hours and offsets.retention.minutes

[Jun Rao (JIRA)]

[ 
https://issues.apache.org/jira/browse/KAFKA-3806?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15321710#comment-15321710
 ] 

Jun Rao commented on KAFKA-3806:


[~wushujames], committing offsets on every poll is likely too frequent and will 
add unnecessary load on the broker. So, committing offsets in some configured 
time interval will be better.

The retention is done per topic/partition. If the last committed offset for a 
given topic/partition hasn't been updated for longer than 
offsets.retention.minutes, the offset will be garbage collected.

> Adjust default values of log.retention.hours and offsets.retention.minutes
> --
>
> Key: KAFKA-3806
> URL: https://issues.apache.org/jira/browse/KAFKA-3806
> Project: Kafka
>  Issue Type: Improvement
>  Components: config
>Affects Versions: 0.9.0.1, 0.10.0.0
>Reporter: Michal Turek
>Priority: Minor
>
> Combination of default values of log.retention.hours (168 hours = 7 days) and 
> offsets.retention.minutes (1440 minutes = 1 day) may be dangerous in special 
> cases. Offset retention should be always greater than log retention.
> We have observed the following scenario and issue:
> - Producing of data to a topic was disabled two days ago by producer update, 
> topic wasn't deleted.
> - Consumer consumed all data and properly committed offsets to Kafka.
> - Consumer made no more offset commits for that topic because there was no 
> more incoming data and there was nothing to confirm. (We have auto-commit 
> disabled, I'm not sure how behaves enabled auto-commit.)
> - After one day: Kafka cleared too old offsets according to 
> offsets.retention.minutes.
> - After two days: Long-term running consumer was restarted after update, it 
> didn't find any committed offsets for that topic since they were deleted by 
> offsets.retention.minutes so it started consuming from the beginning.
> - The messages were still in Kafka due to larger log.retention.hours, about 5 
> days of messages were read again.
> Known workaround to solve this issue:
> - Explicitly configure log.retention.hours and offsets.retention.minutes, 
> don't use defaults.
> Proposals:
> - Prolong default value of offsets.retention.minutes to be at least twice 
> larger than log.retention.hours.
> - Check these values during Kafka startup and log a warning if 
> offsets.retention.minutes is smaller than log.retention.hours.
> - Add a note to migration guide about differences between storing of offsets 
> in ZooKeeper and Kafka (http://kafka.apache.org/documentation.html#upgrade).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (KAFKA-3797) Improve security of __consumer_offsets topic

[Jason Gustafson (JIRA)]

[ 
https://issues.apache.org/jira/browse/KAFKA-3797?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15321678#comment-15321678
 ] 

Jason Gustafson commented on KAFKA-3797:


[~toddpalino] That might make the most sense, but I don't think you could 
currently expect reasonable behavior if an admin tool tried to overwrite 
offsets for a group since:

1. Writes directly to the topic are not reflected in the offset cache until a 
change of leadership.
2. Even if they were, an active consumer wouldn't see the new offset until 
after the next rebalance. 

I also wonder if the external tool could just use the offset API directly in 
this case?

> Improve security of __consumer_offsets topic
> 
>
> Key: KAFKA-3797
> URL: https://issues.apache.org/jira/browse/KAFKA-3797
> Project: Kafka
>  Issue Type: Improvement
>Reporter: Jason Gustafson
>Assignee: Vahid Hashemian
>
> By default, we allow clients to override committed offsets and group metadata 
> using the Produce API as long as they have Write access to the 
> __consumer_offsets topic. From one perspective, this is fine: administrators 
> can restrict access to this topic to trusted users. From another, it seems 
> less than ideal for Write permission on that topic to subsume Group-level 
> permissions for the full cluster. With this access, a user can cause all 
> kinds of mischief including making the group "lose" data by setting offsets 
> ahead of the actual position. This is probably not obvious to administrators 
> who grant access to topics using a wildcard and it increases the risk from 
> incorrectly applying topic patterns (if we ever add support for them). It 
> seems reasonable to consider safer default behavior:
> 1. A simple option to fix this would be to prevent wildcard topic rules from 
> applying to internal topics. To write to an internal topic, you need a 
> separate rule which explicitly grants authorization to that topic.
> 2. A more extreme and perhaps safer option might be to prevent all writes to 
> this topic (and potentially other internal topics) through the Produce API. 
> Do we have any use cases which actually require writing to 
> __consumer_offsets? The only potential case that comes to mind is replication.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (KAFKA-168) Support locality in consumer partition assignment algorithm

[Dominic Chambers (JIRA)]

[ 
https://issues.apache.org/jira/browse/KAFKA-168?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15321545#comment-15321545
 ] 

Dominic Chambers commented on KAFKA-168:


Jay, I'd really appreciate knowing why you closed this issue. I've read 
everything I can find on the web and watched loads of your videos, but still 
can't figure out why co-locating your processing next to the data it's going to 
process isn't a more widely requested feature. Is there some fundamental reason 
why this is a bad idea, is it just not technically possible due to Kafka's 
architecture, or is still something you'd still accept a patch for?

> Support locality in consumer partition assignment algorithm
> ---
>
> Key: KAFKA-168
> URL: https://issues.apache.org/jira/browse/KAFKA-168
> Project: Kafka
>  Issue Type: Sub-task
>  Components: core
>Reporter: Jay Kreps
>
> There are some use-cases where it makes sense to co-locate brokers and 
> consumer processes. In this case it would be nice to optimize the assignment 
> of partitions to consumers so that the consumer preferentially consumes from 
> the broker with which it is co-located.
> If we are going to do KAFKA-167, moving the assignment to the broker, it 
> would make sense to do that first so we only have to change the logic in one 
> place.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (KAFKA-3753) Metrics for StateStores

[Guozhang Wang (JIRA)]

[ 
https://issues.apache.org/jira/browse/KAFKA-3753?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15321511#comment-15321511
 ] 

Guozhang Wang commented on KAFKA-3753:
--

Yeah it's a good idea to add "size" to the KeyValueStore interface.

> Metrics for StateStores
> ---
>
> Key: KAFKA-3753
> URL: https://issues.apache.org/jira/browse/KAFKA-3753
> Project: Kafka
>  Issue Type: Improvement
>  Components: streams
>Reporter: Jeff Klukas
>Assignee: Guozhang Wang
>Priority: Minor
>  Labels: api
> Fix For: 0.10.1.0
>
>
> As a developer building a Kafka Streams application, I'd like to have 
> visibility into what's happening with my state stores. How can I know if a 
> particular store is growing large? How can I know if a particular store is 
> frequently needing to hit disk?
> I'm interested to know if there are existing mechanisms for extracting this 
> information or if other people have thoughts on how we might approach this.
> I can't think of a way to provide metrics generically, so each state store 
> implementation would likely need to handle this separately. Given that the 
> default RocksDBStore will likely be the most-used, it would be a first target 
> for adding metrics.
> I'd be interested in knowing the total number of entries in the store, the 
> total size on disk and in memory, rates of gets and puts, and hit/miss ratio 
> for the MemoryLRUCache. Some of these numbers are likely calculable through 
> the RocksDB API, others may simply not be accessible.
> Would there be value to the wider community in having state stores register 
> metrics?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (KAFKA-2003) Add upgrade tests

[Geoff Anderson (JIRA)]

 [ 
https://issues.apache.org/jira/browse/KAFKA-2003?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Geoff Anderson resolved KAFKA-2003.
---
Resolution: Fixed

Several upgrade tests have been in place now for quite some time.

> Add upgrade tests
> -
>
> Key: KAFKA-2003
> URL: https://issues.apache.org/jira/browse/KAFKA-2003
> Project: Kafka
>  Issue Type: Improvement
>  Components: system tests
>Reporter: Gwen Shapira
>Assignee: Geoff Anderson
>
> To test protocol changes, compatibility and upgrade process, we need a good 
> way to test different versions of the product together and to test end-to-end 
> upgrade process.
> For example, for 0.8.2 to 0.8.3 test we want to check:
> * Can we start a cluster with a mix of 0.8.2 and 0.8.3 brokers?
> * Can a cluster of 0.8.3 brokers bump the protocol level one broker at a time?
> * Can 0.8.2 clients run against a cluster of 0.8.3 brokers?
> There are probably more questions. But an automated framework that can test 
> those and report results will be a good start.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (KAFKA-3805) Running multiple instances of a Streams app on the same machine results in Java_org_rocksdb_RocksDB_write error

[Guozhang Wang (JIRA)]

[ 
https://issues.apache.org/jira/browse/KAFKA-3805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15321500#comment-15321500
 ] 

Guozhang Wang commented on KAFKA-3805:
--

Thanks for reporting this [~agomez].

[~enothereska] Since different tasks have their own sub-directories under the 
application-id directory, I think ideally users should be able to use the same 
{{state.dir}}. Originally we use {{maybeCleanup}} instead of cleaning up the 
state directory right away during rebalance is to optimize the case where 
migrated-out tasks were migrated back shortly after (think of rolling bounce 
the instances for upgrade), and we only delete the task directory ONLY IF the 
thread can successfully grab the file lock. So as long as the other Kafka 
Streams instance's threads hold the file locks, they should not be deleted by 
other threads. Is there anything I missed?


> Running multiple instances of a Streams app on the same machine results in 
> Java_org_rocksdb_RocksDB_write error
> ---
>
> Key: KAFKA-3805
> URL: https://issues.apache.org/jira/browse/KAFKA-3805
> Project: Kafka
>  Issue Type: Bug
>  Components: streams
>Affects Versions: 0.10.0.0
> Environment: * Ubuntu Server 16.04 LTS
> - Java: OpenJDK Runtime Environment (build 
> 1.8.0_91-8u91-b14-0ubuntu4~16.04.1-b14)
> * OSX 10.11.5
>- Java: Java(TM) SE Runtime Environment (build 1.8.0_74-b02)
>Reporter: Andres Gomez Ferrer
>Assignee: Eno Thereska
>Priority: Critical
> Attachments: hs_err_pid23047.log
>
>
> I reproduce the error working with simple two instances of  my kafka streams 
> application reading from one topic writing on other and using a RocksDB store.
> I reproduce it starting one instance, sending some messages on the input 
> topic and later starting another Kafka streams instance. Then, the first 
> instance died and drop a core dump.
> --
>  A fatal error has been detected by the Java Runtime Environment:
>   SIGSEGV (0xb) at pc=0x7f9f488fd8e8, pid=23047, tid=140322171361024
>  JRE version: OpenJDK Runtime Environment (8.0_91-b14) (build 
> 1.8.0_91-8u91-b14-0ubuntu4~16.04.1-b14)
>  Java VM: OpenJDK 64-Bit Server VM (25.91-b14 mixed mode linux-amd64 
> compressed oops)
>  Problematic frame:
>  C  [librocksdbjni712288404493406713..so+0x15b8e8]  
> Java_org_rocksdb_RocksDB_write0+0x48
>  Core dump written. Default location: /root/core or core.23047
>  An error report file with more information is saved as:
>  /root/hs_err_pid23047.log
>  If you would like to submit a bug report, please visit:
>http://bugreport.java.com/bugreport/crash.jsp
>  The crash happened outside the Java Virtual Machine in native code.
>  See problematic frame for where to report the bug.
> --
> The core dump is there: 
> https://s3-eu-west-1.amazonaws.com/arodriguezg-test/core
> Note: The core dump is collected at Ubuntu Server.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (KAFKA-3806) Adjust default values of log.retention.hours and offsets.retention.minutes

[James Cheng (JIRA)]

[ 
https://issues.apache.org/jira/browse/KAFKA-3806?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15321422#comment-15321422
 ] 

James Cheng commented on KAFKA-3806:


Does the offsets.retention.minutes timer apply to a per topic basis? Per 
partition basis? To any topic in for the consumer group?

That is, in the following scenario:
* a consumer listening to a 2 partition topic
* offsets.retention.minutes = 1 day
* partition 1 has no new messages in over a day
* partition 2 has a new message every couple minutes.

Will the offset for partition 1 get deleted, but the offset for partition 2 
remain present?

Similarly, in the following scenario:
* a consumer listening to a 2 topics
* offsets.retention.minutes = 1 day
* topic 1 has no new messages in over a day
* topic 2 has a new message every couple minutes.

Will the offset for topic 1 get deleted, but the offset for topic 2 remain 
present?

-James


> Adjust default values of log.retention.hours and offsets.retention.minutes
> --
>
> Key: KAFKA-3806
> URL: https://issues.apache.org/jira/browse/KAFKA-3806
> Project: Kafka
>  Issue Type: Improvement
>  Components: config
>Affects Versions: 0.9.0.1, 0.10.0.0
>Reporter: Michal Turek
>Priority: Minor
>
> Combination of default values of log.retention.hours (168 hours = 7 days) and 
> offsets.retention.minutes (1440 minutes = 1 day) may be dangerous in special 
> cases. Offset retention should be always greater than log retention.
> We have observed the following scenario and issue:
> - Producing of data to a topic was disabled two days ago by producer update, 
> topic wasn't deleted.
> - Consumer consumed all data and properly committed offsets to Kafka.
> - Consumer made no more offset commits for that topic because there was no 
> more incoming data and there was nothing to confirm. (We have auto-commit 
> disabled, I'm not sure how behaves enabled auto-commit.)
> - After one day: Kafka cleared too old offsets according to 
> offsets.retention.minutes.
> - After two days: Long-term running consumer was restarted after update, it 
> didn't find any committed offsets for that topic since they were deleted by 
> offsets.retention.minutes so it started consuming from the beginning.
> - The messages were still in Kafka due to larger log.retention.hours, about 5 
> days of messages were read again.
> Known workaround to solve this issue:
> - Explicitly configure log.retention.hours and offsets.retention.minutes, 
> don't use defaults.
> Proposals:
> - Prolong default value of offsets.retention.minutes to be at least twice 
> larger than log.retention.hours.
> - Check these values during Kafka startup and log a warning if 
> offsets.retention.minutes is smaller than log.retention.hours.
> - Add a note to migration guide about differences between storing of offsets 
> in ZooKeeper and Kafka (http://kafka.apache.org/documentation.html#upgrade).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (KAFKA-3808) Transient failure in ReplicaVerificationToolTest

[Geoff Anderson (JIRA)]

Geoff Anderson created KAFKA-3808:
-

 Summary: Transient failure in ReplicaVerificationToolTest
 Key: KAFKA-3808
 URL: https://issues.apache.org/jira/browse/KAFKA-3808
 Project: Kafka
  Issue Type: Bug
  Components: system tests
Reporter: Geoff Anderson


{code}
test_id:
2016-05-29--001.kafkatest.tests.tools.replica_verification_test.ReplicaVerificationToolTest.test_replica_lags
status: FAIL
run time:   1 minute 9.231 seconds


Timed out waiting to reach non-zero number of replica lags.
Traceback (most recent call last):
  File 
"/var/lib/jenkins/workspace/system-test-kafka/kafka/venv/local/lib/python2.7/site-packages/ducktape-0.5.1-py2.7.egg/ducktape/tests/runner.py",
 line 106, in run_all_tests
data = self.run_single_test()
  File 
"/var/lib/jenkins/workspace/system-test-kafka/kafka/venv/local/lib/python2.7/site-packages/ducktape-0.5.1-py2.7.egg/ducktape/tests/runner.py",
 line 162, in run_single_test
return self.current_test_context.function(self.current_test)
  File 
"/var/lib/jenkins/workspace/system-test-kafka/kafka/tests/kafkatest/tests/tools/replica_verification_test.py",
 line 88, in test_replica_lags
err_msg="Timed out waiting to reach non-zero number of replica lags.")
  File 
"/var/lib/jenkins/workspace/system-test-kafka/kafka/venv/local/lib/python2.7/site-packages/ducktape-0.5.1-py2.7.egg/ducktape/utils/util.py",
 line 36, in wait_until
raise TimeoutError(err_msg)
TimeoutError: Timed out waiting to reach non-zero number of replica lags.
{code}

http://confluent-kafka-system-test-results.s3-us-west-2.amazonaws.com/2016-05-29--001.1464540508--apache--trunk--404b696/report.html



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (KAFKA-3807) OffsetValidationTest - transient failure on test_broker_rolling_bounce

[Geoff Anderson (JIRA)]

[ 
https://issues.apache.org/jira/browse/KAFKA-3807?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15321404#comment-15321404
 ] 

Geoff Anderson commented on KAFKA-3807:
---

ping [~hachikuji]

> OffsetValidationTest - transient failure on test_broker_rolling_bounce
> --
>
> Key: KAFKA-3807
> URL: https://issues.apache.org/jira/browse/KAFKA-3807
> Project: Kafka
>  Issue Type: Bug
>  Components: system tests
>Reporter: Geoff Anderson
>
> {code}
> test_id:
> 2016-05-28--001.kafkatest.tests.client.consumer_test.OffsetValidationTest.test_broker_rolling_bounce
> status: FAIL
> run time:   3 minutes 8.042 seconds
> Broker rolling bounce caused 2 unexpected group rebalances
> Traceback (most recent call last):
>   File 
> "/var/lib/jenkins/workspace/system-test-kafka/kafka/venv/local/lib/python2.7/site-packages/ducktape-0.5.1-py2.7.egg/ducktape/tests/runner.py",
>  line 106, in run_all_tests
> data = self.run_single_test()
>   File 
> "/var/lib/jenkins/workspace/system-test-kafka/kafka/venv/local/lib/python2.7/site-packages/ducktape-0.5.1-py2.7.egg/ducktape/tests/runner.py",
>  line 162, in run_single_test
> return self.current_test_context.function(self.current_test)
>   File 
> "/var/lib/jenkins/workspace/system-test-kafka/kafka/tests/kafkatest/tests/client/consumer_test.py",
>  line 108, in test_broker_rolling_bounce
> "Broker rolling bounce caused %d unexpected group rebalances" % 
> unexpected_rebalances
> AssertionError: Broker rolling bounce caused 2 unexpected group rebalances
> {code}
> http://confluent-kafka-system-test-results.s3-us-west-2.amazonaws.com/2016-05-28--001.1464455059--apache--trunk--7b7c4a7/report.html



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (KAFKA-3807) OffsetValidationTest - transient failure on test_broker_rolling_bounce

[Geoff Anderson (JIRA)]

Geoff Anderson created KAFKA-3807:
-

 Summary: OffsetValidationTest - transient failure on 
test_broker_rolling_bounce
 Key: KAFKA-3807
 URL: https://issues.apache.org/jira/browse/KAFKA-3807
 Project: Kafka
  Issue Type: Bug
  Components: system tests
Reporter: Geoff Anderson


{code}
test_id:
2016-05-28--001.kafkatest.tests.client.consumer_test.OffsetValidationTest.test_broker_rolling_bounce
status: FAIL
run time:   3 minutes 8.042 seconds


Broker rolling bounce caused 2 unexpected group rebalances
Traceback (most recent call last):
  File 
"/var/lib/jenkins/workspace/system-test-kafka/kafka/venv/local/lib/python2.7/site-packages/ducktape-0.5.1-py2.7.egg/ducktape/tests/runner.py",
 line 106, in run_all_tests
data = self.run_single_test()
  File 
"/var/lib/jenkins/workspace/system-test-kafka/kafka/venv/local/lib/python2.7/site-packages/ducktape-0.5.1-py2.7.egg/ducktape/tests/runner.py",
 line 162, in run_single_test
return self.current_test_context.function(self.current_test)
  File 
"/var/lib/jenkins/workspace/system-test-kafka/kafka/tests/kafkatest/tests/client/consumer_test.py",
 line 108, in test_broker_rolling_bounce
"Broker rolling bounce caused %d unexpected group rebalances" % 
unexpected_rebalances
AssertionError: Broker rolling bounce caused 2 unexpected group rebalances
{code}

http://confluent-kafka-system-test-results.s3-us-west-2.amazonaws.com/2016-05-28--001.1464455059--apache--trunk--7b7c4a7/report.html



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (KAFKA-3806) Adjust default values of log.retention.hours and offsets.retention.minutes

[James Cheng (JIRA)]

[ 
https://issues.apache.org/jira/browse/KAFKA-3806?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15321377#comment-15321377
 ] 

James Cheng commented on KAFKA-3806:


Jun,

Any recommendations on how to structure an application such that it commits 
often enough, but not so much that it spams Kafka with committed offsets?

We had an application here recently that would do something like

{code}
messages = poll(100); // 100ms
process(messages)
commitOffsets();
{code}

If the topic had no new messages, we would end up committing offsets 10 times a 
second. We had 38 instances of this application, each having multiple threads, 
processing a 12-partition topic. We ended up with 12,000 messages per second 
being written to the __consumer_offsets topic. The aggregate size was only 
2MB/sec tho.

On the one hand, Kafka was able to absorb this load fine, and as you say, we 
don't need to worry about offsets going away. On the other hand, it seems 
wasteful.

Any thoughts, or is that just the necessary tradeoff? If we want to spam the 
logs less often, we could change our applications to:
{code}
messages = poll(100); // 100ms
process(messages)
if (messages.length > 0 || enoughTimeHasPassedSinceLastCommit) {
commitOffsets();
}
{code}

That code isn't that difficult, but it is also logic that we would have to 
repeat in all of our consumers.

> Adjust default values of log.retention.hours and offsets.retention.minutes
> --
>
> Key: KAFKA-3806
> URL: https://issues.apache.org/jira/browse/KAFKA-3806
> Project: Kafka
>  Issue Type: Improvement
>  Components: config
>Affects Versions: 0.9.0.1, 0.10.0.0
>Reporter: Michal Turek
>Priority: Minor
>
> Combination of default values of log.retention.hours (168 hours = 7 days) and 
> offsets.retention.minutes (1440 minutes = 1 day) may be dangerous in special 
> cases. Offset retention should be always greater than log retention.
> We have observed the following scenario and issue:
> - Producing of data to a topic was disabled two days ago by producer update, 
> topic wasn't deleted.
> - Consumer consumed all data and properly committed offsets to Kafka.
> - Consumer made no more offset commits for that topic because there was no 
> more incoming data and there was nothing to confirm. (We have auto-commit 
> disabled, I'm not sure how behaves enabled auto-commit.)
> - After one day: Kafka cleared too old offsets according to 
> offsets.retention.minutes.
> - After two days: Long-term running consumer was restarted after update, it 
> didn't find any committed offsets for that topic since they were deleted by 
> offsets.retention.minutes so it started consuming from the beginning.
> - The messages were still in Kafka due to larger log.retention.hours, about 5 
> days of messages were read again.
> Known workaround to solve this issue:
> - Explicitly configure log.retention.hours and offsets.retention.minutes, 
> don't use defaults.
> Proposals:
> - Prolong default value of offsets.retention.minutes to be at least twice 
> larger than log.retention.hours.
> - Check these values during Kafka startup and log a warning if 
> offsets.retention.minutes is smaller than log.retention.hours.
> - Add a note to migration guide about differences between storing of offsets 
> in ZooKeeper and Kafka (http://kafka.apache.org/documentation.html#upgrade).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: [VOTE] KIP-55: Secure quotas for authenticated users

[Rajini Sivaram]

Jun,

Oops, sorry, I hadn't realized that the last note was on the discuss
thread. Thank you for pointing it out. I have sent another note for voting.


On Wed, Jun 8, 2016 at 4:30 PM, Jun Rao  wrote:

> Rajini,
>
> Perhaps it will be clearer if you start the voting in a new thread (with
> VOTE in the subject).
>
> Thanks,
>
> Jun
>
> On Tue, Jun 7, 2016 at 1:55 PM, Rajini Sivaram <
> rajinisiva...@googlemail.com
> > wrote:
>
> > I would like to initiate the vote for KIP-55.
> >
> > The KIP details are here: KIP-55: Secure quotas for authenticated users
> > <
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-55%3A+Secure+Quotas+for+Authenticated+Users
> > >
> > .
> >
> > The JIRA  KAFKA-3492   > >has
> > a draft PR here: https://github.com/apache/kafka/pull/1256.
> >
> > Thank you...
> >
> >
> > Regards,
> >
> > Rajini
> >
>



-- 
Regards,

Rajini

[VOTE] KIP-55: Secure quotas for authenticated users

[Rajini Sivaram]

I would like to initiate the vote for KIP-55.

The KIP details are here: KIP-55: Secure quotas for authenticated users

.

The JIRA  KAFKA-3492  has
a draft PR here: https://github.com/apache/kafka/pull/1256.

Thank you...

Regards,

Rajini

[GitHub] kafka pull request #1481: MINOR: Pass absolute directory path to RocksDB.ope...

[jklukas]

GitHub user jklukas opened a pull request:

https://github.com/apache/kafka/pull/1481

MINOR: Pass absolute directory path to RocksDB.open

The method `RocksDB.open` assumes an absolute file path. If a relative path 
is configured, it leads to an exception like the following:
```
org.apache.kafka.streams.errors.ProcessorStateException: Error opening 
store CustomerIdToUserIdLookup at location 
./tmp/rocksdb/CustomerIdToUserIdLookup
at org.rocksdb.RocksDB.open(Native Method)
at org.rocksdb.RocksDB.open(RocksDB.java:183)
at 
org.apache.kafka.streams.state.internals.RocksDBStore.openDB(RocksDBStore.java:214)
at 
org.apache.kafka.streams.state.internals.RocksDBStore.openDB(RocksDBStore.java:165)
at 
org.apache.kafka.streams.state.internals.RocksDBStore.init(RocksDBStore.java:170)
at 
org.apache.kafka.streams.state.internals.MeteredKeyValueStore.init(MeteredKeyValueStore.java:85)
at 
org.apache.kafka.test.KStreamTestDriver.(KStreamTestDriver.java:64)
at 
org.apache.kafka.test.KStreamTestDriver.(KStreamTestDriver.java:50)
at 
com.simple.estuary.transform.streaming.CartesianTransactionEnrichmentJobTest.testBuilder(CartesianTransactionEnrichmentJobTest.java:41)
```

Is there any risk to always fetching the absolute path as proposed here?

Let me know if you think this requires a JIRA issue or a unit test. I 
started working on a unit test, but don't know of a great solution for writing 
out a file to a relative directory.

This contribution is my original work and I license the work to the project 
under the project's open source license.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/jklukas/kafka rocksdb-abspath

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/kafka/pull/1481.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1481






---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Re: Kafka Streams aggregation store

[Eno Thereska]

Hi Alexander,

I haven't tried Kafka Streams on Windows but did notice that Microsoft has 
merged code into github to make RocksDB available on Windows. Perhaps this is 
useful:
https://blogs.msdn.microsoft.com/bingdevcenter/2015/07/22/open-source-contribution-from-bing-rocksdb-is-now-available-in-windows-platform/

Thanks,
Eno

> On 8 Jun 2016, at 19:47, Alexander Jipa  wrote:
> 
> Hello,
> According to 
> http://www.confluent.io/blog/introducing-kafka-streams-stream-processing-made-simple:
> “In terms of implementation Kafka Streams stores this derived aggregation in 
> a local embedded key-value store (RocksDB by default, but you can plug in 
> anything).”
> 
> So I tried running the world count example on my Windows machine (for local 
> test) and got an error because RocksDB is not available for windows.
> I thought it would be easy to switch to an in-memory store.
> But after awhile I’ve figured out that the KStream aggregation implementation 
> doesn’t allow that.
> It looks like aggregateByKey (and thus countByKey) is always using a 
> persistent store.
> More over that it looks like there’s no way to change the default persistent 
> store…
> 
> Even though I was more or less capable of achieving the goal using manual 
> wiring of a Source, a Producer and a Sink – it doesn’t make it for an easy 
> coding…
> 
> The questions that I have are:
> -  Is there a plan of providing a persistent store support for Kafka 
> Streams on Windows?
> -  Is there a plan of providing KStream API to specify a custom 
> store/factory for aggregations?
> -  Is there a way of changing the default persistent store from 
> RocksDB?
> 
> Best Regards,
> Alexander Jipa

Kafka Streams aggregation store

[Alexander Jipa]

Hello,
According to 
http://www.confluent.io/blog/introducing-kafka-streams-stream-processing-made-simple:
“In terms of implementation Kafka Streams stores this derived aggregation in a 
local embedded key-value store (RocksDB by default, but you can plug in 
anything).”
 
So I tried running the world count example on my Windows machine (for local 
test) and got an error because RocksDB is not available for windows.
I thought it would be easy to switch to an in-memory store.
But after awhile I’ve figured out that the KStream aggregation implementation 
doesn’t allow that.
It looks like aggregateByKey (and thus countByKey) is always using a persistent 
store.
More over that it looks like there’s no way to change the default persistent 
store…
 
Even though I was more or less capable of achieving the goal using manual 
wiring of a Source, a Producer and a Sink – it doesn’t make it for an easy 
coding…
 
The questions that I have are:
-  Is there a plan of providing a persistent store support for Kafka 
Streams on Windows?
-  Is there a plan of providing KStream API to specify a custom 
store/factory for aggregations?
-  Is there a way of changing the default persistent store from RocksDB?
 
Best Regards,
Alexander Jipa

Jenkins build is back to normal : kafka-trunk-jdk8 #690

[Apache Jenkins Server]

See 

[jira] [Commented] (KAFKA-3788) Potential message lost when switching to new segment

[Jay Kreps (JIRA)]

[ 
https://issues.apache.org/jira/browse/KAFKA-3788?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15321156#comment-15321156
 ] 

Jay Kreps commented on KAFKA-3788:
--

You are correct--unix doesn't guarantee either the ordering or writes or even 
the contents of unflushed portions of a file (it is legal to write garbage 
bytes until you call fsync). However, this is handled in the design of the log 
I believe. The flush is asynchronous, but we recover from the last flush point 
and checksum all unflushed messages. If any messages fails to pass the checksum 
validation we truncate the log from that point on and restore off the replicas.

Synchronous flush does not work and is not an option as it would stall the roll 
of the file for the time required to flush. For a 1GB segment this could take 
quite some time.

> Potential message lost when switching to new segment
> 
>
> Key: KAFKA-3788
> URL: https://issues.apache.org/jira/browse/KAFKA-3788
> Project: Kafka
>  Issue Type: Bug
>  Components: log
>Affects Versions: 0.9.0.0, 0.9.0.1, 0.10.0.0
>Reporter: Arkadiusz Firus
>Assignee: Jay Kreps
>Priority: Minor
>  Labels: easyfix
>   Original Estimate: 1h
>  Remaining Estimate: 1h
>
> If a new segment is needed method roll() from class kafka.log.Log is invoked. 
> It prepares new segment and schedules _asynchronous_ flush of the previous 
> segment.
> Asynchronous call can lead to a problematic situation. As far as I know 
> neither Linux nor Windows guarantees that the order of files persisted to 
> disk will be the same as the order of writes to files. This means that 
> records from the new segment can be flushed before the old ones which in case 
> of power outage can lead to gaps between records.
> Changing asynchronous invocation to synchronous one will solve the problem 
> because we have guarantee that all records from the previous segment will be 
> persisted to hard drive before we write any record to the new segment.
> I am guessing that asynchronous invocation was chosen to increase performance 
> but switching between segments is not so often. So it is not a big gain.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Comment Edited] (KAFKA-2904) Consumer Fails to Reconnect after 30s post restarts

[mjuarez (JIRA)]

[ 
https://issues.apache.org/jira/browse/KAFKA-2904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15321094#comment-15321094
 ] 

mjuarez edited comment on KAFKA-2904 at 6/8/16 6:22 PM:


I'm seeing this issue repeatedly when leaving a Kafka consumer running for 
anything longer than a few hours, on a small volume topic (~2400 
messages/second).  This is on Kafka 0.9.0.1 brokers, using the Java 0.9.0.1 
client jars.

{quote}
2016-06-08 09:52:02,633 
org.apache.kafka.clients.consumer.internals.ConsumerCoordinator 
[pool-2-thread-1] ERROR  Error ILLEGAL_GENERATION occurred while committing 
offsets for group TEST1_haymaker_to_hdfs
org.apache.kafka.clients.consumer.CommitFailedException: Commit cannot be 
completed due to group rebalance
{quote}

After that, the app logs start getting flooded with this unhelpful error 
message:

{quote}
2016-06-08 09:52:11,321 
org.apache.kafka.clients.consumer.internals.ConsumerCoordinator 
[pool-2-thread-1] ERROR  Offset commit failed.
org.apache.kafka.clients.consumer.internals.SendFailedException
{quote}

I have confirmed that the application is still consuming and committing offsets 
successfully, but it seems the ConsumerCoordinator is stuck trying to commit an 
offset, and failing repeatedly.


was (Author: mjuarez):
I'm seeing this issue repeatedly when leaving a Kafka consumer running for 
anything longer than a few hours, on a small volume topic (~2400 
messages/second).  This is on Kafka 0.9.0.1 brokers, using the Java 0.9.0.1 
client jars.

{quote}
2016-06-08 09:52:02,633 
org.apache.kafka.clients.consumer.internals.ConsumerCoordinator 
[pool-2-thread-1] ERROR  Error ILLEGAL_GENERATION occurred while committing 
offsets for group TEST1_haymaker_to_hdfs
org.apache.kafka.clients.consumer.CommitFailedException: Commit cannot be 
completed due to group rebalance
{quote}


> Consumer Fails to Reconnect after 30s post restarts
> ---
>
> Key: KAFKA-2904
> URL: https://issues.apache.org/jira/browse/KAFKA-2904
> Project: Kafka
>  Issue Type: Bug
>Reporter: Ben Stopford
>Assignee: Ben Stopford
> Attachments: 2015-11-27--001 (1).tar.gz
>
>
> This problem occurs in around 1 in 20 executions of the security rolling 
> upgrade test. 
> Test scenario is a rolling upgrade where each of the three servers are 
> restarted in turn whilst producer and consumers run. A ten second sleep 
> between start and stop of each node has been added to ensure there is time 
> for failover to occur (re KAFKA-2827). 
> Failure results in no consumed messages after the failure point. 
> Periodically the consumer does not reconnect for its 30s timeout. The 
> consumer’s log at this point is at the bottom of this jira.
> ISR's appear normal at the time of the failure.
> The producer is able to produce throughout this period. 
> *TIMELINE:*
> {quote}
> 20:39:23 - Test starts Consumer and Producer
> 20:39:27 - Consumer starts consuming produced messages
> 20:39:30 - Node 1 shutdown complete
> 20:39:45 - Node 1 restarts
> 20:39:59 - Node 2 shutdown complete
> 20:40:14 - Node 2 restarts 
> 20:40:27 - Consumer stops consuming
> 20:40:28 - Node 2 becomes controller
> 20:40:28 - Node 3 shutdown complete
> 20:40:34 - GroupCoordinator 2: Preparing to restabilize group 
> unique-test-group...
> 20:40:42 - Node 3 restarts
> *20:41:03 - Consumer times out*
> 20:41:03 - GroupCoordinator 2: Stabilized group unique-test-group...
> 20:41:03 - GroupCoordinator 2: Assignment received from leader for group 
> unique-test-group...
> 20:41:03 - GroupCoordinator 2: Preparing to restabilize group 
> unique-test-group...
> 20:41:03 - GroupCoordinator 2: Group unique-test-group... is dead and removed 
> 20:41:53 - Producer shuts down
> {quote}
> Consumer log at time of failure:
> {quote}
> [2015-11-27 20:40:27,268] INFO Current consumption count is 10100 
> (kafka.tools.ConsoleConsumer$)
> [2015-11-27 20:40:27,321] ERROR Error ILLEGAL_GENERATION occurred while 
> committing offsets for group unique-test-group-0.952644842527 
> (org.apache.kafka.clients.consumer.internals.ConsumerCoordinator)
> [2015-11-27 20:40:27,321] WARN Auto offset commit failed: Commit cannot be 
> completed due to group rebalance 
> (org.apache.kafka.clients.consumer.internals.ConsumerCoordinator)
> [2015-11-27 20:40:27,322] ERROR Error ILLEGAL_GENERATION occurred while 
> committing offsets for group unique-test-group-0.952644842527 
> (org.apache.kafka.clients.consumer.internals.ConsumerCoordinator)
> [2015-11-27 20:40:27,322] WARN Auto offset commit failed:  
> (org.apache.kafka.clients.consumer.internals.ConsumerCoordinator)
> [2015-11-27 20:40:27,329] INFO Attempt to join group 
> unique-test-group-0.952644842527 failed due to unknown member id, resetting 
> and retrying. 
> 

Jenkins build is back to normal : kafka-trunk-jdk7 #1355

[Apache Jenkins Server]

See 

[jira] [Commented] (KAFKA-2904) Consumer Fails to Reconnect after 30s post restarts

[mjuarez (JIRA)]

[ 
https://issues.apache.org/jira/browse/KAFKA-2904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15321094#comment-15321094
 ] 

mjuarez commented on KAFKA-2904:


I'm seeing this issue repeatedly when leaving a Kafka consumer running for 
anything longer than a few hours, on a small volume topic (~2400 
messages/second).  This is on Kafka 0.9.0.1 brokers, using the Java 0.9.0.1 
client jars.

{quote}
2016-06-08 09:52:02,633 
org.apache.kafka.clients.consumer.internals.ConsumerCoordinator 
[pool-2-thread-1] ERROR  Error ILLEGAL_GENERATION occurred while committing 
offsets for group TEST1_haymaker_to_hdfs
org.apache.kafka.clients.consumer.CommitFailedException: Commit cannot be 
completed due to group rebalance
{quote}


> Consumer Fails to Reconnect after 30s post restarts
> ---
>
> Key: KAFKA-2904
> URL: https://issues.apache.org/jira/browse/KAFKA-2904
> Project: Kafka
>  Issue Type: Bug
>Reporter: Ben Stopford
>Assignee: Ben Stopford
> Attachments: 2015-11-27--001 (1).tar.gz
>
>
> This problem occurs in around 1 in 20 executions of the security rolling 
> upgrade test. 
> Test scenario is a rolling upgrade where each of the three servers are 
> restarted in turn whilst producer and consumers run. A ten second sleep 
> between start and stop of each node has been added to ensure there is time 
> for failover to occur (re KAFKA-2827). 
> Failure results in no consumed messages after the failure point. 
> Periodically the consumer does not reconnect for its 30s timeout. The 
> consumer’s log at this point is at the bottom of this jira.
> ISR's appear normal at the time of the failure.
> The producer is able to produce throughout this period. 
> *TIMELINE:*
> {quote}
> 20:39:23 - Test starts Consumer and Producer
> 20:39:27 - Consumer starts consuming produced messages
> 20:39:30 - Node 1 shutdown complete
> 20:39:45 - Node 1 restarts
> 20:39:59 - Node 2 shutdown complete
> 20:40:14 - Node 2 restarts 
> 20:40:27 - Consumer stops consuming
> 20:40:28 - Node 2 becomes controller
> 20:40:28 - Node 3 shutdown complete
> 20:40:34 - GroupCoordinator 2: Preparing to restabilize group 
> unique-test-group...
> 20:40:42 - Node 3 restarts
> *20:41:03 - Consumer times out*
> 20:41:03 - GroupCoordinator 2: Stabilized group unique-test-group...
> 20:41:03 - GroupCoordinator 2: Assignment received from leader for group 
> unique-test-group...
> 20:41:03 - GroupCoordinator 2: Preparing to restabilize group 
> unique-test-group...
> 20:41:03 - GroupCoordinator 2: Group unique-test-group... is dead and removed 
> 20:41:53 - Producer shuts down
> {quote}
> Consumer log at time of failure:
> {quote}
> [2015-11-27 20:40:27,268] INFO Current consumption count is 10100 
> (kafka.tools.ConsoleConsumer$)
> [2015-11-27 20:40:27,321] ERROR Error ILLEGAL_GENERATION occurred while 
> committing offsets for group unique-test-group-0.952644842527 
> (org.apache.kafka.clients.consumer.internals.ConsumerCoordinator)
> [2015-11-27 20:40:27,321] WARN Auto offset commit failed: Commit cannot be 
> completed due to group rebalance 
> (org.apache.kafka.clients.consumer.internals.ConsumerCoordinator)
> [2015-11-27 20:40:27,322] ERROR Error ILLEGAL_GENERATION occurred while 
> committing offsets for group unique-test-group-0.952644842527 
> (org.apache.kafka.clients.consumer.internals.ConsumerCoordinator)
> [2015-11-27 20:40:27,322] WARN Auto offset commit failed:  
> (org.apache.kafka.clients.consumer.internals.ConsumerCoordinator)
> [2015-11-27 20:40:27,329] INFO Attempt to join group 
> unique-test-group-0.952644842527 failed due to unknown member id, resetting 
> and retrying. 
> (org.apache.kafka.clients.consumer.internals.AbstractCoordinator)
> [2015-11-27 20:40:27,347] INFO SyncGroup for group 
> unique-test-group-0.952644842527 failed due to UNKNOWN_MEMBER_ID, rejoining 
> the group (org.apache.kafka.clients.consumer.internals.AbstractCoordinator)
> [2015-11-27 20:40:27,357] INFO SyncGroup for group 
> unique-test-group-0.952644842527 failed due to NOT_COORDINATOR_FOR_GROUP, 
> will find new coordinator and rejoin 
> (org.apache.kafka.clients.consumer.internals.AbstractCoordinator)
> [2015-11-27 20:40:27,357] INFO Marking the coordinator 2147483644 dead. 
> (org.apache.kafka.clients.consumer.internals.AbstractCoordinator)
> [2015-11-27 20:40:28,097] INFO Attempt to join group 
> unique-test-group-0.952644842527 failed due to unknown member id, resetting 
> and retrying. 
> (org.apache.kafka.clients.consumer.internals.AbstractCoordinator)
> [2015-11-27 20:40:33,627] INFO Marking the coordinator 2147483646 dead. 
> (org.apache.kafka.clients.consumer.internals.AbstractCoordinator)
> [2015-11-27 20:40:33,627] INFO Attempt to join group 
> unique-test-group-0.952644842527 failed due to obsolete coordinator 
> information, retrying. 
> 

[jira] [Commented] (KAFKA-3788) Potential message lost when switching to new segment

[Bharat Viswanadham (JIRA)]

[ 
https://issues.apache.org/jira/browse/KAFKA-3788?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15321014#comment-15321014
 ] 

Bharat Viswanadham commented on KAFKA-3788:
---

Can I take this task, to work on it?

> Potential message lost when switching to new segment
> 
>
> Key: KAFKA-3788
> URL: https://issues.apache.org/jira/browse/KAFKA-3788
> Project: Kafka
>  Issue Type: Bug
>  Components: log
>Affects Versions: 0.9.0.0, 0.9.0.1, 0.10.0.0
>Reporter: Arkadiusz Firus
>Assignee: Jay Kreps
>Priority: Minor
>  Labels: easyfix
>   Original Estimate: 1h
>  Remaining Estimate: 1h
>
> If a new segment is needed method roll() from class kafka.log.Log is invoked. 
> It prepares new segment and schedules _asynchronous_ flush of the previous 
> segment.
> Asynchronous call can lead to a problematic situation. As far as I know 
> neither Linux nor Windows guarantees that the order of files persisted to 
> disk will be the same as the order of writes to files. This means that 
> records from the new segment can be flushed before the old ones which in case 
> of power outage can lead to gaps between records.
> Changing asynchronous invocation to synchronous one will solve the problem 
> because we have guarantee that all records from the previous segment will be 
> persisted to hard drive before we write any record to the new segment.
> I am guessing that asynchronous invocation was chosen to increase performance 
> but switching between segments is not so often. So it is not a big gain.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (KAFKA-3806) Adjust default values of log.retention.hours and offsets.retention.minutes

[Jun Rao (JIRA)]

[ 
https://issues.apache.org/jira/browse/KAFKA-3806?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15320931#comment-15320931
 ] 

Jun Rao commented on KAFKA-3806:


offsets.retention.minutes is designed to handle the case that a consumer group 
goes away forever. In that case, we don't want to store the offsets for that 
group forever. That retention time potentially can be less than the log 
retention time. In your case, if the consumer is still alive, you would want to 
keep committing the offsets even though they are not changing. Once the 
consumer group coordinator sees the new offset commit, it will extend the time 
to preserve the offsets. This is what auto commit will do. 

> Adjust default values of log.retention.hours and offsets.retention.minutes
> --
>
> Key: KAFKA-3806
> URL: https://issues.apache.org/jira/browse/KAFKA-3806
> Project: Kafka
>  Issue Type: Improvement
>  Components: config
>Affects Versions: 0.9.0.1, 0.10.0.0
>Reporter: Michal Turek
>Priority: Minor
>
> Combination of default values of log.retention.hours (168 hours = 7 days) and 
> offsets.retention.minutes (1440 minutes = 1 day) may be dangerous in special 
> cases. Offset retention should be always greater than log retention.
> We have observed the following scenario and issue:
> - Producing of data to a topic was disabled two days ago by producer update, 
> topic wasn't deleted.
> - Consumer consumed all data and properly committed offsets to Kafka.
> - Consumer made no more offset commits for that topic because there was no 
> more incoming data and there was nothing to confirm. (We have auto-commit 
> disabled, I'm not sure how behaves enabled auto-commit.)
> - After one day: Kafka cleared too old offsets according to 
> offsets.retention.minutes.
> - After two days: Long-term running consumer was restarted after update, it 
> didn't find any committed offsets for that topic since they were deleted by 
> offsets.retention.minutes so it started consuming from the beginning.
> - The messages were still in Kafka due to larger log.retention.hours, about 5 
> days of messages were read again.
> Known workaround to solve this issue:
> - Explicitly configure log.retention.hours and offsets.retention.minutes, 
> don't use defaults.
> Proposals:
> - Prolong default value of offsets.retention.minutes to be at least twice 
> larger than log.retention.hours.
> - Check these values during Kafka startup and log a warning if 
> offsets.retention.minutes is smaller than log.retention.hours.
> - Add a note to migration guide about differences between storing of offsets 
> in ZooKeeper and Kafka (http://kafka.apache.org/documentation.html#upgrade).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (KAFKA-3800) java client can`t poll msg

[Ewen Cheslack-Postava (JIRA)]

[ 
https://issues.apache.org/jira/browse/KAFKA-3800?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15320886#comment-15320886
 ] 

Ewen Cheslack-Postava commented on KAFKA-3800:
--

[~frank_zg] Can you modify your code to loop and call poll repeatedly? You may 
simply not have received the data within the timeout you specified.

> java client can`t poll msg
> --
>
> Key: KAFKA-3800
> URL: https://issues.apache.org/jira/browse/KAFKA-3800
> Project: Kafka
>  Issue Type: Bug
>  Components: consumer
>Affects Versions: 0.9.0.1
> Environment: java8,win7 64
>Reporter: frank
>Assignee: Neha Narkhede
>
> i use hump topic name, after poll msg is null.eg: Test_4 why?
> all low char is ok. i`m try nodejs,kafka-console-consumers.bat is ok



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (KAFKA-3711) Allow configuration of MetricsReporter subclasses

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/KAFKA-3711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15320881#comment-15320881
 ] 

ASF GitHub Bot commented on KAFKA-3711:
---

Github user asfgit closed the pull request at:

https://github.com/apache/kafka/pull/1479


> Allow configuration of MetricsReporter subclasses
> -
>
> Key: KAFKA-3711
> URL: https://issues.apache.org/jira/browse/KAFKA-3711
> Project: Kafka
>  Issue Type: Improvement
>  Components: clients, streams
>Reporter: Jeff Klukas
>Assignee: Guozhang Wang
>Priority: Minor
> Fix For: 0.10.1.0
>
>
> The current interface for attaching metrics reporters to clients allows only 
> defining a list of class names, but provides no means for configuring those 
> reporters.
> There is at least one existing project 
> (https://github.com/apakulov/kafka-graphite) that solves this problem by 
> passing additional properties into the client, which then get passed on to 
> the reporter. This seems to work quite well, but it generates warnings like 
> {{The configuration kafka.graphite.metrics.prefix = foo was supplied but 
> isn't a known config.}}
> Should passing arbitrary additional parameters like this be officially 
> supported as the way to configure metrics reporters? Should these warnings 
> about unrecognized parameters be removed?
> Perhaps there should be some mechanism for registering additional 
> configuration parameters for clients to expect?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (KAFKA-3711) Allow configuration of MetricsReporter subclasses

[Ewen Cheslack-Postava (JIRA)]

 [ 
https://issues.apache.org/jira/browse/KAFKA-3711?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ewen Cheslack-Postava resolved KAFKA-3711.
--
Resolution: Fixed

Issue resolved by pull request 1479
[https://github.com/apache/kafka/pull/1479]

> Allow configuration of MetricsReporter subclasses
> -
>
> Key: KAFKA-3711
> URL: https://issues.apache.org/jira/browse/KAFKA-3711
> Project: Kafka
>  Issue Type: Improvement
>  Components: clients, streams
>Reporter: Jeff Klukas
>Assignee: Guozhang Wang
>Priority: Minor
> Fix For: 0.10.1.0
>
>
> The current interface for attaching metrics reporters to clients allows only 
> defining a list of class names, but provides no means for configuring those 
> reporters.
> There is at least one existing project 
> (https://github.com/apakulov/kafka-graphite) that solves this problem by 
> passing additional properties into the client, which then get passed on to 
> the reporter. This seems to work quite well, but it generates warnings like 
> {{The configuration kafka.graphite.metrics.prefix = foo was supplied but 
> isn't a known config.}}
> Should passing arbitrary additional parameters like this be officially 
> supported as the way to configure metrics reporters? Should these warnings 
> about unrecognized parameters be removed?
> Perhaps there should be some mechanism for registering additional 
> configuration parameters for clients to expect?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] kafka pull request #1479: KAFKA-3711: Ensure a RecordingMap is passed to con...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/kafka/pull/1479


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Comment Edited] (KAFKA-3805) Running multiple instances of a Streams app on the same machine results in Java_org_rocksdb_RocksDB_write error

[Andres Gomez Ferrer (JIRA)]

[ 
https://issues.apache.org/jira/browse/KAFKA-3805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15320871#comment-15320871
 ] 

Andres Gomez Ferrer edited comment on KAFKA-3805 at 6/8/16 4:31 PM:


[~enothereska] I'm trying to use different data.dir by Kafka streams 
application and the issue persists, and now I'm also trying to run the 
applications inside Linux containers so that they work in different disk spaces 
but the issue persist too. 


was (Author: agomez):
In addition: 

I'm trying to use different data.dir by Kafka streams application and the issue 
persists, and now I'm also trying to run the applications inside Linux 
containers so that they work in different disk spaces but the issue persist 
too. 

> Running multiple instances of a Streams app on the same machine results in 
> Java_org_rocksdb_RocksDB_write error
> ---
>
> Key: KAFKA-3805
> URL: https://issues.apache.org/jira/browse/KAFKA-3805
> Project: Kafka
>  Issue Type: Bug
>  Components: streams
>Affects Versions: 0.10.0.0
> Environment: * Ubuntu Server 16.04 LTS
> - Java: OpenJDK Runtime Environment (build 
> 1.8.0_91-8u91-b14-0ubuntu4~16.04.1-b14)
> * OSX 10.11.5
>- Java: Java(TM) SE Runtime Environment (build 1.8.0_74-b02)
>Reporter: Andres Gomez Ferrer
>Assignee: Eno Thereska
>Priority: Critical
> Attachments: hs_err_pid23047.log
>
>
> I reproduce the error working with simple two instances of  my kafka streams 
> application reading from one topic writing on other and using a RocksDB store.
> I reproduce it starting one instance, sending some messages on the input 
> topic and later starting another Kafka streams instance. Then, the first 
> instance died and drop a core dump.
> --
>  A fatal error has been detected by the Java Runtime Environment:
>   SIGSEGV (0xb) at pc=0x7f9f488fd8e8, pid=23047, tid=140322171361024
>  JRE version: OpenJDK Runtime Environment (8.0_91-b14) (build 
> 1.8.0_91-8u91-b14-0ubuntu4~16.04.1-b14)
>  Java VM: OpenJDK 64-Bit Server VM (25.91-b14 mixed mode linux-amd64 
> compressed oops)
>  Problematic frame:
>  C  [librocksdbjni712288404493406713..so+0x15b8e8]  
> Java_org_rocksdb_RocksDB_write0+0x48
>  Core dump written. Default location: /root/core or core.23047
>  An error report file with more information is saved as:
>  /root/hs_err_pid23047.log
>  If you would like to submit a bug report, please visit:
>http://bugreport.java.com/bugreport/crash.jsp
>  The crash happened outside the Java Virtual Machine in native code.
>  See problematic frame for where to report the bug.
> --
> The core dump is there: 
> https://s3-eu-west-1.amazonaws.com/arodriguezg-test/core
> Note: The core dump is collected at Ubuntu Server.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (KAFKA-3805) Running multiple instances of a Streams app on the same machine results in Java_org_rocksdb_RocksDB_write error

[Andres Gomez Ferrer (JIRA)]

[ 
https://issues.apache.org/jira/browse/KAFKA-3805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15320871#comment-15320871
 ] 

Andres Gomez Ferrer commented on KAFKA-3805:


In addition: 

I'm trying to use different data.dir by Kafka streams application and the issue 
persists, and now I'm also trying to run the applications inside Linux 
containers so that they work in different disk spaces but the issue persist 
too. 

> Running multiple instances of a Streams app on the same machine results in 
> Java_org_rocksdb_RocksDB_write error
> ---
>
> Key: KAFKA-3805
> URL: https://issues.apache.org/jira/browse/KAFKA-3805
> Project: Kafka
>  Issue Type: Bug
>  Components: streams
>Affects Versions: 0.10.0.0
> Environment: * Ubuntu Server 16.04 LTS
> - Java: OpenJDK Runtime Environment (build 
> 1.8.0_91-8u91-b14-0ubuntu4~16.04.1-b14)
> * OSX 10.11.5
>- Java: Java(TM) SE Runtime Environment (build 1.8.0_74-b02)
>Reporter: Andres Gomez Ferrer
>Assignee: Eno Thereska
>Priority: Critical
> Attachments: hs_err_pid23047.log
>
>
> I reproduce the error working with simple two instances of  my kafka streams 
> application reading from one topic writing on other and using a RocksDB store.
> I reproduce it starting one instance, sending some messages on the input 
> topic and later starting another Kafka streams instance. Then, the first 
> instance died and drop a core dump.
> --
>  A fatal error has been detected by the Java Runtime Environment:
>   SIGSEGV (0xb) at pc=0x7f9f488fd8e8, pid=23047, tid=140322171361024
>  JRE version: OpenJDK Runtime Environment (8.0_91-b14) (build 
> 1.8.0_91-8u91-b14-0ubuntu4~16.04.1-b14)
>  Java VM: OpenJDK 64-Bit Server VM (25.91-b14 mixed mode linux-amd64 
> compressed oops)
>  Problematic frame:
>  C  [librocksdbjni712288404493406713..so+0x15b8e8]  
> Java_org_rocksdb_RocksDB_write0+0x48
>  Core dump written. Default location: /root/core or core.23047
>  An error report file with more information is saved as:
>  /root/hs_err_pid23047.log
>  If you would like to submit a bug report, please visit:
>http://bugreport.java.com/bugreport/crash.jsp
>  The crash happened outside the Java Virtual Machine in native code.
>  See problematic frame for where to report the bug.
> --
> The core dump is there: 
> https://s3-eu-west-1.amazonaws.com/arodriguezg-test/core
> Note: The core dump is collected at Ubuntu Server.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (KAFKA-3805) Running multiple instances of a Streams app on the same machine results in Java_org_rocksdb_RocksDB_write error

[Eno Thereska (JIRA)]

[ 
https://issues.apache.org/jira/browse/KAFKA-3805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15320770#comment-15320770
 ] 

Eno Thereska commented on KAFKA-3805:
-

[~agomez] The issue seems to be that in StreamThread.java, the maybeClean 
method destroys everything in the `state.dir` if partitions are revoked from a 
task. Unfortunately, with 2 apps on the same machine, that will mean that a new 
task's files are also deleted (from the old task). The workaround for now is to 
use different state.dirs for each application (using 
StreamsConfig.STATE_DIR_CONFIG). Alternatively, parallelism is also achieved by 
using one app and multiple threads (using 
StreamsConfig.NUM_STREAM_THREADS_CONFIG). The default number of threads is one 
and you might want to increase that to match your number of CPUs.

> Running multiple instances of a Streams app on the same machine results in 
> Java_org_rocksdb_RocksDB_write error
> ---
>
> Key: KAFKA-3805
> URL: https://issues.apache.org/jira/browse/KAFKA-3805
> Project: Kafka
>  Issue Type: Bug
>  Components: streams
>Affects Versions: 0.10.0.0
> Environment: * Ubuntu Server 16.04 LTS
> - Java: OpenJDK Runtime Environment (build 
> 1.8.0_91-8u91-b14-0ubuntu4~16.04.1-b14)
> * OSX 10.11.5
>- Java: Java(TM) SE Runtime Environment (build 1.8.0_74-b02)
>Reporter: Andres Gomez Ferrer
>Assignee: Eno Thereska
>Priority: Critical
> Attachments: hs_err_pid23047.log
>
>
> I reproduce the error working with simple two instances of  my kafka streams 
> application reading from one topic writing on other and using a RocksDB store.
> I reproduce it starting one instance, sending some messages on the input 
> topic and later starting another Kafka streams instance. Then, the first 
> instance died and drop a core dump.
> --
>  A fatal error has been detected by the Java Runtime Environment:
>   SIGSEGV (0xb) at pc=0x7f9f488fd8e8, pid=23047, tid=140322171361024
>  JRE version: OpenJDK Runtime Environment (8.0_91-b14) (build 
> 1.8.0_91-8u91-b14-0ubuntu4~16.04.1-b14)
>  Java VM: OpenJDK 64-Bit Server VM (25.91-b14 mixed mode linux-amd64 
> compressed oops)
>  Problematic frame:
>  C  [librocksdbjni712288404493406713..so+0x15b8e8]  
> Java_org_rocksdb_RocksDB_write0+0x48
>  Core dump written. Default location: /root/core or core.23047
>  An error report file with more information is saved as:
>  /root/hs_err_pid23047.log
>  If you would like to submit a bug report, please visit:
>http://bugreport.java.com/bugreport/crash.jsp
>  The crash happened outside the Java Virtual Machine in native code.
>  See problematic frame for where to report the bug.
> --
> The core dump is there: 
> https://s3-eu-west-1.amazonaws.com/arodriguezg-test/core
> Note: The core dump is collected at Ubuntu Server.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: [VOTE] KIP-55: Secure quotas for authenticated users

[Jun Rao]

Rajini,

Perhaps it will be clearer if you start the voting in a new thread (with
VOTE in the subject).

Thanks,

Jun

On Tue, Jun 7, 2016 at 1:55 PM, Rajini Sivaram  wrote:

> I would like to initiate the vote for KIP-55.
>
> The KIP details are here: KIP-55: Secure quotas for authenticated users
> <
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-55%3A+Secure+Quotas+for+Authenticated+Users
> >
> .
>
> The JIRA  KAFKA-3492   >has
> a draft PR here: https://github.com/apache/kafka/pull/1256.
>
> Thank you...
>
>
> Regards,
>
> Rajini
>

[jira] [Created] (KAFKA-3806) Adjust default values of log.retention.hours and offsets.retention.minutes

[Michal Turek (JIRA)]

Michal Turek created KAFKA-3806:
---

 Summary: Adjust default values of log.retention.hours and 
offsets.retention.minutes
 Key: KAFKA-3806
 URL: https://issues.apache.org/jira/browse/KAFKA-3806
 Project: Kafka
  Issue Type: Improvement
  Components: config
Affects Versions: 0.10.0.0, 0.9.0.1
Reporter: Michal Turek
Priority: Minor


Combination of default values of log.retention.hours (168 hours = 7 days) and 
offsets.retention.minutes (1440 minutes = 1 day) may be dangerous in special 
cases. Offset retention should be always greater than log retention.

We have observed the following scenario and issue:

- Producing of data to a topic was disabled two days ago by producer update, 
topic wasn't deleted.
- Consumer consumed all data and properly committed offsets to Kafka.
- Consumer made no more offset commits for that topic because there was no more 
incoming data and there was nothing to confirm. (We have auto-commit disabled, 
I'm not sure how behaves enabled auto-commit.)
- After one day: Kafka cleared too old offsets according to 
offsets.retention.minutes.
- After two days: Long-term running consumer was restarted after update, it 
didn't find any committed offsets for that topic since they were deleted by 
offsets.retention.minutes so it started consuming from the beginning.
- The messages were still in Kafka due to larger log.retention.hours, about 5 
days of messages were read again.


Known workaround to solve this issue:

- Explicitly configure log.retention.hours and offsets.retention.minutes, don't 
use defaults.


Proposals:

- Prolong default value of offsets.retention.minutes to be at least twice 
larger than log.retention.hours.
- Check these values during Kafka startup and log a warning if 
offsets.retention.minutes is smaller than log.retention.hours.
- Add a note to migration guide about differences between storing of offsets in 
ZooKeeper and Kafka (http://kafka.apache.org/documentation.html#upgrade).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

kafka 9 group offset

[揣立武]

Hello! Now I am using kafka 0.9. The offset of group does not on zookeeper 
again, so how can I get the offset of group. 

1. when I use commond: "bin/kafka-run-class.sh 
kafka.tools.ConsumerOffsetChecker --zookeeper ..."
error like this: "ConsumerOffsetChecker is deprecated and will be dropped in 
releases following 0.9.0. Use ConsumerGroupCommand instead. 
(kafka.tools.ConsumerOffsetChecker$)"

2. when I use commond "bin/kafka-run-class.sh kafka.admin.ConsumerGroupCommand 
--zookeeper ..."
error like this: "Error while executing consumer group command 
org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = NoNode 
for /consumers"

can you help me? thx!

-- 
揣立武
美团网技术工程部基础服务组 
手机： 13522216605 
邮箱： chuail...@meituan.com 
地址： 北京市朝阳区望京东路6号望京国际研发园三期F座 
邮编： 100102

[jira] [Commented] (KAFKA-3805) Running multiple instances of a Streams app on the same machine results in Java_org_rocksdb_RocksDB_write error

[Andres Gomez Ferrer (JIRA)]

[ 
https://issues.apache.org/jira/browse/KAFKA-3805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15320570#comment-15320570
 ] 

Andres Gomez Ferrer commented on KAFKA-3805:


[~enothereska] you're welcome! :)

> Running multiple instances of a Streams app on the same machine results in 
> Java_org_rocksdb_RocksDB_write error
> ---
>
> Key: KAFKA-3805
> URL: https://issues.apache.org/jira/browse/KAFKA-3805
> Project: Kafka
>  Issue Type: Bug
>  Components: streams
>Affects Versions: 0.10.0.0
> Environment: * Ubuntu Server 16.04 LTS
> - Java: OpenJDK Runtime Environment (build 
> 1.8.0_91-8u91-b14-0ubuntu4~16.04.1-b14)
> * OSX 10.11.5
>- Java: Java(TM) SE Runtime Environment (build 1.8.0_74-b02)
>Reporter: Andres Gomez Ferrer
>Assignee: Eno Thereska
>Priority: Critical
> Attachments: hs_err_pid23047.log
>
>
> I reproduce the error working with simple two instances of  my kafka streams 
> application reading from one topic writing on other and using a RocksDB store.
> I reproduce it starting one instance, sending some messages on the input 
> topic and later starting another Kafka streams instance. Then, the first 
> instance died and drop a core dump.
> --
>  A fatal error has been detected by the Java Runtime Environment:
>   SIGSEGV (0xb) at pc=0x7f9f488fd8e8, pid=23047, tid=140322171361024
>  JRE version: OpenJDK Runtime Environment (8.0_91-b14) (build 
> 1.8.0_91-8u91-b14-0ubuntu4~16.04.1-b14)
>  Java VM: OpenJDK 64-Bit Server VM (25.91-b14 mixed mode linux-amd64 
> compressed oops)
>  Problematic frame:
>  C  [librocksdbjni712288404493406713..so+0x15b8e8]  
> Java_org_rocksdb_RocksDB_write0+0x48
>  Core dump written. Default location: /root/core or core.23047
>  An error report file with more information is saved as:
>  /root/hs_err_pid23047.log
>  If you would like to submit a bug report, please visit:
>http://bugreport.java.com/bugreport/crash.jsp
>  The crash happened outside the Java Virtual Machine in native code.
>  See problematic frame for where to report the bug.
> --
> The core dump is there: 
> https://s3-eu-west-1.amazonaws.com/arodriguezg-test/core
> Note: The core dump is collected at Ubuntu Server.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (KAFKA-3805) Running multiple instances of a Streams app on the same machine results in Java_org_rocksdb_RocksDB_write error

[Eno Thereska (JIRA)]

[ 
https://issues.apache.org/jira/browse/KAFKA-3805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15320550#comment-15320550
 ] 

Eno Thereska commented on KAFKA-3805:
-

[~agomez] Ok, your usage makes sense. I'll take a look, thanks for reporting.

> Running multiple instances of a Streams app on the same machine results in 
> Java_org_rocksdb_RocksDB_write error
> ---
>
> Key: KAFKA-3805
> URL: https://issues.apache.org/jira/browse/KAFKA-3805
> Project: Kafka
>  Issue Type: Bug
>  Components: streams
>Affects Versions: 0.10.0.0
> Environment: * Ubuntu Server 16.04 LTS
> - Java: OpenJDK Runtime Environment (build 
> 1.8.0_91-8u91-b14-0ubuntu4~16.04.1-b14)
> * OSX 10.11.5
>- Java: Java(TM) SE Runtime Environment (build 1.8.0_74-b02)
>Reporter: Andres Gomez Ferrer
>Assignee: Eno Thereska
>Priority: Critical
> Attachments: hs_err_pid23047.log
>
>
> I reproduce the error working with simple two instances of  my kafka streams 
> application reading from one topic writing on other and using a RocksDB store.
> I reproduce it starting one instance, sending some messages on the input 
> topic and later starting another Kafka streams instance. Then, the first 
> instance died and drop a core dump.
> --
>  A fatal error has been detected by the Java Runtime Environment:
>   SIGSEGV (0xb) at pc=0x7f9f488fd8e8, pid=23047, tid=140322171361024
>  JRE version: OpenJDK Runtime Environment (8.0_91-b14) (build 
> 1.8.0_91-8u91-b14-0ubuntu4~16.04.1-b14)
>  Java VM: OpenJDK 64-Bit Server VM (25.91-b14 mixed mode linux-amd64 
> compressed oops)
>  Problematic frame:
>  C  [librocksdbjni712288404493406713..so+0x15b8e8]  
> Java_org_rocksdb_RocksDB_write0+0x48
>  Core dump written. Default location: /root/core or core.23047
>  An error report file with more information is saved as:
>  /root/hs_err_pid23047.log
>  If you would like to submit a bug report, please visit:
>http://bugreport.java.com/bugreport/crash.jsp
>  The crash happened outside the Java Virtual Machine in native code.
>  See problematic frame for where to report the bug.
> --
> The core dump is there: 
> https://s3-eu-west-1.amazonaws.com/arodriguezg-test/core
> Note: The core dump is collected at Ubuntu Server.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Assigned] (KAFKA-3805) Running multiple instances of a Streams app on the same machine results in Java_org_rocksdb_RocksDB_write error

[Eno Thereska (JIRA)]

 [ 
https://issues.apache.org/jira/browse/KAFKA-3805?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Eno Thereska reassigned KAFKA-3805:
---

Assignee: Eno Thereska  (was: Guozhang Wang)

> Running multiple instances of a Streams app on the same machine results in 
> Java_org_rocksdb_RocksDB_write error
> ---
>
> Key: KAFKA-3805
> URL: https://issues.apache.org/jira/browse/KAFKA-3805
> Project: Kafka
>  Issue Type: Bug
>  Components: streams
>Affects Versions: 0.10.0.0
> Environment: * Ubuntu Server 16.04 LTS
> - Java: OpenJDK Runtime Environment (build 
> 1.8.0_91-8u91-b14-0ubuntu4~16.04.1-b14)
> * OSX 10.11.5
>- Java: Java(TM) SE Runtime Environment (build 1.8.0_74-b02)
>Reporter: Andres Gomez Ferrer
>Assignee: Eno Thereska
>Priority: Critical
> Attachments: hs_err_pid23047.log
>
>
> I reproduce the error working with simple two instances of  my kafka streams 
> application reading from one topic writing on other and using a RocksDB store.
> I reproduce it starting one instance, sending some messages on the input 
> topic and later starting another Kafka streams instance. Then, the first 
> instance died and drop a core dump.
> --
>  A fatal error has been detected by the Java Runtime Environment:
>   SIGSEGV (0xb) at pc=0x7f9f488fd8e8, pid=23047, tid=140322171361024
>  JRE version: OpenJDK Runtime Environment (8.0_91-b14) (build 
> 1.8.0_91-8u91-b14-0ubuntu4~16.04.1-b14)
>  Java VM: OpenJDK 64-Bit Server VM (25.91-b14 mixed mode linux-amd64 
> compressed oops)
>  Problematic frame:
>  C  [librocksdbjni712288404493406713..so+0x15b8e8]  
> Java_org_rocksdb_RocksDB_write0+0x48
>  Core dump written. Default location: /root/core or core.23047
>  An error report file with more information is saved as:
>  /root/hs_err_pid23047.log
>  If you would like to submit a bug report, please visit:
>http://bugreport.java.com/bugreport/crash.jsp
>  The crash happened outside the Java Virtual Machine in native code.
>  See problematic frame for where to report the bug.
> --
> The core dump is there: 
> https://s3-eu-west-1.amazonaws.com/arodriguezg-test/core
> Note: The core dump is collected at Ubuntu Server.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (KAFKA-3805) Running multiple instances of a Streams app on the same machine results in Java_org_rocksdb_RocksDB_write error

[Michael Noll (JIRA)]

 [ 
https://issues.apache.org/jira/browse/KAFKA-3805?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael Noll updated KAFKA-3805:

Summary: Running multiple instances of a Streams app on the same machine 
results in Java_org_rocksdb_RocksDB_write error  (was: 
Java_org_rocksdb_RocksDB_write error)

> Running multiple instances of a Streams app on the same machine results in 
> Java_org_rocksdb_RocksDB_write error
> ---
>
> Key: KAFKA-3805
> URL: https://issues.apache.org/jira/browse/KAFKA-3805
> Project: Kafka
>  Issue Type: Bug
>  Components: streams
>Affects Versions: 0.10.0.0
> Environment: * Ubuntu Server 16.04 LTS
> - Java: OpenJDK Runtime Environment (build 
> 1.8.0_91-8u91-b14-0ubuntu4~16.04.1-b14)
> * OSX 10.11.5
>- Java: Java(TM) SE Runtime Environment (build 1.8.0_74-b02)
>Reporter: Andres Gomez Ferrer
>Assignee: Guozhang Wang
>Priority: Critical
> Attachments: hs_err_pid23047.log
>
>
> I reproduce the error working with simple two instances of  my kafka streams 
> application reading from one topic writing on other and using a RocksDB store.
> I reproduce it starting one instance, sending some messages on the input 
> topic and later starting another Kafka streams instance. Then, the first 
> instance died and drop a core dump.
> --
>  A fatal error has been detected by the Java Runtime Environment:
>   SIGSEGV (0xb) at pc=0x7f9f488fd8e8, pid=23047, tid=140322171361024
>  JRE version: OpenJDK Runtime Environment (8.0_91-b14) (build 
> 1.8.0_91-8u91-b14-0ubuntu4~16.04.1-b14)
>  Java VM: OpenJDK 64-Bit Server VM (25.91-b14 mixed mode linux-amd64 
> compressed oops)
>  Problematic frame:
>  C  [librocksdbjni712288404493406713..so+0x15b8e8]  
> Java_org_rocksdb_RocksDB_write0+0x48
>  Core dump written. Default location: /root/core or core.23047
>  An error report file with more information is saved as:
>  /root/hs_err_pid23047.log
>  If you would like to submit a bug report, please visit:
>http://bugreport.java.com/bugreport/crash.jsp
>  The crash happened outside the Java Virtual Machine in native code.
>  See problematic frame for where to report the bug.
> --
> The core dump is there: 
> https://s3-eu-west-1.amazonaws.com/arodriguezg-test/core
> Note: The core dump is collected at Ubuntu Server.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Comment Edited] (KAFKA-3805) Java_org_rocksdb_RocksDB_write error

[Andres Gomez Ferrer (JIRA)]

[ 
https://issues.apache.org/jira/browse/KAFKA-3805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15320525#comment-15320525
 ] 

Andres Gomez Ferrer edited comment on KAFKA-3805 at 6/8/16 1:26 PM:


[~ijuma] yeah, this is a possibility. But .. when the partitions are rebalanced 
the cache always are rebalanced, isn't?

for example: If I have 4 partitions and to the same application id 
(kafka-streams-7778), I have some like this on my filesystem:

root@ubuntu:~# ls /tmp/kafka-streams/kafka-streams-7778/
 0_0  0_1 0_2  0_3

And I suppose that when I will launch another Kafka stream application the 
partitions will be rebalanced and one application write inside [0_0  0_1] and 
the another on [0_2  0_3], isn't?




was (Author: agomez):
[~ijuma] yeah, this is a possibility

> Java_org_rocksdb_RocksDB_write error
> 
>
> Key: KAFKA-3805
> URL: https://issues.apache.org/jira/browse/KAFKA-3805
> Project: Kafka
>  Issue Type: Bug
>  Components: streams
>Affects Versions: 0.10.0.0
> Environment: * Ubuntu Server 16.04 LTS
> - Java: OpenJDK Runtime Environment (build 
> 1.8.0_91-8u91-b14-0ubuntu4~16.04.1-b14)
> * OSX 10.11.5
>- Java: Java(TM) SE Runtime Environment (build 1.8.0_74-b02)
>Reporter: Andres Gomez Ferrer
>Assignee: Guozhang Wang
>Priority: Critical
> Attachments: hs_err_pid23047.log
>
>
> I reproduce the error working with simple two instances of  my kafka streams 
> application reading from one topic writing on other and using a RocksDB store.
> I reproduce it starting one instance, sending some messages on the input 
> topic and later starting another Kafka streams instance. Then, the first 
> instance died and drop a core dump.
> --
>  A fatal error has been detected by the Java Runtime Environment:
>   SIGSEGV (0xb) at pc=0x7f9f488fd8e8, pid=23047, tid=140322171361024
>  JRE version: OpenJDK Runtime Environment (8.0_91-b14) (build 
> 1.8.0_91-8u91-b14-0ubuntu4~16.04.1-b14)
>  Java VM: OpenJDK 64-Bit Server VM (25.91-b14 mixed mode linux-amd64 
> compressed oops)
>  Problematic frame:
>  C  [librocksdbjni712288404493406713..so+0x15b8e8]  
> Java_org_rocksdb_RocksDB_write0+0x48
>  Core dump written. Default location: /root/core or core.23047
>  An error report file with more information is saved as:
>  /root/hs_err_pid23047.log
>  If you would like to submit a bug report, please visit:
>http://bugreport.java.com/bugreport/crash.jsp
>  The crash happened outside the Java Virtual Machine in native code.
>  See problematic frame for where to report the bug.
> --
> The core dump is there: 
> https://s3-eu-west-1.amazonaws.com/arodriguezg-test/core
> Note: The core dump is collected at Ubuntu Server.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Comment Edited] (KAFKA-3805) Java_org_rocksdb_RocksDB_write error

[Andres Gomez Ferrer (JIRA)]

[ 
https://issues.apache.org/jira/browse/KAFKA-3805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15320525#comment-15320525
 ] 

Andres Gomez Ferrer edited comment on KAFKA-3805 at 6/8/16 1:18 PM:


[~ijuma] yeah, this is a possibility


was (Author: agomez):
[~ijuma]yeah, this is a possibility

> Java_org_rocksdb_RocksDB_write error
> 
>
> Key: KAFKA-3805
> URL: https://issues.apache.org/jira/browse/KAFKA-3805
> Project: Kafka
>  Issue Type: Bug
>  Components: streams
>Affects Versions: 0.10.0.0
> Environment: * Ubuntu Server 16.04 LTS
> - Java: OpenJDK Runtime Environment (build 
> 1.8.0_91-8u91-b14-0ubuntu4~16.04.1-b14)
> * OSX 10.11.5
>- Java: Java(TM) SE Runtime Environment (build 1.8.0_74-b02)
>Reporter: Andres Gomez Ferrer
>Assignee: Guozhang Wang
>Priority: Critical
> Attachments: hs_err_pid23047.log
>
>
> I reproduce the error working with simple two instances of  my kafka streams 
> application reading from one topic writing on other and using a RocksDB store.
> I reproduce it starting one instance, sending some messages on the input 
> topic and later starting another Kafka streams instance. Then, the first 
> instance died and drop a core dump.
> --
>  A fatal error has been detected by the Java Runtime Environment:
>   SIGSEGV (0xb) at pc=0x7f9f488fd8e8, pid=23047, tid=140322171361024
>  JRE version: OpenJDK Runtime Environment (8.0_91-b14) (build 
> 1.8.0_91-8u91-b14-0ubuntu4~16.04.1-b14)
>  Java VM: OpenJDK 64-Bit Server VM (25.91-b14 mixed mode linux-amd64 
> compressed oops)
>  Problematic frame:
>  C  [librocksdbjni712288404493406713..so+0x15b8e8]  
> Java_org_rocksdb_RocksDB_write0+0x48
>  Core dump written. Default location: /root/core or core.23047
>  An error report file with more information is saved as:
>  /root/hs_err_pid23047.log
>  If you would like to submit a bug report, please visit:
>http://bugreport.java.com/bugreport/crash.jsp
>  The crash happened outside the Java Virtual Machine in native code.
>  See problematic frame for where to report the bug.
> --
> The core dump is there: 
> https://s3-eu-west-1.amazonaws.com/arodriguezg-test/core
> Note: The core dump is collected at Ubuntu Server.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (KAFKA-3805) Java_org_rocksdb_RocksDB_write error

[Andres Gomez Ferrer (JIRA)]

[ 
https://issues.apache.org/jira/browse/KAFKA-3805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15320525#comment-15320525
 ] 

Andres Gomez Ferrer commented on KAFKA-3805:


[~ijuma]yeah, this is a possibility

> Java_org_rocksdb_RocksDB_write error
> 
>
> Key: KAFKA-3805
> URL: https://issues.apache.org/jira/browse/KAFKA-3805
> Project: Kafka
>  Issue Type: Bug
>  Components: streams
>Affects Versions: 0.10.0.0
> Environment: * Ubuntu Server 16.04 LTS
> - Java: OpenJDK Runtime Environment (build 
> 1.8.0_91-8u91-b14-0ubuntu4~16.04.1-b14)
> * OSX 10.11.5
>- Java: Java(TM) SE Runtime Environment (build 1.8.0_74-b02)
>Reporter: Andres Gomez Ferrer
>Assignee: Guozhang Wang
>Priority: Critical
> Attachments: hs_err_pid23047.log
>
>
> I reproduce the error working with simple two instances of  my kafka streams 
> application reading from one topic writing on other and using a RocksDB store.
> I reproduce it starting one instance, sending some messages on the input 
> topic and later starting another Kafka streams instance. Then, the first 
> instance died and drop a core dump.
> --
>  A fatal error has been detected by the Java Runtime Environment:
>   SIGSEGV (0xb) at pc=0x7f9f488fd8e8, pid=23047, tid=140322171361024
>  JRE version: OpenJDK Runtime Environment (8.0_91-b14) (build 
> 1.8.0_91-8u91-b14-0ubuntu4~16.04.1-b14)
>  Java VM: OpenJDK 64-Bit Server VM (25.91-b14 mixed mode linux-amd64 
> compressed oops)
>  Problematic frame:
>  C  [librocksdbjni712288404493406713..so+0x15b8e8]  
> Java_org_rocksdb_RocksDB_write0+0x48
>  Core dump written. Default location: /root/core or core.23047
>  An error report file with more information is saved as:
>  /root/hs_err_pid23047.log
>  If you would like to submit a bug report, please visit:
>http://bugreport.java.com/bugreport/crash.jsp
>  The crash happened outside the Java Virtual Machine in native code.
>  See problematic frame for where to report the bug.
> --
> The core dump is there: 
> https://s3-eu-west-1.amazonaws.com/arodriguezg-test/core
> Note: The core dump is collected at Ubuntu Server.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Questions about Kafka Scripts

[Kamal C]

Hi,

A. What is the usage of `kafka-replica-verification.sh` script ? I don't
find any documentations about it in [1] and [2].

I have a topic `test` with 10 partitions. Ran the above script, it
continuously prints the below results.

[kamal@tcltest1 bin]$ sh kafka-replica-verification.sh --time -2
--topic-white-list test --report-interval-ms 1000 --broker-list
localhost:9092
2016-06-07 12:49:13,664: verification process is started.
2016-06-07 12:49:14,632: max lag is 0 for partition [test,6] at offset
39008 among 10 partitions
2016-06-07 12:49:15,633: max lag is 0 for partition [test,6] at offset
39008 among 10 partitions
2016-06-07 12:49:16,634: max lag is 0 for partition [test,6] at offset
39008 among 10 partitions
2016-06-07 12:49:17,635: max lag is 0 for partition [test,6] at offset
39008 among 10 partitions
2016-06-07 12:49:18,636: max lag is 0 for partition [test,6] at offset
39008 among 10 partitions
2016-06-07 12:49:19,637: max lag is 0 for partition [test,6] at offset
39008 among 10 partitions

B. `kafka-replay-log-producer.sh` script throws ConsumerTimeoutException.
What is wrong with the below command ?

[kamal@tcltest1 bin]$ sh kafka-replay-log-producer.sh --broker-list
localhost:9092 --inputtopic test --outputtopic xyz --reporting-interval
1000 --zookeeper localhost:2181
[2016-06-07 12:42:10,925] ERROR consumer thread timing out
(kafka.tools.ReplayLogProducer$ZKConsumerThread)
kafka.consumer.ConsumerTimeoutException
at kafka.consumer.ConsumerIterator.makeNext(ConsumerIterator.scala:69)
at kafka.consumer.ConsumerIterator.makeNext(ConsumerIterator.scala:33)
at
kafka.utils.IteratorTemplate.maybeComputeNext(IteratorTemplate.scala:64)
at kafka.utils.IteratorTemplate.hasNext(IteratorTemplate.scala:56)
at scala.collection.Iterator$class.foreach(Iterator.scala:893)
at kafka.utils.IteratorTemplate.foreach(IteratorTemplate.scala:30)
at scala.collection.IterableLike$class.foreach(IterableLike.scala:72)
at kafka.consumer.KafkaStream.foreach(KafkaStream.scala:25)
at
kafka.tools.ReplayLogProducer$ZKConsumerThread.run(ReplayLogProducer.scala:140)


[1]: http://kafka.apache.org/documentation.html
[2]: https://cwiki.apache.org/confluence/display/KAFKA/System+Tools

[jira] [Commented] (KAFKA-3805) Java_org_rocksdb_RocksDB_write error

[Ismael Juma (JIRA)]

[ 
https://issues.apache.org/jira/browse/KAFKA-3805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15320517#comment-15320517
 ] 

Ismael Juma commented on KAFKA-3805:


[~agomez], you probably need to use a different `state.dir` for each instance 
in that case.

> Java_org_rocksdb_RocksDB_write error
> 
>
> Key: KAFKA-3805
> URL: https://issues.apache.org/jira/browse/KAFKA-3805
> Project: Kafka
>  Issue Type: Bug
>  Components: streams
>Affects Versions: 0.10.0.0
> Environment: * Ubuntu Server 16.04 LTS
> - Java: OpenJDK Runtime Environment (build 
> 1.8.0_91-8u91-b14-0ubuntu4~16.04.1-b14)
> * OSX 10.11.5
>- Java: Java(TM) SE Runtime Environment (build 1.8.0_74-b02)
>Reporter: Andres Gomez Ferrer
>Assignee: Guozhang Wang
>Priority: Critical
> Attachments: hs_err_pid23047.log
>
>
> I reproduce the error working with simple two instances of  my kafka streams 
> application reading from one topic writing on other and using a RocksDB store.
> I reproduce it starting one instance, sending some messages on the input 
> topic and later starting another Kafka streams instance. Then, the first 
> instance died and drop a core dump.
> --
>  A fatal error has been detected by the Java Runtime Environment:
>   SIGSEGV (0xb) at pc=0x7f9f488fd8e8, pid=23047, tid=140322171361024
>  JRE version: OpenJDK Runtime Environment (8.0_91-b14) (build 
> 1.8.0_91-8u91-b14-0ubuntu4~16.04.1-b14)
>  Java VM: OpenJDK 64-Bit Server VM (25.91-b14 mixed mode linux-amd64 
> compressed oops)
>  Problematic frame:
>  C  [librocksdbjni712288404493406713..so+0x15b8e8]  
> Java_org_rocksdb_RocksDB_write0+0x48
>  Core dump written. Default location: /root/core or core.23047
>  An error report file with more information is saved as:
>  /root/hs_err_pid23047.log
>  If you would like to submit a bug report, please visit:
>http://bugreport.java.com/bugreport/crash.jsp
>  The crash happened outside the Java Virtual Machine in native code.
>  See problematic frame for where to report the bug.
> --
> The core dump is there: 
> https://s3-eu-west-1.amazonaws.com/arodriguezg-test/core
> Note: The core dump is collected at Ubuntu Server.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Comment Edited] (KAFKA-3805) Java_org_rocksdb_RocksDB_write error

[Andres Gomez Ferrer (JIRA)]

[ 
https://issues.apache.org/jira/browse/KAFKA-3805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15320507#comment-15320507
 ] 

Andres Gomez Ferrer edited comment on KAFKA-3805 at 6/8/16 1:09 PM:


[~enothereska] I use the same application id, because I want that the two 
instances balance the Kafka topics partitions.

In the docs:

application.id - An identifier for the stream processing application. Must be 
unique within the Kafka cluster. It is used as 1) the default client-id prefix, 
2) the group-id for membership management, 3) the changelog topic prefix.

The number 2) told that the application id is used to make the group-id and it  
must be the same if I want that the two instances balance the topic partitions, 
isn't??

I think that, if I use differents applications id both Kafka stream 
applications process all messages from all partitions.


was (Author: agomez):
[~enothereska] I use the same application id, because I want that the two 
instances balance the Kafka topics partitions.

In the docs:

application.id - An identifier for the stream processing application. Must be 
unique within the Kafka cluster. It is used as 1) the default client-id prefix, 
2) the group-id for membership management, 3) the changelog topic prefix.

The number 2) told that the application id is used to make the group-id and it  
must be the same if I want that the two instances balance the topic partitions, 
isn't??

I think that, if I use differents applications id both Kafka stream 
applications read all messages from all partitions.

> Java_org_rocksdb_RocksDB_write error
> 
>
> Key: KAFKA-3805
> URL: https://issues.apache.org/jira/browse/KAFKA-3805
> Project: Kafka
>  Issue Type: Bug
>  Components: streams
>Affects Versions: 0.10.0.0
> Environment: * Ubuntu Server 16.04 LTS
> - Java: OpenJDK Runtime Environment (build 
> 1.8.0_91-8u91-b14-0ubuntu4~16.04.1-b14)
> * OSX 10.11.5
>- Java: Java(TM) SE Runtime Environment (build 1.8.0_74-b02)
>Reporter: Andres Gomez Ferrer
>Assignee: Guozhang Wang
>Priority: Critical
> Attachments: hs_err_pid23047.log
>
>
> I reproduce the error working with simple two instances of  my kafka streams 
> application reading from one topic writing on other and using a RocksDB store.
> I reproduce it starting one instance, sending some messages on the input 
> topic and later starting another Kafka streams instance. Then, the first 
> instance died and drop a core dump.
> --
>  A fatal error has been detected by the Java Runtime Environment:
>   SIGSEGV (0xb) at pc=0x7f9f488fd8e8, pid=23047, tid=140322171361024
>  JRE version: OpenJDK Runtime Environment (8.0_91-b14) (build 
> 1.8.0_91-8u91-b14-0ubuntu4~16.04.1-b14)
>  Java VM: OpenJDK 64-Bit Server VM (25.91-b14 mixed mode linux-amd64 
> compressed oops)
>  Problematic frame:
>  C  [librocksdbjni712288404493406713..so+0x15b8e8]  
> Java_org_rocksdb_RocksDB_write0+0x48
>  Core dump written. Default location: /root/core or core.23047
>  An error report file with more information is saved as:
>  /root/hs_err_pid23047.log
>  If you would like to submit a bug report, please visit:
>http://bugreport.java.com/bugreport/crash.jsp
>  The crash happened outside the Java Virtual Machine in native code.
>  See problematic frame for where to report the bug.
> --
> The core dump is there: 
> https://s3-eu-west-1.amazonaws.com/arodriguezg-test/core
> Note: The core dump is collected at Ubuntu Server.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Comment Edited] (KAFKA-3805) Java_org_rocksdb_RocksDB_write error

[Andres Gomez Ferrer (JIRA)]

[ 
https://issues.apache.org/jira/browse/KAFKA-3805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15320507#comment-15320507
 ] 

Andres Gomez Ferrer edited comment on KAFKA-3805 at 6/8/16 1:04 PM:


[~enothereska] I use the same application id, because I want that the two 
instances balance the Kafka topics partitions.

In the docs:

application.id - An identifier for the stream processing application. Must be 
unique within the Kafka cluster. It is used as 1) the default client-id prefix, 
2) the group-id for membership management, 3) the changelog topic prefix.

The number 2) told that the application id is used to make the group-id and it  
must be the same if I want that the two instances balance the topic partitions, 
isn't??

I think that, if I use differents applications id both Kafka stream 
applications read all messages from all partitions.


was (Author: agomez):
I use the same application id, because I want that the two instances balance 
the Kafka topics partitions.

In the docs:

application.id - An identifier for the stream processing application. Must be 
unique within the Kafka cluster. It is used as 1) the default client-id prefix, 
2) the group-id for membership management, 3) the changelog topic prefix.

The number 2) told that the application id is used to make the group-id and it  
must be the same if I want that the two instances balance the topic partitions, 
isn't??

I think that, if I use differents applications id both Kafka stream 
applications read all messages from all partitions.

> Java_org_rocksdb_RocksDB_write error
> 
>
> Key: KAFKA-3805
> URL: https://issues.apache.org/jira/browse/KAFKA-3805
> Project: Kafka
>  Issue Type: Bug
>  Components: streams
>Affects Versions: 0.10.0.0
> Environment: * Ubuntu Server 16.04 LTS
> - Java: OpenJDK Runtime Environment (build 
> 1.8.0_91-8u91-b14-0ubuntu4~16.04.1-b14)
> * OSX 10.11.5
>- Java: Java(TM) SE Runtime Environment (build 1.8.0_74-b02)
>Reporter: Andres Gomez Ferrer
>Assignee: Guozhang Wang
>Priority: Critical
> Attachments: hs_err_pid23047.log
>
>
> I reproduce the error working with simple two instances of  my kafka streams 
> application reading from one topic writing on other and using a RocksDB store.
> I reproduce it starting one instance, sending some messages on the input 
> topic and later starting another Kafka streams instance. Then, the first 
> instance died and drop a core dump.
> --
>  A fatal error has been detected by the Java Runtime Environment:
>   SIGSEGV (0xb) at pc=0x7f9f488fd8e8, pid=23047, tid=140322171361024
>  JRE version: OpenJDK Runtime Environment (8.0_91-b14) (build 
> 1.8.0_91-8u91-b14-0ubuntu4~16.04.1-b14)
>  Java VM: OpenJDK 64-Bit Server VM (25.91-b14 mixed mode linux-amd64 
> compressed oops)
>  Problematic frame:
>  C  [librocksdbjni712288404493406713..so+0x15b8e8]  
> Java_org_rocksdb_RocksDB_write0+0x48
>  Core dump written. Default location: /root/core or core.23047
>  An error report file with more information is saved as:
>  /root/hs_err_pid23047.log
>  If you would like to submit a bug report, please visit:
>http://bugreport.java.com/bugreport/crash.jsp
>  The crash happened outside the Java Virtual Machine in native code.
>  See problematic frame for where to report the bug.
> --
> The core dump is there: 
> https://s3-eu-west-1.amazonaws.com/arodriguezg-test/core
> Note: The core dump is collected at Ubuntu Server.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (KAFKA-3805) Java_org_rocksdb_RocksDB_write error

[Andres Gomez Ferrer (JIRA)]

[ 
https://issues.apache.org/jira/browse/KAFKA-3805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15320507#comment-15320507
 ] 

Andres Gomez Ferrer commented on KAFKA-3805:


I use the same application id, because I want that the two instances balance 
the Kafka topics partitions.

In the docs:

application.id - An identifier for the stream processing application. Must be 
unique within the Kafka cluster. It is used as 1) the default client-id prefix, 
2) the group-id for membership management, 3) the changelog topic prefix.

The number 2) told that the application id is used to make the group-id and it  
must be the same if I want that the two instances balance the topic partitions, 
isn't??

I think that, if I use differents applications id both Kafka stream 
applications read all messages from all partitions.

> Java_org_rocksdb_RocksDB_write error
> 
>
> Key: KAFKA-3805
> URL: https://issues.apache.org/jira/browse/KAFKA-3805
> Project: Kafka
>  Issue Type: Bug
>  Components: streams
>Affects Versions: 0.10.0.0
> Environment: * Ubuntu Server 16.04 LTS
> - Java: OpenJDK Runtime Environment (build 
> 1.8.0_91-8u91-b14-0ubuntu4~16.04.1-b14)
> * OSX 10.11.5
>- Java: Java(TM) SE Runtime Environment (build 1.8.0_74-b02)
>Reporter: Andres Gomez Ferrer
>Assignee: Guozhang Wang
>Priority: Critical
> Attachments: hs_err_pid23047.log
>
>
> I reproduce the error working with simple two instances of  my kafka streams 
> application reading from one topic writing on other and using a RocksDB store.
> I reproduce it starting one instance, sending some messages on the input 
> topic and later starting another Kafka streams instance. Then, the first 
> instance died and drop a core dump.
> --
>  A fatal error has been detected by the Java Runtime Environment:
>   SIGSEGV (0xb) at pc=0x7f9f488fd8e8, pid=23047, tid=140322171361024
>  JRE version: OpenJDK Runtime Environment (8.0_91-b14) (build 
> 1.8.0_91-8u91-b14-0ubuntu4~16.04.1-b14)
>  Java VM: OpenJDK 64-Bit Server VM (25.91-b14 mixed mode linux-amd64 
> compressed oops)
>  Problematic frame:
>  C  [librocksdbjni712288404493406713..so+0x15b8e8]  
> Java_org_rocksdb_RocksDB_write0+0x48
>  Core dump written. Default location: /root/core or core.23047
>  An error report file with more information is saved as:
>  /root/hs_err_pid23047.log
>  If you would like to submit a bug report, please visit:
>http://bugreport.java.com/bugreport/crash.jsp
>  The crash happened outside the Java Virtual Machine in native code.
>  See problematic frame for where to report the bug.
> --
> The core dump is there: 
> https://s3-eu-west-1.amazonaws.com/arodriguezg-test/core
> Note: The core dump is collected at Ubuntu Server.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (KAFKA-3805) Java_org_rocksdb_RocksDB_write error

[Ismael Juma (JIRA)]

[ 
https://issues.apache.org/jira/browse/KAFKA-3805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15320492#comment-15320492
 ] 

Ismael Juma commented on KAFKA-3805:


[~enothereska], sounds like we should be acquiring a file system lock or 
something along those lines to fail in a nicer way.

> Java_org_rocksdb_RocksDB_write error
> 
>
> Key: KAFKA-3805
> URL: https://issues.apache.org/jira/browse/KAFKA-3805
> Project: Kafka
>  Issue Type: Bug
>  Components: streams
>Affects Versions: 0.10.0.0
> Environment: * Ubuntu Server 16.04 LTS
> - Java: OpenJDK Runtime Environment (build 
> 1.8.0_91-8u91-b14-0ubuntu4~16.04.1-b14)
> * OSX 10.11.5
>- Java: Java(TM) SE Runtime Environment (build 1.8.0_74-b02)
>Reporter: Andres Gomez Ferrer
>Assignee: Guozhang Wang
>Priority: Critical
> Attachments: hs_err_pid23047.log
>
>
> I reproduce the error working with simple two instances of  my kafka streams 
> application reading from one topic writing on other and using a RocksDB store.
> I reproduce it starting one instance, sending some messages on the input 
> topic and later starting another Kafka streams instance. Then, the first 
> instance died and drop a core dump.
> --
>  A fatal error has been detected by the Java Runtime Environment:
>   SIGSEGV (0xb) at pc=0x7f9f488fd8e8, pid=23047, tid=140322171361024
>  JRE version: OpenJDK Runtime Environment (8.0_91-b14) (build 
> 1.8.0_91-8u91-b14-0ubuntu4~16.04.1-b14)
>  Java VM: OpenJDK 64-Bit Server VM (25.91-b14 mixed mode linux-amd64 
> compressed oops)
>  Problematic frame:
>  C  [librocksdbjni712288404493406713..so+0x15b8e8]  
> Java_org_rocksdb_RocksDB_write0+0x48
>  Core dump written. Default location: /root/core or core.23047
>  An error report file with more information is saved as:
>  /root/hs_err_pid23047.log
>  If you would like to submit a bug report, please visit:
>http://bugreport.java.com/bugreport/crash.jsp
>  The crash happened outside the Java Virtual Machine in native code.
>  See problematic frame for where to report the bug.
> --
> The core dump is there: 
> https://s3-eu-west-1.amazonaws.com/arodriguezg-test/core
> Note: The core dump is collected at Ubuntu Server.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (KAFKA-3802) log mtimes reset on broker restart

[Ismael Juma (JIRA)]

[ 
https://issues.apache.org/jira/browse/KAFKA-3802?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15320467#comment-15320467
 ] 

Ismael Juma commented on KAFKA-3802:


Thanks for the investigation [~moritz]. Good job finding the source of the 
issue. The documentation of `truncate` says:

{code}
*  If the given size is less than the file's current size then the file
 * is truncated, discarding any bytes beyond the new end of the file.  If
 * the given size is greater than or equal to the file's current size then
 * the file is not modified.  In either case, if this channel's file
 * position is greater than the given size then it is set to that size.
{code}

So, it looks like a bug in the JDK, potentially fixed in JDK 8.

As an aside, it's a bit odd that `truncateTo` sets the position unconditionally 
to `targetSize`, but that's a topic for another JIRA, I guess.



> log mtimes reset on broker restart
> --
>
> Key: KAFKA-3802
> URL: https://issues.apache.org/jira/browse/KAFKA-3802
> Project: Kafka
>  Issue Type: Bug
>Affects Versions: 0.9.0.1
>Reporter: Andrew Otto
>
> Folks over in 
> http://mail-archives.apache.org/mod_mbox/kafka-users/201605.mbox/%3CCAO8=cz0ragjad1acx4geqcwj+rkd1gmdavkjwytwthkszfg...@mail.gmail.com%3E
>  are commenting about this issue.
> In 0.9, any data log file that was on
> disk before the broker has it's mtime modified to the time of the broker
> restart.
> This causes problems with log retention, as all the files then look like
> they contain recent data to kafka.  We use the default log retention of 7
> days, but if all the files are touched at the same time, this can cause us
> to retain up to 2 weeks of log data, which can fill up our disks.
> This happens *most* of the time, but seemingly not all.  We have seen broker 
> restarts where mtimes were not changed.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (KAFKA-3805) Java_org_rocksdb_RocksDB_write error

[Andres Gomez Ferrer (JIRA)]

[ 
https://issues.apache.org/jira/browse/KAFKA-3805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15320424#comment-15320424
 ] 

Andres Gomez Ferrer commented on KAFKA-3805:


Yes .. I think so.

> Java_org_rocksdb_RocksDB_write error
> 
>
> Key: KAFKA-3805
> URL: https://issues.apache.org/jira/browse/KAFKA-3805
> Project: Kafka
>  Issue Type: Bug
>  Components: streams
>Affects Versions: 0.10.0.0
> Environment: * Ubuntu Server 16.04 LTS
> - Java: OpenJDK Runtime Environment (build 
> 1.8.0_91-8u91-b14-0ubuntu4~16.04.1-b14)
> * OSX 10.11.5
>- Java: Java(TM) SE Runtime Environment (build 1.8.0_74-b02)
>Reporter: Andres Gomez Ferrer
>Assignee: Guozhang Wang
>Priority: Critical
> Attachments: hs_err_pid23047.log
>
>
> I reproduce the error working with simple two instances of  my kafka streams 
> application reading from one topic writing on other and using a RocksDB store.
> I reproduce it starting one instance, sending some messages on the input 
> topic and later starting another Kafka streams instance. Then, the first 
> instance died and drop a core dump.
> --
>  A fatal error has been detected by the Java Runtime Environment:
>   SIGSEGV (0xb) at pc=0x7f9f488fd8e8, pid=23047, tid=140322171361024
>  JRE version: OpenJDK Runtime Environment (8.0_91-b14) (build 
> 1.8.0_91-8u91-b14-0ubuntu4~16.04.1-b14)
>  Java VM: OpenJDK 64-Bit Server VM (25.91-b14 mixed mode linux-amd64 
> compressed oops)
>  Problematic frame:
>  C  [librocksdbjni712288404493406713..so+0x15b8e8]  
> Java_org_rocksdb_RocksDB_write0+0x48
>  Core dump written. Default location: /root/core or core.23047
>  An error report file with more information is saved as:
>  /root/hs_err_pid23047.log
>  If you would like to submit a bug report, please visit:
>http://bugreport.java.com/bugreport/crash.jsp
>  The crash happened outside the Java Virtual Machine in native code.
>  See problematic frame for where to report the bug.
> --
> The core dump is there: 
> https://s3-eu-west-1.amazonaws.com/arodriguezg-test/core
> Note: The core dump is collected at Ubuntu Server.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (KAFKA-3805) Java_org_rocksdb_RocksDB_write error

[Ismael Juma (JIRA)]

[ 
https://issues.apache.org/jira/browse/KAFKA-3805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15320420#comment-15320420
 ] 

Ismael Juma commented on KAFKA-3805:


I was just trying to clarify, it looks like a bug.

> Java_org_rocksdb_RocksDB_write error
> 
>
> Key: KAFKA-3805
> URL: https://issues.apache.org/jira/browse/KAFKA-3805
> Project: Kafka
>  Issue Type: Bug
>  Components: streams
>Affects Versions: 0.10.0.0
> Environment: * Ubuntu Server 16.04 LTS
> - Java: OpenJDK Runtime Environment (build 
> 1.8.0_91-8u91-b14-0ubuntu4~16.04.1-b14)
> * OSX 10.11.5
>- Java: Java(TM) SE Runtime Environment (build 1.8.0_74-b02)
>Reporter: Andres Gomez Ferrer
>Assignee: Guozhang Wang
>Priority: Critical
> Attachments: hs_err_pid23047.log
>
>
> I reproduce the error working with simple two instances of  my kafka streams 
> application reading from one topic writing on other and using a RocksDB store.
> I reproduce it starting one instance, sending some messages on the input 
> topic and later starting another Kafka streams instance. Then, the first 
> instance died and drop a core dump.
> --
>  A fatal error has been detected by the Java Runtime Environment:
>   SIGSEGV (0xb) at pc=0x7f9f488fd8e8, pid=23047, tid=140322171361024
>  JRE version: OpenJDK Runtime Environment (8.0_91-b14) (build 
> 1.8.0_91-8u91-b14-0ubuntu4~16.04.1-b14)
>  Java VM: OpenJDK 64-Bit Server VM (25.91-b14 mixed mode linux-amd64 
> compressed oops)
>  Problematic frame:
>  C  [librocksdbjni712288404493406713..so+0x15b8e8]  
> Java_org_rocksdb_RocksDB_write0+0x48
>  Core dump written. Default location: /root/core or core.23047
>  An error report file with more information is saved as:
>  /root/hs_err_pid23047.log
>  If you would like to submit a bug report, please visit:
>http://bugreport.java.com/bugreport/crash.jsp
>  The crash happened outside the Java Virtual Machine in native code.
>  See problematic frame for where to report the bug.
> --
> The core dump is there: 
> https://s3-eu-west-1.amazonaws.com/arodriguezg-test/core
> Note: The core dump is collected at Ubuntu Server.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Comment Edited] (KAFKA-3805) Java_org_rocksdb_RocksDB_write error

[Andres Gomez Ferrer (JIRA)]

[ 
https://issues.apache.org/jira/browse/KAFKA-3805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15320400#comment-15320400
 ] 

Andres Gomez Ferrer edited comment on KAFKA-3805 at 6/8/16 11:49 AM:
-

Yes, I started another Kafka Streams on the same machine. Is this a problem??? 

I put two environments because I reproduce the problem in both.


was (Author: agomez):
Yes, I started another Kafka Streams in the same machine. Is this a problem??? 

> Java_org_rocksdb_RocksDB_write error
> 
>
> Key: KAFKA-3805
> URL: https://issues.apache.org/jira/browse/KAFKA-3805
> Project: Kafka
>  Issue Type: Bug
>  Components: streams
>Affects Versions: 0.10.0.0
> Environment: * Ubuntu Server 16.04 LTS
> - Java: OpenJDK Runtime Environment (build 
> 1.8.0_91-8u91-b14-0ubuntu4~16.04.1-b14)
> * OSX 10.11.5
>- Java: Java(TM) SE Runtime Environment (build 1.8.0_74-b02)
>Reporter: Andres Gomez Ferrer
>Assignee: Guozhang Wang
>Priority: Critical
> Attachments: hs_err_pid23047.log
>
>
> I reproduce the error working with simple two instances of  my kafka streams 
> application reading from one topic writing on other and using a RocksDB store.
> I reproduce it starting one instance, sending some messages on the input 
> topic and later starting another Kafka streams instance. Then, the first 
> instance died and drop a core dump.
> --
>  A fatal error has been detected by the Java Runtime Environment:
>   SIGSEGV (0xb) at pc=0x7f9f488fd8e8, pid=23047, tid=140322171361024
>  JRE version: OpenJDK Runtime Environment (8.0_91-b14) (build 
> 1.8.0_91-8u91-b14-0ubuntu4~16.04.1-b14)
>  Java VM: OpenJDK 64-Bit Server VM (25.91-b14 mixed mode linux-amd64 
> compressed oops)
>  Problematic frame:
>  C  [librocksdbjni712288404493406713..so+0x15b8e8]  
> Java_org_rocksdb_RocksDB_write0+0x48
>  Core dump written. Default location: /root/core or core.23047
>  An error report file with more information is saved as:
>  /root/hs_err_pid23047.log
>  If you would like to submit a bug report, please visit:
>http://bugreport.java.com/bugreport/crash.jsp
>  The crash happened outside the Java Virtual Machine in native code.
>  See problematic frame for where to report the bug.
> --
> The core dump is there: 
> https://s3-eu-west-1.amazonaws.com/arodriguezg-test/core
> Note: The core dump is collected at Ubuntu Server.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Comment Edited] (KAFKA-3805) Java_org_rocksdb_RocksDB_write error

[Andres Gomez Ferrer (JIRA)]

[ 
https://issues.apache.org/jira/browse/KAFKA-3805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15320400#comment-15320400
 ] 

Andres Gomez Ferrer edited comment on KAFKA-3805 at 6/8/16 11:49 AM:
-

Yes, I started another Kafka Streams on the same machine. Is this a problem??? 

I put two environments because I have reproduced the problem in both.


was (Author: agomez):
Yes, I started another Kafka Streams on the same machine. Is this a problem??? 

I put two environments because I reproduce the problem in both.

> Java_org_rocksdb_RocksDB_write error
> 
>
> Key: KAFKA-3805
> URL: https://issues.apache.org/jira/browse/KAFKA-3805
> Project: Kafka
>  Issue Type: Bug
>  Components: streams
>Affects Versions: 0.10.0.0
> Environment: * Ubuntu Server 16.04 LTS
> - Java: OpenJDK Runtime Environment (build 
> 1.8.0_91-8u91-b14-0ubuntu4~16.04.1-b14)
> * OSX 10.11.5
>- Java: Java(TM) SE Runtime Environment (build 1.8.0_74-b02)
>Reporter: Andres Gomez Ferrer
>Assignee: Guozhang Wang
>Priority: Critical
> Attachments: hs_err_pid23047.log
>
>
> I reproduce the error working with simple two instances of  my kafka streams 
> application reading from one topic writing on other and using a RocksDB store.
> I reproduce it starting one instance, sending some messages on the input 
> topic and later starting another Kafka streams instance. Then, the first 
> instance died and drop a core dump.
> --
>  A fatal error has been detected by the Java Runtime Environment:
>   SIGSEGV (0xb) at pc=0x7f9f488fd8e8, pid=23047, tid=140322171361024
>  JRE version: OpenJDK Runtime Environment (8.0_91-b14) (build 
> 1.8.0_91-8u91-b14-0ubuntu4~16.04.1-b14)
>  Java VM: OpenJDK 64-Bit Server VM (25.91-b14 mixed mode linux-amd64 
> compressed oops)
>  Problematic frame:
>  C  [librocksdbjni712288404493406713..so+0x15b8e8]  
> Java_org_rocksdb_RocksDB_write0+0x48
>  Core dump written. Default location: /root/core or core.23047
>  An error report file with more information is saved as:
>  /root/hs_err_pid23047.log
>  If you would like to submit a bug report, please visit:
>http://bugreport.java.com/bugreport/crash.jsp
>  The crash happened outside the Java Virtual Machine in native code.
>  See problematic frame for where to report the bug.
> --
> The core dump is there: 
> https://s3-eu-west-1.amazonaws.com/arodriguezg-test/core
> Note: The core dump is collected at Ubuntu Server.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (KAFKA-3805) Java_org_rocksdb_RocksDB_write error

[Andres Gomez Ferrer (JIRA)]

 [ 
https://issues.apache.org/jira/browse/KAFKA-3805?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andres Gomez Ferrer updated KAFKA-3805:
---
Description: 
I reproduce the error working with simple two instances of  my kafka streams 
application reading from one topic writing on other and using a RocksDB store.

I reproduce it starting one instance, sending some messages on the input topic 
and later starting another Kafka streams instance. Then, the first instance 
died and drop a core dump.

--
 A fatal error has been detected by the Java Runtime Environment:

  SIGSEGV (0xb) at pc=0x7f9f488fd8e8, pid=23047, tid=140322171361024

 JRE version: OpenJDK Runtime Environment (8.0_91-b14) (build 
1.8.0_91-8u91-b14-0ubuntu4~16.04.1-b14)
 Java VM: OpenJDK 64-Bit Server VM (25.91-b14 mixed mode linux-amd64 compressed 
oops)
 Problematic frame:
 C  [librocksdbjni712288404493406713..so+0x15b8e8]  
Java_org_rocksdb_RocksDB_write0+0x48

 Core dump written. Default location: /root/core or core.23047

 An error report file with more information is saved as:
 /root/hs_err_pid23047.log

 If you would like to submit a bug report, please visit:
   http://bugreport.java.com/bugreport/crash.jsp
 The crash happened outside the Java Virtual Machine in native code.
 See problematic frame for where to report the bug.
--

The core dump is there: https://s3-eu-west-1.amazonaws.com/arodriguezg-test/core

Note: The core dump is collected at Ubuntu Server.

  was:
I reproduce the error working with simple two instances of  my kafka streams 
application reading from one topic writing on other and using a RocksDB store.

I reproduce it starting one instance, sending some messages on the input topic 
and later starting another Kafka streams instance. Then, the first instance 
died and drop a core dump.

--
 A fatal error has been detected by the Java Runtime Environment:

  SIGSEGV (0xb) at pc=0x7f9f488fd8e8, pid=23047, tid=140322171361024

 JRE version: OpenJDK Runtime Environment (8.0_91-b14) (build 
1.8.0_91-8u91-b14-0ubuntu4~16.04.1-b14)
 Java VM: OpenJDK 64-Bit Server VM (25.91-b14 mixed mode linux-amd64 compressed 
oops)
 Problematic frame:
 C  [librocksdbjni712288404493406713..so+0x15b8e8]  
Java_org_rocksdb_RocksDB_write0+0x48

 Core dump written. Default location: /root/core or core.23047

 An error report file with more information is saved as:
 /root/hs_err_pid23047.log

 If you would like to submit a bug report, please visit:
   http://bugreport.java.com/bugreport/crash.jsp
 The crash happened outside the Java Virtual Machine in native code.
 See problematic frame for where to report the bug.
--

The core dump is there: https://s3-eu-west-1.amazonaws.com/arodriguezg-test/core


> Java_org_rocksdb_RocksDB_write error
> 
>
> Key: KAFKA-3805
> URL: https://issues.apache.org/jira/browse/KAFKA-3805
> Project: Kafka
>  Issue Type: Bug
>  Components: streams
>Affects Versions: 0.10.0.0
> Environment: * Ubuntu Server 16.04 LTS
> - Java: OpenJDK Runtime Environment (build 
> 1.8.0_91-8u91-b14-0ubuntu4~16.04.1-b14)
> * OSX 10.11.5
>- Java: Java(TM) SE Runtime Environment (build 1.8.0_74-b02)
>Reporter: Andres Gomez Ferrer
>Assignee: Guozhang Wang
>Priority: Critical
> Attachments: hs_err_pid23047.log
>
>
> I reproduce the error working with simple two instances of  my kafka streams 
> application reading from one topic writing on other and using a RocksDB store.
> I reproduce it starting one instance, sending some messages on the input 
> topic and later starting another Kafka streams instance. Then, the first 
> instance died and drop a core dump.
> --
>  A fatal error has been detected by the Java Runtime Environment:
>   SIGSEGV (0xb) at pc=0x7f9f488fd8e8, pid=23047, tid=140322171361024
>  JRE version: OpenJDK Runtime Environment (8.0_91-b14) (build 
> 1.8.0_91-8u91-b14-0ubuntu4~16.04.1-b14)
>  Java VM: OpenJDK 64-Bit Server VM (25.91-b14 mixed mode linux-amd64 
> compressed oops)
>  Problematic frame:
>  C  [librocksdbjni712288404493406713..so+0x15b8e8]  
> Java_org_rocksdb_RocksDB_write0+0x48
>  Core dump written. Default location: /root/core or core.23047
>  An error report file with more information is saved as:
>  

[jira] [Commented] (KAFKA-3805) Java_org_rocksdb_RocksDB_write error

[Andres Gomez Ferrer (JIRA)]

[ 
https://issues.apache.org/jira/browse/KAFKA-3805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15320400#comment-15320400
 ] 

Andres Gomez Ferrer commented on KAFKA-3805:


Yes, I started another Kafka Streams in the same machine. Is this a problem??? 

> Java_org_rocksdb_RocksDB_write error
> 
>
> Key: KAFKA-3805
> URL: https://issues.apache.org/jira/browse/KAFKA-3805
> Project: Kafka
>  Issue Type: Bug
>  Components: streams
>Affects Versions: 0.10.0.0
> Environment: * Ubuntu Server 16.04 LTS
> - Java: OpenJDK Runtime Environment (build 
> 1.8.0_91-8u91-b14-0ubuntu4~16.04.1-b14)
> * OSX 10.11.5
>- Java: Java(TM) SE Runtime Environment (build 1.8.0_74-b02)
>Reporter: Andres Gomez Ferrer
>Assignee: Guozhang Wang
>Priority: Critical
> Attachments: hs_err_pid23047.log
>
>
> I reproduce the error working with simple two instances of  my kafka streams 
> application reading from one topic writing on other and using a RocksDB store.
> I reproduce it starting one instance, sending some messages on the input 
> topic and later starting another Kafka streams instance. Then, the first 
> instance died and drop a core dump.
> --
>  A fatal error has been detected by the Java Runtime Environment:
>   SIGSEGV (0xb) at pc=0x7f9f488fd8e8, pid=23047, tid=140322171361024
>  JRE version: OpenJDK Runtime Environment (8.0_91-b14) (build 
> 1.8.0_91-8u91-b14-0ubuntu4~16.04.1-b14)
>  Java VM: OpenJDK 64-Bit Server VM (25.91-b14 mixed mode linux-amd64 
> compressed oops)
>  Problematic frame:
>  C  [librocksdbjni712288404493406713..so+0x15b8e8]  
> Java_org_rocksdb_RocksDB_write0+0x48
>  Core dump written. Default location: /root/core or core.23047
>  An error report file with more information is saved as:
>  /root/hs_err_pid23047.log
>  If you would like to submit a bug report, please visit:
>http://bugreport.java.com/bugreport/crash.jsp
>  The crash happened outside the Java Virtual Machine in native code.
>  See problematic frame for where to report the bug.
> --
> The core dump is there: 
> https://s3-eu-west-1.amazonaws.com/arodriguezg-test/core



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (KAFKA-3805) Java_org_rocksdb_RocksDB_write error

[Andres Gomez Ferrer (JIRA)]

 [ 
https://issues.apache.org/jira/browse/KAFKA-3805?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andres Gomez Ferrer updated KAFKA-3805:
---
Description: 
I reproduce the error working with simple two instances of  my kafka streams 
application reading from one topic writing on other and using a RocksDB store.

I reproduce it starting one instance, sending some messages on the input topic 
and later starting another Kafka streams instance. Then, the first instance 
died and drop a core dump.

--
 A fatal error has been detected by the Java Runtime Environment:

  SIGSEGV (0xb) at pc=0x7f9f488fd8e8, pid=23047, tid=140322171361024

 JRE version: OpenJDK Runtime Environment (8.0_91-b14) (build 
1.8.0_91-8u91-b14-0ubuntu4~16.04.1-b14)
 Java VM: OpenJDK 64-Bit Server VM (25.91-b14 mixed mode linux-amd64 compressed 
oops)
 Problematic frame:
 C  [librocksdbjni712288404493406713..so+0x15b8e8]  
Java_org_rocksdb_RocksDB_write0+0x48

 Core dump written. Default location: /root/core or core.23047

 An error report file with more information is saved as:
 /root/hs_err_pid23047.log

 If you would like to submit a bug report, please visit:
   http://bugreport.java.com/bugreport/crash.jsp
 The crash happened outside the Java Virtual Machine in native code.
 See problematic frame for where to report the bug.
--

The core dump is there: https://s3-eu-west-1.amazonaws.com/arodriguezg-test/core

  was:
I reproduce the error working with simple two instances of  my kafka streams 
application reading from one topic writing on other and using a RocksDB store.

I reproduce it starting one instance, sending some messages on the input topic 
and later starting another Kafka streams instance. Then, the first instance 
died and drop a core dump.

--
 A fatal error has been detected by the Java Runtime Environment:

  SIGSEGV (0xb) at pc=0x7f9f488fd8e8, pid=23047, tid=140322171361024

 JRE version: OpenJDK Runtime Environment (8.0_91-b14) (build 
1.8.0_91-8u91-b14-0ubuntu4~16.04.1-b14)
 Java VM: OpenJDK 64-Bit Server VM (25.91-b14 mixed mode linux-amd64 compressed 
oops)
 Problematic frame:
 C  [librocksdbjni712288404493406713..so+0x15b8e8]  
Java_org_rocksdb_RocksDB_write0+0x48

 Core dump written. Default location: /root/core or core.23047

 An error report file with more information is saved as:
 /root/hs_err_pid23047.log

 If you would like to submit a bug report, please visit:
   http://bugreport.java.com/bugreport/crash.jsp
 The crash happened outside the Java Virtual Machine in native code.
 See problematic frame for where to report the bug.
--


> Java_org_rocksdb_RocksDB_write error
> 
>
> Key: KAFKA-3805
> URL: https://issues.apache.org/jira/browse/KAFKA-3805
> Project: Kafka
>  Issue Type: Bug
>  Components: streams
>Affects Versions: 0.10.0.0
> Environment: * Ubuntu Server 16.04 LTS
> - Java: OpenJDK Runtime Environment (build 
> 1.8.0_91-8u91-b14-0ubuntu4~16.04.1-b14)
> * OSX 10.11.5
>- Java: Java(TM) SE Runtime Environment (build 1.8.0_74-b02)
>Reporter: Andres Gomez Ferrer
>Assignee: Guozhang Wang
>Priority: Critical
> Attachments: hs_err_pid23047.log
>
>
> I reproduce the error working with simple two instances of  my kafka streams 
> application reading from one topic writing on other and using a RocksDB store.
> I reproduce it starting one instance, sending some messages on the input 
> topic and later starting another Kafka streams instance. Then, the first 
> instance died and drop a core dump.
> --
>  A fatal error has been detected by the Java Runtime Environment:
>   SIGSEGV (0xb) at pc=0x7f9f488fd8e8, pid=23047, tid=140322171361024
>  JRE version: OpenJDK Runtime Environment (8.0_91-b14) (build 
> 1.8.0_91-8u91-b14-0ubuntu4~16.04.1-b14)
>  Java VM: OpenJDK 64-Bit Server VM (25.91-b14 mixed mode linux-amd64 
> compressed oops)
>  Problematic frame:
>  C  [librocksdbjni712288404493406713..so+0x15b8e8]  
> Java_org_rocksdb_RocksDB_write0+0x48
>  Core dump written. Default location: /root/core or core.23047
>  An error report file with more information is saved as:
>  /root/hs_err_pid23047.log
>  If you would like to submit a bug report, please visit:
>http://bugreport.java.com/bugreport/crash.jsp
>  The 

[jira] [Updated] (KAFKA-3805) Java_org_rocksdb_RocksDB_write error

[Andres Gomez Ferrer (JIRA)]

 [ 
https://issues.apache.org/jira/browse/KAFKA-3805?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andres Gomez Ferrer updated KAFKA-3805:
---
Attachment: hs_err_pid23047.log

> Java_org_rocksdb_RocksDB_write error
> 
>
> Key: KAFKA-3805
> URL: https://issues.apache.org/jira/browse/KAFKA-3805
> Project: Kafka
>  Issue Type: Bug
>  Components: streams
>Affects Versions: 0.10.0.0
> Environment: * Ubuntu Server 16.04 LTS
> - Java: OpenJDK Runtime Environment (build 
> 1.8.0_91-8u91-b14-0ubuntu4~16.04.1-b14)
> * OSX 10.11.5
>- Java: Java(TM) SE Runtime Environment (build 1.8.0_74-b02)
>Reporter: Andres Gomez Ferrer
>Assignee: Guozhang Wang
>Priority: Critical
> Attachments: hs_err_pid23047.log
>
>
> I reproduce the error working with simple two instances of  my kafka streams 
> application reading from one topic writing on other and using a RocksDB store.
> I reproduce it starting one instance, sending some messages on the input 
> topic and later starting another Kafka streams instance. Then, the first 
> instance died and drop a core dump.
> --
>  A fatal error has been detected by the Java Runtime Environment:
>   SIGSEGV (0xb) at pc=0x7f9f488fd8e8, pid=23047, tid=140322171361024
>  JRE version: OpenJDK Runtime Environment (8.0_91-b14) (build 
> 1.8.0_91-8u91-b14-0ubuntu4~16.04.1-b14)
>  Java VM: OpenJDK 64-Bit Server VM (25.91-b14 mixed mode linux-amd64 
> compressed oops)
>  Problematic frame:
>  C  [librocksdbjni712288404493406713..so+0x15b8e8]  
> Java_org_rocksdb_RocksDB_write0+0x48
>  Core dump written. Default location: /root/core or core.23047
>  An error report file with more information is saved as:
>  /root/hs_err_pid23047.log
>  If you would like to submit a bug report, please visit:
>http://bugreport.java.com/bugreport/crash.jsp
>  The crash happened outside the Java Virtual Machine in native code.
>  See problematic frame for where to report the bug.
> --



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (KAFKA-3805) Java_org_rocksdb_RocksDB_write error

[Andres Gomez Ferrer (JIRA)]

 [ 
https://issues.apache.org/jira/browse/KAFKA-3805?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andres Gomez Ferrer updated KAFKA-3805:
---
Description: 
I reproduce the error working with simple two instances of  my kafka streams 
application reading from one topic writing on other and using a RocksDB store.

I reproduce it starting one instance, sending some messages on the input topic 
and later starting another Kafka streams instance. Then, the first instance 
died and drop a core dump.

--
 A fatal error has been detected by the Java Runtime Environment:

  SIGSEGV (0xb) at pc=0x7f9f488fd8e8, pid=23047, tid=140322171361024

 JRE version: OpenJDK Runtime Environment (8.0_91-b14) (build 
1.8.0_91-8u91-b14-0ubuntu4~16.04.1-b14)
 Java VM: OpenJDK 64-Bit Server VM (25.91-b14 mixed mode linux-amd64 compressed 
oops)
 Problematic frame:
 C  [librocksdbjni712288404493406713..so+0x15b8e8]  
Java_org_rocksdb_RocksDB_write0+0x48

 Core dump written. Default location: /root/core or core.23047

 An error report file with more information is saved as:
 /root/hs_err_pid23047.log

 If you would like to submit a bug report, please visit:
   http://bugreport.java.com/bugreport/crash.jsp
 The crash happened outside the Java Virtual Machine in native code.
 See problematic frame for where to report the bug.
--

  was:
I reproduce the error working with simple two instances of  my kafka streams 
application reading from one topic writing on other and using a RocksDB store.

I reproduce it starting one instance, sending some messages on the input topic 
and later starting another Kafka streams instance. Then, the first instance 
died and drop a core dump.

#
# A fatal error has been detected by the Java Runtime Environment:
#
#  SIGSEGV (0xb) at pc=0x7f9f488fd8e8, pid=23047, tid=140322171361024
#
# JRE version: OpenJDK Runtime Environment (8.0_91-b14) (build 
1.8.0_91-8u91-b14-0ubuntu4~16.04.1-b14)
# Java VM: OpenJDK 64-Bit Server VM (25.91-b14 mixed mode linux-amd64 
compressed oops)
# Problematic frame:
# C  [librocksdbjni712288404493406713..so+0x15b8e8]  
Java_org_rocksdb_RocksDB_write0+0x48
#
# Core dump written. Default location: /root/core or core.23047
#
# An error report file with more information is saved as:
# /root/hs_err_pid23047.log
#
# If you would like to submit a bug report, please visit:
#   http://bugreport.java.com/bugreport/crash.jsp
# The crash happened outside the Java Virtual Machine in native code.
# See problematic frame for where to report the bug.
#


> Java_org_rocksdb_RocksDB_write error
> 
>
> Key: KAFKA-3805
> URL: https://issues.apache.org/jira/browse/KAFKA-3805
> Project: Kafka
>  Issue Type: Bug
>  Components: streams
>Affects Versions: 0.10.0.0
> Environment: * Ubuntu Server 16.04 LTS
> - Java: OpenJDK Runtime Environment (build 
> 1.8.0_91-8u91-b14-0ubuntu4~16.04.1-b14)
> * OSX 10.11.5
>- Java: Java(TM) SE Runtime Environment (build 1.8.0_74-b02)
>Reporter: Andres Gomez Ferrer
>Assignee: Guozhang Wang
>Priority: Critical
> Attachments: hs_err_pid23047.log
>
>
> I reproduce the error working with simple two instances of  my kafka streams 
> application reading from one topic writing on other and using a RocksDB store.
> I reproduce it starting one instance, sending some messages on the input 
> topic and later starting another Kafka streams instance. Then, the first 
> instance died and drop a core dump.
> --
>  A fatal error has been detected by the Java Runtime Environment:
>   SIGSEGV (0xb) at pc=0x7f9f488fd8e8, pid=23047, tid=140322171361024
>  JRE version: OpenJDK Runtime Environment (8.0_91-b14) (build 
> 1.8.0_91-8u91-b14-0ubuntu4~16.04.1-b14)
>  Java VM: OpenJDK 64-Bit Server VM (25.91-b14 mixed mode linux-amd64 
> compressed oops)
>  Problematic frame:
>  C  [librocksdbjni712288404493406713..so+0x15b8e8]  
> Java_org_rocksdb_RocksDB_write0+0x48
>  Core dump written. Default location: /root/core or core.23047
>  An error report file with more information is saved as:
>  /root/hs_err_pid23047.log
>  If you would like to submit a bug report, please visit:
>http://bugreport.java.com/bugreport/crash.jsp
>  The crash happened outside the Java Virtual Machine in native code.
>  See problematic frame for where to report the bug.
> --



--
This message was sent by Atlassian JIRA

[jira] [Updated] (KAFKA-3805) Java_org_rocksdb_RocksDB_write error

[Andres Gomez Ferrer (JIRA)]

 [ 
https://issues.apache.org/jira/browse/KAFKA-3805?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andres Gomez Ferrer updated KAFKA-3805:
---
Description: 
I reproduce the error working with simple two instances of  my kafka streams 
application reading from one topic writing on other and using a RocksDB store.

I reproduce it starting one instance, sending some messages on the input topic 
and later starting another Kafka streams instance. Then, the first instance 
died and drop a core dump.

#
# A fatal error has been detected by the Java Runtime Environment:
#
#  SIGSEGV (0xb) at pc=0x7f9f488fd8e8, pid=23047, tid=140322171361024
#
# JRE version: OpenJDK Runtime Environment (8.0_91-b14) (build 
1.8.0_91-8u91-b14-0ubuntu4~16.04.1-b14)
# Java VM: OpenJDK 64-Bit Server VM (25.91-b14 mixed mode linux-amd64 
compressed oops)
# Problematic frame:
# C  [librocksdbjni712288404493406713..so+0x15b8e8]  
Java_org_rocksdb_RocksDB_write0+0x48
#
# Core dump written. Default location: /root/core or core.23047
#
# An error report file with more information is saved as:
# /root/hs_err_pid23047.log
#
# If you would like to submit a bug report, please visit:
#   http://bugreport.java.com/bugreport/crash.jsp
# The crash happened outside the Java Virtual Machine in native code.
# See problematic frame for where to report the bug.
#

  was:
I reproduce the error working with simple two instances of  my kafka streams 
application reading from one topic writing on other and using a RocksDB store.

I reproduce it starting one instance, sending some messages on the input topic 
and later starting another Kafka streams instance. Then, the first instance 
died and drop a core dump.

#
# A fatal error has been detected by the Java Runtime Environment:
#
#  SIGSEGV (0xb) at pc=0x00012102b7ad, pid=7288, tid=22531
#
# JRE version: Java(TM) SE Runtime Environment (8.0_74-b02) (build 1.8.0_74-b02)
# Java VM: Java HotSpot(TM) 64-Bit Server VM (25.74-b02 mixed mode bsd-amd64 
compressed oops)
# Problematic frame:
# C  [librocksdbjni6949968954399704135..jnilib+0x117ad]  
Java_org_rocksdb_RocksDB_write0+0x1d
#
# Core dump written. Default location: /cores/core or core.7288
#
# If you would like to submit a bug report, please visit:
#   http://bugreport.java.com/bugreport/crash.jsp
# The crash happened outside the Java Virtual Machine in native code.
# See problematic frame for where to report the bug.
#


> Java_org_rocksdb_RocksDB_write error
> 
>
> Key: KAFKA-3805
> URL: https://issues.apache.org/jira/browse/KAFKA-3805
> Project: Kafka
>  Issue Type: Bug
>  Components: streams
>Affects Versions: 0.10.0.0
> Environment: * Ubuntu Server 16.04 LTS
> - Java: OpenJDK Runtime Environment (build 
> 1.8.0_91-8u91-b14-0ubuntu4~16.04.1-b14)
> * OSX 10.11.5
>- Java: Java(TM) SE Runtime Environment (build 1.8.0_74-b02)
>Reporter: Andres Gomez Ferrer
>Assignee: Guozhang Wang
>Priority: Critical
>
> I reproduce the error working with simple two instances of  my kafka streams 
> application reading from one topic writing on other and using a RocksDB store.
> I reproduce it starting one instance, sending some messages on the input 
> topic and later starting another Kafka streams instance. Then, the first 
> instance died and drop a core dump.
> #
> # A fatal error has been detected by the Java Runtime Environment:
> #
> #  SIGSEGV (0xb) at pc=0x7f9f488fd8e8, pid=23047, tid=140322171361024
> #
> # JRE version: OpenJDK Runtime Environment (8.0_91-b14) (build 
> 1.8.0_91-8u91-b14-0ubuntu4~16.04.1-b14)
> # Java VM: OpenJDK 64-Bit Server VM (25.91-b14 mixed mode linux-amd64 
> compressed oops)
> # Problematic frame:
> # C  [librocksdbjni712288404493406713..so+0x15b8e8]  
> Java_org_rocksdb_RocksDB_write0+0x48
> #
> # Core dump written. Default location: /root/core or core.23047
> #
> # An error report file with more information is saved as:
> # /root/hs_err_pid23047.log
> #
> # If you would like to submit a bug report, please visit:
> #   http://bugreport.java.com/bugreport/crash.jsp
> # The crash happened outside the Java Virtual Machine in native code.
> # See problematic frame for where to report the bug.
> #



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (KAFKA-3805) Java_org_rocksdb_RocksDB_write error

[Andres Gomez Ferrer (JIRA)]

 [ 
https://issues.apache.org/jira/browse/KAFKA-3805?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andres Gomez Ferrer updated KAFKA-3805:
---
Attachment: (was: hs_err_pid7288.log)

> Java_org_rocksdb_RocksDB_write error
> 
>
> Key: KAFKA-3805
> URL: https://issues.apache.org/jira/browse/KAFKA-3805
> Project: Kafka
>  Issue Type: Bug
>  Components: streams
>Affects Versions: 0.10.0.0
> Environment: * Ubuntu Server 16.04 LTS
> - Java: OpenJDK Runtime Environment (build 
> 1.8.0_91-8u91-b14-0ubuntu4~16.04.1-b14)
> * OSX 10.11.5
>- Java: Java(TM) SE Runtime Environment (build 1.8.0_74-b02)
>Reporter: Andres Gomez Ferrer
>Assignee: Guozhang Wang
>Priority: Critical
>
> I reproduce the error working with simple two instances of  my kafka streams 
> application reading from one topic writing on other and using a RocksDB store.
> I reproduce it starting one instance, sending some messages on the input 
> topic and later starting another Kafka streams instance. Then, the first 
> instance died and drop a core dump.
> #
> # A fatal error has been detected by the Java Runtime Environment:
> #
> #  SIGSEGV (0xb) at pc=0x00012102b7ad, pid=7288, tid=22531
> #
> # JRE version: Java(TM) SE Runtime Environment (8.0_74-b02) (build 
> 1.8.0_74-b02)
> # Java VM: Java HotSpot(TM) 64-Bit Server VM (25.74-b02 mixed mode bsd-amd64 
> compressed oops)
> # Problematic frame:
> # C  [librocksdbjni6949968954399704135..jnilib+0x117ad]  
> Java_org_rocksdb_RocksDB_write0+0x1d
> #
> # Core dump written. Default location: /cores/core or core.7288
> #
> # If you would like to submit a bug report, please visit:
> #   http://bugreport.java.com/bugreport/crash.jsp
> # The crash happened outside the Java Virtual Machine in native code.
> # See problematic frame for where to report the bug.
> #



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (KAFKA-3805) Java_org_rocksdb_RocksDB_write error

[Ismael Juma (JIRA)]

[ 
https://issues.apache.org/jira/browse/KAFKA-3805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15320398#comment-15320398
 ] 

Ismael Juma commented on KAFKA-3805:


Are you starting another Kafka Streams instance on the same machine?

> Java_org_rocksdb_RocksDB_write error
> 
>
> Key: KAFKA-3805
> URL: https://issues.apache.org/jira/browse/KAFKA-3805
> Project: Kafka
>  Issue Type: Bug
>  Components: streams
>Affects Versions: 0.10.0.0
> Environment: * Ubuntu Server 16.04 LTS
> - Java: OpenJDK Runtime Environment (build 
> 1.8.0_91-8u91-b14-0ubuntu4~16.04.1-b14)
> * OSX 10.11.5
>- Java: Java(TM) SE Runtime Environment (build 1.8.0_74-b02)
>Reporter: Andres Gomez Ferrer
>Assignee: Guozhang Wang
>Priority: Critical
> Attachments: hs_err_pid7288.log
>
>
> I reproduce the error working with simple two instances of  my kafka streams 
> application reading from one topic writing on other and using a RocksDB store.
> I reproduce it starting one instance, sending some messages on the input 
> topic and later starting another Kafka streams instance. Then, the first 
> instance died and drop a core dump.
> #
> # A fatal error has been detected by the Java Runtime Environment:
> #
> #  SIGSEGV (0xb) at pc=0x00012102b7ad, pid=7288, tid=22531
> #
> # JRE version: Java(TM) SE Runtime Environment (8.0_74-b02) (build 
> 1.8.0_74-b02)
> # Java VM: Java HotSpot(TM) 64-Bit Server VM (25.74-b02 mixed mode bsd-amd64 
> compressed oops)
> # Problematic frame:
> # C  [librocksdbjni6949968954399704135..jnilib+0x117ad]  
> Java_org_rocksdb_RocksDB_write0+0x1d
> #
> # Core dump written. Default location: /cores/core or core.7288
> #
> # If you would like to submit a bug report, please visit:
> #   http://bugreport.java.com/bugreport/crash.jsp
> # The crash happened outside the Java Virtual Machine in native code.
> # See problematic frame for where to report the bug.
> #



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (KAFKA-3802) log mtimes reset on broker restart

[Moritz Siuts (JIRA)]

[ 
https://issues.apache.org/jira/browse/KAFKA-3802?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15320391#comment-15320391
 ] 

Moritz Siuts commented on KAFKA-3802:
-

This 
https://github.com/emetriq/kafka/commit/94bca24bcef5d479f1e4350c65faad2bf7a09246
 seems to fix the issue. Before I do a PR I need to add a good unit tests for 
the change, but maybe you already have feedback.

> log mtimes reset on broker restart
> --
>
> Key: KAFKA-3802
> URL: https://issues.apache.org/jira/browse/KAFKA-3802
> Project: Kafka
>  Issue Type: Bug
>Affects Versions: 0.9.0.1
>Reporter: Andrew Otto
>
> Folks over in 
> http://mail-archives.apache.org/mod_mbox/kafka-users/201605.mbox/%3CCAO8=cz0ragjad1acx4geqcwj+rkd1gmdavkjwytwthkszfg...@mail.gmail.com%3E
>  are commenting about this issue.
> In 0.9, any data log file that was on
> disk before the broker has it's mtime modified to the time of the broker
> restart.
> This causes problems with log retention, as all the files then look like
> they contain recent data to kafka.  We use the default log retention of 7
> days, but if all the files are touched at the same time, this can cause us
> to retain up to 2 weeks of log data, which can fill up our disks.
> This happens *most* of the time, but seemingly not all.  We have seen broker 
> restarts where mtimes were not changed.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (KAFKA-3805) Java_org_rocksdb_RocksDB_write error

[Andres Gomez Ferrer (JIRA)]

 [ 
https://issues.apache.org/jira/browse/KAFKA-3805?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andres Gomez Ferrer updated KAFKA-3805:
---
Attachment: hs_err_pid7288.log

> Java_org_rocksdb_RocksDB_write error
> 
>
> Key: KAFKA-3805
> URL: https://issues.apache.org/jira/browse/KAFKA-3805
> Project: Kafka
>  Issue Type: Bug
>  Components: streams
>Affects Versions: 0.10.0.0
> Environment: * Ubuntu Server 16.04 LTS
> - Java: OpenJDK Runtime Environment (build 
> 1.8.0_91-8u91-b14-0ubuntu4~16.04.1-b14)
> * OSX 10.11.5
>- Java: Java(TM) SE Runtime Environment (build 1.8.0_74-b02)
>Reporter: Andres Gomez Ferrer
>Assignee: Guozhang Wang
>Priority: Critical
> Attachments: hs_err_pid7288.log
>
>
> I reproduce the error working with simple two instances of  my kafka streams 
> application reading from one topic writing on other and using a RocksDB store.
> I reproduce it starting one instance, sending some messages on the input 
> topic and later starting another Kafka streams instance. Then, the first 
> instance died and drop a core dump.
> #
> # A fatal error has been detected by the Java Runtime Environment:
> #
> #  SIGSEGV (0xb) at pc=0x00012102b7ad, pid=7288, tid=22531
> #
> # JRE version: Java(TM) SE Runtime Environment (8.0_74-b02) (build 
> 1.8.0_74-b02)
> # Java VM: Java HotSpot(TM) 64-Bit Server VM (25.74-b02 mixed mode bsd-amd64 
> compressed oops)
> # Problematic frame:
> # C  [librocksdbjni6949968954399704135..jnilib+0x117ad]  
> Java_org_rocksdb_RocksDB_write0+0x1d
> #
> # Core dump written. Default location: /cores/core or core.7288
> #
> # If you would like to submit a bug report, please visit:
> #   http://bugreport.java.com/bugreport/crash.jsp
> # The crash happened outside the Java Virtual Machine in native code.
> # See problematic frame for where to report the bug.
> #



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (KAFKA-3805) Java_org_rocksdb_RocksDB_write error

[Andres Gomez Ferrer (JIRA)]

Andres Gomez Ferrer created KAFKA-3805:
--

 Summary: Java_org_rocksdb_RocksDB_write error
 Key: KAFKA-3805
 URL: https://issues.apache.org/jira/browse/KAFKA-3805
 Project: Kafka
  Issue Type: Bug
  Components: streams
Affects Versions: 0.10.0.0
 Environment: * Ubuntu Server 16.04 LTS
- Java: OpenJDK Runtime Environment (build 
1.8.0_91-8u91-b14-0ubuntu4~16.04.1-b14)

* OSX 10.11.5
   - Java: Java(TM) SE Runtime Environment (build 1.8.0_74-b02)
Reporter: Andres Gomez Ferrer
Assignee: Guozhang Wang
Priority: Critical


I reproduce the error working with simple two instances of  my kafka streams 
application reading from one topic writing on other and using a RocksDB store.

I reproduce it starting one instance, sending some messages on the input topic 
and later starting another Kafka streams instance. Then, the first instance 
died and drop a core dump.

#
# A fatal error has been detected by the Java Runtime Environment:
#
#  SIGSEGV (0xb) at pc=0x00012102b7ad, pid=7288, tid=22531
#
# JRE version: Java(TM) SE Runtime Environment (8.0_74-b02) (build 1.8.0_74-b02)
# Java VM: Java HotSpot(TM) 64-Bit Server VM (25.74-b02 mixed mode bsd-amd64 
compressed oops)
# Problematic frame:
# C  [librocksdbjni6949968954399704135..jnilib+0x117ad]  
Java_org_rocksdb_RocksDB_write0+0x1d
#
# Core dump written. Default location: /cores/core or core.7288
#
# If you would like to submit a bug report, please visit:
#   http://bugreport.java.com/bugreport/crash.jsp
# The crash happened outside the Java Virtual Machine in native code.
# See problematic frame for where to report the bug.
#



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (KAFKA-3802) log mtimes reset on broker restart

[Moritz Siuts (JIRA)]

[ 
https://issues.apache.org/jira/browse/KAFKA-3802?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15320162#comment-15320162
 ] 

Moritz Siuts commented on KAFKA-3802:
-

I have been able to reproduce the problem with a small test program and to hunt 
it down to a specific change. The problem occurs when Kafka is shutted down, so 
when the logfiles are closed.

The problem seems to be introduced with KAFKA-1646 which added {{trim()}} to 
the {{close()}} method in the {{FileMessageSet}}.
The trim method calls {{channel.truncate()}} which on some systems (I can 
reproduce it  on Ubuntu 12.04 with Java7 but not on Mac OS X with Java 8) 
modifies the mtime. If I delete the truncate code in my PoC below the problem 
does not occur.

I think one could fix this, by checking in {{truncateTo()}} that the targetSize 
is different from channel.size before calling truncate on the channel, but I 
was not able to find the time to test this. 

Because the code was not changed in Kafka 0.10.0 it should have the same 
problems.

Code for reproducing (watch the mtime of {{/tmp/kafka.txt}} while it is 
sleeping for 2 Minutes:

{noformat}
import java.io.File;
import java.io.RandomAccessFile;
import java.nio.ByteBuffer;
import java.nio.channels.FileChannel;
import java.util.concurrent.TimeUnit;

public class Main {

public static void main(String[] args) throws Exception {
File file = new File("/tmp/kafka.txt");

FileChannel channel = new RandomAccessFile(file, "rw").getChannel();


channel.write(ByteBuffer.wrap("Kafka".getBytes("UTF-8")));

System.out.println("Going to sleep.");
Thread.sleep(TimeUnit.MINUTES.toMillis(2));

System.out.println("Going to close the channel.");
channel.force(true);
channel.truncate(channel.size()); // problem is here

channel.close();
}
}

{noformat}

> log mtimes reset on broker restart
> --
>
> Key: KAFKA-3802
> URL: https://issues.apache.org/jira/browse/KAFKA-3802
> Project: Kafka
>  Issue Type: Bug
>Affects Versions: 0.9.0.1
>Reporter: Andrew Otto
>
> Folks over in 
> http://mail-archives.apache.org/mod_mbox/kafka-users/201605.mbox/%3CCAO8=cz0ragjad1acx4geqcwj+rkd1gmdavkjwytwthkszfg...@mail.gmail.com%3E
>  are commenting about this issue.
> In 0.9, any data log file that was on
> disk before the broker has it's mtime modified to the time of the broker
> restart.
> This causes problems with log retention, as all the files then look like
> they contain recent data to kafka.  We use the default log retention of 7
> days, but if all the files are touched at the same time, this can cause us
> to retain up to 2 weeks of log data, which can fill up our disks.
> This happens *most* of the time, but seemingly not all.  We have seen broker 
> restarts where mtimes were not changed.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)