[jira] [Commented] (KAFKA-3775) Throttle maximum number of tasks assigned to a single KafkaStreams

[ https://issues.apache.org/jira/browse/KAFKA-3775?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15313773#comment-15313773 ] The Data Lorax commented on KAFKA-3775: --- I wonder how surah a design would maintain

[jira] [Comment Edited] (KAFKA-3775) Throttle maximum number of tasks assigned to a single KafkaStreams

[ https://issues.apache.org/jira/browse/KAFKA-3775?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15313773#comment-15313773 ] The Data Lorax edited comment on KAFKA-3775 at 6/3/16 7:24 AM: -

[jira] [Commented] (KAFKA-3775) Throttle maximum number of tasks assigned to a single KafkaStreams

[ https://issues.apache.org/jira/browse/KAFKA-3775?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15313828#comment-15313828 ] Yuto Kawamura commented on KAFKA-3775: -- Thanks for feedback [~BigAndy] . > With the

[jira] [Comment Edited] (KAFKA-3775) Throttle maximum number of tasks assigned to a single KafkaStreams

[ https://issues.apache.org/jira/browse/KAFKA-3775?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15313828#comment-15313828 ] Yuto Kawamura edited comment on KAFKA-3775 at 6/3/16 8:17 AM: --

[GitHub] kafka pull request #1464: KAFKA-3785; Fetcher spending unnecessary time duri...

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/1464 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[jira] [Commented] (KAFKA-3785) Fetcher spending unnecessary time during metrics recording

[ https://issues.apache.org/jira/browse/KAFKA-3785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15313862#comment-15313862 ] ASF GitHub Bot commented on KAFKA-3785: --- Github user asfgit closed the pull request

[jira] [Resolved] (KAFKA-3785) Fetcher spending unnecessary time during metrics recording

[ https://issues.apache.org/jira/browse/KAFKA-3785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma resolved KAFKA-3785. Resolution: Fixed > Fetcher spending unnecessary time during metrics recording > ---

[jira] [Updated] (KAFKA-3785) Fetcher spending unnecessary time during metrics recording

[ https://issues.apache.org/jira/browse/KAFKA-3785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-3785: --- Fix Version/s: 0.10.0.1 > Fetcher spending unnecessary time during metrics recording > ---

[jira] [Updated] (KAFKA-3785) Fetcher spending unnecessary time during metrics recording

[ https://issues.apache.org/jira/browse/KAFKA-3785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-3785: --- Assignee: Greg Fodor > Fetcher spending unnecessary time during metrics recording > --

[jira] [Created] (KAFKA-3787) Preserve message timestamp in mirror mkaer

TAO XIAO created KAFKA-3787: --- Summary: Preserve message timestamp in mirror mkaer Key: KAFKA-3787 URL: https://issues.apache.org/jira/browse/KAFKA-3787 Project: Kafka Issue Type: Bug Comp

[GitHub] kafka pull request #1466: KAFKA-3787: Preserve the message timestamp in mirr...

GitHub user xiaotao183 opened a pull request: https://github.com/apache/kafka/pull/1466 KAFKA-3787: Preserve the message timestamp in mirror maker The timestamp of messages consumed by mirror maker is not preserved after sending to target cluster. The correct behavior is to keep cre

[jira] [Commented] (KAFKA-3787) Preserve message timestamp in mirror mkaer

[ https://issues.apache.org/jira/browse/KAFKA-3787?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15313886#comment-15313886 ] ASF GitHub Bot commented on KAFKA-3787: --- GitHub user xiaotao183 opened a pull reques

[jira] [Updated] (KAFKA-3787) Preserve message timestamp in mirror mkaer

[ https://issues.apache.org/jira/browse/KAFKA-3787?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] TAO XIAO updated KAFKA-3787: Status: Patch Available (was: Open) > Preserve message timestamp in mirror mkaer >

Build failed in Jenkins: kafka-0.10.0-jdk7 #113

See Changes: [ismael] KAFKA-3785; Fetcher spending unnecessary time during metrics recording -- Started by an SCM change [EnvInject] - Loading node environment variables. Building remotely on H1

[jira] [Updated] (KAFKA-3787) Preserve message timestamp in mirror mkaer

[ https://issues.apache.org/jira/browse/KAFKA-3787?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-3787: --- Fix Version/s: 0.10.0.1 > Preserve message timestamp in mirror mkaer > ---

[GitHub] kafka pull request #1456: MINOR: Add user overridden test logging events

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/1456 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

Build failed in Jenkins: kafka-trunk-jdk7 #1336

See Changes: [ismael] KAFKA-3785; Fetcher spending unnecessary time during metrics recording [ismael] MINOR: Add user overridden test logging events -- Started by an SCM change [EnvInject] - Lo

Jenkins build is back to normal : kafka-trunk-jdk8 #672

See

Build failed in Jenkins: kafka-trunk-jdk8 #673

See Changes: [ismael] MINOR: Add user overridden test logging events -- Started by an SCM change [EnvInject] - Loading node environment variables. Building remotely on ubuntu-us1 (Ubuntu ubuntu u

Build failed in Jenkins: kafka-0.10.0-jdk7 #114

See Changes: [ismael] MINOR: Add user overridden test logging events -- Started by an SCM change [EnvInject] - Loading node environment variables. Building remotely on ubuntu-us1 (Ubuntu ubuntu

[jira] [Created] (KAFKA-3788) Potential message lost when switching to new segment

Arkadiusz Firus created KAFKA-3788: -- Summary: Potential message lost when switching to new segment Key: KAFKA-3788 URL: https://issues.apache.org/jira/browse/KAFKA-3788 Project: Kafka Issue

[jira] [Commented] (KAFKA-3775) Throttle maximum number of tasks assigned to a single KafkaStreams

[ https://issues.apache.org/jira/browse/KAFKA-3775?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15314258#comment-15314258 ] The Data Lorax commented on KAFKA-3775: --- Yes, the risk I'm taking about is unassigne

[GitHub] kafka pull request #1413: MINOR: set charset of Javadoc to UTF-8

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/1413 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[jira] [Commented] (KAFKA-3680) Make Java client classloading more flexible

[ https://issues.apache.org/jira/browse/KAFKA-3680?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15314411#comment-15314411 ] Avi Flax commented on KAFKA-3680: - I’m very much looking forward to this fix! I’m using Ka

[jira] [Created] (KAFKA-3789) Upgrade Snappy to fix snappy decompression errors

Grant Henke created KAFKA-3789: -- Summary: Upgrade Snappy to fix snappy decompression errors Key: KAFKA-3789 URL: https://issues.apache.org/jira/browse/KAFKA-3789 Project: Kafka Issue Type: Bug

Build failed in Jenkins: kafka-trunk-jdk8 #674

See Changes: [me] MINOR: set charset of Javadoc to UTF-8 -- Started by an SCM change [EnvInject] - Loading node environment variables. Building remotely on ubuntu3 (Ubuntu ubuntu legacy-ubuntu) i

[jira] [Created] (KAFKA-3790) Default options when removing ACLs do not comply with documentation

Sébastien Launay created KAFKA-3790: --- Summary: Default options when removing ACLs do not comply with documentation Key: KAFKA-3790 URL: https://issues.apache.org/jira/browse/KAFKA-3790 Project: Kafk

Re: KStreams Rewind Offset

Hello Michael, Just want to clarify a few thing before I can provide some suggestions in simplifying your pipeline with Kafka Streams: as for the "race condition between reading the stream data and the data being removed", do you mean that the stream records is actually mutable, hence after they a

[GitHub] kafka pull request #1467: KAFKA-3789: Upgrade Snappy to fix snappy decompres...

GitHub user granthenke opened a pull request: https://github.com/apache/kafka/pull/1467 KAFKA-3789: Upgrade Snappy to fix snappy decompression errors You can merge this pull request into a Git repository by running: $ git pull https://github.com/granthenke/kafka snappy-fix Al

[jira] [Commented] (KAFKA-3789) Upgrade Snappy to fix snappy decompression errors

[ https://issues.apache.org/jira/browse/KAFKA-3789?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15314443#comment-15314443 ] ASF GitHub Bot commented on KAFKA-3789: --- GitHub user granthenke opened a pull reques

[jira] [Updated] (KAFKA-3789) Upgrade Snappy to fix snappy decompression errors

[ https://issues.apache.org/jira/browse/KAFKA-3789?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Grant Henke updated KAFKA-3789: --- Status: Patch Available (was: Open) > Upgrade Snappy to fix snappy decompression errors > ---

[GitHub] kafka pull request #1468: KAFKA-3790: Allow for removal of non specific ACLs

GitHub user slaunay opened a pull request: https://github.com/apache/kafka/pull/1468 KAFKA-3790: Allow for removal of non specific ACLs - remove ACLs with `aclMatch()` rather than `Object#equals(Object)` - remove unused session argument from `aclMatch()` to reuse it in `removeAc

[jira] [Commented] (KAFKA-3790) Default options when removing ACLs do not comply with documentation

[ https://issues.apache.org/jira/browse/KAFKA-3790?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15314474#comment-15314474 ] ASF GitHub Bot commented on KAFKA-3790: --- GitHub user slaunay opened a pull request:

Jenkins build is back to normal : kafka-trunk-jdk7 #1337

See

Re: [DISCUSS] KIP-47 - Add timestamp-based log deletion policy

Bumping this thread so Wes can reply to it. Ignore this mail. 2016-02-24 0:36 GMT+01:00 Joel Koshy : > Great - thanks for clarifying. > > Joel > > On Tue, Feb 23, 2016 at 1:47 PM, Bill Warshaw wrote: > > > Sorry that I didn't see this comment before the meeting Joel. I'll try > to > > clarify w

Re: [DISCUSS] KIP-54 Sticky Partition Assignment Strategy

Sorry for being late on this thread. The assign() function is auto-triggered during the rebalance by one of the consumers when it receives all subscription information collected from the server-side coordinator. More details can be found here: https://cwiki.apache.org/confluence/display/KAFKA/Kaf

[jira] [Commented] (KAFKA-3777) Extract the LRU cache out of RocksDBStore

[ https://issues.apache.org/jira/browse/KAFKA-3777?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15314561#comment-15314561 ] Guozhang Wang commented on KAFKA-3777: -- Adding to what Jay mentioned, we are already

[jira] [Updated] (KAFKA-3784) TimeWindows#windowsFor misidentifies some windows if TimeWindows#advanceBy is used

[ https://issues.apache.org/jira/browse/KAFKA-3784?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang updated KAFKA-3784: - Component/s: (was: core) streams > TimeWindows#windowsFor misidentifies some

[jira] [Updated] (KAFKA-3784) TimeWindows#windowsFor misidentifies some windows if TimeWindows#advanceBy is used

[ https://issues.apache.org/jira/browse/KAFKA-3784?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang updated KAFKA-3784: - Assignee: Tom Rybak > TimeWindows#windowsFor misidentifies some windows if TimeWindows#advanceBy i

Re: [DISCUSS] KIP-62: Allow consumer to send heartbeats from a background thread

Hey Ewen, I confess your comments caught me off guard. It never occurred to me that anyone would ask for a rebalance timeout so that it could be set _larger_ than the process timeout. Even with buffered or batch processing, I would usually expect flushing before a rebalance to take no more time th

[jira] [Commented] (KAFKA-3775) Throttle maximum number of tasks assigned to a single KafkaStreams

[ https://issues.apache.org/jira/browse/KAFKA-3775?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15314606#comment-15314606 ] Guozhang Wang commented on KAFKA-3775: -- Hi [~mjsax] [~kawamuray], great discussion re

[jira] [Commented] (KAFKA-3705) Support non-key joining in KTable

[ https://issues.apache.org/jira/browse/KAFKA-3705?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15314643#comment-15314643 ] Guozhang Wang commented on KAFKA-3705: -- Another thing we may want to consider is to u

[jira] [Commented] (KAFKA-3775) Throttle maximum number of tasks assigned to a single KafkaStreams

[ https://issues.apache.org/jira/browse/KAFKA-3775?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15314721#comment-15314721 ] Jay Kreps commented on KAFKA-3775: -- I'd have to agree I think this is a memory management

[GitHub] kafka pull request #1462: KAFKA-3784 TimeWindows#windowsFor misidentifies so...

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/1462 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[jira] [Commented] (KAFKA-3784) TimeWindows#windowsFor misidentifies some windows if TimeWindows#advanceBy is used

[ https://issues.apache.org/jira/browse/KAFKA-3784?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15314736#comment-15314736 ] ASF GitHub Bot commented on KAFKA-3784: --- Github user asfgit closed the pull request

[jira] [Updated] (KAFKA-3784) TimeWindows#windowsFor misidentifies some windows if TimeWindows#advanceBy is used

[ https://issues.apache.org/jira/browse/KAFKA-3784?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang updated KAFKA-3784: - Labels: architecture (was: ) > TimeWindows#windowsFor misidentifies some windows if TimeWindows#a

[jira] [Resolved] (KAFKA-3784) TimeWindows#windowsFor misidentifies some windows if TimeWindows#advanceBy is used

[ https://issues.apache.org/jira/browse/KAFKA-3784?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang resolved KAFKA-3784. -- Resolution: Fixed Fix Version/s: 0.10.0.1 Issue resolved by pull request 1462 [https://gi

[GitHub] kafka pull request #1425: KAFKA-3728 EndToEndAuthorizationTest offsets_topic...

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/1425 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[jira] [Commented] (KAFKA-3728) EndToEndAuthorizationTest offsets_topic misconfigured

[ https://issues.apache.org/jira/browse/KAFKA-3728?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15314748#comment-15314748 ] ASF GitHub Bot commented on KAFKA-3728: --- Github user asfgit closed the pull request

Re: [DISCUSS] KIP-63: Unify store and downstream caching in streams

Just to clarify: "cache.max.bytes.buffering" is per processor? On Thu, Jun 2, 2016 at 11:30 AM, Eno Thereska wrote: > Hi there, > > I have created KIP-63: Unify store and downstream caching in streams > https://cwiki.apache.org/confluence/display/KAFKA/KIP-63%3A+Unify+store+and+downstream+cachin

Build failed in Jenkins: kafka-trunk-jdk7 #1338

See Changes: [wangguoz] KAFKA-3784: TimeWindows#windowsFor calculation is incorrect [ismael] KAFKA-3728; EndToEndAuthorizationTest offsets_topic misconfigured -- Started by an SCM change [EnvIn

[jira] [Resolved] (KAFKA-3728) EndToEndAuthorizationTest offsets_topic misconfigured

[ https://issues.apache.org/jira/browse/KAFKA-3728?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma resolved KAFKA-3728. Resolution: Fixed Fix Version/s: 0.10.0.1 Issue resolved by pull request 1425 [https://github

Kafka 0.10.0.0 broker configuration to support older consumer/producer clients...

We are in the process of upgrading the broker to Kafka 0.10.0.0. The following documentation is not too clear to us: "By setting the message format version, one certifies that all existing messages are on or below that message format version. Otherwise consumers before 0.10.0.0 might break." http:

Build failed in Jenkins: kafka-trunk-jdk8 #675

See Changes: [wangguoz] KAFKA-3784: TimeWindows#windowsFor calculation is incorrect [ismael] KAFKA-3728; EndToEndAuthorizationTest offsets_topic misconfigured -- Started by an SCM change [EnvInj

Build failed in Jenkins: kafka-0.10.0-jdk7 #115

See Changes: [wangguoz] KAFKA-3784: TimeWindows#windowsFor calculation is incorrect [ismael] KAFKA-3728; EndToEndAuthorizationTest offsets_topic misconfigured -- Started by an SCM change [EnvIn

Re: [DISCUSS] KIP-63: Unify store and downstream caching in streams

Hi Gwen, Yes. As an example, if cache.max.bytes.buffering set to X, and if users have A aggregation operators and T KTable.to() operators, then X*(A + T) total bytes will be allocated for caching. Eno > On 3 Jun 2016, at 21:37, Gwen Shapira wrote: > > Just to clarify: "cache.max.bytes.buffer

Re: [DISCUSS] KIP-63: Unify store and downstream caching in streams

Hi Eno, Does this mean that Kafka Streams will disable the RocksDB write buffer? Is it currently safe to recover a Kafka Streams application after SIGKILL on the same machine? If not, will this make it safe to do so? If RocksDB is not flushed before offsets are commited in Kafka and is killed w

Re: [DISCUSS] KIP-63: Unify store and downstream caching in streams

Hey Eno, Should the config be the global memory use rather than the per-processor? That is, let’s say I know I have fixed a 1GB heap because that is what I set for Java, and want to use 100MB for caching, it seems like right now I’d have to do some math that depends on my knowing a bit about how c

[jira] [Commented] (KAFKA-3693) Race condition between highwatermark-checkpoint thread and handleLeaderAndIsrRequest at broker start-up

[ https://issues.apache.org/jira/browse/KAFKA-3693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15315055#comment-15315055 ] Maysam Yabandeh commented on KAFKA-3693: Thanks [~junrao]. My understanding is tha

[jira] [Commented] (KAFKA-3693) Race condition between highwatermark-checkpoint thread and handleLeaderAndIsrRequest at broker start-up

[ https://issues.apache.org/jira/browse/KAFKA-3693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15315086#comment-15315086 ] Jun Rao commented on KAFKA-3693: Hmm, if broker 16 was down at 06:17:02,012, it shouldn't

[jira] [Commented] (KAFKA-3693) Race condition between highwatermark-checkpoint thread and handleLeaderAndIsrRequest at broker start-up

[ https://issues.apache.org/jira/browse/KAFKA-3693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15315112#comment-15315112 ] Maysam Yabandeh commented on KAFKA-3693: As mentioned above: bq. We managed to rep