Build failed in Jenkins: kafka-trunk-jdk10 #467

See -- Started by an SCM change [EnvInject] - Loading node environment variables. Building remotely on H23 (ubuntu xenial) in workspace

Build failed in Jenkins: kafka-trunk-jdk8 #2955

See Changes: [ismael] MINOR: Use LATEST_1_1 instead of V_1_1_0 in quota_test (#5636) -- [...truncated 2.69 MB...] org.apache.kafka.streams.TopologyTestDriverTest >

Re: [DISCUSS] KIP 368: Allow SASL Connections to Periodically Re-Authenticate

Hi everyone. I've updated the PR to reflect the latest conclusions from this ongoing discussion. The KIP still needs the suggested updates; I will do that later this week. II agree with Rajini that some additional feedback from the community at large would be very helpful at this point in time.

Build failed in Jenkins: kafka-trunk-jdk8 #2954

See Changes: [lindong28] KAFKA-5887; Replace findBugs with spotBugs and upgrade to Gradle 4.10 -- [...truncated 2.69 MB...] org.apache.kafka.streams.MockTimeTest >

Re: KIP-213 - Scalable/Usable Foreign-Key KTable joins - Rebooted.

Hello Adam, Could you elaborate how would the hwm be defined for each record, and the detailed logic of Highwater Mark Processor (e.g. how to bound the space usage of the Highwater Mark store? would it be growing indefinitly)? Maybe it's already included in your running PR but I cannot fully

Re: [DISCUSS] KIP-362: Dynamic Session Window Support

Hello Lei, As Matthias mentioned, the key question here is that because of the late arrivals of records which may indicate a shorter session gap interval, some session windows may be "mistakenly" merged and hence need to be undone the merge, i.e. to split them again. Back to my example, you are

Re: [DISCUSS] KIP-358: Migrate Streams API to Duration instead of long ms times

Hi Nikolay, I'm a +1 to points 1 and 2 above from Guozhang. Thanks, Bill On Mon, Sep 10, 2018 at 6:58 PM Guozhang Wang wrote: > Hello Nikolay, > > Thanks for picking this up! Just sharing my two cents: > > 1) I'd suggest keeping `Punctuator#punctuate(long timestamp)` as is since > comparing

Jenkins build is back to normal : kafka-trunk-jdk10 #465

See

Re: [DISCUSS] KIP-358: Migrate Streams API to Duration instead of long ms times

Hello Nikolay, Thanks for picking this up! Just sharing my two cents: 1) I'd suggest keeping `Punctuator#punctuate(long timestamp)` as is since comparing with other places where we are replacing with Duration and Instant, this is not a user specified value as part of the DSL but rather a

Re: [DISCUSS] KIP-370: Remove Orphan Partitions

Here is the implementation for the KIP 370. https://github.com/xiowu0/kafka/commit/f1bd3085639f41a7af02567550a8e3018cfac3e9 The purpose is to do one time cleanup (after a configured delay) of orphan partitions when a broker starts up. Xiongqi (Wesley) Wu On Wed, Sep 5, 2018 at 10:51 AM

Re: [VOTE] KIP-291: Have separate queues for control requests and data requests

To be clear, we can only remove configs in major new versions. Otherwise, we can only deprecate them. Ismael On Mon, Sep 10, 2018 at 10:47 AM Jun Rao wrote: > Hi, Lucas, > > For the network idlePct, your understanding is correct. Currently, > networkIdlePct metric is calculated as the average

Build failed in Jenkins: kafka-trunk-jdk8 #2953

See Changes: [wangguoz] MINOR: Mark new Scala streams tests as integration tests (KIP-270 -- [...truncated 2.69 MB...] org.apache.kafka.streams.test.OutputVerifierTest >

Re: [VOTE] KIP-354 Time-based log compaction policy

On Mon, Sep 10, 2018, at 11:44, xiongqi wu wrote: > Thank you for comments. I will use '0' for now. > > If we create topics through admin client in the future, we might perform > some useful checks. (but the assumption is all brokers in the same cluster > have the same default configurations

Build failed in Jenkins: kafka-trunk-jdk10 #464

See Changes: [wangguoz] MINOR: Mark new Scala streams tests as integration tests (KIP-270 -- [...truncated 2.21 MB...] org.apache.kafka.streams.test.OutputVerifierTest >

Re: [VOTE] KIP-354 Time-based log compaction policy

Thank you for comments. I will use '0' for now. If we create topics through admin client in the future, we might perform some useful checks. (but the assumption is all brokers in the same cluster have the same default configurations value, otherwise,it might still be tricky to do such cross

[jira] [Created] (KAFKA-7395) Add fencing to OffsetsForLeaderEpoch API (KIP-320)

Jason Gustafson created KAFKA-7395: -- Summary: Add fencing to OffsetsForLeaderEpoch API (KIP-320) Key: KAFKA-7395 URL: https://issues.apache.org/jira/browse/KAFKA-7395 Project: Kafka Issue

Re: [VOTE] KIP-354 Time-based log compaction policy

I don't have a strong opinion. But I think we should probably be consistent with how segment.ms works, and just use 0. best, Colin On Wed, Sep 5, 2018, at 21:19, Brett Rann wrote: > OK thanks for that clarification. I see why you're uncomfortable with 0 now. > > I'm not really fussed. I just

Re: [VOTE] KIP-354 Time-based log compaction policy

On Wed, Sep 5, 2018, at 10:26, xiongqi wu wrote: > Colin, > > When a user creates a topic, the user doesn't know broker's configures. If > there is 100 brokers, the user can't contact all the brokers to find out > the minimum allowed value for segment.ms. Today, the topic creation is > done via

Re: [DISCUSS] KIP-358: Migrate Streams API to Duration instead of long ms times

Hello, Matthias. > (4) While I agree that we might want to deprecate it, I am not sure if this > should be part of this KIP? > Seems to be unrelated? > Should this have been part of KIP-319? > If yes, we might still want to updated this other KIP? WDYT? OK, I removed this deprecation from

Re: [DISCUSS] KIP-349 Priorities for Source Topics

Oh, also, I am -1 on disabling incremental fetch requests when the prioritization feature is used. Users often have performance problems that are difficult to understand when they use various combinations of features. Of course as implmentors we "know" what the right combination of features

Re: [DISCUSS] KIP-349 Priorities for Source Topics

On Thu, Sep 6, 2018, at 05:24, Jan Filipiak wrote: > > On 05.09.2018 17:18, Colin McCabe wrote: > > Hi all, > > > > I agree that DISCUSS is more appropriate than VOTE at this point, since I > > don't remember the last discussion coming to a definite conclusion. > > > > I guess my concern is that

Re: [VOTE] KIP-349 Priorities for Source Topics

Hmm. My understanding is that streams doesn't need anything like this since streams pauses topics when it doesn't need more data from them. (Matthias, can you confirm?) best, Colin On Mon, Aug 20, 2018, at 06:01, Thomas Becker wrote: > I agree with Jan. A strategy interface for choosing

Re: [DISCUSS] KIP-349 Priorities for Source Topics

On Thu, Sep 6, 2018, at 20:21, Matthias J. Sax wrote: > I am still not sure how Samza's MessageChooser actually works and how > this would align with KafkaConsumer fetch requests. > > > Maybe I can give some background (conceptually); @Colin, please correct > me if I say anything wrong: > > >

Re: [VOTE] KIP-291: Have separate queues for control requests and data requests

Hi, Lucas, For the network idlePct, your understanding is correct. Currently, networkIdlePct metric is calculated as the average of (1 - io-ratio) in the selector of all network threads. The metrics part looks good to me in the updated KIP. I am not still not quite sure about the configs. 100.

[jira] [Created] (KAFKA-7394) Allow OffsetsForLeaderEpoch requests with topic describe ACL (KIP-320)

Jason Gustafson created KAFKA-7394: -- Summary: Allow OffsetsForLeaderEpoch requests with topic describe ACL (KIP-320) Key: KAFKA-7394 URL: https://issues.apache.org/jira/browse/KAFKA-7394 Project:

[jira] [Resolved] (KAFKA-4978) kafka.metrics.reporters missing from the documentation

[ https://issues.apache.org/jira/browse/KAFKA-4978?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manikumar resolved KAFKA-4978. -- Resolution: Duplicate Resolving as duplicated of KAFKA-5066 which will add docs for KafkaMetricsConfig

[jira] [Resolved] (KAFKA-2244) Document Kafka metrics configuration properties

[ https://issues.apache.org/jira/browse/KAFKA-2244?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manikumar resolved KAFKA-2244. -- Resolution: Duplicate Resolving as duplicated of KAFKA-5066 which will add docs for KafkaMetricsConfig

Re: [VOTE] KIP-363: Allow performance tools to print final results to output file

+1 (non-binding) Thanks for the KIP. On Wed, Sep 5, 2018 at 8:24 PM Viktor Somogyi-Vass wrote: > +1 (non-binding), > > Thanks for the KIP! > > Viktor > > On Wed, Sep 5, 2018 at 10:14 AM Attila Sasvári > wrote: > > > Hi All, > > > > I'd like to start the vote on KIP-363: > > * > > >

[jira] [Resolved] (KAFKA-1833) OfflinePartitionLeaderSelector may return null leader when ISR and Assgined Broker have no common

[ https://issues.apache.org/jira/browse/KAFKA-1833?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manikumar resolved KAFKA-1833. -- Resolution: Auto Closed Closing as PartitionLeaderSelector class is removed/refactored. >

[jira] [Created] (KAFKA-7393) Consider making suppression node names independent of topologically numbered names.

John Roesler created KAFKA-7393: --- Summary: Consider making suppression node names independent of topologically numbered names. Key: KAFKA-7393 URL: https://issues.apache.org/jira/browse/KAFKA-7393

Re: [VOTE] KIP-357: Add support to list ACLs per principal

Bump! On Tue, Sep 4, 2018 at 11:58 PM Adam Bellemare wrote: > +1 (non binding) - would really like to see this one. > > On Mon, Sep 3, 2018 at 12:18 PM, Mickael Maison > wrote: > > > +1 (non binding) > > On Mon, Sep 3, 2018 at 3:14 PM Manikumar > > wrote: > > > > > > bump up! waiting for 2

Re: [DISCUSS] KIP 368: Allow SASL Connections to Periodically Re-Authenticate

Hi Ron, Thank you for the analysis. Yes, I agree with the differences you have pointed out between the two approaches. 1. Re-authenticating idle connections (or connections nearing idle timeout): With the lower-level approach, there is nothing stopping us from re-authenticating idle

Re: [VOTE] KIP-367 Introduce close(Duration) to Producer and AdminClient instead of close(long, TimeUnit)

+1. (Non-binding) On Mon, Sep 10, 2018 at 4:13 AM Matthias J. Sax wrote: > Thanks a lot for the KIP. > > +1 (binding) > > > -Matthias > > > On 9/8/18 11:27 AM, Chia-Ping Tsai wrote: > > Hi All, > > > > I'd like to put KIP-367 to the vote. > > > > >

Re: [ANNOUNCE] Kafka Committers Guidelines

Comprehensive, Clear and Helpful. Thanks for your great work! Best, Dongjin On Fri, Sep 7, 2018 at 6:36 AM Guozhang Wang wrote: > Hello folks, > > Many of the Kafka PMC members have received emails asking about "how can I > become a committer in Apache Kafka", and so we figured it is a good

Re: [DISCUSS] KIP-110: Add Codec for ZStandard Compression (Updated)

Hi Jason, You are right. Explicit statements are always better. I updated the document following your suggestion. @Magnus Thanks for the inspection. It seems like a new error code is not a problem. Thanks, Dongjin On Fri, Sep 7, 2018 at 3:23 AM Jason Gustafson wrote: > Hi Dongjin, > > The