Jenkins build is back to normal : kafka-trunk-jdk10 #94

See

Build failed in Jenkins: kafka-trunk-jdk7 #3415

See Changes: [junrao] KAFKA-6361: Fix log divergence between leader and follower after fast -- [...truncated 416.38 KB...] kafka.coordinator.transaction.TransactionCoord

Re: Use of a formatter like Scalafmt

It certainly annoys me every time I open the code and my linter starts highlighting that some code is indented with spaces and some with tabs... increasing consistency across the codebase would be appreciated. On Wed, May 9, 2018 at 9:10 PM, Ismael Juma wrote: > Sounds good about doing this for

Re: Use of a formatter like Scalafmt

Sounds good about doing this for Kafka streams scala first. Core is a bit more complicated so may require more discussion. Ismael On Wed, 9 May 2018, 16:59 Matthias J. Sax, wrote: > Joan, > > thanks for starting this initiative. I am overall +1 > > However, I am worried about applying it to `co

Build failed in Jenkins: kafka-trunk-jdk10 #93

See -- Started by an SCM change Started by an SCM change [EnvInject] - Loading node environment variables. Building remotely on H24 (ubuntu xenial) in workspace

Jenkins build is back to normal : kafka-trunk-jdk7 #3414

See

Jenkins build is back to normal : kafka-1.1-jdk7 #129

See

Build failed in Jenkins: kafka-trunk-jdk8 #2634

See Changes: [wangguoz] KAFKA-6878 Switch the order of underlying.init and initInternal (#4988) [github] KAFKA-6813: Remove deprecated APIs in KIP-182, Part II (#4976) [junrao] KAFKA-6361: Fix log divergence

[jira] [Resolved] (KAFKA-6361) Fast leader fail over can lead to log divergence between leader and follower

[ https://issues.apache.org/jira/browse/KAFKA-6361?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao resolved KAFKA-6361. Resolution: Fixed Fix Version/s: 2.0.0 Merged the PR to trunk. > Fast leader fail over can lead to l

Jenkins build is back to normal : kafka-trunk-jdk8 #2633

See

Jenkins build is back to normal : kafka-trunk-jdk10 #91

See

[jira] [Resolved] (KAFKA-6878) NPE when querying global state store not in READY state

[ https://issues.apache.org/jira/browse/KAFKA-6878?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang resolved KAFKA-6878. -- Resolution: Fixed Fix Version/s: 1.1.1 2.0.0 > NPE when querying globa

Re: Use of a formatter like Scalafmt

Joan, thanks for starting this initiative. I am overall +1 However, I am worried about applying it to `core` module as the change is massive. For the Kafka Streams Scala module, that is new and was not released yet, I am +1. A personal thing about the style: the 2-space indention for JavaDocs is

Re: Use of a formatter like Scalafmt

Hi Ted, As far as I understand this is an issue for PRs and back porting the changes to other branches. Applying the tool to the other branches should also resolve the conflicts as the formattings will match, leaving only the actual changes in the diffs. That's what we did sometime ago at my work

Re: [DISCUSS] KIP-297: Externalizing Secrets for Connect Configurations

Hi Magesh, I updated the KIP with a link to a PR for a working prototype. The prototype does not yet use the Connect plugin machinery for class loader isolation, but should give you an idea of what the final implementation will look like. Here is the link: https://github.com/apache/kafka/pull/49

Re: Use of a formatter like Scalafmt

Applying the tool across code base would result in massive changes. How would this be handled ? Original message From: Joan Goyeau Date: 5/9/18 3:31 PM (GMT-08:00) To: dev@kafka.apache.org Subject: Use of a formatter like Scalafmt Hi, Contributing to Kafka Streams' Scala API

Build failed in Jenkins: kafka-trunk-jdk7 #3413

See Changes: [jason] KAFKA-6299; Fix AdminClient error handling when metadata changes (#4295) -- [...truncated 1.28 MB...] at org.gradle.internal.concurrent.Threa

[jira] [Created] (KAFKA-6891) send.buffer.bytes should be allowed to set -1 in KafkaConnect

Oleg Kuznetsov created KAFKA-6891: - Summary: send.buffer.bytes should be allowed to set -1 in KafkaConnect Key: KAFKA-6891 URL: https://issues.apache.org/jira/browse/KAFKA-6891 Project: Kafka

Use of a formatter like Scalafmt

Hi, Contributing to Kafka Streams' Scala API, I've been kinda lost on how should I format my code. I know formatting is the start of religion wars but personally I have no preference at all. I just want consistency across the codebase, no unnecessary formatting diffs in PRs and offload the formatt

[jira] [Created] (KAFKA-6890) Add connector level configurability for producer/consumer client configs

Allen Tang created KAFKA-6890: - Summary: Add connector level configurability for producer/consumer client configs Key: KAFKA-6890 URL: https://issues.apache.org/jira/browse/KAFKA-6890 Project: Kafka

Re: [VOTE] KIP-266: Add TimeoutException for KafkaConsumer#position

+1 from me, thanks! Guozhang On Wed, May 9, 2018 at 10:46 AM, Jason Gustafson wrote: > Thanks for the KIP, +1 (binding). > > One small correction: the KIP mentions that close() will be deprecated, but > we do not want to do this because it is needed by the Closeable interface. > We only want t

[jira] [Created] (KAFKA-6889) Fix Streams system test to only specify used log files

Matthias J. Sax created KAFKA-6889: -- Summary: Fix Streams system test to only specify used log files Key: KAFKA-6889 URL: https://issues.apache.org/jira/browse/KAFKA-6889 Project: Kafka Issu

[jira] [Resolved] (KAFKA-6299) Fix AdminClient error handling when metadata changes

[ https://issues.apache.org/jira/browse/KAFKA-6299?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Gustafson resolved KAFKA-6299. Resolution: Fixed Fix Version/s: 2.0.0 > Fix AdminClient error handling when metadata

Re: [VOTE] KIP-283: Efficient Memory Usage for Down-Conversion

Thanks for the feedback, Jason and Ismael. I renamed the config to "message.downconversion.enable". Also, as an update, I found a potential problem with one of the suggestions the KIP made, specifically about the case where the size of messages after down-conversion is greater than the size before

Build failed in Jenkins: kafka-trunk-jdk8 #2632

See Changes: [wangguoz] KAFKA 6673: Implemented missing override equals method (#4745) [wangguoz] KAFKA-6878: NPE when querying global state store not in READY state -- S

Build failed in Jenkins: kafka-1.1-jdk7 #128

See Changes: [wangguoz] KAFKA-6878: NPE when querying global state store not in READY state [matthias] HOTFIX: remove empty line to pass checkstyle -- Started by an SCM chan

Build failed in Jenkins: kafka-trunk-jdk10 #90

See Changes: [wangguoz] KAFKA 6673: Implemented missing override equals method (#4745) [wangguoz] KAFKA-6878: NPE when querying global state store not in READY state -- St

Build failed in Jenkins: kafka-trunk-jdk7 #3412

See Changes: [wangguoz] KAFKA 6673: Implemented missing override equals method (#4745) [wangguoz] KAFKA-6878: NPE when querying global state store not in READY state -- S

Re: [VOTE] KIP-222 - Add "describe consumer group" to KafkaAdminClient

Hi All, We found one more gap in this KIP. It is useful for the describeConsumerGroups API to expose the group state so that users can tell when a group is rebalancing. Colin submitted a PR with the proposed changes here: https://github.com/apache/kafka/pull/4980. Please review and let us know if

Re: [DISCUSS] KIP-280: Enhanced log compaction

I have thought about being consistency in strategy v.s. practical concerns about storage convenience to its impact on compaction effectiveness. The different between timestamp and the header key-value pairs is that for the latter, as I mentioned before, "it is arguably out of Kafka's control, and

Re: [DISCUSS] KIP-278: Add version option to Kafka's commands

+1. Thanks, Sasaki. Colin On Wed, May 9, 2018, at 09:15, Jason Gustafson wrote: > Hi Sasaki, > > Thanks for the update. The KIP looks good to me. I'd suggest moving to a > vote. > > Thanks, > Jason > > On Mon, May 7, 2018 at 7:08 AM, Sasaki Toru > wrote: > > > Hi Manikumar, Colin, > > > > Th

Re: [DISCUSS] KIP-273 Kafka to support using ETCD beside Zookeeper

Hi Molnar, The points Ismael brought up earlier (and that were brought up on KIP-30) are still relevant here. As Ismael said, the goal is to get rid of external dependencies here. We're going to post more about this soon (sorry for the delay) thanks, Colin On Wed, May 9, 2018, at 07:29, M

Re: [VOTE] KIP-266: Add TimeoutException for KafkaConsumer#position

Thanks for the KIP, +1 (binding). One small correction: the KIP mentions that close() will be deprecated, but we do not want to do this because it is needed by the Closeable interface. We only want to deprecate close(long, TimeUnit) in favor of close(Duration). -Jason On Tue, May 8, 2018 at 12:4

[jira] [Resolved] (KAFKA-6673) Segment and Stamped implement Comparable, but don't override equals.

[ https://issues.apache.org/jira/browse/KAFKA-6673?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang resolved KAFKA-6673. -- Resolution: Fixed Fix Version/s: 2.0.0 > Segment and Stamped implement Comparable, but do

[jira] [Reopened] (KAFKA-4327) Move Reset Tool from core to streams

[ https://issues.apache.org/jira/browse/KAFKA-4327?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax reopened KAFKA-4327: I discussed with [~guozhang] about this because I think we should actually move the tool to str

Kafka offset problem when using Spark Streaming...

Hi all, We're facing some problems with ours Spark Streaming jobs, from yesterday we have got the following error into our logs when the jobs fail: java.lang.AssertionError: assertion failed: Beginning offset 562747 is after the ending offset 562743 for topic elk-topic partition 0. Any help ab

Re: [DISCUSS] KIP-297: Externalizing Secrets for Connect Configurations

Hi Ron, Thanks for the pointer! I'll take a look at KIP 269. I will also put together a PR to demonstrate what the implementation might look like. Thanks, Robert On Wed, May 9, 2018 at 6:44 AM, Ron Dagostino wrote: > Hi Robert. You may find KIP-269: Substitution Within Configuration Values

Re: [DISCUSS] KIP-297: Externalizing Secrets for Connect Configurations

Hi Magesh, Thanks for the feedback! I will put together a PR to demonstrate what the implementation might look like, as well as a reference FileConfigProvider. 1. The delayMs for a (potentially) scheduled reload is determined by the ConfigProvider. For example, a (hypothetical) VaultConfigProv

Re: [VOTE] KIP-294 - Enable TLS hostname verification by default

+1, thanks for the KIP! On Wed, May 9, 2018 at 4:41 PM, Ted Yu wrote: > +1 > > On Wed, May 9, 2018 at 8:28 AM, Rajini Sivaram > wrote: > >> Hi all, >> >> Since there have been no objections on this straightforward KIP, I would >> like to initiate the voting process. KIP-294 proposes to use a sec

Re: [VOTE] KIP-281: ConsumerPerformance: Increase Polling Loop Timeout and Make It Reachable by the End User

+1 (binding) Thanks for the KIP, Alex! Regards, Rajini On Sat, Apr 21, 2018 at 3:35 AM, Ted Yu wrote: > You need two more binding votes. > > FYI > > On Thu, Apr 19, 2018 at 10:45 PM, Alex Dunayevsky > wrote: > > > +1 > > > > > > 4 votes total: > > > > 1 binding vote (Jason Gustafson) > > >

Re: [DISCUSS] KIP-278: Add version option to Kafka's commands

Hi Sasaki, Thanks for the update. The KIP looks good to me. I'd suggest moving to a vote. Thanks, Jason On Mon, May 7, 2018 at 7:08 AM, Sasaki Toru wrote: > Hi Manikumar, Colin, > > Thank you for your comment. > > As Colin said, I proposed adding an option to show version information of > "loc

Re: [VOTE] KIP-283: Efficient Memory Usage for Down-Conversion

Maybe it should message instead of record to be consistent with message.format.version. On Wed, 9 May 2018, 09:04 Jason Gustafson, wrote: > Hi Dhruvil, > > Thanks for the KIP. +1 from me. Just a minor nitpick on the name of the new > config. I would suggest "record.downconversion.enable". The "r

Re: [VOTE] KIP-283: Efficient Memory Usage for Down-Conversion

Hi Dhruvil, Thanks for the KIP. +1 from me. Just a minor nitpick on the name of the new config. I would suggest "record.downconversion.enable". The "record" prefix emphasizes what is being down-converted and similar existing configs use "enable" rather than "enabled." -Jason On Wed, May 2, 2018

Re: [VOTE] KIP-294 - Enable TLS hostname verification by default

+1 On Wed, May 9, 2018 at 8:28 AM, Rajini Sivaram wrote: > Hi all, > > Since there have been no objections on this straightforward KIP, I would > like to initiate the voting process. KIP-294 proposes to use a secure > default value for endpoint identification when using SSL as the security > pro

[VOTE] KIP-294 - Enable TLS hostname verification by default

Hi all, Since there have been no objections on this straightforward KIP, I would like to initiate the voting process. KIP-294 proposes to use a secure default value for endpoint identification when using SSL as the security protocol. The KIP Is here: https://cwiki.apache.org/confluence/display/KA

Re: [DISCUSS] KIP-273 Kafka to support using ETCD beside Zookeeper

Hi, I just rebased the Etcd implementation proposal on trunk. Pinging to see if anyone has feedback on my questions from my previous email. Molnár Bálint ezt írta (időpont: 2018. ápr. 4., Sze, 10:08): > Hi, > Thanks again for the feedback. > > Is there already ongoing work for having an own cons

[jira] [Created] (KAFKA-6888) java.lang.UnsatisfiedLinkError: librocksdbjni4271925274440341234.dll: âÑX

Sandeep Kapoor created KAFKA-6888: - Summary: java.lang.UnsatisfiedLinkError: librocksdbjni4271925274440341234.dll: âÑX Key: KAFKA-6888 URL: https://issues.apache.org/jira/browse/KAFKA-6888 Project: K

Re: [DISCUSS] KIP-297: Externalizing Secrets for Connect Configurations

Hi Robert. You may find KIP-269: Substitution Within Configuration Values ( https://cwiki.apache.org/confluence/display/KAFKA/KIP+ 269+Substitution+Within+Configuration+Values) to be interesting. I don't know if it is relevant here -- I agree with Magesh, more concrete details would help, includi

Re: [VOTE] #2 KIP-248: Create New ConfigCommand That Uses The New AdminClient

Hi Colin, > We are going to need to create a new version of AlterConfigsRequest to add > the "incremental" boolean. So while we're doing that, maybe we can change > the type to NULLABLE_STRING. I was just talking to a colleague yesterday and we came to the conclusion that we should keep the bo

Build failed in Jenkins: kafka-trunk-jdk8 #2631

See Changes: [github] KAFKA-6834: Handle compaction with batches bigger than max.message.bytes -- [...truncated 420.30 KB...] kafka.admin.ResetConsumerGroupOffsetTest > t

Build failed in Jenkins: kafka-trunk-jdk7 #3411

See Changes: [github] KAFKA-6834: Handle compaction with batches bigger than max.message.bytes -- Started by an SCM change [EnvInject] - Loading node environment variable

[jira] [Resolved] (KAFKA-6834) log cleaner should handle the case when the size of a message set is larger than the max message size

[ https://issues.apache.org/jira/browse/KAFKA-6834?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajini Sivaram resolved KAFKA-6834. --- Resolution: Fixed Reviewer: Jason Gustafson Fix Version/s: 2.0.0 > log cleaner

[jira] [Resolved] (KAFKA-6885) DescribeConfigs synonyms are are identical to parent entry for BROKER resources

[ https://issues.apache.org/jira/browse/KAFKA-6885?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Magnus Edenhill resolved KAFKA-6885. Resolution: Invalid > DescribeConfigs synonyms are are identical to parent entry for BROKER

Build failed in Jenkins: kafka-trunk-jdk8 #2630

See Changes: [ismael] MINOR: Fix trace logging in ReplicaManager (#4916) -- [...truncated 419.72 KB...] kafka.admin.ResetConsumerGroupOffsetTest > testResetOffsetsShiftPl

[DISCUSS] KIP-298: Error Handling in Connect

All, I'd like to start a discussion on adding ways to handle and report record processing errors in Connect. Please find a KIP here: https://cwiki.apache.org/confluence/display/KAFKA/KIP-298%3A+Error+Handling+in+Connect Any feedback will be highly appreciated. Thanks very much, Arjun