[GitHub] kafka pull request #3519: KAFKA-5576: increase the rocksDB version to 5.5.1 ...

GitHub user yussufsh opened a pull request: https://github.com/apache/kafka/pull/3519 KAFKA-5576: increase the rocksDB version to 5.5.1 for Power support You can merge this pull request into a Git repository by running: $ git pull https://github.com/yussufsh/kafka rockdbupgrad

[GitHub] kafka pull request #3518: WIP: SimpleRegexAclAuthorizer

GitHub user nnordrum opened a pull request: https://github.com/apache/kafka/pull/3518 WIP: SimpleRegexAclAuthorizer I extended the SimpleAclAuthorizer to support regex. In order to keep backwards compatibility, I added a prefix to the Resource name (default is "r:") to allow otherw

Re: [VOTE] KIP-134: Delay initial consumer group rebalance

Found this thread after posting an alternative idea after we starting hitting this issue ourselves for a job that has a lot of state stores and topic partitions. My suggestion was to have consumer groups have a configurable minimum member count before consumption begins, but that has its own trade

Build failed in Jenkins: kafka-trunk-jdk8 #1801

See -- [...truncated 2.62 MB...] org.apache.kafka.connect.runtime.distributed.DistributedHerderTest > testRestartConnectorRedirectToOwner PASSED org.apache.kafka.connect.runtime.distri

[jira] [Created] (KAFKA-5581) Streams can be smarter in deciding when to create changelog topics for state stores

Guozhang Wang created KAFKA-5581: Summary: Streams can be smarter in deciding when to create changelog topics for state stores Key: KAFKA-5581 URL: https://issues.apache.org/jira/browse/KAFKA-5581 Pro

New page request: Commercial support for Apache Kafka

Hi All, My company, Dattell , offers expert commercial support for Kafka, and we would like to make our support known to those in the Kafka community. Would you consider adding a Commercial Support page to the Kafka website so that users looking for help know where to find s

Re: [DISCUSS] KIP-173: Add prefix to StreamsConfig to enable setting default internal topic configs

Thanks Damian. LGTM. Guozhang On Mon, Jul 10, 2017 at 1:35 PM, Damian Guy wrote: > Thanks Guozhang, i added a couple of example usages to the KIP > > On Fri, 30 Jun 2017 at 17:06 Guozhang Wang wrote: > > > Thanks for the proposal Damian. > > > > The PR looks promising to me. One minor comment

[GitHub] kafka pull request #3461: MINOR: need to be backwards compatible with deprec...

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/3461 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

Jenkins build is back to normal : kafka-trunk-jdk8 #1799

See

Re: Consumer group suggestion: minimum consumer count

Apologies, just discovered KIP-134, which would address our specific problem directly via delaying consumer join. On Mon, Jul 10, 2017 at 4:53 PM, Greg Fodor wrote: > With Kafka Streams, it's common to spin up and shut down clusters of > consumers by performing a graceful shutdown and restart du

Consumer group suggestion: minimum consumer count

With Kafka Streams, it's common to spin up and shut down clusters of consumers by performing a graceful shutdown and restart during a deploy. One thing we've been running into is that during the startup and shutdown of a kafka streams clusters you often can have multiple rebalances as the consumer

Re: [DISCUSS] KIP-172 Add regular-expression topic support for sink connector

Kenji, This looks great. I'd probably make a couple of minor changes for clarity: 1. In public interfaces section, I'd either round it out with client configuration via REST proxy or just simplify to "Adds an optional field to sink connector configuration" 2. Maybe clarify that if you specify bot

[jira] [Created] (KAFKA-5579) SchemaBuilder.type(Schema.Type) should not allow null.

Jeremy Custenborder created KAFKA-5579: -- Summary: SchemaBuilder.type(Schema.Type) should not allow null. Key: KAFKA-5579 URL: https://issues.apache.org/jira/browse/KAFKA-5579 Project: Kafka

[jira] [Created] (KAFKA-5580) Group rebalance delay only used with generation 0

Jason Gustafson created KAFKA-5580: -- Summary: Group rebalance delay only used with generation 0 Key: KAFKA-5580 URL: https://issues.apache.org/jira/browse/KAFKA-5580 Project: Kafka Issue Typ

[GitHub] kafka pull request #3504: KAFKA-5566: fixed race condition between flush and...

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/3504 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[GitHub] kafka pull request #3517: KAFKA-5579 check for null.

GitHub user jcustenborder opened a pull request: https://github.com/apache/kafka/pull/3517 KAFKA-5579 check for null. You can merge this pull request into a Git repository by running: $ git pull https://github.com/jcustenborder/kafka KAFKA-5579 Alternatively you can review an

Build failed in Jenkins: kafka-trunk-jdk8 #1798

See Changes: [damian.guy] KAFKA-5157; Options for handling corrupt data during deserialization -- [...truncated 4.70 MB...] org.apache.kafka.streams.kstream.internals.KSt

Re: [DISCUSS] KIP-168: Add TotalTopicCount metric per cluster

Hi Abhishek, Thanks for your response. Even though, as you suggested, this metric can be calculated using other present metrics, I still feel that adding the 'number of partitions' metrics helps in better representing a Kafka cluster. I don't feel too strong about it though (in case it is consid

Re: [DISCUSS] KIP-168: Add TotalTopicCount metric per cluster

Hi Vahid, Thanks for the response. Kafka currently has a partition leader count metric per broker. The total partition count metric can be built by summing the partition leader count across the cluster. This information may not be consistent but it'll be eventually consistent in steady state. Bu

Re: Aggressive log compaction ratio appears to have no negative effect on log-compacted topics

Jeff, The discussion thread from a while back on KIP-58 has some discussion around "log.cleaner.min.cleanable.ratio". KIP-58 page: https://cwiki.apache.org/confluence/display/KAFKA/KIP-58+-+Make+Log+Compaction+Point+Configurable Discussion thread (linked off that page): http://mail-archives.ap

Re: [DISCUSS] KIP-168: Add TotalTopicCount metric per cluster

Thanks Dong! On Wed, Jun 28, 2017 at 9:46 AM, Dong Lin wrote: > Hey Abhishek, > > GlobalTopicCount sounds good to me. > > Thanks, > Dong > > On Wed, Jun 28, 2017 at 8:39 AM, Abhishek Mendhekar < > abhishek.mendhe...@gmail.com> wrote: > > > Hi Joel, > > > > Thanks for your comments. > > > > To ke

Re: [DISCUSS] KIP-173: Add prefix to StreamsConfig to enable setting default internal topic configs

Thanks Guozhang, i added a couple of example usages to the KIP On Fri, 30 Jun 2017 at 17:06 Guozhang Wang wrote: > Thanks for the proposal Damian. > > The PR looks promising to me. One minor comment for the wiki page is that > you can add some example on how to call the proposed interface (e.g.

Re: [DISCUSS] KIP-163: Lower the Minimum Required ACL Permission of OffsetFetch

I'm bumping this up again to get some feedback, especially from some of the committers, on the KIP and on the note below. Thanks. --Vahid From: "Vahid S Hashemian" To: dev@kafka.apache.org Cc: "Kafka User" Date: 06/21/2017 12:49 PM Subject:Re: [DISCUSS] KIP-163: Lower th

[jira] [Resolved] (KAFKA-5157) Options for handling corrupt data during deserialization

[ https://issues.apache.org/jira/browse/KAFKA-5157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Damian Guy resolved KAFKA-5157. --- Resolution: Fixed Issue resolved by pull request 3423 [https://github.com/apache/kafka/pull/3423] > O

[GitHub] kafka pull request #3423: KAFKA-5157: Options for handling corrupt data duri...

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/3423 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[jira] [Created] (KAFKA-5578) Streams Task Assignor should consider the staleness of state directories when allocating tasks

Damian Guy created KAFKA-5578: - Summary: Streams Task Assignor should consider the staleness of state directories when allocating tasks Key: KAFKA-5578 URL: https://issues.apache.org/jira/browse/KAFKA-5578

Contention on CachedSchemaRegistryClient#getBy*

We've noticed that there's a lot of contention on get/set of schemas by ID in CachedSchemaRegistryClient if that object is shared across threads. All of the methods on CachedSchemaRegistryClient are synchronized and getBySubjectAndId is called when deserializing a record: https://github.com/conflue

[GitHub] kafka pull request #3510: HOTFIX: fix a few typos on streams quickstart

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/3510 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[GitHub] kafka pull request #3516: KAFKA-5562: execute state dir cleanup on single th...

GitHub user dguy opened a pull request: https://github.com/apache/kafka/pull/3516 KAFKA-5562: execute state dir cleanup on single thread Use a single `StateDirectory` per streams instance. Use threadId to determine which thread owns the lock. Only allow the owning thread to u

[GitHub] kafka pull request #3515: MINOR: Make streams quick start more interactive

GitHub user guozhangwang opened a pull request: https://github.com/apache/kafka/pull/3515 MINOR: Make streams quick start more interactive 1. Make the WordCountDemo application to not stop automatically but via "ctrl-C". 2. Update the quickstart html file to let users type input

[jira] [Created] (KAFKA-5577) WindowedStreamPartitioner does not provide topic name to serializer

Matthias J. Sax created KAFKA-5577: -- Summary: WindowedStreamPartitioner does not provide topic name to serializer Key: KAFKA-5577 URL: https://issues.apache.org/jira/browse/KAFKA-5577 Project: Kafka

[jira] [Resolved] (KAFKA-5577) WindowedStreamPartitioner does not provide topic name to serializer

[ https://issues.apache.org/jira/browse/KAFKA-5577?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax resolved KAFKA-5577. Resolution: Fixed Fixed via https://github.com/apache/kafka/pull/2776 > WindowedStreamParti

Re: [ANNOUNCE] New Kafka PMC member Ismael Juma

Very well deserved. Congratulations Ismael! On Mon, Jul 10, 2017 at 6:33 AM Viktor Somogyi wrote: > Congrats Ismael :) > > On Fri, Jul 7, 2017 at 6:59 PM, Konstantine Karantasis < > konstant...@confluent.io> wrote: > > > Congratulations Ismael! > > > > > > On Fri, Jul 7, 2017 at 8:26 AM Eno There

Re: [ANNOUNCE] New Kafka PMC member Ismael Juma

Congrats Ismael :) On Fri, Jul 7, 2017 at 6:59 PM, Konstantine Karantasis < konstant...@confluent.io> wrote: > Congratulations Ismael! > > > On Fri, Jul 7, 2017 at 8:26 AM Eno Thereska > wrote: > > > Congrats! > > > > Eno > > > On 7 Jul 2017, at 16:13, Kamal C > wrote: > > > > > > Congratulatio

Re: From Scala to Java based tools : joptsimple vs argparse4j

Maybe I can just add the following line : to the import-control.xml for the "tools" subpackage, but I'd like to know if we could agree on the tool to use for parsing command line arguments for the Java tools. FYI, I have just opened a starting PR (to not merge, just evaluating and dis

Re: From Scala to Java based tools : joptsimple vs argparse4j

I found a quite interesting thing running the check style : Disallowed import - joptsimple.OptionParser.20 Disallowed import - joptsimple.OptionSet. 21 Disallowed import - joptsimple.OptionSpec. 22 It's not allowed to use the joptsimple library in the Java code. Why ? Paolo Pat

[GitHub] kafka pull request #3514: KAFKA-5561: Rewrite TopicCommand using the new Adm...

GitHub user ppatierno opened a pull request: https://github.com/apache/kafka/pull/3514 KAFKA-5561: Rewrite TopicCommand using the new Admin client (** WIP do not merge **) Start on porting TopicCommand from Scala to Java using Admin Client API as well Please do not merge i

[GitHub] kafka pull request #3513: Updated consumer properties

GitHub user klalafaryan opened a pull request: https://github.com/apache/kafka/pull/3513 Updated consumer properties This consumer properties is for the old consumer. Updated to the new one. (it uses boostrap servers not the zookeeper) You can merge this pull request into a Git rep

[jira] [Created] (KAFKA-5576) Support Power platform by updating rocksdb

Yussuf Shaikh created KAFKA-5576: Summary: Support Power platform by updating rocksdb Key: KAFKA-5576 URL: https://issues.apache.org/jira/browse/KAFKA-5576 Project: Kafka Issue Type: Bug

Re: From Scala to Java based tools : joptsimple vs argparse4j

Hi Tom, no I have no specific reason but I'd like to know why it was chosen for the Scala based tools. Maybe the devs can say if there was a specific reason. For sure, using the same library for the Java tools makes the migration too smooth with the same way of parsing, checking arguments and

Re: From Scala to Java based tools : joptsimple vs argparse4j

Hi Paolo, I don't have an opinion about which you should use, but I certainly agree that two option parsing dependencies appears to be 1 too many. Is there a reason why you prefer joptsimple? Cheers, Tom On 10 July 2017 at 08:38, Paolo Patierno wrote: > Hi devs, > > > working on re-writing t

[jira] [Created] (KAFKA-5574) kafka-consumer-perf-test.sh report header has one less column in show-detailed-stats mode

Yuexin Zhang created KAFKA-5574: --- Summary: kafka-consumer-perf-test.sh report header has one less column in show-detailed-stats mode Key: KAFKA-5574 URL: https://issues.apache.org/jira/browse/KAFKA-5574

From Scala to Java based tools : joptsimple vs argparse4j

Hi devs, working on re-writing the TopicCommand tool in Java using the Admin Client, I was asking if there is any specific reason why the Scala tools use joptsimple for parsing command line arguments while the current Java based tools use argparse4j. I'd like to use the joptsimple even for th