Re: [DISCUSS] KIP-885: Expose Broker's Name and Version to Clients Skip to end of metadata

Hi Travis and thanks for the KIP, two comments below: Den fre 11 nov. 2022 kl 13:37 skrev David Jacot : > 02: I am a bit concerned by clients that could misuse these information. > For instance, one may be tempted to rely on the version to decide whether a > feature is enabled or not. The api

Re: [DISCUSS] (continued) KIP-848: The Next Generation of the Consumer Rebalance Protocol

Hi, one minor comment on the latest update: Den mån 24 okt. 2022 kl 16:26 skrev David Jacot : > * Jason pointed out that the member id handling is a tad weird. The > group coordinator generates the member id and then trusts the member > when it rejoins the group. This also implies that the

Re: [VOTE] KIP-848: The Next Generation of the Consumer Rebalance Protocol

Great work on the KIP, David. +1 (nonbinding) Den fre 14 okt. 2022 kl 11:50 skrev Luke Chen : > Hi David, > > I made a final pass and LGTM now. > +1 from me. > > Luke > > On Wed, Oct 5, 2022 at 12:32 AM Guozhang Wang wrote: > > > Hello David, > > > > I've made my final pass on the doc and I

Re: [DISCUSS] (continued) KIP-848: The Next Generation of the Consumer Rebalance Protocol

Hi David, kudos on a very well designed KIP, this will make a ton of difference for the maintainability of client implementations and operational aspects of consumer groups! Some comments: 1. Maybe I missed this, but where does max.poll.interval.ms fit into the new protocol? It seems like

Re: [DISCUSS] KIP-714: Client metrics and observability

; > > different from other existing metrics also seems a bit confusing. It > > seems > > > that the main benefit of having standard metric names across clients is > > for > > > better server side monitoring. Could we do the standardization in the > > > plugin on the server?

Re: [DISCUSS] KIP-714: Client metrics and observability

ross clients seem to cause more confusion during > > troubleshooting. > > > > Thanks, > > > > Jun > > > > On Tue, May 31, 2022 at 5:02 AM Magnus Edenhill > > wrote: > > > >> Den fre 20 maj 2022 kl 01:23 skrev Jun Rao : > >> >

Re: [DISCUSS] KIP-714: Client metrics and observability

table standard metric a client can still provide what special metrics it has. Thanks, Magnus On Thu, May 19, 2022 at 10:39 AM Magnus Edenhill wrote: > > > Den ons 18 maj 2022 kl 19:57 skrev Jun Rao : > > > > > Hi, Magnus, > > > > > > > Hi Ju

Re: [DISCUSS] KIP-714: Client metrics and observability

Den ons 18 maj 2022 kl 19:57 skrev Jun Rao : > Hi, Magnus, > Hi Jun > > Thanks for the updated KIP. Just a couple of more comments. > > 50. To troubleshoot a particular client issue, I imagine that the client > needs to identify its client_instance_id. How does the client find this > out? Do

Re: [VOTE] KIP-714: Client Metrics and Observability

t;> their feeds. They need to be able to take one of the many kafka > clients > > and > > >> go off to the races. > > >> > > >> Being able to retrieve client metrics would be invaluable here as it's > > hard > > >> and time consuming to comm

Re: [DISCUSS] KIP-714: Client metrics and observability

Hey Ismael, > > The PushTelemetryRequest handler decompresses the payload before passing > it > > to the metrics plugin. > > This was done to avoid having to expose a public decompression interface > to > > metrics plugin developers. > > > > Are there cases where the metrics plugin developers

Re: [DISCUSS] KIP-714: Client metrics and observability

>> including groups, tags, etc? We typically don't add the broker_id > >> label for > >> broker metrics. Also, brokers use Yammer metrics, which doesn't have > >> type > >> Sum. > >> > >> 29. There are several client me

Re: [DISCUSS] KIP-714: Client metrics and observability

ght not want to maintain a bunch of histograms for each partition, since that could be quite space consuming, this client.io.wait.time is a single metric per client instance and can thus afford a Histogram representation. Thanks, Magnus > Thanks, > > Jun > > On Wed, Jan 26,

Re: [VOTE] KIP-714: Client Metrics and Observability

e enterprise. Our clients like the performance > and data safeties related to the kafka connection. The observability has > been a problem... > > Jonathan Rivers > jrivers...@gmail.com > > > > > On Mon, Oct 18, 2021 at 11:56 PM Ryanne Dolan > wrote: > > >

Re: [DISCUSS] KIP-714: Client metrics and observability

ed idempotence=true. This is more of a > configuration > > > than a metric. Are we including that as a metric? What other > > configurations > > > are we including? Should we separate the configurations from the > metrics? > > > > > > Thanks, >

Re: [DISCUSS] KIP-714: Client metrics and observability

etrics about the subscription cache itself? That I > think > > would improve its usability and debuggability as we'd be able to see its > > performance, hit/miss rates, eviction counts and others. > > > > Best, > > Viktor > > > > On Thu, Nov 18, 2021

Re: [DISCUSS] KIP-714: Client metrics and observability

the KIP but > what about adding metrics about the subscription cache itself? That I think > would improve its usability and debuggability as we'd be able to see its > performance, hit/miss rates, eviction counts and others. > > Best, > Viktor > > On Thu, Nov 18, 2021 at 5:12 PM

Re: [DISCUSS] KIP-784: Add top-level error code field to DescribeLogDirsResponse

Hey Mickael and thanks for the KIP. I'm a bit late to the game, but one small nit for consistency: The global response error codes are typically at the start of the response, usually right after the ThrottleTime. This also makes it easier for the response parser to assign the global error for

Re: [DISCUSS] KIP-714: Client metrics and observability

efore enabling it > everywhere. > Right, and I agree on being defensive, but since this feature still requires manual enabling on the brokers before actually being used, I think that gives enough control to opt-in or out of this feature as needed. Thanks for your comments! Regards, Magnus

Re: [DISCUSS] KIP-714: Client metrics and observability

ient instance ID proposal was to construct the client > instance ID using clientId as a prefix. Now that the client instance ID is > a UUID, would we want to add clientId as a supported selector? > Let me know what you think. > > David > > On Tue, Oct 19, 2021 at 12:33 PM Mag

Re: [DISCUSS] KIP-714: Client metrics and observability

the client is producing to/consuming from. I'll add some sizes to the KIP for some typical use-cases. Thanks, Magnus > Thanks > > On Tue, Oct 19, 2021 at 5:06 PM Magnus Edenhill > wrote: > > > > Den tis 19 okt. 2021 kl 13:22 skrev Tom Bentley : > > > > > Hi

Re: [DISCUSS] KIP-714: Client metrics and observability

; that may be included as a metrics label if it has been set by the user", so > I'm confused whether non-Kafka Streams clients should set an application_id > or not. > I'll clarify this in the KIP, but basically we would need to add an ` application.id` config property for non-st

[VOTE] KIP-714: Client Metrics and Observability

Hi all, I'd like to start a vote on KIP-714. https://cwiki.apache.org/confluence/x/2xRRCg Discussion thread: https://www.mail-archive.com/dev@kafka.apache.org/msg119000.html Thanks, Magnus

Re: [DISCUSS] KIP-714: Client metrics and observability

. Regards, Magnus Den mån 4 okt. 2021 kl 20:57 skrev Magnus Edenhill : > Hi Gwen, > > I'm finishing up the KIP based on the last couple of discussion points in > this thread > and will call the Vote later this week. > > Best, > Magnus > > Den lör 2 okt. 20

Re: [DISCUSS] KIP-714: Client metrics and observability

't > find the vote thread. Is there one that I'm missing? > > Gwen > > On Wed, Sep 22, 2021 at 4:58 AM Magnus Edenhill > wrote: > > > Den tis 21 sep. 2021 kl 06:58 skrev Colin McCabe : > > > > > On Mon, Sep 20, 2021, at 17:35, Feng Min wrot

Re: [DISCUSS] KIP-714: Client metrics and observability

Den tis 21 sep. 2021 kl 06:58 skrev Colin McCabe : > On Mon, Sep 20, 2021, at 17:35, Feng Min wrote: > > Thanks Magnus & Colin for the discussion. > > > > Based on KIP-714's stateless design, Client can pretty much use any > > connection to any broker to send metrics. We are not associating >

Re: [DISCUSS] KIP-714: Client metrics and observability

Den mån 20 sep. 2021 kl 20:41 skrev Colin McCabe : > On Tue, Sep 14, 2021, at 00:47, Magnus Edenhill wrote: > > Thanks for your feedback Colin, see my updated proposal below. > > ... > > Hi Magnus, > > Thanks for the update. > > > > > Splitting up the A

Re: [DISCUSS] KIP-714: Client metrics and observability

Thanks for your feedback Colin, see my updated proposal below. Den tors 22 juli 2021 kl 03:17 skrev Colin McCabe : > On Tue, Jun 29, 2021, at 07:22, Magnus Edenhill wrote: > > Den tors 17 juni 2021 kl 00:52 skrev Colin McCabe : > > > A few critiques: > > > > &

Re: [VOTE] 3.0.0 RC1

Hi Konstantine, Some findings from running 3.0.0-RC1 with the librdkafka test suite: * Compaction seems to take slightly longer to kick in when segment sizes exceed their threshold. (Used to take less than 20 seconds, now takes 20..30 seconds.) * CreateTopic seems to take slightly longer to

Re: [DISCUSS] KIP-714: Client metrics and observability

Hey Tom, Den mån 21 juni 2021 kl 21:08 skrev Tom Bentley : > > 1. Did you consider using a `default ClientTelemetryReceiver > clientReceiver() { return null; }` method on the existing MetricsReporter > interface, avoiding the need for the ClientTelemetry trait? > I'll let Xavier answer this one

Re: [DISCUSS] KIP-714: Client metrics and observability

) scenario is true, then hooks would still be a > great > > > alternative, because then Confluent could provide drop-in hooks for > any client > > > and the end result of easy-panels would be the same. > > > > > > > In general, if a feature

Re: [DISCUSS] KIP-714: Client metrics and observability

Den tors 17 juni 2021 kl 00:52 skrev Colin McCabe : > Hi Magnus, > > Thanks for the KIP. This is certainly something I've been wishing for for > a while. > > Maybe we should emphasize more that the metrics that are being gathered > here are Kafka metrics, not general application business logic

Re: [DISCUSS] KIP-714: Client metrics and observability

Thanks for your feedback, Colin, see response below. Den tors 17 juni 2021 kl 00:28 skrev Colin McCabe : > On Sun, Jun 13, 2021, at 21:51, Travis Bischel wrote: > ... > > Another downside is that by dictating the important metrics, this KIP > either > > has two choices: try to choose what is

Re: [DISCUSS] KIP-714: Client metrics and observability

tor", so the opt-out appears to be the only motivation. Am I missing > something? > > Ryanne > > On Wed, Jun 2, 2021 at 7:46 AM Magnus Edenhill wrote: > > > Hey all, > > > > I'm proposing KIP-714 to add remote Client metrics and observability. > > This

Re: [DISCUSS] KIP-714: Client metrics and observability

; client > and the end result of easy-panels would be the same. > > In summary, > > - Metrics are more of an organizational concern, not specifically a broker > operator concern. > > - The proposal seems to hijack how metrics are gathered within > organizations > > -

[DISCUSS] KIP-714: Client metrics and observability

Hey all, I'm proposing KIP-714 to add remote Client metrics and observability. This functionality will allow centralized monitoring and troubleshooting of clients and their internals. Please see https://cwiki.apache.org/confluence/display/KAFKA/KIP-714%3A+Client+metrics+and+observability

[jira] [Created] (KAFKA-12712) KRaft: Missing controller.quorom.voters config not properly handled

Magnus Edenhill created KAFKA-12712: --- Summary: KRaft: Missing controller.quorom.voters config not properly handled Key: KAFKA-12712 URL: https://issues.apache.org/jira/browse/KAFKA-12712 Project

Re: [DISCUSS] KIP-709: Extend OffsetFetch requests to accept multiple group ids

Hey Thomas, I'm late to the game. It looks like the "top level" ErrorCode moved from the top-level to the Group array, which makes sense, but it would be good if it was marked as // MOVED in the KIP and also a note that top level errors that are unrelated to the group will be returned as

Re: [DISCUSSION] Upgrade system tests to python 3

;> > https://github.com/apache/kafka/pull/9196/files#diff-9235a7bdb1ca9268681c0e56f3f3609bR39 > >>> > >>>> 16 сент. 2020 г., в 07:32, Guozhang Wang > >>> написал(а): > >>>> > >>>> Hello Nikolay, > >>>> > >>

Re: [DISCUSS] KIP-363

Hi Tom, I think it would be useful with some real world (or made up!) numbers on how much relative/% space is saved for the most error-dense protocol requests. E.g., an OffsetCommitResponse with 10 topics and 100 failing partitions would reduce the overall size by % bytes. Thanks, Magnus Den

Re: [DISCUSS] KIP-551: Expose disk read and write metrics

Hi Colin, aren't those counters (ever increasing), rather than gauges (fluctuating)? You also mention CPU usage as a side note, you could use getrusage(2)'s ru_utime (user) and ru_stime (sys) to allow the broker to monitor its own CPU usage. /Magnus Den tis 10 dec. 2019 kl 19:33 skrev Colin

[jira] [Resolved] (KAFKA-9180) Broker won't start with empty log dir

[ https://issues.apache.org/jira/browse/KAFKA-9180?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Magnus Edenhill resolved KAFKA-9180. Resolution: Invalid Turned out to be old client jars making a mess.   > Broker wo

[jira] [Created] (KAFKA-9180) Broker won't start with empty log dir

Magnus Edenhill created KAFKA-9180: -- Summary: Broker won't start with empty log dir Key: KAFKA-9180 URL: https://issues.apache.org/jira/browse/KAFKA-9180 Project: Kafka Issue Type: Bug

Re: [VOTE] KIP-482: The Kafka Protocol should Support Optional Tagged Fields

Den mån 23 sep. 2019 kl 14:42 skrev Colin McCabe : > On Fri, Sep 20, 2019, at 18:05, Jun Rao wrote: > > 101. We already use varInt in the message format. I assume that the > > protocol uses the same varInt representation? > > It uses a slightly different varint representation. Basically, the >

Re: [VOTE] KIP-511: Collect and Expose Client's Name and Version in the Brokers

> I think we should force client software names and versions to follow a regular expression and disconnect if they do not. Disconnecting is not really a great error propagation method since it leaves the client oblivious to what went wrong. Instead suggest we return an ApiVersionResponse with an

Re: [DISCUSS] KIP-467: Augment ProduceResponse error messaging

. 2019 kl 08:49 skrev Magnus Edenhill : > Hey Guozhang, > > I'm late to the game, again, but if it is not too late I'd recommend that > instead > of having a single error code for all failed records, instead provide > per-record > error codes by having an array of [relati

Re: [DISCUSS] KIP-467: Augment ProduceResponse error messaging

Hey Guozhang, I'm late to the game, again, but if it is not too late I'd recommend that instead of having a single error code for all failed records, instead provide per-record error codes by having an array of [relative_offset, error_code], where they're both varint-encoded for space-efficiency.

Re: [DISCUSS] KIP-482: The Kafka Protocol should Support Optional Fields

Great KIP as always, Colin! Some comments: > If the flexible versions are not specified, it is assumed that all versions are flexible. This is ambiguous, if a protocol-generator is pointed to an older Kafka protocol specification it can't know if the lack of flexibleVersions field means

Re: [DISCUSS] KIP-511: Collect and Expose Client's Name and Version in the Brokers

Great proposal, this feature is well overdue! 1) >From an operator's perspective I don't think the kafka client implementation name and version are sufficient, I also believe the application name and version are of interest. You could have all applications in your cluster run the same kafka

Re: [VOTE] KIP-360: Improve handling of unknown producer when using EOS

+1 (non-binding) Den ons 10 apr. 2019 kl 02:38 skrev Guozhang Wang : > +1 (binding). Thanks for the written KIP! The approach lgtm. > > One minor thing: the name of "last epoch" maybe a bit misleading (although > it is for internal usage only and will not be exposed to users) for future >

Re: [DISCUSS] KIP-430 - Return Authorized Operations in Describe Responses

Hey Rajini, good work on the KIP! I'm personally not thrilled with piggy-backing the authorized_operations on existing APIs, it seems like a mix of concerns. Since there is already the notion of generic ResourceType,ResourceName tuples in the Admin API, I believe it would make more sense to add

Re: [VOTE] 2.1.1 RC2

+1 Passes librdkafka test suite. Den fre 8 feb. 2019 kl 21:02 skrev Colin McCabe : > Hi all, > > This is the third candidate for release of Apache Kafka 2.1.1. This > release includes many bug fixes for Apache Kafka 2.1. > > Compared to rc1, this release includes the following changes: > *

Re: [VOTE] KIP-228 Negative record timestamp support

Sorry for getting in the game this late, and on the wrong thread! I think negative timestamps makes sense and is a good addition, but I have a couple of concerns with the proposal: 1. I believe any change to the protocol format or semantics require a protocol bump, in this case for

Re: [VOTE] 2.0.1 RC0

+1 (non-binding) Passes librdkafka integration test suite Den mån 29 okt. 2018 kl 18:08 skrev Manikumar : > Hi Eno, > > This looks like an existing issue occuring only on source artifacts. We > are able to generate aggregate docs on cloned repo. > I am getting similar error on previous release

Re: [VOTE] 2.1.0 RC0

+1 (non-binding) passes librdkafka integration test suite Den fre 26 okt. 2018 kl 15:58 skrev Manikumar : > minor observation: config sections are empty in the documentation page. > http://kafka.apache.org/21/documentation.html#producerconfigs > > On Wed, Oct 24, 2018 at 10:49 PM Ted Yu wrote:

[jira] [Created] (KAFKA-7549) Old ProduceRequest with zstd compression does not return error to client

Magnus Edenhill created KAFKA-7549: -- Summary: Old ProduceRequest with zstd compression does not return error to client Key: KAFKA-7549 URL: https://issues.apache.org/jira/browse/KAFKA-7549 Project

Re: [DISCUSS] KIP-110: Add Codec for ZStandard Compression (Updated)

> Ismael wrote: > Jason, that's an interesting point regarding the Java client. Do we know > what clients in other languages do in these cases? librdkafka (and its bindings) passes unknown/future errors through to the application, the error code remains intact while the error string will be set

Re: [DISCUSS]: KIP-339: Create a new ModifyConfigs API

Thanks for driving this KIP, Colin. I agree with Dong that a new similar modifyConfigs API (and protocol API) is confusing and that we should try to extend the current alterConfigs interface to support the incremental mode instead, deprecating the non-incremental mode in the process. Another

Re: [VOTE] KIP-322: Return new error code for DeleteTopics API when topic deletion disabled.

+1 (non-binding) 2018-07-04 13:40 GMT+02:00 Satish Duggana : > +1 > > Thanks, > Satish. > > On Wed, Jul 4, 2018 at 4:11 PM, Daniele Ascione > wrote: > > > +1 > > > > Thanks, > > Daniele > > > > Il giorno mar 3 lug 2018 alle ore 23:55 Harsha ha > > scritto: > > > > > +1. > > > > > > Thanks, > >

Re: [VOTE] #2 KIP-248: Create New ConfigCommand That Uses The New AdminClient

t; >> >> in ZK.> >> >> From this perspective I'd leave the the > > command there but put a> >> >> warning that the tool is deprecated and > > should only be used for> >> >> setting up SCRAM credentials. > > >>> >>>

Re: TLS settings in librdkafka

ltimate goal > of the PR is allowing us to avoid even offering insecure algorithms > during the TLS handshake. I.e: avoid the usage of certificate > signature algorithms involving SHA1. > > Magnus Edenhill mentioned in PR[3] some ongoing work/discussion about > keeping librdkafka

[jira] [Resolved] (KAFKA-6885) DescribeConfigs synonyms are are identical to parent entry for BROKER resources

[ https://issues.apache.org/jira/browse/KAFKA-6885?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Magnus Edenhill resolved KAFKA-6885. Resolution: Invalid > DescribeConfigs synonyms are are identical to parent entry for BRO

[jira] [Created] (KAFKA-6885) DescribeConfigs synonyms are are identical to parent entry for BROKER resources

Magnus Edenhill created KAFKA-6885: -- Summary: DescribeConfigs synonyms are are identical to parent entry for BROKER resources Key: KAFKA-6885 URL: https://issues.apache.org/jira/browse/KAFKA-6885

Re: [VOTE] #2 KIP-248: Create New ConfigCommand That Uses The New AdminClient

t; > order, into this buffer at the current position, and then increments > > the > > > > position by eight. > > > > > > > > The implementation will be defined in > > > > org.apache.kafka.common.protocol.types with the other protocol types

[jira] [Created] (KAFKA-6778) DescribeConfigs does not return error for non-existent topic

Magnus Edenhill created KAFKA-6778: -- Summary: DescribeConfigs does not return error for non-existent topic Key: KAFKA-6778 URL: https://issues.apache.org/jira/browse/KAFKA-6778 Project: Kafka

Re: [VOTE] #2 KIP-248: Create New ConfigCommand That Uses The New AdminClient

Hi Viktor, since serialization of floats isn't as straight forward as integers, please specify the exact serialization format of DOUBLE in the protocol docs (e.g., IEEE 754), including endianness (big-endian please). This will help the non-java client ecosystem. Thanks, Magnus 2018-04-09

Re: [DISCUSS] KIP-191: KafkaConsumer.subscribe() overload that takes just Pattern

For what it is worth: librdkafka-based clients will treat any subscribed topic string that is prefixed with the regexp start-of-line anchor ("^") as a regexp, thus providing regex and actual topic subscriptions with the same API, also allowing a mix of actuals and regexps: E.g.:

[jira] [Commented] (KAFKA-4340) Change the default value of log.message.timestamp.difference.max.ms to the same as log.retention.ms

[ https://issues.apache.org/jira/browse/KAFKA-4340?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16027366#comment-16027366 ] Magnus Edenhill commented on KAFKA-4340: Since this is a change to the protocol API (change

[jira] [Commented] (KAFKA-4340) Change the default value of log.message.timestamp.difference.max.ms to the same as log.retention.ms

[ https://issues.apache.org/jira/browse/KAFKA-4340?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16023668#comment-16023668 ] Magnus Edenhill commented on KAFKA-4340: Generally I would agree, but in this case I don't think

[jira] [Commented] (KAFKA-4340) Change the default value of log.message.timestamp.difference.max.ms to the same as log.retention.ms

[ https://issues.apache.org/jira/browse/KAFKA-4340?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16022945#comment-16022945 ] Magnus Edenhill commented on KAFKA-4340: While the idea behind this JIRA is good (as a means

[jira] [Reopened] (KAFKA-4340) Change the default value of log.message.timestamp.difference.max.ms to the same as log.retention.ms

[ https://issues.apache.org/jira/browse/KAFKA-4340?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Magnus Edenhill reopened KAFKA-4340: > Change the default value of log.message.timestamp.difference.max.ms to the >

Re: [DISCUSS] KIP-152 - Improve diagnostics for SASL authentication failures

Hi Rajini, great KIP! This solution was proposed on the original KIP-43 thread but voted down, so let's hope it does better this time :) /Magnus 2017-05-04 13:37 GMT+02:00 Rajini Sivaram : > Hi all, > > I have created a KIP to improve diagnostics for SASL

Re: [DISCUSS] KIP-140: Add administrative RPCs for adding, deleting, and listing ACLs

Hey Colin, good KIP! Some comments: 1a. For operation, permission_type and resource_type: is there any reason for having the any and unknown enums as negative values? Since neither of these fields has an integer significance (unlike for example offsets which use negative offsets for logical

[jira] [Issue Comment Deleted] (KAFKA-4842) Streams integration tests occasionally fail with connection error

[ https://issues.apache.org/jira/browse/KAFKA-4842?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Magnus Edenhill updated KAFKA-4842: --- Comment: was deleted (was: Happened again on a trunk PR, https://github.com/apache/kafka

Re: [VOTE] 0.10.2.1 RC2

+1 (non-binding) - passes librdkafka regression tests 2017-04-18 17:38 GMT+02:00 Edoardo Comar : > Thanks Gwen > KAFKA-5075 is not included in the > http://home.apache.org/~gwenshap/kafka-0.10.2.1-rc2/RELEASE_NOTES.html > > -- >

[jira] [Updated] (KAFKA-4983) Test failure: kafka.api.ConsumerBounceTest.testSubscribeWhenTopicUnavailable

[ https://issues.apache.org/jira/browse/KAFKA-4983?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Magnus Edenhill updated KAFKA-4983: --- Description: The PR builder encountered this test failure

[jira] [Created] (KAFKA-4983) Test failure: kafka.api.ConsumerBounceTest.testSubscribeWhenTopicUnavailable

Magnus Edenhill created KAFKA-4983: -- Summary: Test failure: kafka.api.ConsumerBounceTest.testSubscribeWhenTopicUnavailable Key: KAFKA-4983 URL: https://issues.apache.org/jira/browse/KAFKA-4983

[jira] [Commented] (KAFKA-4476) Kafka Streams gets stuck if metadata is missing

[ https://issues.apache.org/jira/browse/KAFKA-4476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15949128#comment-15949128 ] Magnus Edenhill commented on KAFKA-4476: Directed here from KAFKA-4482. Happened again on trunk

[jira] [Commented] (KAFKA-4842) Streams integration tests occasionally fail with connection error

[ https://issues.apache.org/jira/browse/KAFKA-4842?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15948898#comment-15948898 ] Magnus Edenhill commented on KAFKA-4842: Happened again on a trunk PR, https://github.com/apache

[jira] [Updated] (KAFKA-4974) System test failure in 0.8.2.2 upgrade tests

[ https://issues.apache.org/jira/browse/KAFKA-4974?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Magnus Edenhill updated KAFKA-4974: --- Description: The 0.10.2 system test failed in one of the upgrade tests from 0.8.2.2: http

[jira] [Created] (KAFKA-4974) System test failure in 0.8.2.2 upgrade tests

Magnus Edenhill created KAFKA-4974: -- Summary: System test failure in 0.8.2.2 upgrade tests Key: KAFKA-4974 URL: https://issues.apache.org/jira/browse/KAFKA-4974 Project: Kafka Issue Type

Re: [DISCUSS] KIP-82 - Add Record Headers

Big +1 on VarInts. CPUs are fast, memory is slow. I agree with Jason that we'll want to continue verifying messages, including their headers, so while I appreciate the idea of the opaque header blob it won't be useful in practice. /Magnus 2017-02-17 10:41 GMT-08:00 Jason Gustafson

Re: [DISCUSS] KIP-110: Add Codec for ZStandard Compression

Hi Dongjin and good work on the KIP, I understand that ZStandard is generally considered an improvement over LZ4, but the benchmark you provided on the KIP-110 wiki doesn't really reflect that, and even makes a note that they are comparable: *> As you can see above, ZStandard shows outstanding

Re: [DISCUSS] KIP-82 - Add Record Headers

> >> >> >>> message. > >> >> >>>>> > >> >> >>>>> > >> >> >>>>> > >> >> >>>>> I love strings as much as the next guy (we had them in Flume), >

Re: [DISCUSS] KIP-84: Support SASL/SCRAM mechanisms

Hey, I'm probably going to argue against bumping the SaslHandshake version, it is redundant to the existing SaslHandshakeResponse.enabled_mechanisms field. With SaslHandshake version bump: * Client performs ApiVersionRequest * If SaslHandshake >= v1: use SCRAM * If SaslHandshake < v1: use

Re: [DISCUSS] KIP-82 - Add Record Headers

Hi, I'm +1 for adding generic message headers, but I do share the concerns previously aired on this thread and during the KIP meeting. So let me propose a slimmer alternative that does not require any sort of global header registry, does not affect broker performance or operations, and adds as

Re: [DISCUSS] KIP-87 - Add Compaction Tombstone Flag

the magic byte > > to 1 for this change. > > But since we are not adding an actual field, we can do away with bumping > up > > the magic byte. > > > > If we really want to go the standard route of bumping up the magic byte > for > > any change to message format

Re: [DISCUSS] KIP-87 - Add Compaction Tombstone Flag

would do this. > I dont believe this is needed since it is already attributed through the request's API version. Producer: * if a client sends ProduceRequest V4 then attributes.bit5 indicates a tombstone * if a clients sends ProduceRequest On Tue, Oct 25, 2016 at 10:17 AM, Magnus E

Re: [DISCUSS] KIP-87 - Add Compaction Tombstone Flag

ric knowledge so was hoping it would be flagged up in this discussion > thread ☺ > > Cheers > Mike > > On 10/25/16, 5:36 PM, "Magnus Edenhill" <mag...@edenhill.se> wrote: > > Hi Michael, > > With the version bumps for Produce and Fetch request

Re: [DISCUSS] KIP-87 - Add Compaction Tombstone Flag

Hi Michael, With the version bumps for Produce and Fetch requests, do you really need to bump MagicByte too? Regards, Magnus 2016-10-25 18:09 GMT+02:00 Michael Pearce : > Hi All, > > I would like to discuss the following KIP proposal: >

Re: [VOTE] 0.10.1.0 RC3

+1 (non-binding) passes librdkafka test suites 2016-10-19 15:55 GMT+02:00 Ismael Juma : > +1 (non-binding). > > Verified source and Scala 2.11 binary artifacts, ran ./gradlew test with > JDK 7u80, quick start on source artifact and Scala 2.11 binary artifacts. > > Thanks for

Re: [DISCUSS] KIP-79 - ListOffsetRequest v1 and offsetForTime() method in new consumer.

Good write-up Qin, the API looks promising. I have one comment: 2016-09-03 5:20 GMT+02:00 Becket Qin : > The currently offsetsForTimes() API obviously does not support querying > multiple timestamps for the same partition. It doesn't seems a feature for > ListOffsetRequest

[jira] [Commented] (KAFKA-1588) Offset response does not support two requests for the same topic/partition combo

[ https://issues.apache.org/jira/browse/KAFKA-1588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15415000#comment-15415000 ] Magnus Edenhill commented on KAFKA-1588: I see now that this behaviour is infact documented

[jira] [Commented] (KAFKA-1588) Offset response does not support two requests for the same topic/partition combo

[ https://issues.apache.org/jira/browse/KAFKA-1588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15414995#comment-15414995 ] Magnus Edenhill commented on KAFKA-1588: Thanks [~ijuma]. But I dont see how the bug fix changes

[jira] [Commented] (KAFKA-1588) Offset response does not support two requests for the same topic/partition combo

[ https://issues.apache.org/jira/browse/KAFKA-1588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15414868#comment-15414868 ] Magnus Edenhill commented on KAFKA-1588: [~guozhang] Why would this fix require a protocol change

Re: [DISCUSS] KIP-47 - Add timestamp-based log deletion policy

Bumping this thread so Wes can reply to it. Ignore this mail. 2016-02-24 0:36 GMT+01:00 Joel Koshy : > Great - thanks for clarifying. > > Joel > > On Tue, Feb 23, 2016 at 1:47 PM, Bill Warshaw wrote: > > > Sorry that I didn't see this comment before the

[jira] [Created] (KAFKA-3743) kafka-server-start.sh: Unhelpful error message

Magnus Edenhill created KAFKA-3743: -- Summary: kafka-server-start.sh: Unhelpful error message Key: KAFKA-3743 URL: https://issues.apache.org/jira/browse/KAFKA-3743 Project: Kafka Issue Type

Re: [VOTE] KIP-57: Interoperable LZ4 Framing

I have verified this functionality on kafka trunk using librdkafka lz4 branch and it works as intended. 2016-05-07 18:07 GMT+02:00 Dana Powers : > Vote Passed! I will update the wiki. > > -Dana > On May 7, 2016 3:48 AM, "Ismael Juma" wrote: > > >

Re: [VOTE] KIP-57: Interoperable LZ4 Framing

Good work on this Dana. I'll test it with librdkafka (which uses the official liblz4) and report back. 2016-05-03 20:02 GMT+02:00 Dana Powers : > Yes, great point. The intent of adding "naive" support for the > remaining LZ4 header flags (contentsize and contentchecksum)

Re: [DISCUSS] KIP-43: Kafka SASL enhancements

updated KIP-43 to include a section with the handshake > request/response format. Have also added some more text to distinguish the > actual authentication flow from the Kafka handshake/request flow. > > Thank you, > > Rajini > > > On Mon, Apr 25, 2016 at 3:41 AM, Ma

Re: [VOTE] KIP-57: Interoperable LZ4 Framing

+1 (non-binding) 2016-04-25 14:26 GMT-07:00 Dana Powers : > Hi all, > > Initiating a vote thread because the KIP-57 proposal is specific to > the 0.10 release. > > KIP-57 can be accessed here: > < >

  1   2   >