[jira] [Commented] (KAFKA-9181) Flaky test kafka.api.SaslGssapiSslEndToEndAuthorizationTest.testNoConsumeWithoutDescribeAclViaSubscribe
[
https://issues.apache.org/jira/browse/KAFKA-9181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17118524#comment-17118524
]
Rajini Sivaram commented on KAFKA-9181:
---
[~hai_lin] Thanks for reporting this issue. Opened KAFKA-10056 to follow this
up.
> Flaky test
> kafka.api.SaslGssapiSslEndToEndAuthorizationTest.testNoConsumeWithoutDescribeAclViaSubscribe
> ---
>
> Key: KAFKA-9181
> URL: https://issues.apache.org/jira/browse/KAFKA-9181
> Project: Kafka
> Issue Type: Test
> Components: core
>Reporter: Bill Bejeck
>Assignee: Rajini Sivaram
>Priority: Major
> Labels: flaky-test, tests
> Fix For: 2.5.0
>
>
> Failed in
> [https://builds.apache.org/job/kafka-pr-jdk8-scala2.11/26571/testReport/junit/kafka.api/SaslGssapiSslEndToEndAuthorizationTest/testNoConsumeWithoutDescribeAclViaSubscribe/]
>
> {noformat}
> Error Messageorg.apache.kafka.common.errors.TopicAuthorizationException: Not
> authorized to access topics:
> [topic2]Stacktraceorg.apache.kafka.common.errors.TopicAuthorizationException:
> Not authorized to access topics: [topic2]
> Standard OutputAdding ACLs for resource
> `ResourcePattern(resourceType=CLUSTER, name=kafka-cluster,
> patternType=LITERAL)`:
> (principal=User:kafka, host=*, operation=CLUSTER_ACTION,
> permissionType=ALLOW)
> Current ACLs for resource `Cluster:LITERAL:kafka-cluster`:
> User:kafka has Allow permission for operations: ClusterAction from
> hosts: *
> Adding ACLs for resource `ResourcePattern(resourceType=TOPIC, name=*,
> patternType=LITERAL)`:
> (principal=User:kafka, host=*, operation=READ, permissionType=ALLOW)
> Current ACLs for resource `Topic:LITERAL:*`:
> User:kafka has Allow permission for operations: Read from hosts: *
> Debug is true storeKey true useTicketCache false useKeyTab true doNotPrompt
> false ticketCache is null isInitiator true KeyTab is
> /tmp/kafka6494439724844851846.tmp refreshKrb5Config is false principal is
> kafka/localh...@example.com tryFirstPass is false useFirstPass is false
> storePass is false clearPass is false
> principal is kafka/localh...@example.com
> Will use keytab
> Commit Succeeded
> [2019-11-13 04:43:16,187] ERROR [ReplicaFetcher replicaId=1, leaderId=0,
> fetcherId=0] Error for partition __consumer_offsets-0 at offset 0
> (kafka.server.ReplicaFetcherThread:76)
> org.apache.kafka.common.errors.UnknownTopicOrPartitionException: This server
> does not host this topic-partition.
> [2019-11-13 04:43:16,191] ERROR [ReplicaFetcher replicaId=2, leaderId=0,
> fetcherId=0] Error for partition __consumer_offsets-0 at offset 0
> (kafka.server.ReplicaFetcherThread:76)
> org.apache.kafka.common.errors.UnknownTopicOrPartitionException: This server
> does not host this topic-partition.
> [2019-11-13 04:43:16,384] ERROR [ReplicaFetcher replicaId=1, leaderId=2,
> fetcherId=0] Error for partition e2etopic-0 at offset 0
> (kafka.server.ReplicaFetcherThread:76)
> org.apache.kafka.common.errors.UnknownTopicOrPartitionException: This server
> does not host this topic-partition.
> [2019-11-13 04:43:16,384] ERROR [ReplicaFetcher replicaId=0, leaderId=2,
> fetcherId=0] Error for partition e2etopic-0 at offset 0
> (kafka.server.ReplicaFetcherThread:76)
> org.apache.kafka.common.errors.UnknownTopicOrPartitionException: This server
> does not host this topic-partition.
> Adding ACLs for resource `ResourcePattern(resourceType=TOPIC, name=e2etopic,
> patternType=LITERAL)`:
> (principal=User:client, host=*, operation=WRITE, permissionType=ALLOW)
> (principal=User:client, host=*, operation=DESCRIBE,
> permissionType=ALLOW)
> (principal=User:client, host=*, operation=CREATE, permissionType=ALLOW)
> Current ACLs for resource `Topic:LITERAL:e2etopic`:
> User:client has Allow permission for operations: Describe from hosts: *
> User:client has Allow permission for operations: Write from hosts: *
> User:client has Allow permission for operations: Create from hosts: *
> Adding ACLs for resource `ResourcePattern(resourceType=TOPIC, name=e2etopic,
> patternType=LITERAL)`:
> (principal=User:client, host=*, operation=READ, permissionType=ALLOW)
> (principal=User:client, host=*, operation=DESCRIBE,
> permissionType=ALLOW)
> Adding ACLs for resource `ResourcePattern(resourceType=GROUP, name=group,
> patternType=LITERAL)`:
> (principal=User:client, host=*, operation=READ, permissionType=ALLOW)
> Current ACLs for resource `Topic:LITERAL:e2etopic`:
> User:client has Allow permission for operations: Read from hosts: *
> User:client has Allow permission for operations: Describe from hosts: *
> User:client has Allow permission for operations: Write
[jira] [Commented] (KAFKA-9181) Flaky test kafka.api.SaslGssapiSslEndToEndAuthorizationTest.testNoConsumeWithoutDescribeAclViaSubscribe
[
https://issues.apache.org/jira/browse/KAFKA-9181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17096800#comment-17096800
]
Hai Lin commented on KAFKA-9181:
I did notice some issue after this patch, here is what I observe.
Consumer metadata might skip first metadata update, cause grouopSubscription is
not reset. In my case, the consumer coordinator thread hijack the update by
calling newMetadataRequestAndVersion with outdated groupSubscription before
joinPrepare() happen. The groupSubscription will get reset later and it will
eventually get update later, and this won't be an issue for initial consumer
subscribe(since the groupSubscription is empty anyway), but it might happen the
following subscribe when groupSubscription is not empty. This will create a
discrepancy between subscription and groupSubscription, if any new metadata
request happened in between, metadataTopics will return outdated group
information.
h4. The happy path
* Consumer call subscribe -> Update {{needUpdated}}, bump up
{{requestVersion}} and update {{subscription}} in {{SubscriptionState}} ->
{{prepareJoin()}} was call in first {{poll()}} to reset {{groupSubscription}}
-> next time when metadata update was call and {{metadataTopics()}} returns
{{subscription}} since {{groupSubscription}} is empty -> update call issue to
broker to fetch partition information for new topic
h4. In our case
* Consumer call subscribe -> Update {{needUpdated}}, bump up
{{requestVersion}} and update {{subscription}}(not {{groupSubscription}}) in
{{SubscriptionState}} -> Consumer Coordinator heartbeat thread call metadata
request and {{SubscriptionState}} gave away the current requestVersion and
outdated {{groupSubscription}} -> making request for metadata update with
outdated subscription -> request comes back to client and since
{{requestVersion}} is up to latest, it reset {{needUpdated}} flag ->
{{joinPrepare()}} called and reset {{groupSubscription}} -> no new metadata
update request follow cause {{needUpdated}} was reset -> metadata request will
happen when {{metadata.max.age}} reaches.
I saw some discussion in the pull request, don't know if I miss anything here.
> Flaky test
> kafka.api.SaslGssapiSslEndToEndAuthorizationTest.testNoConsumeWithoutDescribeAclViaSubscribe
> ---
>
> Key: KAFKA-9181
> URL: https://issues.apache.org/jira/browse/KAFKA-9181
> Project: Kafka
> Issue Type: Test
> Components: core
>Reporter: Bill Bejeck
>Assignee: Rajini Sivaram
>Priority: Major
> Labels: flaky-test, tests
> Fix For: 2.5.0
>
>
> Failed in
> [https://builds.apache.org/job/kafka-pr-jdk8-scala2.11/26571/testReport/junit/kafka.api/SaslGssapiSslEndToEndAuthorizationTest/testNoConsumeWithoutDescribeAclViaSubscribe/]
>
> {noformat}
> Error Messageorg.apache.kafka.common.errors.TopicAuthorizationException: Not
> authorized to access topics:
> [topic2]Stacktraceorg.apache.kafka.common.errors.TopicAuthorizationException:
> Not authorized to access topics: [topic2]
> Standard OutputAdding ACLs for resource
> `ResourcePattern(resourceType=CLUSTER, name=kafka-cluster,
> patternType=LITERAL)`:
> (principal=User:kafka, host=*, operation=CLUSTER_ACTION,
> permissionType=ALLOW)
> Current ACLs for resource `Cluster:LITERAL:kafka-cluster`:
> User:kafka has Allow permission for operations: ClusterAction from
> hosts: *
> Adding ACLs for resource `ResourcePattern(resourceType=TOPIC, name=*,
> patternType=LITERAL)`:
> (principal=User:kafka, host=*, operation=READ, permissionType=ALLOW)
> Current ACLs for resource `Topic:LITERAL:*`:
> User:kafka has Allow permission for operations: Read from hosts: *
> Debug is true storeKey true useTicketCache false useKeyTab true doNotPrompt
> false ticketCache is null isInitiator true KeyTab is
> /tmp/kafka6494439724844851846.tmp refreshKrb5Config is false principal is
> kafka/localh...@example.com tryFirstPass is false useFirstPass is false
> storePass is false clearPass is false
> principal is kafka/localh...@example.com
> Will use keytab
> Commit Succeeded
> [2019-11-13 04:43:16,187] ERROR [ReplicaFetcher replicaId=1, leaderId=0,
> fetcherId=0] Error for partition __consumer_offsets-0 at offset 0
> (kafka.server.ReplicaFetcherThread:76)
> org.apache.kafka.common.errors.UnknownTopicOrPartitionException: This server
> does not host this topic-partition.
> [2019-11-13 04:43:16,191] ERROR [ReplicaFetcher replicaId=2, leaderId=0,
> fetcherId=0] Error for partition __consumer_offsets-0 at offset 0
> (kafka.server.ReplicaFetcherThread:76)
> org.apache.kafka.common.errors.UnknownTopicOrPartitionException: This server
> does not host this topic-partition.
> [2019-11-13
[jira] [Commented] (KAFKA-9181) Flaky test kafka.api.SaslGssapiSslEndToEndAuthorizationTest.testNoConsumeWithoutDescribeAclViaSubscribe
[
https://issues.apache.org/jira/browse/KAFKA-9181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17034677#comment-17034677
]
ASF GitHub Bot commented on KAFKA-9181:
---
hachikuji commented on pull request #8084: KAFKA-9181; Maintain clean
separation between local and group subscriptions in consumer's
SubscriptionState (#7941)
URL: https://github.com/apache/kafka/pull/8084
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Flaky test
> kafka.api.SaslGssapiSslEndToEndAuthorizationTest.testNoConsumeWithoutDescribeAclViaSubscribe
> ---
>
> Key: KAFKA-9181
> URL: https://issues.apache.org/jira/browse/KAFKA-9181
> Project: Kafka
> Issue Type: Test
> Components: core
>Reporter: Bill Bejeck
>Assignee: Rajini Sivaram
>Priority: Major
> Labels: flaky-test, tests
> Fix For: 2.5.0
>
>
> Failed in
> [https://builds.apache.org/job/kafka-pr-jdk8-scala2.11/26571/testReport/junit/kafka.api/SaslGssapiSslEndToEndAuthorizationTest/testNoConsumeWithoutDescribeAclViaSubscribe/]
>
> {noformat}
> Error Messageorg.apache.kafka.common.errors.TopicAuthorizationException: Not
> authorized to access topics:
> [topic2]Stacktraceorg.apache.kafka.common.errors.TopicAuthorizationException:
> Not authorized to access topics: [topic2]
> Standard OutputAdding ACLs for resource
> `ResourcePattern(resourceType=CLUSTER, name=kafka-cluster,
> patternType=LITERAL)`:
> (principal=User:kafka, host=*, operation=CLUSTER_ACTION,
> permissionType=ALLOW)
> Current ACLs for resource `Cluster:LITERAL:kafka-cluster`:
> User:kafka has Allow permission for operations: ClusterAction from
> hosts: *
> Adding ACLs for resource `ResourcePattern(resourceType=TOPIC, name=*,
> patternType=LITERAL)`:
> (principal=User:kafka, host=*, operation=READ, permissionType=ALLOW)
> Current ACLs for resource `Topic:LITERAL:*`:
> User:kafka has Allow permission for operations: Read from hosts: *
> Debug is true storeKey true useTicketCache false useKeyTab true doNotPrompt
> false ticketCache is null isInitiator true KeyTab is
> /tmp/kafka6494439724844851846.tmp refreshKrb5Config is false principal is
> kafka/localh...@example.com tryFirstPass is false useFirstPass is false
> storePass is false clearPass is false
> principal is kafka/localh...@example.com
> Will use keytab
> Commit Succeeded
> [2019-11-13 04:43:16,187] ERROR [ReplicaFetcher replicaId=1, leaderId=0,
> fetcherId=0] Error for partition __consumer_offsets-0 at offset 0
> (kafka.server.ReplicaFetcherThread:76)
> org.apache.kafka.common.errors.UnknownTopicOrPartitionException: This server
> does not host this topic-partition.
> [2019-11-13 04:43:16,191] ERROR [ReplicaFetcher replicaId=2, leaderId=0,
> fetcherId=0] Error for partition __consumer_offsets-0 at offset 0
> (kafka.server.ReplicaFetcherThread:76)
> org.apache.kafka.common.errors.UnknownTopicOrPartitionException: This server
> does not host this topic-partition.
> [2019-11-13 04:43:16,384] ERROR [ReplicaFetcher replicaId=1, leaderId=2,
> fetcherId=0] Error for partition e2etopic-0 at offset 0
> (kafka.server.ReplicaFetcherThread:76)
> org.apache.kafka.common.errors.UnknownTopicOrPartitionException: This server
> does not host this topic-partition.
> [2019-11-13 04:43:16,384] ERROR [ReplicaFetcher replicaId=0, leaderId=2,
> fetcherId=0] Error for partition e2etopic-0 at offset 0
> (kafka.server.ReplicaFetcherThread:76)
> org.apache.kafka.common.errors.UnknownTopicOrPartitionException: This server
> does not host this topic-partition.
> Adding ACLs for resource `ResourcePattern(resourceType=TOPIC, name=e2etopic,
> patternType=LITERAL)`:
> (principal=User:client, host=*, operation=WRITE, permissionType=ALLOW)
> (principal=User:client, host=*, operation=DESCRIBE,
> permissionType=ALLOW)
> (principal=User:client, host=*, operation=CREATE, permissionType=ALLOW)
> Current ACLs for resource `Topic:LITERAL:e2etopic`:
> User:client has Allow permission for operations: Describe from hosts: *
> User:client has Allow permission for operations: Write from hosts: *
> User:client has Allow permission for operations: Create from hosts: *
> Adding ACLs for resource `ResourcePattern(resourceType=TOPIC, name=e2etopic,
> patternType=LITERAL)`:
> (principal=User:client, host=*, operation=READ, permissionType=ALLOW)
> (principal=User:client, host=*, operation=DESCRIBE,
> permissionType=ALLOW)
>
[jira] [Commented] (KAFKA-9181) Flaky test kafka.api.SaslGssapiSslEndToEndAuthorizationTest.testNoConsumeWithoutDescribeAclViaSubscribe
[
https://issues.apache.org/jira/browse/KAFKA-9181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17034043#comment-17034043
]
ASF GitHub Bot commented on KAFKA-9181:
---
soondenana commented on pull request #8084: KAFKA-9181; Maintain clean
separation between local and group subscriptions in consumer's
SubscriptionState (#7941)
URL: https://github.com/apache/kafka/pull/8084
Reviewers: Jason Gustafson , Guozhang Wang
(cherry picked from commit a565d1a182cc69c9994c4512b5e9877e97f06cdf)
*More detailed description of your change,
if necessary. The PR title and PR message become
the squashed commit message, so use a separate
comment to ping reviewers.*
*Summary of testing strategy (including rationale)
for the feature or bug fix. Unit and/or integration
tests are expected for any behaviour change and
system tests should be considered for larger changes.*
### Committer Checklist (excluded from commit message)
- [ ] Verify design and implementation
- [ ] Verify test coverage and CI build status
- [ ] Verify documentation (including upgrade notes)
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Flaky test
> kafka.api.SaslGssapiSslEndToEndAuthorizationTest.testNoConsumeWithoutDescribeAclViaSubscribe
> ---
>
> Key: KAFKA-9181
> URL: https://issues.apache.org/jira/browse/KAFKA-9181
> Project: Kafka
> Issue Type: Test
> Components: core
>Reporter: Bill Bejeck
>Assignee: Rajini Sivaram
>Priority: Major
> Labels: flaky-test, tests
> Fix For: 2.5.0
>
>
> Failed in
> [https://builds.apache.org/job/kafka-pr-jdk8-scala2.11/26571/testReport/junit/kafka.api/SaslGssapiSslEndToEndAuthorizationTest/testNoConsumeWithoutDescribeAclViaSubscribe/]
>
> {noformat}
> Error Messageorg.apache.kafka.common.errors.TopicAuthorizationException: Not
> authorized to access topics:
> [topic2]Stacktraceorg.apache.kafka.common.errors.TopicAuthorizationException:
> Not authorized to access topics: [topic2]
> Standard OutputAdding ACLs for resource
> `ResourcePattern(resourceType=CLUSTER, name=kafka-cluster,
> patternType=LITERAL)`:
> (principal=User:kafka, host=*, operation=CLUSTER_ACTION,
> permissionType=ALLOW)
> Current ACLs for resource `Cluster:LITERAL:kafka-cluster`:
> User:kafka has Allow permission for operations: ClusterAction from
> hosts: *
> Adding ACLs for resource `ResourcePattern(resourceType=TOPIC, name=*,
> patternType=LITERAL)`:
> (principal=User:kafka, host=*, operation=READ, permissionType=ALLOW)
> Current ACLs for resource `Topic:LITERAL:*`:
> User:kafka has Allow permission for operations: Read from hosts: *
> Debug is true storeKey true useTicketCache false useKeyTab true doNotPrompt
> false ticketCache is null isInitiator true KeyTab is
> /tmp/kafka6494439724844851846.tmp refreshKrb5Config is false principal is
> kafka/localh...@example.com tryFirstPass is false useFirstPass is false
> storePass is false clearPass is false
> principal is kafka/localh...@example.com
> Will use keytab
> Commit Succeeded
> [2019-11-13 04:43:16,187] ERROR [ReplicaFetcher replicaId=1, leaderId=0,
> fetcherId=0] Error for partition __consumer_offsets-0 at offset 0
> (kafka.server.ReplicaFetcherThread:76)
> org.apache.kafka.common.errors.UnknownTopicOrPartitionException: This server
> does not host this topic-partition.
> [2019-11-13 04:43:16,191] ERROR [ReplicaFetcher replicaId=2, leaderId=0,
> fetcherId=0] Error for partition __consumer_offsets-0 at offset 0
> (kafka.server.ReplicaFetcherThread:76)
> org.apache.kafka.common.errors.UnknownTopicOrPartitionException: This server
> does not host this topic-partition.
> [2019-11-13 04:43:16,384] ERROR [ReplicaFetcher replicaId=1, leaderId=2,
> fetcherId=0] Error for partition e2etopic-0 at offset 0
> (kafka.server.ReplicaFetcherThread:76)
> org.apache.kafka.common.errors.UnknownTopicOrPartitionException: This server
> does not host this topic-partition.
> [2019-11-13 04:43:16,384] ERROR [ReplicaFetcher replicaId=0, leaderId=2,
> fetcherId=0] Error for partition e2etopic-0 at offset 0
> (kafka.server.ReplicaFetcherThread:76)
> org.apache.kafka.common.errors.UnknownTopicOrPartitionException: This server
> does not host this topic-partition.
> Adding ACLs for resource `ResourcePattern(resourceType=TOPIC, name=e2etopic,
> patternType=LITERAL)`:
> (principal=User:client, host=*, operation=WRITE,
[jira] [Commented] (KAFKA-9181) Flaky test kafka.api.SaslGssapiSslEndToEndAuthorizationTest.testNoConsumeWithoutDescribeAclViaSubscribe
[
https://issues.apache.org/jira/browse/KAFKA-9181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17022848#comment-17022848
]
ASF GitHub Bot commented on KAFKA-9181:
---
rajinisivaram commented on pull request #7941: KAFKA-9181; Maintain clean
separation between local and group subscriptions in consumer's SubscriptionState
URL: https://github.com/apache/kafka/pull/7941
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Flaky test
> kafka.api.SaslGssapiSslEndToEndAuthorizationTest.testNoConsumeWithoutDescribeAclViaSubscribe
> ---
>
> Key: KAFKA-9181
> URL: https://issues.apache.org/jira/browse/KAFKA-9181
> Project: Kafka
> Issue Type: Test
> Components: core
>Reporter: Bill Bejeck
>Assignee: Rajini Sivaram
>Priority: Major
> Labels: flaky-test, tests
>
> Failed in
> [https://builds.apache.org/job/kafka-pr-jdk8-scala2.11/26571/testReport/junit/kafka.api/SaslGssapiSslEndToEndAuthorizationTest/testNoConsumeWithoutDescribeAclViaSubscribe/]
>
> {noformat}
> Error Messageorg.apache.kafka.common.errors.TopicAuthorizationException: Not
> authorized to access topics:
> [topic2]Stacktraceorg.apache.kafka.common.errors.TopicAuthorizationException:
> Not authorized to access topics: [topic2]
> Standard OutputAdding ACLs for resource
> `ResourcePattern(resourceType=CLUSTER, name=kafka-cluster,
> patternType=LITERAL)`:
> (principal=User:kafka, host=*, operation=CLUSTER_ACTION,
> permissionType=ALLOW)
> Current ACLs for resource `Cluster:LITERAL:kafka-cluster`:
> User:kafka has Allow permission for operations: ClusterAction from
> hosts: *
> Adding ACLs for resource `ResourcePattern(resourceType=TOPIC, name=*,
> patternType=LITERAL)`:
> (principal=User:kafka, host=*, operation=READ, permissionType=ALLOW)
> Current ACLs for resource `Topic:LITERAL:*`:
> User:kafka has Allow permission for operations: Read from hosts: *
> Debug is true storeKey true useTicketCache false useKeyTab true doNotPrompt
> false ticketCache is null isInitiator true KeyTab is
> /tmp/kafka6494439724844851846.tmp refreshKrb5Config is false principal is
> kafka/localh...@example.com tryFirstPass is false useFirstPass is false
> storePass is false clearPass is false
> principal is kafka/localh...@example.com
> Will use keytab
> Commit Succeeded
> [2019-11-13 04:43:16,187] ERROR [ReplicaFetcher replicaId=1, leaderId=0,
> fetcherId=0] Error for partition __consumer_offsets-0 at offset 0
> (kafka.server.ReplicaFetcherThread:76)
> org.apache.kafka.common.errors.UnknownTopicOrPartitionException: This server
> does not host this topic-partition.
> [2019-11-13 04:43:16,191] ERROR [ReplicaFetcher replicaId=2, leaderId=0,
> fetcherId=0] Error for partition __consumer_offsets-0 at offset 0
> (kafka.server.ReplicaFetcherThread:76)
> org.apache.kafka.common.errors.UnknownTopicOrPartitionException: This server
> does not host this topic-partition.
> [2019-11-13 04:43:16,384] ERROR [ReplicaFetcher replicaId=1, leaderId=2,
> fetcherId=0] Error for partition e2etopic-0 at offset 0
> (kafka.server.ReplicaFetcherThread:76)
> org.apache.kafka.common.errors.UnknownTopicOrPartitionException: This server
> does not host this topic-partition.
> [2019-11-13 04:43:16,384] ERROR [ReplicaFetcher replicaId=0, leaderId=2,
> fetcherId=0] Error for partition e2etopic-0 at offset 0
> (kafka.server.ReplicaFetcherThread:76)
> org.apache.kafka.common.errors.UnknownTopicOrPartitionException: This server
> does not host this topic-partition.
> Adding ACLs for resource `ResourcePattern(resourceType=TOPIC, name=e2etopic,
> patternType=LITERAL)`:
> (principal=User:client, host=*, operation=WRITE, permissionType=ALLOW)
> (principal=User:client, host=*, operation=DESCRIBE,
> permissionType=ALLOW)
> (principal=User:client, host=*, operation=CREATE, permissionType=ALLOW)
> Current ACLs for resource `Topic:LITERAL:e2etopic`:
> User:client has Allow permission for operations: Describe from hosts: *
> User:client has Allow permission for operations: Write from hosts: *
> User:client has Allow permission for operations: Create from hosts: *
> Adding ACLs for resource `ResourcePattern(resourceType=TOPIC, name=e2etopic,
> patternType=LITERAL)`:
> (principal=User:client, host=*, operation=READ, permissionType=ALLOW)
> (principal=User:client, host=*, operation=DESCRIBE,
> permissionType=ALLOW)
> Adding ACLs for resource
[jira] [Commented] (KAFKA-9181) Flaky test kafka.api.SaslGssapiSslEndToEndAuthorizationTest.testNoConsumeWithoutDescribeAclViaSubscribe
[
https://issues.apache.org/jira/browse/KAFKA-9181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17013814#comment-17013814
]
ASF GitHub Bot commented on KAFKA-9181:
---
rajinisivaram commented on pull request #7941: KAFKA-9181; Ensure
SubscriptionState.groupSubscription is updated even if onJoinPrepare not
invoked after subscribe
URL: https://github.com/apache/kafka/pull/7941
As described in KAFKA-9181,
kafka.api.SaslGssapiSslEndToEndAuthorizationTest.testNoConsumeWithoutDescribeAclViaSubscribe
occasionally hits unexpected TopicAuthorizationException even after the topic
is removed from the subscription. The test uses small metadata refresh time and
hence can see metadata responses before JoinGroup is processed. We currently
rely on `onJoinPrepare` to reset SubscriptionState.groupSubscription, which
accumulates topics until reset. If we process JoinGroup after a subscribe
without a new `onJoinPrepare`, we leave the topic in
`SubscriptionState.groupSubscription` and hence in metadata. This PR resets
`groupSubscription` when sending JoinGroup request, when
`ConsumerCoordinator.joinedSubscription` is updated.
### Committer Checklist (excluded from commit message)
- [ ] Verify design and implementation
- [ ] Verify test coverage and CI build status
- [ ] Verify documentation (including upgrade notes)
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Flaky test
> kafka.api.SaslGssapiSslEndToEndAuthorizationTest.testNoConsumeWithoutDescribeAclViaSubscribe
> ---
>
> Key: KAFKA-9181
> URL: https://issues.apache.org/jira/browse/KAFKA-9181
> Project: Kafka
> Issue Type: Test
> Components: core
>Reporter: Bill Bejeck
>Assignee: Rajini Sivaram
>Priority: Major
> Labels: flaky-test, tests
>
> Failed in
> [https://builds.apache.org/job/kafka-pr-jdk8-scala2.11/26571/testReport/junit/kafka.api/SaslGssapiSslEndToEndAuthorizationTest/testNoConsumeWithoutDescribeAclViaSubscribe/]
>
> {noformat}
> Error Messageorg.apache.kafka.common.errors.TopicAuthorizationException: Not
> authorized to access topics:
> [topic2]Stacktraceorg.apache.kafka.common.errors.TopicAuthorizationException:
> Not authorized to access topics: [topic2]
> Standard OutputAdding ACLs for resource
> `ResourcePattern(resourceType=CLUSTER, name=kafka-cluster,
> patternType=LITERAL)`:
> (principal=User:kafka, host=*, operation=CLUSTER_ACTION,
> permissionType=ALLOW)
> Current ACLs for resource `Cluster:LITERAL:kafka-cluster`:
> User:kafka has Allow permission for operations: ClusterAction from
> hosts: *
> Adding ACLs for resource `ResourcePattern(resourceType=TOPIC, name=*,
> patternType=LITERAL)`:
> (principal=User:kafka, host=*, operation=READ, permissionType=ALLOW)
> Current ACLs for resource `Topic:LITERAL:*`:
> User:kafka has Allow permission for operations: Read from hosts: *
> Debug is true storeKey true useTicketCache false useKeyTab true doNotPrompt
> false ticketCache is null isInitiator true KeyTab is
> /tmp/kafka6494439724844851846.tmp refreshKrb5Config is false principal is
> kafka/localh...@example.com tryFirstPass is false useFirstPass is false
> storePass is false clearPass is false
> principal is kafka/localh...@example.com
> Will use keytab
> Commit Succeeded
> [2019-11-13 04:43:16,187] ERROR [ReplicaFetcher replicaId=1, leaderId=0,
> fetcherId=0] Error for partition __consumer_offsets-0 at offset 0
> (kafka.server.ReplicaFetcherThread:76)
> org.apache.kafka.common.errors.UnknownTopicOrPartitionException: This server
> does not host this topic-partition.
> [2019-11-13 04:43:16,191] ERROR [ReplicaFetcher replicaId=2, leaderId=0,
> fetcherId=0] Error for partition __consumer_offsets-0 at offset 0
> (kafka.server.ReplicaFetcherThread:76)
> org.apache.kafka.common.errors.UnknownTopicOrPartitionException: This server
> does not host this topic-partition.
> [2019-11-13 04:43:16,384] ERROR [ReplicaFetcher replicaId=1, leaderId=2,
> fetcherId=0] Error for partition e2etopic-0 at offset 0
> (kafka.server.ReplicaFetcherThread:76)
> org.apache.kafka.common.errors.UnknownTopicOrPartitionException: This server
> does not host this topic-partition.
> [2019-11-13 04:43:16,384] ERROR [ReplicaFetcher replicaId=0, leaderId=2,
> fetcherId=0] Error for partition e2etopic-0 at offset 0
> (kafka.server.ReplicaFetcherThread:76)
> org.apache.kafka.common.errors.UnknownTopicOrPartitionException: This server
> does not
[jira] [Commented] (KAFKA-9181) Flaky test kafka.api.SaslGssapiSslEndToEndAuthorizationTest.testNoConsumeWithoutDescribeAclViaSubscribe
[
https://issues.apache.org/jira/browse/KAFKA-9181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16974343#comment-16974343
]
Bill Bejeck commented on KAFKA-9181:
Failed again in [https://builds.apache.org/job/kafka-pr-jdk11-scala2.13/3358/]
on 11/14/19
{noformat}
Error Messageorg.apache.kafka.common.errors.TopicAuthorizationException: Not
authorized to access topics:
[topic2]Stacktraceorg.apache.kafka.common.errors.TopicAuthorizationException:
Not authorized to access topics: [topic2]
Standard OutputAdding ACLs for resource `ResourcePattern(resourceType=CLUSTER,
name=kafka-cluster, patternType=LITERAL)`:
(principal=User:kafka, host=*, operation=CLUSTER_ACTION,
permissionType=ALLOW)
Current ACLs for resource `Cluster:LITERAL:kafka-cluster`:
User:kafka has Allow permission for operations: ClusterAction from
hosts: *
Adding ACLs for resource `ResourcePattern(resourceType=TOPIC, name=*,
patternType=LITERAL)`:
(principal=User:kafka, host=*, operation=READ, permissionType=ALLOW)
Current ACLs for resource `Topic:LITERAL:*`:
User:kafka has Allow permission for operations: Read from hosts: *
Debug is true storeKey true useTicketCache false useKeyTab true doNotPrompt
false ticketCache is null isInitiator true KeyTab is
/tmp/kafka15290895624380393501.tmp refreshKrb5Config is false principal is
kafka/localh...@example.com tryFirstPass is false useFirstPass is false
storePass is false clearPass is false
principal is kafka/localh...@example.com
Will use keytab
Commit Succeeded
[2019-11-14 00:58:37,287] ERROR [ReplicaFetcher replicaId=1, leaderId=0,
fetcherId=0] Error for partition __consumer_offsets-0 at offset 0
(kafka.server.ReplicaFetcherThread:76)
org.apache.kafka.common.errors.UnknownTopicOrPartitionException: This server
does not host this topic-partition.
[2019-11-14 00:58:37,304] ERROR [ReplicaFetcher replicaId=2, leaderId=0,
fetcherId=0] Error for partition __consumer_offsets-0 at offset 0
(kafka.server.ReplicaFetcherThread:76)
org.apache.kafka.common.errors.UnknownTopicOrPartitionException: This server
does not host this topic-partition.
[2019-11-14 00:58:37,474] ERROR [ReplicaFetcher replicaId=1, leaderId=0,
fetcherId=0] Error for partition e2etopic-0 at offset 0
(kafka.server.ReplicaFetcherThread:76)
org.apache.kafka.common.errors.UnknownTopicOrPartitionException: This server
does not host this topic-partition.
[2019-11-14 00:58:37,490] ERROR [ReplicaFetcher replicaId=2, leaderId=0,
fetcherId=0] Error for partition e2etopic-0 at offset 0
(kafka.server.ReplicaFetcherThread:76)
org.apache.kafka.common.errors.UnknownTopicOrPartitionException: This server
does not host this topic-partition.
Adding ACLs for resource `ResourcePattern(resourceType=TOPIC, name=e2etopic,
patternType=LITERAL)`:
(principal=User:client, host=*, operation=CREATE, permissionType=ALLOW)
(principal=User:client, host=*, operation=WRITE, permissionType=ALLOW)
(principal=User:client, host=*, operation=DESCRIBE,
permissionType=ALLOW)
Current ACLs for resource `Topic:LITERAL:e2etopic`:
User:client has Allow permission for operations: Describe from hosts: *
User:client has Allow permission for operations: Create from hosts: *
User:client has Allow permission for operations: Write from hosts: *
Adding ACLs for resource `ResourcePattern(resourceType=TOPIC, name=e2etopic,
patternType=LITERAL)`:
(principal=User:client, host=*, operation=READ, permissionType=ALLOW)
(principal=User:client, host=*, operation=DESCRIBE,
permissionType=ALLOW)
Adding ACLs for resource `ResourcePattern(resourceType=GROUP, name=group,
patternType=LITERAL)`:
(principal=User:client, host=*, operation=READ, permissionType=ALLOW)
Current ACLs for resource `Topic:LITERAL:e2etopic`:
User:client has Allow permission for operations: Describe from hosts: *
User:client has Allow permission for operations: Create from hosts: *
User:client has Allow permission for operations: Write from hosts: *
User:client has Allow permission for operations: Read from hosts: *
Current ACLs for resource `Group:LITERAL:group`:
User:client has Allow permission for operations: Read from hosts: *
Debug is true storeKey true useTicketCache false useKeyTab true doNotPrompt
false ticketCache is null isInitiator true KeyTab is
/tmp/kafka6561302204353356505.tmp refreshKrb5Config is false principal is
cli...@example.com tryFirstPass is false useFirstPass is false storePass is
false clearPass is false
principal is cli...@example.com
Will use keytab
Commit Succeeded
Debug is true storeKey true useTicketCache false useKeyTab true doNotPrompt
false ticketCache is null isInitiator true KeyTab is
/tmp/kafka6561302204353356505.tmp refreshKrb5Config is false principal is
clie...@example.com tryFirstPass is false useFirstPass is false
