Re: Kafka Topic Permissions Failure

[Razin Bouzar via user]

Hey Hang,

We may have encountered a bug. It seems like deleting the job and
redeploying "fixed" the issue. We will work on recreating this scenario and
will follow-up at some point.

On Mon, Nov 13, 2023 at 3:35 AM Hang Ruan  wrote:

> Hi, Razin.
>
> It seems like the issue you shared is a different problem from yours. They
> have different error messages.
> Have you ever tried to consume this topic using the Kafka java client[1]
> by yourself to make sure you could access the topic normally?
>
> Best,
> Hang
>
> [1] https://developer.confluent.io/get-started/java/#build-consumer
>
> Razin Bouzar via user  于2023年11月11日周六 04:33写道：
>
>> Hello,
>>
>> We seem to be encountering a bug wherein we see the error
>> TopicAuthorizationException: Not authorized to access topics: [topic] for
>> a topic wave R/W/D access to the topic and are using the correct principal
>> (MTLS auth). The only somewhat related FLINK bug I found was FLINK-27041
>> <https://issues.apache.org/jira/browse/FLINK-27041>. Some partitions are
>> empty, but we are running a newer version of Flink (1.16.1).
>>
>> Are there any suggestions on what else to check?
>>
>>
>> *Full stack trace:*
>>
>> java.lang.RuntimeException: One or more fetchers have encountered
>> exception
>> at org.apache.flink.connector.base.source.reader.fetcher.
>> SplitFetcherManager.checkErrors(SplitFetcherManager.java:225)
>> at org.apache.flink.connector.base.source.reader.SourceReaderBase
>> .getNextFetch(SourceReaderBase.java:169)
>> at org.apache.flink.connector.base.source.reader.SourceReaderBase
>> .pollNext(SourceReaderBase.java:130)
>> at org.apache.flink.streaming.api.operators.SourceOperator.emitNext(
>> SourceOperator.java:385)
>> at org.apache.flink.streaming.runtime.io.StreamTaskSourceInput.emitNext(
>> StreamTaskSourceInput.java:68)
>> at org.apache.flink.streaming.runtime.io.StreamOneInputProcessor
>> .processInput(StreamOneInputProcessor.java:65)
>> at org.apache.flink.streaming.runtime.tasks.StreamTask.processInput(
>> StreamTask.java:542)
>> at org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor
>> .runMailboxLoop(MailboxProcessor.java:231)
>> at org.apache.flink.streaming.runtime.tasks.StreamTask.runMailboxLoop(
>> StreamTask.java:831)
>> at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask
>> .java:780)
>> at org.apache.flink.runtime.taskmanager.Task.runWithSystemExitMonitoring(
>> Task.java:935)
>> at org.apache.flink.runtime.taskmanager.Task.restoreAndInvoke(Task.java:
>> 914)
>> at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:728)
>> at org.apache.flink.runtime.taskmanager.Task.run(Task.java:550)
>> at java.base/java.lang.Thread.run(Thread.java:829)
>> Caused by: java.lang.RuntimeException: SplitFetcher thread 0 received
>> unexpected exception while polling the records
>> at org.apache.flink.connector.base.source.reader.fetcher.SplitFetcher
>> .runOnce(SplitFetcher.java:150)
>> at org.apache.flink.connector.base.source.reader.fetcher.SplitFetcher
>> .run(SplitFetcher.java:105)
>> at java.base/java.util.concurrent.Executors$RunnableAdapter.call(
>> Executors.java:515)
>> at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
>> at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(
>> ThreadPoolExecutor.java:1128)
>> at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(
>> ThreadPoolExecutor.java:628)
>> ... 1 more
>> Caused by: org.apache.kafka.common.errors.TopicAuthorizationException:
>>
>> --
>> RAZIN BOUZAR
>> Monitoring Cloud | Salesforce
>> Mobile: 317-502-8995
>>
>> <https://smart.salesforce.com/sig/rbouzar//us_mb/default/link.html>
>>
>

-- 
RAZIN BOUZAR
Monitoring Cloud | Salesforce
Mobile: 317-502-8995

<https://smart.salesforce.com/sig/rbouzar//us_mb/default/link.html>

Kafka Topic Permissions Failure

[Razin Bouzar via user]

Hello,

We seem to be encountering a bug wherein we see the error
TopicAuthorizationException: Not authorized to access topics: [topic] for a
topic wave R/W/D access to the topic and are using the correct principal
(MTLS auth). The only somewhat related FLINK bug I found was FLINK-27041
. Some partitions are
empty, but we are running a newer version of Flink (1.16.1).

Are there any suggestions on what else to check?


*Full stack trace:*

java.lang.RuntimeException: One or more fetchers have encountered exception
at org.apache.flink.connector.base.source.reader.fetcher.SplitFetcherManager
.checkErrors(SplitFetcherManager.java:225)
at org.apache.flink.connector.base.source.reader.SourceReaderBase
.getNextFetch(SourceReaderBase.java:169)
at org.apache.flink.connector.base.source.reader.SourceReaderBase.pollNext(
SourceReaderBase.java:130)
at org.apache.flink.streaming.api.operators.SourceOperator.emitNext(
SourceOperator.java:385)
at org.apache.flink.streaming.runtime.io.StreamTaskSourceInput.emitNext(
StreamTaskSourceInput.java:68)
at org.apache.flink.streaming.runtime.io.StreamOneInputProcessor
.processInput(StreamOneInputProcessor.java:65)
at org.apache.flink.streaming.runtime.tasks.StreamTask.processInput(
StreamTask.java:542)
at org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor
.runMailboxLoop(MailboxProcessor.java:231)
at org.apache.flink.streaming.runtime.tasks.StreamTask.runMailboxLoop(
StreamTask.java:831)
at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask
.java:780)
at org.apache.flink.runtime.taskmanager.Task.runWithSystemExitMonitoring(
Task.java:935)
at org.apache.flink.runtime.taskmanager.Task.restoreAndInvoke(Task.java:914)
at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:728)
at org.apache.flink.runtime.taskmanager.Task.run(Task.java:550)
at java.base/java.lang.Thread.run(Thread.java:829)
Caused by: java.lang.RuntimeException: SplitFetcher thread 0 received
unexpected exception while polling the records
at org.apache.flink.connector.base.source.reader.fetcher.SplitFetcher
.runOnce(SplitFetcher.java:150)
at org.apache.flink.connector.base.source.reader.fetcher.SplitFetcher.run(
SplitFetcher.java:105)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors
.java:515)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(
ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(
ThreadPoolExecutor.java:628)
... 1 more
Caused by: org.apache.kafka.common.errors.TopicAuthorizationException:

-- 
RAZIN BOUZAR
Monitoring Cloud | Salesforce
Mobile: 317-502-8995

Admin Client Configs

[Razin Bouzar via user]

Looking for some guidance on overriding Kafka admin client configs. The
partition discovery feature sometimes fails to connect with kafka and we'd
like to implement retries. There is no clear documentation on which
settings need to be changed?admin client configs of note:

   - default.api.timeout.ms -- applies for client ops which do not specify
   a timeout param.
   - reconnect.backoff.max
   

   - request.timeout.ms
   - retries
   

   - retry.backoff.ms

CVE-2019-14887

[Razin Bouzar via user]

Hello,

CVE-2019-14887
 is flagged
as a critical vulnerability in Wildfly 1.0.7.Final. It exists within:

   1. flink-azure-fs-hadoop
   

   2. flink-s3-fs-hadoop
   

   3. flink-s3-fs-presto
   


There is a fix for this CVE in Widlfly v1.0.10.Final. Can we create a Jira
and/or assist with upgrading to 1.0.10?

Thank you,
Razin

-- 
RAZIN BOUZAR
Senior Engineer - Monitoring Cloud | Salesforce
Mobile: 317-502-8995