RE: Identifying a flink dashboard

2023-06-27 Thread Schwalbe Matthias
Good Morning Mike, As a quick fix, sort of, you could use an Ingress on nginx-ingress (instead of the port-forward) and Add a sub_filter rule to patch the HTML response. I use this to add a tag to the header and for the Flink-Dashboard I experience no glitches. As to point 3. … you don’t need

Re:Kafka source with idleness and alignment stops consuming

2023-06-27 Thread haishui
Hi, I have the same trouble. This is really a bug. `shouldWaitForAlignment` needs to be another change. By the way, a source will be marked as idle, when the source has waiting for alignment for a long time. Is this a bug? 在 2023-06-27 23:25:38,"Alexis Sarda-Espinosa" 写道: He

Re: RocksDB State Backend GET returns null intermittently

2023-06-27 Thread Prabhu Joseph
Thanks Hangxiang and Alex for the pointers. Have added audit logs into RocsDBValueState (GET call: value() and PUT call: update()) and found nothing wrong on the RocsDB side. It never sends Null to the GET call for the key, which was PUT earlier. Then we added audit logs into the CX application and

Re: Very long launch of the Flink application in BATCH mode

2023-06-27 Thread Shammon FY
Hi Brendan, I think you may need to confirm which stage the job is blocked, the client is submitting job or resourcemanage is scheduling job or tasks are launching in TM? May be you need provide more information to help us to figure the issue Best, Shammon FY On Tuesday, June 27, 2023, Weihua Hu

Re: Identifying a flink dashboard

2023-06-27 Thread Mike Phillips
G'day Alex, Thanks! 1 - hmm maybe beyond my capabilities presently 2 - Yuck! :-) Will look at this 3 - Not possible, the dashboards are not accessible via the internet, so we use kube and port forward, URL looks like http://wobbegong:3/ the port changes 4 - I think this requires the dashb

Kafka source with idleness and alignment stops consuming

2023-06-27 Thread Alexis Sarda-Espinosa
Hello, I am currently evaluating idleness and alignment with Flink 1.17.1 and the externalized Kafka connector. My job has 3 sources whose watermark strategies are defined like this: WatermarkStrategy.forBoundedOutOfOrderness(maxAllowedWatermarkDrift) .withIdleness(idleTimeout)

Re: Very long launch of the Flink application in BATCH mode

2023-06-27 Thread Weihua Hu
Hi, Brendan It looks like it's invoking your main method referring to the log. You can add more logs in the main method to figure out which part takes too long. Best, Weihua On Tue, Jun 27, 2023 at 5:06 AM Brendan Cortez wrote: > No, I'm using a collection source + 20 same JDBC lookups + Kafk

Re: RocksDB State Backend GET returns null intermittently

2023-06-27 Thread Alexander Fedulov
Hi Prabhu, make sure that the key you use is the same for both records and try to reproduce the issue with the level of parallelism of 1. Best, Alex On Sun, 25 Jun 2023 at 04:29, Hangxiang Yu wrote: > Hi, Prabhu. > > This is a correctness issue. IIUC, It should not be related to the size of >

Re: Identifying a flink dashboard

2023-06-27 Thread Alexander Fedulov
Hi Mike, no, it is currently hard-coded https://github.com/apache/flink/blob/master/flink-runtime-web/web-dashboard/src/app/app.component.html#L23 Your options are: 1. Contribute a change to make it configurable 2. Use some browser plugin that allows renaming page titles 3. Always use different p

Invalid AMRMToken from appattempy

2023-06-27 Thread 湘晗刚
Caused by :org.apache.hadoop.ipc.RemoteException:Invalid AMRMToken from appattempy_162**_1178_1 at org.apache.hadoop.ipc.Client.getRpcResponse(Client.java:1563) … Flink 1.10 on yarn Thanks in advance 湘晗刚