Re: Weird Flink SQL error

2022-11-24 Thread Dan Hill 
Thanks, Leonard!

I just solved this also by replacing "ROW(" with "(" from the create view
statement.

CREATE TEMPORARY VIEW `test_content_metrics_view` AS
SELECT
DATE_FORMAT(TUMBLE_ROWTIME(rowtime, INTERVAL '1' DAY), '-MM-dd'),
(
platform_id,
content_id
)
FROM content_event
GROUP BY
platform_id,
content_id,
TUMBLE(rowtime, INTERVAL '1' DAY)



On Thu, Nov 24, 2022 at 7:37 PM Leonard Xu  wrote:

> Do not trust the line number from sql parser exception, you should use
> ROW in your DDL when you declare a composite row type,
> try the following:
>
> CREATE TABLE test_content_metrics (
>dt STRING NOT NULL,
>`body` ROW<
>`platform_id` BIGINT,
>`content_id` STRING
>>
> ) PARTITIONED BY (dt) WITH (
>'connector' = 'filesystem',
>'path' = 'etl/test_content_metrics',
>'format' = 'json',
> )
>
>
> Best,
> Leonard
>
>
> On Nov 25, 2022, at 11:20 AM, Dan Hill  wrote:
>
> Also, if I try to do an aggregate inside the ROW, I get an error.  I don't
> get the error if it's not wrapped in.a Row.
>
> ROW(
> SUM(view_count)
> ) AS body,
>
>
>  Caused by: org.apache.flink.table.api.SqlParserException: SQL parse 
> failed. Encountered "SUM" at line 8, column 5.
> Was expecting one of:
> 
> "EXCEPT" ...
> "FETCH" ...
> "FROM" ...
> "INTERSECT" ...
> "LIMIT" ...
> "OFFSET" ...
> "ORDER" ...
> "MINUS" ...
> "UNION" ...
> "," ...
>
>
> org.apache.flink.table.planner.parse.CalciteParser.parse(CalciteParser.java:56)
>
> org.apache.flink.table.planner.delegation.ParserImpl.parse(ParserImpl.java:98)
>
> org.apache.flink.table.api.internal.TableEnvironmentImpl.executeSql(TableEnvironmentImpl.java:736)
>
> ai.promoted.metrics.logprocessor.job.contentmetrics.ContentMetricsJob.executeSqlFromResource(ContentMetricsJob.java:148)
>[...]
>  Caused by: org.apache.calcite.sql.parser.SqlParseException: Encountered 
> "SUM" at line 8, column 5.
> Was expecting one of:
> 
> "EXCEPT" ...
> "FETCH" ...
> "FROM" ...
> "INTERSECT" ...
> "LIMIT" ...
> "OFFSET" ...
> "ORDER" ...
> "MINUS" ...
> "UNION" ...
> "," ...
>
>
> org.apache.flink.sql.parser.impl.FlinkSqlParserImpl.convertException(FlinkSqlParserImpl.java:462)
>
> org.apache.flink.sql.parser.impl.FlinkSqlParserImpl.normalizeException(FlinkSqlParserImpl.java:225)
>
> org.apache.calcite.sql.parser.SqlParser.handleException(SqlParser.java:140)
>org.apache.calcite.sql.parser.SqlParser.parseQuery(SqlParser.java:155)
>org.apache.calcite.sql.parser.SqlParser.parseStmt(SqlParser.java:180)
>[...]
>  Caused by: org.apache.flink.sql.parser.impl.ParseException: Encountered 
> "SUM" at line 8, column 5.
> Was expecting one of:
> 
> "EXCEPT" ...
> "FETCH" ...
> "FROM" ...
> "INTERSECT" ...
> "LIMIT" ...
> "OFFSET" ...
> "ORDER" ...
> "MINUS" ...
> "UNION" ...
> "," ...
>
>
> org.apache.flink.sql.parser.impl.FlinkSqlParserImpl.generateParseException(FlinkSqlParserImpl.java:40981)
>
> org.apache.flink.sql.parser.impl.FlinkSqlParserImpl.jj_consume_token(FlinkSqlParserImpl.java:40792)
>
> org.apache.flink.sql.parser.impl.FlinkSqlParserImpl.SqlStmtEof(FlinkSqlParserImpl.java:3981)
>
> org.apache.flink.sql.parser.impl.FlinkSqlParserImpl.parseSqlStmtEof(FlinkSqlParserImpl.java:273)
>org.apache.calcite.sql.parser.SqlParser.parseQuery(SqlParser.java:153)
>[...]
>
>
> On Thu, Nov 24, 2022 at 6:41 PM Dan Hill  wrote:
>
>> Here's the full stack trace.
>>
>> => org.apache.flink.table.api.SqlParserException: SQL parse failed. 
>> Encountered "." at line 1, column 336.
>> Was expecting one of:
>> ")" ...
>> "," ...
>>
>>
>> org.apache.flink.table.planner.parse.CalciteParser.parse(CalciteParser.java:56)
>>
>> org.apache.flink.table.planner.calcite.FlinkPlannerImpl$ToRelContextImpl.expandView(FlinkPlannerImpl.scala:270)
>>
>> org.apache.calcite.plan.ViewExpanders$1.expandView(ViewExpanders.java:52)
>>
>> org.apache.flink.table.planner.catalog.SqlCatalogViewTable.convertToRel(SqlCatalogViewTable.java:58)
>>
>> org.apache.flink.table.planner.plan.schema.ExpandingPreparingTable.expand(ExpandingPreparingTable.java:59)
>>[...]
>>  Caused by: org.apache.calcite.sql.parser.SqlParseException: Encountered 
>> "." at line 1, column 336.
>> Was expecting one of:
>> ")" ...
>> "," ...
>>
>>
>> org.apache.flink.sql.parser.impl.FlinkSqlParserImpl.convertException(FlinkSqlParserImpl.java:462)
>>
>> org.apache.flink.sql.parser.impl.FlinkSqlParserImpl.normalizeException(FlinkSqlParserImpl.java:225)
>>
>> org.apache.calcite.sql.parser.SqlParser.handleException(SqlParser.java:140)
>>org.apache.calcite.sql.parser.SqlParser.parseQuery(SqlParser.java:155)
>>org.apache.calcite.

Support for higher-than-millisecond resolution event-time timestamps

2022-11-24 Thread Salva Alcántara 
As mentioned in the docs

:

> Attention: Both timestamps and watermarks are specified as milliseconds
since the Java epoch of 1970-01-01T00:00:00Z.

Are there any plans for supporting higher time resolutions?

Also, internally, Flink uses the `long` type for the timestamps, so maybe
the milliseconds assumption is arbitrary and things would actually work
just fine for higher resolutions provided that they fit into the long type
(???). I found this SO post:

https://stackoverflow.com/questions/54402759/streaming-data-processing-and-nano-second-time-resolution

which touches upon this but it's a bit old already and there seems to be no
clear answer in the end. So maybe we could touch base on it.

Regards,

Salva

Re: Weird Flink SQL error

2022-11-24 Thread Leonard Xu 
Do not trust the line number from sql parser exception, you should use ROW in your DDL when you declare a composite row type, try the 
following:
CREATE TABLE test_content_metrics (
   dt STRING NOT NULL,
   `body` ROW<
   `platform_id` BIGINT,
   `content_id` STRING
   >
) PARTITIONED BY (dt) WITH (
   'connector' = 'filesystem',
   'path' = 'etl/test_content_metrics',
   'format' = 'json',
)

Best,
Leonard


> On Nov 25, 2022, at 11:20 AM, Dan Hill  wrote:
> 
> Also, if I try to do an aggregate inside the ROW, I get an error.  I don't 
> get the error if it's not wrapped in.a Row.
> 
> ROW(
> SUM(view_count)
> ) AS body,
> 
>  Caused by: org.apache.flink.table.api.SqlParserException: SQL parse 
> failed. Encountered "SUM" at line 8, column 5.
> Was expecting one of:
>  
> "EXCEPT" ...
> "FETCH" ...
> "FROM" ...
> "INTERSECT" ...
> "LIMIT" ...
> "OFFSET" ...
> "ORDER" ...
> "MINUS" ...
> "UNION" ...
> "," ...
> 
>
> org.apache.flink.table.planner.parse.CalciteParser.parse(CalciteParser.java:56)
>
> org.apache.flink.table.planner.delegation.ParserImpl.parse(ParserImpl.java:98)
>
> org.apache.flink.table.api.internal.TableEnvironmentImpl.executeSql(TableEnvironmentImpl.java:736)
>
> ai.promoted.metrics.logprocessor.job.contentmetrics.ContentMetricsJob.executeSqlFromResource(ContentMetricsJob.java:148)
>[...]
>  Caused by: org.apache.calcite.sql.parser.SqlParseException: Encountered 
> "SUM" at line 8, column 5.
> Was expecting one of:
>  
> "EXCEPT" ...
> "FETCH" ...
> "FROM" ...
> "INTERSECT" ...
> "LIMIT" ...
> "OFFSET" ...
> "ORDER" ...
> "MINUS" ...
> "UNION" ...
> "," ...
> 
>
> org.apache.flink.sql.parser.impl.FlinkSqlParserImpl.convertException(FlinkSqlParserImpl.java:462)
>
> org.apache.flink.sql.parser.impl.FlinkSqlParserImpl.normalizeException(FlinkSqlParserImpl.java:225)
>
> org.apache.calcite.sql.parser.SqlParser.handleException(SqlParser.java:140)
>org.apache.calcite.sql.parser.SqlParser.parseQuery(SqlParser.java:155)
>org.apache.calcite.sql.parser.SqlParser.parseStmt(SqlParser.java:180)
>[...]
>  Caused by: org.apache.flink.sql.parser.impl.ParseException: Encountered 
> "SUM" at line 8, column 5.
> Was expecting one of:
>  
> "EXCEPT" ...
> "FETCH" ...
> "FROM" ...
> "INTERSECT" ...
> "LIMIT" ...
> "OFFSET" ...
> "ORDER" ...
> "MINUS" ...
> "UNION" ...
> "," ...
> 
>
> org.apache.flink.sql.parser.impl.FlinkSqlParserImpl.generateParseException(FlinkSqlParserImpl.java:40981)
>
> org.apache.flink.sql.parser.impl.FlinkSqlParserImpl.jj_consume_token(FlinkSqlParserImpl.java:40792)
>
> org.apache.flink.sql.parser.impl.FlinkSqlParserImpl.SqlStmtEof(FlinkSqlParserImpl.java:3981)
>
> org.apache.flink.sql.parser.impl.FlinkSqlParserImpl.parseSqlStmtEof(FlinkSqlParserImpl.java:273)
>org.apache.calcite.sql.parser.SqlParser.parseQuery(SqlParser.java:153)
>[...]
> 
> On Thu, Nov 24, 2022 at 6:41 PM Dan Hill  > wrote:
> Here's the full stack trace.
> 
> => org.apache.flink.table.api.SqlParserException: SQL parse failed. 
> Encountered "." at line 1, column 336.
> Was expecting one of:
> ")" ...
> "," ...
> 
>
> org.apache.flink.table.planner.parse.CalciteParser.parse(CalciteParser.java:56)
>
> org.apache.flink.table.planner.calcite.FlinkPlannerImpl$ToRelContextImpl.expandView(FlinkPlannerImpl.scala:270)
>
> org.apache.calcite.plan.ViewExpanders$1.expandView(ViewExpanders.java:52)
>
> org.apache.flink.table.planner.catalog.SqlCatalogViewTable.convertToRel(SqlCatalogViewTable.java:58)
>
> org.apache.flink.table.planner.plan.schema.ExpandingPreparingTable.expand(ExpandingPreparingTable.java:59)
>[...]
>  Caused by: org.apache.calcite.sql.parser.SqlParseException: Encountered 
> "." at line 1, column 336.
> Was expecting one of:
> ")" ...
> "," ...
> 
>
> org.apache.flink.sql.parser.impl.FlinkSqlParserImpl.convertException(FlinkSqlParserImpl.java:462)
>
> org.apache.flink.sql.parser.impl.FlinkSqlParserImpl.normalizeException(FlinkSqlParserImpl.java:225)
>
> org.apache.calcite.sql.parser.SqlParser.handleException(SqlParser.java:140)
>org.apache.calcite.sql.parser.SqlParser.parseQuery(SqlParser.java:155)
>org.apache.calcite.sql.parser.SqlParser.parseStmt(SqlParser.java:180)
>[...]
>  Caused by: org.apache.flink.sql.parser.impl.ParseException: Encountered 
> "." at line 1, column 336.
> Was expecting one of:
> ")" ...
> "," ...
> 
>
> org.apache.flink.sql.parser.impl.FlinkSqlParserImpl.generateParseException(FlinkSqlParserImpl.java:40981)
>
> org.apache.flink.sql.parser.impl.FlinkSqlParserImpl.jj_consume_token(FlinkSqlParserImpl.jav

Re: Weird Flink SQL error

2022-11-24 Thread Dan Hill 
Also, if I try to do an aggregate inside the ROW, I get an error.  I don't
get the error if it's not wrapped in.a Row.

ROW(
SUM(view_count)
) AS body,


 Caused by: org.apache.flink.table.api.SqlParserException: SQL
parse failed. Encountered "SUM" at line 8, column 5.
Was expecting one of:

"EXCEPT" ...
"FETCH" ...
"FROM" ...
"INTERSECT" ...
"LIMIT" ...
"OFFSET" ...
"ORDER" ...
"MINUS" ...
"UNION" ...
"," ...

   
org.apache.flink.table.planner.parse.CalciteParser.parse(CalciteParser.java:56)
   
org.apache.flink.table.planner.delegation.ParserImpl.parse(ParserImpl.java:98)
   
org.apache.flink.table.api.internal.TableEnvironmentImpl.executeSql(TableEnvironmentImpl.java:736)
   
ai.promoted.metrics.logprocessor.job.contentmetrics.ContentMetricsJob.executeSqlFromResource(ContentMetricsJob.java:148)
   [...]
 Caused by: org.apache.calcite.sql.parser.SqlParseException:
Encountered "SUM" at line 8, column 5.
Was expecting one of:

"EXCEPT" ...
"FETCH" ...
"FROM" ...
"INTERSECT" ...
"LIMIT" ...
"OFFSET" ...
"ORDER" ...
"MINUS" ...
"UNION" ...
"," ...

   
org.apache.flink.sql.parser.impl.FlinkSqlParserImpl.convertException(FlinkSqlParserImpl.java:462)
   
org.apache.flink.sql.parser.impl.FlinkSqlParserImpl.normalizeException(FlinkSqlParserImpl.java:225)
   
org.apache.calcite.sql.parser.SqlParser.handleException(SqlParser.java:140)
   org.apache.calcite.sql.parser.SqlParser.parseQuery(SqlParser.java:155)
   org.apache.calcite.sql.parser.SqlParser.parseStmt(SqlParser.java:180)
   [...]
 Caused by: org.apache.flink.sql.parser.impl.ParseException:
Encountered "SUM" at line 8, column 5.
Was expecting one of:

"EXCEPT" ...
"FETCH" ...
"FROM" ...
"INTERSECT" ...
"LIMIT" ...
"OFFSET" ...
"ORDER" ...
"MINUS" ...
"UNION" ...
"," ...

   
org.apache.flink.sql.parser.impl.FlinkSqlParserImpl.generateParseException(FlinkSqlParserImpl.java:40981)
   
org.apache.flink.sql.parser.impl.FlinkSqlParserImpl.jj_consume_token(FlinkSqlParserImpl.java:40792)
   
org.apache.flink.sql.parser.impl.FlinkSqlParserImpl.SqlStmtEof(FlinkSqlParserImpl.java:3981)
   
org.apache.flink.sql.parser.impl.FlinkSqlParserImpl.parseSqlStmtEof(FlinkSqlParserImpl.java:273)
   org.apache.calcite.sql.parser.SqlParser.parseQuery(SqlParser.java:153)
   [...]


On Thu, Nov 24, 2022 at 6:41 PM Dan Hill  wrote:

> Here's the full stack trace.
>
> => org.apache.flink.table.api.SqlParserException: SQL parse failed. 
> Encountered "." at line 1, column 336.
> Was expecting one of:
> ")" ...
> "," ...
>
>
> org.apache.flink.table.planner.parse.CalciteParser.parse(CalciteParser.java:56)
>
> org.apache.flink.table.planner.calcite.FlinkPlannerImpl$ToRelContextImpl.expandView(FlinkPlannerImpl.scala:270)
>
> org.apache.calcite.plan.ViewExpanders$1.expandView(ViewExpanders.java:52)
>
> org.apache.flink.table.planner.catalog.SqlCatalogViewTable.convertToRel(SqlCatalogViewTable.java:58)
>
> org.apache.flink.table.planner.plan.schema.ExpandingPreparingTable.expand(ExpandingPreparingTable.java:59)
>[...]
>  Caused by: org.apache.calcite.sql.parser.SqlParseException: Encountered 
> "." at line 1, column 336.
> Was expecting one of:
> ")" ...
> "," ...
>
>
> org.apache.flink.sql.parser.impl.FlinkSqlParserImpl.convertException(FlinkSqlParserImpl.java:462)
>
> org.apache.flink.sql.parser.impl.FlinkSqlParserImpl.normalizeException(FlinkSqlParserImpl.java:225)
>
> org.apache.calcite.sql.parser.SqlParser.handleException(SqlParser.java:140)
>org.apache.calcite.sql.parser.SqlParser.parseQuery(SqlParser.java:155)
>org.apache.calcite.sql.parser.SqlParser.parseStmt(SqlParser.java:180)
>[...]
>  Caused by: org.apache.flink.sql.parser.impl.ParseException: Encountered 
> "." at line 1, column 336.
> Was expecting one of:
> ")" ...
> "," ...
>
>
> org.apache.flink.sql.parser.impl.FlinkSqlParserImpl.generateParseException(FlinkSqlParserImpl.java:40981)
>
> org.apache.flink.sql.parser.impl.FlinkSqlParserImpl.jj_consume_token(FlinkSqlParserImpl.java:40792)
>
> org.apache.flink.sql.parser.impl.FlinkSqlParserImpl.ParenthesizedSimpleIdentifierList(FlinkSqlParserImpl.java:25220)
>
> org.apache.flink.sql.parser.impl.FlinkSqlParserImpl.Expression3(FlinkSqlParserImpl.java:19925)
>
> org.apache.flink.sql.parser.impl.FlinkSqlParserImpl.Expression2b(FlinkSqlParserImpl.java:19581)
>[...]
>
>
>
> On Wed, Nov 23, 2022 at 1:55 PM Dan Hill  wrote:
>
>> If I remove the "TEMPORARY VIEW" and just inline the SQL, this works
>> fine.  This seems like a bug with temporary views.
>>
>> On Wed, Nov 23, 2022 at 1:38 PM Dan Hill  wrote:
>>
>>> Looks related to this issue.
>>> https://lists.apache.org/thread/1sb5bos6tjv39fh0wjkvmv

Re: Weird Flink SQL error

2022-11-24 Thread Dan Hill 
Here's the full stack trace.

=> org.apache.flink.table.api.SqlParserException: SQL parse
failed. Encountered "." at line 1, column 336.
Was expecting one of:
")" ...
"," ...

   
org.apache.flink.table.planner.parse.CalciteParser.parse(CalciteParser.java:56)
   
org.apache.flink.table.planner.calcite.FlinkPlannerImpl$ToRelContextImpl.expandView(FlinkPlannerImpl.scala:270)
   org.apache.calcite.plan.ViewExpanders$1.expandView(ViewExpanders.java:52)
   
org.apache.flink.table.planner.catalog.SqlCatalogViewTable.convertToRel(SqlCatalogViewTable.java:58)
   
org.apache.flink.table.planner.plan.schema.ExpandingPreparingTable.expand(ExpandingPreparingTable.java:59)
   [...]
 Caused by: org.apache.calcite.sql.parser.SqlParseException:
Encountered "." at line 1, column 336.
Was expecting one of:
")" ...
"," ...

   
org.apache.flink.sql.parser.impl.FlinkSqlParserImpl.convertException(FlinkSqlParserImpl.java:462)
   
org.apache.flink.sql.parser.impl.FlinkSqlParserImpl.normalizeException(FlinkSqlParserImpl.java:225)
   
org.apache.calcite.sql.parser.SqlParser.handleException(SqlParser.java:140)
   org.apache.calcite.sql.parser.SqlParser.parseQuery(SqlParser.java:155)
   org.apache.calcite.sql.parser.SqlParser.parseStmt(SqlParser.java:180)
   [...]
 Caused by: org.apache.flink.sql.parser.impl.ParseException:
Encountered "." at line 1, column 336.
Was expecting one of:
")" ...
"," ...

   
org.apache.flink.sql.parser.impl.FlinkSqlParserImpl.generateParseException(FlinkSqlParserImpl.java:40981)
   
org.apache.flink.sql.parser.impl.FlinkSqlParserImpl.jj_consume_token(FlinkSqlParserImpl.java:40792)
   
org.apache.flink.sql.parser.impl.FlinkSqlParserImpl.ParenthesizedSimpleIdentifierList(FlinkSqlParserImpl.java:25220)
   
org.apache.flink.sql.parser.impl.FlinkSqlParserImpl.Expression3(FlinkSqlParserImpl.java:19925)
   
org.apache.flink.sql.parser.impl.FlinkSqlParserImpl.Expression2b(FlinkSqlParserImpl.java:19581)
   [...]



On Wed, Nov 23, 2022 at 1:55 PM Dan Hill  wrote:

> If I remove the "TEMPORARY VIEW" and just inline the SQL, this works
> fine.  This seems like a bug with temporary views.
>
> On Wed, Nov 23, 2022 at 1:38 PM Dan Hill  wrote:
>
>> Looks related to this issue.
>> https://lists.apache.org/thread/1sb5bos6tjv39fh0wjkvmvht0824r4my
>>
>> In my case, it doesn't seem like it's a sink issue.  Even if I change my
>> minicluster test to SELECT * it, it fails the same way.
>>
>> CREATE TEMPORARY VIEW `test_content_metrics_view` AS
>> SELECT
>> DATE_FORMAT(TUMBLE_ROWTIME(rowtime, INTERVAL '1' DAY), '-MM-dd'),
>> ROW(
>> platform_id,
>> content_id
>> )
>> FROM content_event
>> GROUP BY
>> platform_id,
>> content_id,
>> TUMBLE(rowtime, INTERVAL '1' DAY)
>>
>> SELECT * FROM test_content_metrics_view
>>
>>
>>
>> On Wed, Nov 23, 2022 at 1:19 PM Dan Hill  wrote:
>>
>>> I upgraded to Flink v1.16.0 and I get the same error.
>>>
>>> On Wed, Nov 23, 2022 at 9:47 AM Dan Hill  wrote:
>>>
 For the error `Encountered "." at line 1, column 119.`, here are the
 confusing parts:

 1. The error happens when I executed the last part of the sql query:

 INSERT INTO `test_content_metrics`
 SELECT * FROM `test_content_metrics_view`

 2. Line 1 column 119 doesn't exist in that SQL statement.
 3. None of the SQL that I've written has a period "." in it.



 On Wed, Nov 23, 2022 at 8:32 AM Dan Hill  wrote:

> I'm using Flink 1.14.4
>
> On Wed, Nov 23, 2022, 02:28 yuxia  wrote:
>
>> Hi, Dan.
>> I'm wondering what type of error you expect. IMO, I think most
>> engines throw parse error in such way which tell you encounter an
>> unexpected token.
>>
>> Best regards,
>> Yuxia
>>
>> --
>> *发件人: *"Dan Hill" 
>> *收件人: *"User" 
>> *发送时间: *星期三, 2022年 11 月 23日 下午 1:55:20
>> *主题: *Weird Flink SQL error
>>
>> Hi.  I'm hitting an obfuscated Flink SQL parser error.  Is there a
>> way to get better errors for Flink SQL?  I'm hitting it when I wrap some 
>> of
>> the fields on an inner Row.
>>
>>
>> *Works*
>>
>> CREATE TEMPORARY VIEW `test_content_metrics_view` AS
>> SELECT
>> DATE_FORMAT(TUMBLE_ROWTIME(rowtime, INTERVAL '1' DAY), '-MM-dd'),
>> platform_id,
>> content_id
>> FROM content_event
>> GROUP BY
>> platform_id,
>> content_id,
>> TUMBLE(rowtime, INTERVAL '1' DAY)
>>
>> CREATE TABLE test_content_metrics (
>>dt STRING NOT NULL,
>>`platform_id` BIGINT,
>>`content_id` STRING
>> ) PARTITIONED BY (dt) WITH (
>>'connector' = 'filesystem',
>>'path' = 'etl/test_content_metrics',
>>'format' = 'json',
>> )
>>
>> INSERT INTO `test_content_metrics`
>> SELECT * FROM `test_conten

Re: Stand alone K8s HA mode with Static Tokens Used by Service Accounts

2022-11-24 Thread Yang Wang 
IIUC, the fabric8 Kubernetes-client 5.5.0 should already support to reload
the latest kube config if received 401 error.
Refer to the following PR[1] for more information.

Please share your feedback here if it still could not work.

[1]. https://github.com/fabric8io/kubernetes-client/pull/2731

Best,
Yang

Berkay Polat via user  于2022年11月23日周三 01:57写道：

> Hi team,
>
> Bumping this up again, from the AWS docs, the suggested approach is to
> simply upgrade the K8s java SDK client (
> https://github.com/kubernetes-client/java/) being used. However, in
> Flink's case with the io.fabric8 K8s client, I am not sure how to handle
> it. Any help and guidance would be much appreciated.
>
> Thanks,
>
> -- Forwarded message -
> From: Berkay Polat 
> Date: Thu, Nov 17, 2022 at 12:36 PM
> Subject: Stand alone K8s HA mode with Static Tokens Used by Service
> Accounts
> To: 
>
>
> Hi,
>
> Our team has been using flink 1.15 and we have a stand alone K8s flink
> setup that uses K8s HA services for its HA mode. Recently, our organization
> is in the works of updating their EKS clusters' Kubernetes versions to 1.21
> or later. We received a request from our support team that the service
> accounts associated with our stand alone flink cluster have been using
> static tokens, which is not permitted for newer K8s versions. Instead, they
> requested us to switch to a refresh token approach (
> https://docs.aws.amazon.com/eks/latest/userguide/service-accounts.html#identify-pods-using-stale-tokens
> ).
>
> From what I understand, in flink 1.15, HA mode is using version 5.5.0 of
> io.fabric8's kubernetes client and it seems that it is compatible with K8s
> 1.21.1 and later (
> https://github.com/fabric8io/kubernetes-client#compatibility-matrix) so I
> am not sure what the underlying limitation/issue is here.
>
> The AWS doc link I referred to earlier recommends upgrading versions for
> Kubernetes Client SDKs but it refers to io.kubernetes's client SDKs, not
> io.fabric8.
>
> Could someone shed some light on it? Would it be worth it to request a
> change to upgrade the io.fabric8 kubernetes client version to a newer
> version?
>
> Thanks,
> --
> *BERKAY POLAT*
> Software Engineer SMTS | MuleSoft at Salesforce
> Mobile: 443-710-7021
>
> 
>
>
> --
> *BERKAY POLAT*
> Software Engineer SMTS | MuleSoft at Salesforce
> Mobile: 443-710-7021
>

Re: Query about flink job manager dashboard

2022-11-24 Thread naga sudhakar 
Thank you for your response.

On Thu, 24 Nov, 2022, 2:07 PM Martijn Visser, 
wrote:

> Hi,
>
> 1) No, that's currently not possible.
> 2) You could consider disabling to disallow uploading new JARs and/or
> cancelling jobs from the UI. See
> https://nightlies.apache.org/flink/flink-docs-master/docs/deployment/config/#advanced-options-for-flink-web-ui
>
> Best regards,
>
> Martijn
>
> On Thu, Nov 24, 2022 at 3:44 AM naga sudhakar 
> wrote:
>
>> Hi Team,
>>> Greetings!!!
>>> I am a software developer using apache flink and deploying flink jobs
>>> using the same. I have two queries about flink job manager dashboard. Can
>>> you please help with below?
>>>
>>> 1) is it possible to add login mechanism for the flink job manager dash
>>> board and have a role based mechanism for viewing running jobs, cancelling
>>> jobs, adding the jobs?
>>> 2) is it possible to disable to dash bord display but use api to do the
>>> same operations using API?
>>>
>>>
>>> Thanks,
>>> Nagasudhakar.
>>>
>>

Re: Query about flink job manager dashboard

2022-11-24 Thread Martijn Visser 
Hi,

1) No, that's currently not possible.
2) You could consider disabling to disallow uploading new JARs and/or
cancelling jobs from the UI. See
https://nightlies.apache.org/flink/flink-docs-master/docs/deployment/config/#advanced-options-for-flink-web-ui

Best regards,

Martijn

On Thu, Nov 24, 2022 at 3:44 AM naga sudhakar 
wrote:

> Hi Team,
>> Greetings!!!
>> I am a software developer using apache flink and deploying flink jobs
>> using the same. I have two queries about flink job manager dashboard. Can
>> you please help with below?
>>
>> 1) is it possible to add login mechanism for the flink job manager dash
>> board and have a role based mechanism for viewing running jobs, cancelling
>> jobs, adding the jobs?
>> 2) is it possible to disable to dash bord display but use api to do the
>> same operations using API?
>>
>>
>> Thanks,
>> Nagasudhakar.
>>
>