[jira] [Comment Edited] (FLINK-32666) ASM rewrite class lead to package failed.
[
https://issues.apache.org/jira/browse/FLINK-32666?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17747305#comment-17747305
]
lizhiqiang edited comment on FLINK-32666 at 7/26/23 6:55 AM:
-
I see this restriction in the maven version in the pom.xml file of the master
branch
{code:java}
org.apache.maven.plugins
maven-enforcer-plugin
enforce-maven
enforce
(,3.3)
1.8.0
{code}
And maven-enforcer-plugin does not to restrict minor versions of jdk 1.8 .
We would prefer to develop a smaller version of java labeled on the
documentation.
was (Author: JIRAUSER286148):
I see this restriction in the maven version in the pom.xml file of the master
branch
{code:java}
org.apache.maven.plugins
maven-enforcer-plugin
enforce-maven
enforce
(,3.3)
1.8.0
{code}
And maven-enforcer-plugin has no way to restrict minor versions of jdk 1.8 .
We would prefer to develop a smaller version of java labeled on the
documentation.
> ASM rewrite class lead to package failed.
> -
>
> Key: FLINK-32666
> URL: https://issues.apache.org/jira/browse/FLINK-32666
> Project: Flink
> Issue Type: Bug
> Components: Table SQL / Planner
>Affects Versions: 1.18.0
>Reporter: lizhiqiang
>Priority: Major
>
> {code:java}
> [DEBUG] Processing JAR
> /Users/lzq/Desktop/Projects/Flink/flink/flink-master/flink-table/flink-table-planner/target/flink-table-planner_2.12-1.17-SNAPSHOT.jar
> [DEBUG] Rewrote class bytecode:
> org/apache/calcite/interpreter/JaninoRexCompiler.class
> [DEBUG] Rewrote class bytecode:
> org/apache/calcite/tools/RelBuilder$Frame.class
> [DEBUG] Rewrote class bytecode:
> org/apache/calcite/tools/ImmutableRelBuilder$Config.class
> [DEBUG] Rewrote class bytecode: org/apache/calcite/tools/RelBuilder.class
> [DEBUG] Rewrote class bytecode:
> org/apache/calcite/tools/RelBuilder$OverCallImpl.class
> [DEBUG] Rewrote class bytecode:
> org/apache/calcite/tools/RelBuilder$GroupKey.class
> [DEBUG] Keeping original class bytecode:
> org/apache/calcite/tools/RelBuilder$OverCall.class
> [DEBUG] Rewrote class bytecode:
> org/apache/calcite/tools/RelBuilder$RelOptTableFinder.class
> [DEBUG] Rewrote class bytecode:
> org/apache/calcite/tools/ImmutableRelBuilder$Config$Builder.class
> [DEBUG] Rewrote class bytecode:
> org/apache/calcite/tools/RelBuilder$Registrar.class
> [DEBUG] Keeping original class bytecode:
> org/apache/calcite/tools/RelBuilder$OverCallImpl$1.class
> [DEBUG] Rewrote class bytecode:
> org/apache/calcite/tools/RelBuilder$AggCallImpl.class
> [DEBUG] Keeping original class bytecode:
> org/apache/calcite/tools/ImmutableRelBuilder.class
> [DEBUG] Rewrote class bytecode:
> org/apache/calcite/tools/RelBuilder$AggCall.class
> [DEBUG] Keeping original class bytecode:
> org/apache/calcite/tools/ImmutableRelBuilder$1.class
> [DEBUG] Keeping original class bytecode:
> org/apache/calcite/tools/RelBuilder$1.class
> [DEBUG] Rewrote class bytecode:
> org/apache/calcite/tools/RelBuilder$GroupKeyImpl.class
> [DEBUG] Keeping original class bytecode:
> org/apache/calcite/tools/RelBuilder$Config.class
> [DEBUG] Keeping original class bytecode:
> org/apache/calcite/tools/ImmutableRelBuilder$Config$InitShim.class
> [DEBUG] Rewrote class bytecode:
> org/apache/calcite/tools/RelBuilder$AggCallPlus.class
> [DEBUG] Rewrote class bytecode:
> org/apache/calcite/tools/RelBuilder$AggCallImpl2.class
> [DEBUG] Keeping original class bytecode:
> org/apache/calcite/tools/RelBuilder$Shifter.class
> [DEBUG] Rewrote class bytecode:
> org/apache/calcite/tools/RelBuilder$Field.class
> [DEBUG] Keeping original class bytecode:
> org/apache/calcite/tools/RelBuilder$2.class
> [DEBUG] Rewrote class bytecode:
> org/apache/calcite/util/javac/JaninoCompiler$JaninoCompilerArgs.class
> [DEBUG] Rewrote class bytecode:
> org/apache/calcite/util/javac/JaninoCompiler.class
> [DEBUG] Rewrote class bytecode:
> org/apache/calcite/util/javac/JaninoCompiler$AccountingClassLoader.class
> [DEBUG] Rewrote class bytecode:
> org/apache/calcite/adapter/enumerable/EnumerableInterpretable$1$1.class
> [DEBUG] Rewrote class bytecode:
> org/apache/calcite/adapter/enumerable/EnumerableInterpretable$EnumerableNode.clas
[jira] [Created] (FLINK-32679) Filter conditions cannot be pushed to JOIN in some case
grandfisher created FLINK-32679:
---
Summary: Filter conditions cannot be pushed to JOIN in some case
Key: FLINK-32679
URL: https://issues.apache.org/jira/browse/FLINK-32679
Project: Flink
Issue Type: Improvement
Components: Table SQL / Planner
Reporter: grandfisher
There is a case
{code:java}
SELECT a.id, b.id, c.id, d.id, e.id
, f.id
FROM `table-v1` a
INNER JOIN `table-v2` b ON a.id = b.id
INNER JOIN `table-v3` c ON b.id = c.id
INNER JOIN `table-v4` d ON c.id = d.id
INNER JOIN `table-v5` e ON d.id = e.id
INNER JOIN `table-v6` f ON a.id = f.id
WHERE f.id = 0
{code}
In this sql, each table should have a condition {*}id=0{*}, but actually only
table *f* and *a* has this condition.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Assigned] (FLINK-32678) Test FLIP-285 LeaderElection
[ https://issues.apache.org/jira/browse/FLINK-32678?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias Pohl reassigned FLINK-32678: - Assignee: Yang Wang > Test FLIP-285 LeaderElection > > > Key: FLINK-32678 > URL: https://issues.apache.org/jira/browse/FLINK-32678 > Project: Flink > Issue Type: Technical Debt > Components: Runtime / Coordination >Affects Versions: 1.18.0 >Reporter: Matthias Pohl >Assignee: Yang Wang >Priority: Blocker > Labels: release-testing > Fix For: 1.18.0 > > > We decided to do another round of testing for the LeaderElection refactoring > which happened in > [FLIP-285|https://cwiki.apache.org/confluence/display/FLINK/FLIP-285%3A+refactoring+leaderelection+to+make+flink+support+multi-component+leader+election+out-of-the-box]. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (FLINK-32666) ASM rewrite class lead to package failed.
[
https://issues.apache.org/jira/browse/FLINK-32666?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17747305#comment-17747305
]
lizhiqiang commented on FLINK-32666:
I see this restriction in the maven version in the pom.xml file of the master
branch
{code:java}
org.apache.maven.plugins
maven-enforcer-plugin
enforce-maven
enforce
(,3.3)
1.8.0
{code}
And maven-enforcer-plugin has no way to restrict minor versions of jdk 1.8 .
We would prefer to develop a smaller version of java labeled on the
documentation.
> ASM rewrite class lead to package failed.
> -
>
> Key: FLINK-32666
> URL: https://issues.apache.org/jira/browse/FLINK-32666
> Project: Flink
> Issue Type: Bug
> Components: Table SQL / Planner
>Affects Versions: 1.18.0
>Reporter: lizhiqiang
>Priority: Major
>
> {code:java}
> [DEBUG] Processing JAR
> /Users/lzq/Desktop/Projects/Flink/flink/flink-master/flink-table/flink-table-planner/target/flink-table-planner_2.12-1.17-SNAPSHOT.jar
> [DEBUG] Rewrote class bytecode:
> org/apache/calcite/interpreter/JaninoRexCompiler.class
> [DEBUG] Rewrote class bytecode:
> org/apache/calcite/tools/RelBuilder$Frame.class
> [DEBUG] Rewrote class bytecode:
> org/apache/calcite/tools/ImmutableRelBuilder$Config.class
> [DEBUG] Rewrote class bytecode: org/apache/calcite/tools/RelBuilder.class
> [DEBUG] Rewrote class bytecode:
> org/apache/calcite/tools/RelBuilder$OverCallImpl.class
> [DEBUG] Rewrote class bytecode:
> org/apache/calcite/tools/RelBuilder$GroupKey.class
> [DEBUG] Keeping original class bytecode:
> org/apache/calcite/tools/RelBuilder$OverCall.class
> [DEBUG] Rewrote class bytecode:
> org/apache/calcite/tools/RelBuilder$RelOptTableFinder.class
> [DEBUG] Rewrote class bytecode:
> org/apache/calcite/tools/ImmutableRelBuilder$Config$Builder.class
> [DEBUG] Rewrote class bytecode:
> org/apache/calcite/tools/RelBuilder$Registrar.class
> [DEBUG] Keeping original class bytecode:
> org/apache/calcite/tools/RelBuilder$OverCallImpl$1.class
> [DEBUG] Rewrote class bytecode:
> org/apache/calcite/tools/RelBuilder$AggCallImpl.class
> [DEBUG] Keeping original class bytecode:
> org/apache/calcite/tools/ImmutableRelBuilder.class
> [DEBUG] Rewrote class bytecode:
> org/apache/calcite/tools/RelBuilder$AggCall.class
> [DEBUG] Keeping original class bytecode:
> org/apache/calcite/tools/ImmutableRelBuilder$1.class
> [DEBUG] Keeping original class bytecode:
> org/apache/calcite/tools/RelBuilder$1.class
> [DEBUG] Rewrote class bytecode:
> org/apache/calcite/tools/RelBuilder$GroupKeyImpl.class
> [DEBUG] Keeping original class bytecode:
> org/apache/calcite/tools/RelBuilder$Config.class
> [DEBUG] Keeping original class bytecode:
> org/apache/calcite/tools/ImmutableRelBuilder$Config$InitShim.class
> [DEBUG] Rewrote class bytecode:
> org/apache/calcite/tools/RelBuilder$AggCallPlus.class
> [DEBUG] Rewrote class bytecode:
> org/apache/calcite/tools/RelBuilder$AggCallImpl2.class
> [DEBUG] Keeping original class bytecode:
> org/apache/calcite/tools/RelBuilder$Shifter.class
> [DEBUG] Rewrote class bytecode:
> org/apache/calcite/tools/RelBuilder$Field.class
> [DEBUG] Keeping original class bytecode:
> org/apache/calcite/tools/RelBuilder$2.class
> [DEBUG] Rewrote class bytecode:
> org/apache/calcite/util/javac/JaninoCompiler$JaninoCompilerArgs.class
> [DEBUG] Rewrote class bytecode:
> org/apache/calcite/util/javac/JaninoCompiler.class
> [DEBUG] Rewrote class bytecode:
> org/apache/calcite/util/javac/JaninoCompiler$AccountingClassLoader.class
> [DEBUG] Rewrote class bytecode:
> org/apache/calcite/adapter/enumerable/EnumerableInterpretable$1$1.class
> [DEBUG] Rewrote class bytecode:
> org/apache/calcite/adapter/enumerable/EnumerableInterpretable$EnumerableNode.class
> [DEBUG] Rewrote class bytecode:
> org/apache/calcite/adapter/enumerable/EnumerableInterpretable$1.class
> [DEBUG] Keeping original class bytecode:
> org/apache/calcite/adapter/enumerable/EnumerableInterpretable$StaticFieldDetector.class
> [DEBUG] Rewrote class bytecode:
> org/apache/calcite/adapter/enumerable/EnumerableInterpretable.class
> [DEBUG] Rewrote class bytecode:
> org/apache/calcite/adapter/enumerable/EnumerableInterpretable$1$1$1.class
> [DEBUG] Keeping original class bytecode:
> org/apache/calcite/jdbc/CalciteSchemaBuilder.class
> [DEBUG] Rewrote class bytecode:
> org/apache/calcite/rex/RexSimplify$SafeRexVisitor.class
> [DEBUG] Rewrote class bytecode:
> org/apache/calcite/rex/RexSimplify$SargCollector.class
> [D
[jira] [Commented] (FLINK-27054) Elasticsearch SQL connector SSL issue
[ https://issues.apache.org/jira/browse/FLINK-27054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17747304#comment-17747304 ] Martijn Visser commented on FLINK-27054: [~legendtkl] It all depends on a contributor who's willing to make the fix. It can also be that it depends on how this would work with the new Java Client from Elasticsearch, since that's needed if we ever want to be able to support ES 8 > Elasticsearch SQL connector SSL issue > - > > Key: FLINK-27054 > URL: https://issues.apache.org/jira/browse/FLINK-27054 > Project: Flink > Issue Type: Bug > Components: Connectors / ElasticSearch >Reporter: ricardo >Priority: Major > > The current Flink ElasticSearch SQL connector > https://nightlies.apache.org/flink/flink-docs-master/docs/connectors/table/elasticsearch/ > is missing SSL options, can't connect to ES clusters which require SSL > certificate. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (FLINK-32678) Test FLIP-285 LeaderElection
Matthias Pohl created FLINK-32678: - Summary: Test FLIP-285 LeaderElection Key: FLINK-32678 URL: https://issues.apache.org/jira/browse/FLINK-32678 Project: Flink Issue Type: Technical Debt Components: Runtime / Coordination Affects Versions: 1.18.0 Reporter: Matthias Pohl Fix For: 1.18.0 We decided to do another round of testing for the LeaderElection refactoring which happened in [FLIP-285|https://cwiki.apache.org/confluence/display/FLINK/FLIP-285%3A+refactoring+leaderelection+to+make+flink+support+multi-component+leader+election+out-of-the-box]. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (FLINK-32677) flink-benchmarks-regression-check failed to send slack messages since 2023.07.17
[
https://issues.apache.org/jira/browse/FLINK-32677?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17747303#comment-17747303
]
Martijn Visser commented on FLINK-32677:
[~Yanfei Lei] I think that was because the ASF Slack workspace had too many
integrations active, so I removed some of them already on the 17th and re-added
the integration for Jenkins.
> flink-benchmarks-regression-check failed to send slack messages since
> 2023.07.17
>
>
> Key: FLINK-32677
> URL: https://issues.apache.org/jira/browse/FLINK-32677
> Project: Flink
> Issue Type: Bug
> Components: Benchmarks
>Reporter: Yanfei Lei
>Priority: Critical
>
> {code:java}
> Response Code: 404 rel="alternate" type="text/html"
> href="http://codespeed.dak8s.net:8080/log"/>2717802023-07-19T11:38:29Z2023-07-19T11:38:29ZJul
> 19, 2023 11:38:29 AM jenkins.plugins.slack.StandardSlackService publish
> WARNING: Response Code: 404
> Slack post may have failed. Response:
> null href="http://codespeed.dak8s.net:8080/log"/>2717792023-07-19T11:38:29Z2023-07-19T11:38:29ZJul
> 19, 2023 11:38:29 AM jenkins.plugins.slack.StandardSlackService publish
> WARNING: Slack post may have failed. Response: null {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[GitHub] [flink] Tartarus0zm commented on pull request #23060: [FLINK-32519][docs] Add doc for [CREATE OR] REPLACE TABLE AS statement
Tartarus0zm commented on PR #23060: URL: https://github.com/apache/flink/pull/23060#issuecomment-1651049951 @luoyuxia thanks for your review! I have update the pr, PTAL, and I will add the docs for atomicity CTAS/RTAS in [FLINK-32581](https://issues.apache.org/jira/browse/FLINK-32581) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink-web] Myasuka commented on pull request #665: [FLINK-32611] Redirect to Apache Paimon's link instead of legacy flink table store
Myasuka commented on PR #665: URL: https://github.com/apache/flink-web/pull/665#issuecomment-1651049361 @JingsongLi would you please take a look？ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (FLINK-32677) flink-benchmarks-regression-check failed to send slack messages since 2023.07.17
[
https://issues.apache.org/jira/browse/FLINK-32677?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17747292#comment-17747292
]
Yanfei Lei commented on FLINK-32677:
[~MartijnVisser] It seems that the Jenkins plugin has been reset since
2023.07.17, could you please help take a look?
> flink-benchmarks-regression-check failed to send slack messages since
> 2023.07.17
>
>
> Key: FLINK-32677
> URL: https://issues.apache.org/jira/browse/FLINK-32677
> Project: Flink
> Issue Type: Bug
> Components: Benchmarks
>Reporter: Yanfei Lei
>Priority: Critical
>
> {code:java}
> Response Code: 404 rel="alternate" type="text/html"
> href="http://codespeed.dak8s.net:8080/log"/>2717802023-07-19T11:38:29Z2023-07-19T11:38:29ZJul
> 19, 2023 11:38:29 AM jenkins.plugins.slack.StandardSlackService publish
> WARNING: Response Code: 404
> Slack post may have failed. Response:
> null href="http://codespeed.dak8s.net:8080/log"/>2717792023-07-19T11:38:29Z2023-07-19T11:38:29ZJul
> 19, 2023 11:38:29 AM jenkins.plugins.slack.StandardSlackService publish
> WARNING: Slack post may have failed. Response: null {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (FLINK-32677) flink-benchmarks-regression-check failed to send slack messages since 2023.07.17
[
https://issues.apache.org/jira/browse/FLINK-32677?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yanfei Lei updated FLINK-32677:
---
Description:
{code:java}
Response Code: 404http://codespeed.dak8s.net:8080/log"/>2717802023-07-19T11:38:29Z2023-07-19T11:38:29ZJul
19, 2023 11:38:29 AM jenkins.plugins.slack.StandardSlackService publish
WARNING: Response Code: 404
Slack post may have failed. Response:
nullhttp://codespeed.dak8s.net:8080/log"/>2717792023-07-19T11:38:29Z2023-07-19T11:38:29ZJul
19, 2023 11:38:29 AM jenkins.plugins.slack.StandardSlackService publish
WARNING: Slack post may have failed. Response: null {code}
> flink-benchmarks-regression-check failed to send slack messages since
> 2023.07.17
>
>
> Key: FLINK-32677
> URL: https://issues.apache.org/jira/browse/FLINK-32677
> Project: Flink
> Issue Type: Bug
> Components: Benchmarks
>Reporter: Yanfei Lei
>Priority: Critical
>
> {code:java}
> Response Code: 404 rel="alternate" type="text/html"
> href="http://codespeed.dak8s.net:8080/log"/>2717802023-07-19T11:38:29Z2023-07-19T11:38:29ZJul
> 19, 2023 11:38:29 AM jenkins.plugins.slack.StandardSlackService publish
> WARNING: Response Code: 404
> Slack post may have failed. Response:
> null href="http://codespeed.dak8s.net:8080/log"/>2717792023-07-19T11:38:29Z2023-07-19T11:38:29ZJul
> 19, 2023 11:38:29 AM jenkins.plugins.slack.StandardSlackService publish
> WARNING: Slack post may have failed. Response: null {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (FLINK-32677) flink-benchmarks-regression-check failed to send slack messages since 2023.07.17
Yanfei Lei created FLINK-32677: -- Summary: flink-benchmarks-regression-check failed to send slack messages since 2023.07.17 Key: FLINK-32677 URL: https://issues.apache.org/jira/browse/FLINK-32677 Project: Flink Issue Type: Bug Components: Benchmarks Reporter: Yanfei Lei -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Closed] (FLINK-32551) Provide the possibility to take a savepoint when deleting a flinkdeployment
[ https://issues.apache.org/jira/browse/FLINK-32551?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gyula Fora closed FLINK-32551. -- Fix Version/s: kubernetes-operator-1.6.0 Resolution: Fixed merged to main 18903878cbeff57fa1ad10ccd66cec3d68de7aa2 > Provide the possibility to take a savepoint when deleting a flinkdeployment > --- > > Key: FLINK-32551 > URL: https://issues.apache.org/jira/browse/FLINK-32551 > Project: Flink > Issue Type: Improvement > Components: Kubernetes Operator >Reporter: Nicolas Fraison >Assignee: Nicolas Fraison >Priority: Major > Labels: pull-request-available > Fix For: kubernetes-operator-1.6.0 > > > Currently if a flinkdeployment is deleted all the HA metadata is removed and > no savepoint is taken. > It would be great (for ex. in case of fat finger) to be able to configure > deployment in order to take a savepoint if the deployment is deleted -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [flink-kubernetes-operator] gyfora merged pull request #634: [FLINK-32551] Add option to take a savepoint when deleting a flinkdeployment/flinksessionjob
gyfora merged PR #634: URL: https://github.com/apache/flink-kubernetes-operator/pull/634 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] snuyanzin commented on pull request #23074: [BP-1.17][FLINK-30629][Client/Job Submission] Increase clientHeartbeatTimeout …
snuyanzin commented on PR #23074: URL: https://github.com/apache/flink/pull/23074#issuecomment-1651026104 @flinkbot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (FLINK-32611) Redirect to Apache Paimon's link instead of legacy flink table store
[ https://issues.apache.org/jira/browse/FLINK-32611?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated FLINK-32611: --- Labels: pull-request-available (was: ) > Redirect to Apache Paimon's link instead of legacy flink table store > > > Key: FLINK-32611 > URL: https://issues.apache.org/jira/browse/FLINK-32611 > Project: Flink > Issue Type: Improvement > Components: Documentation, Project Website >Reporter: Yun Tang >Assignee: Yun Tang >Priority: Major > Labels: pull-request-available > Fix For: 1.18.0 > > > Current Flink's official web site would always point to the legacy flink > table store. However, we should point to the new Apache Paimon website and > docs. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [flink-web] Myasuka opened a new pull request, #665: [FLINK-32611] Redirect to Apache Paimon's link instead of legacy flink table store
Myasuka opened a new pull request, #665: URL: https://github.com/apache/flink-web/pull/665 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (FLINK-32666) ASM rewrite class lead to package failed.
[
https://issues.apache.org/jira/browse/FLINK-32666?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17747279#comment-17747279
]
lizhiqiang commented on FLINK-32666:
thx [~martijnvisser] [~chesnay] reply.
maven I tried 3.2.5 3.6.3 3.8.8 3.9.1 java I upgraded from jdk8_144 to jdk8_372.
jdk8_144 and maven 3.2.5 , 3.6.3 , 3.9.1 has the same problem.
jdk8_372 + maven 3.8.8 / 3.9.1 package succeed. this is a jdk problem.
> ASM rewrite class lead to package failed.
> -
>
> Key: FLINK-32666
> URL: https://issues.apache.org/jira/browse/FLINK-32666
> Project: Flink
> Issue Type: Bug
> Components: Table SQL / Planner
>Affects Versions: 1.18.0
>Reporter: lizhiqiang
>Priority: Major
>
> {code:java}
> [DEBUG] Processing JAR
> /Users/lzq/Desktop/Projects/Flink/flink/flink-master/flink-table/flink-table-planner/target/flink-table-planner_2.12-1.17-SNAPSHOT.jar
> [DEBUG] Rewrote class bytecode:
> org/apache/calcite/interpreter/JaninoRexCompiler.class
> [DEBUG] Rewrote class bytecode:
> org/apache/calcite/tools/RelBuilder$Frame.class
> [DEBUG] Rewrote class bytecode:
> org/apache/calcite/tools/ImmutableRelBuilder$Config.class
> [DEBUG] Rewrote class bytecode: org/apache/calcite/tools/RelBuilder.class
> [DEBUG] Rewrote class bytecode:
> org/apache/calcite/tools/RelBuilder$OverCallImpl.class
> [DEBUG] Rewrote class bytecode:
> org/apache/calcite/tools/RelBuilder$GroupKey.class
> [DEBUG] Keeping original class bytecode:
> org/apache/calcite/tools/RelBuilder$OverCall.class
> [DEBUG] Rewrote class bytecode:
> org/apache/calcite/tools/RelBuilder$RelOptTableFinder.class
> [DEBUG] Rewrote class bytecode:
> org/apache/calcite/tools/ImmutableRelBuilder$Config$Builder.class
> [DEBUG] Rewrote class bytecode:
> org/apache/calcite/tools/RelBuilder$Registrar.class
> [DEBUG] Keeping original class bytecode:
> org/apache/calcite/tools/RelBuilder$OverCallImpl$1.class
> [DEBUG] Rewrote class bytecode:
> org/apache/calcite/tools/RelBuilder$AggCallImpl.class
> [DEBUG] Keeping original class bytecode:
> org/apache/calcite/tools/ImmutableRelBuilder.class
> [DEBUG] Rewrote class bytecode:
> org/apache/calcite/tools/RelBuilder$AggCall.class
> [DEBUG] Keeping original class bytecode:
> org/apache/calcite/tools/ImmutableRelBuilder$1.class
> [DEBUG] Keeping original class bytecode:
> org/apache/calcite/tools/RelBuilder$1.class
> [DEBUG] Rewrote class bytecode:
> org/apache/calcite/tools/RelBuilder$GroupKeyImpl.class
> [DEBUG] Keeping original class bytecode:
> org/apache/calcite/tools/RelBuilder$Config.class
> [DEBUG] Keeping original class bytecode:
> org/apache/calcite/tools/ImmutableRelBuilder$Config$InitShim.class
> [DEBUG] Rewrote class bytecode:
> org/apache/calcite/tools/RelBuilder$AggCallPlus.class
> [DEBUG] Rewrote class bytecode:
> org/apache/calcite/tools/RelBuilder$AggCallImpl2.class
> [DEBUG] Keeping original class bytecode:
> org/apache/calcite/tools/RelBuilder$Shifter.class
> [DEBUG] Rewrote class bytecode:
> org/apache/calcite/tools/RelBuilder$Field.class
> [DEBUG] Keeping original class bytecode:
> org/apache/calcite/tools/RelBuilder$2.class
> [DEBUG] Rewrote class bytecode:
> org/apache/calcite/util/javac/JaninoCompiler$JaninoCompilerArgs.class
> [DEBUG] Rewrote class bytecode:
> org/apache/calcite/util/javac/JaninoCompiler.class
> [DEBUG] Rewrote class bytecode:
> org/apache/calcite/util/javac/JaninoCompiler$AccountingClassLoader.class
> [DEBUG] Rewrote class bytecode:
> org/apache/calcite/adapter/enumerable/EnumerableInterpretable$1$1.class
> [DEBUG] Rewrote class bytecode:
> org/apache/calcite/adapter/enumerable/EnumerableInterpretable$EnumerableNode.class
> [DEBUG] Rewrote class bytecode:
> org/apache/calcite/adapter/enumerable/EnumerableInterpretable$1.class
> [DEBUG] Keeping original class bytecode:
> org/apache/calcite/adapter/enumerable/EnumerableInterpretable$StaticFieldDetector.class
> [DEBUG] Rewrote class bytecode:
> org/apache/calcite/adapter/enumerable/EnumerableInterpretable.class
> [DEBUG] Rewrote class bytecode:
> org/apache/calcite/adapter/enumerable/EnumerableInterpretable$1$1$1.class
> [DEBUG] Keeping original class bytecode:
> org/apache/calcite/jdbc/CalciteSchemaBuilder.class
> [DEBUG] Rewrote class bytecode:
> org/apache/calcite/rex/RexSimplify$SafeRexVisitor.class
> [DEBUG] Rewrote class bytecode:
> org/apache/calcite/rex/RexSimplify$SargCollector.class
> [DEBUG] Rewrote class bytecode:
> org/apache/calcite/rex/RexSimplify$RexSargBuilder.class
> [DEBUG] Rewrote class bytecode:
> org/apache/calcite/rex/RexSimplify$IsPredicate.class
> [DEBUG] Rewrote class bytecode:
> org/apache/calcite/rex/RexSimplify$Predicate.class
> [DEBUG] Rewrote class bytecode:
> org/apache/calcite/rex/RexSimplify$Comparison.class
> [DEBUG] Keeping original class bytecode:
>
[GitHub] [flink] flinkbot commented on pull request #23078: [FLINK-32676][doc] Add doc for catalog modification listener
flinkbot commented on PR #23078: URL: https://github.com/apache/flink/pull/23078#issuecomment-1650945339 ## CI report: * e7334d5e5d953f137655d50504d6c43cd68f2d05 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Assigned] (FLINK-32676) Add doc for catalog modification listener
[ https://issues.apache.org/jira/browse/FLINK-32676?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fang Yong reassigned FLINK-32676: - Assignee: Fang Yong > Add doc for catalog modification listener > - > > Key: FLINK-32676 > URL: https://issues.apache.org/jira/browse/FLINK-32676 > Project: Flink > Issue Type: Improvement > Components: Documentation, Table SQL / Runtime >Affects Versions: 1.18.0 >Reporter: Fang Yong >Assignee: Fang Yong >Priority: Major > Labels: pull-request-available > > Add doc for catalog modification listener -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [flink] Tartarus0zm commented on a diff in pull request #23060: [FLINK-32519][docs] Add doc for [CREATE OR] REPLACE TABLE AS statement
Tartarus0zm commented on code in PR #23060:
URL: https://github.com/apache/flink/pull/23060#discussion_r1274348810
##
docs/content.zh/docs/dev/table/sql/create.md:
##
@@ -557,6 +558,58 @@ INSERT INTO my_ctas_table SELECT id, name, age FROM
source_table WHERE mod(id, 1
{{< top >}}
+## CREATE TABLE [USING]
Review Comment:
I suggest keeping the RTAS syntax section separate and not putting it under
AS select_statement; if we put it under AS select_statement, we would need to
modify the current CREATE TABLE syntax to be compatible with [CREATE OR]
REPLACE, which would not be clear enough and would increase the cost of
understanding for the user.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Updated] (FLINK-32676) Add doc for catalog modification listener
[ https://issues.apache.org/jira/browse/FLINK-32676?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated FLINK-32676: --- Labels: pull-request-available (was: ) > Add doc for catalog modification listener > - > > Key: FLINK-32676 > URL: https://issues.apache.org/jira/browse/FLINK-32676 > Project: Flink > Issue Type: Improvement > Components: Documentation, Table SQL / Runtime >Affects Versions: 1.18.0 >Reporter: Fang Yong >Priority: Major > Labels: pull-request-available > > Add doc for catalog modification listener -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [flink] FangYongs opened a new pull request, #23078: [FLINK-32676][doc] Add doc for catalog modification listener
FangYongs opened a new pull request, #23078: URL: https://github.com/apache/flink/pull/23078 ## What is the purpose of the change Add doc for catalog modification listener ## Brief change log - Add doc for catalog modification listener ## Verifying this change This change is a trivial rework / code cleanup without any test coverage. ## Does this pull request potentially affect one of the following parts: - Dependencies (does it add or upgrade a dependency): (yes / no) no - The public API, i.e., is any changed class annotated with `@Public(Evolving)`: (yes / no) no - The serializers: (yes / no / don't know) no - The runtime per-record code paths (performance sensitive): (yes / no / don't know) no - Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: (yes / no / don't know) no - The S3 file system connector: (yes / no / don't know) no ## Documentation - Does this pull request introduce a new feature? (yes / no) no - If yes, how is the feature documented? (not applicable / docs / JavaDocs / not documented) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot commented on pull request #23077: [FLINK-32675][doc] Add document for the tiered storage of hybrid shuffle
flinkbot commented on PR #23077: URL: https://github.com/apache/flink/pull/23077#issuecomment-1650927484 ## CI report: * 83cab86fbe6d152754f20ebc561ae3412fde69cc UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (FLINK-32675) Add doc for the tiered storage of hybrid shuffle
[ https://issues.apache.org/jira/browse/FLINK-32675?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated FLINK-32675: --- Labels: pull-request-available (was: ) > Add doc for the tiered storage of hybrid shuffle > > > Key: FLINK-32675 > URL: https://issues.apache.org/jira/browse/FLINK-32675 > Project: Flink > Issue Type: Improvement > Components: Documentation, Runtime / Network >Affects Versions: 1.18.0 >Reporter: Yuxin Tan >Assignee: Yuxin Tan >Priority: Major > Labels: pull-request-available > > The new Hybrid Shuffle mode supporting remote storage > (https://issues.apache.org/jira/browse/FLINK-31634) has finished, we should > also update the Flink doc of Hybrid Shuffle. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [flink] TanYuxin-tyx opened a new pull request, #23077: [FLINK-32675][doc] Add document for the tiered storage of hybrid shuffle
TanYuxin-tyx opened a new pull request, #23077: URL: https://github.com/apache/flink/pull/23077 ## What is the purpose of the change *Add document for the tiered storage of hybrid shuffle* ## Brief change log - *Add document for the tiered storage of hybrid shuffle* ## Verifying this change This change is a doc update without any test coverage. ## Does this pull request potentially affect one of the following parts: - Dependencies (does it add or upgrade a dependency): (no) - The public API, i.e., is any changed class annotated with `@Public(Evolving)`: (no) - The serializers: (no) - The runtime per-record code paths (performance sensitive): (no) - Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: (no) - The S3 file system connector: (no) ## Documentation - Does this pull request introduce a new feature? (no) - If yes, how is the feature documented? (docs) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Comment Edited] (FLINK-32668) fix up watchdog timeout error msg in common.sh(e2e test)
[
https://issues.apache.org/jira/browse/FLINK-32668?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17747240#comment-17747240
]
Hongshun Wang edited comment on FLINK-32668 at 7/26/23 3:17 AM:
Thanks for your advise, [~mapohl] .
In fact, it's also hard for me to decide whether to throw an error, which
depends on whether a timeout test case meets our requirements.
* If we think test case needs to meet the required time performance
requirements, we need to throw an error here.
* If we think that even if the timeout is met, it's still ok. The reason why
kills this test is that {*}we just want to execute the next one quickly{*},
then there is no need to throw an error here.A warning is well enough.
What do you think?
was (Author: JIRAUSER298968):
[~mapohl]
In fact, it's also hard for me to decide whether to throw an error, which
depends on whether a timeout test case meets our requirements.
* If we think test case needs to meet the required time performance
requirements, we need to throw an error here.
* If we think that even if the timeout is met, it's still ok. The reason why
kills this test is that {*}we just want to execute the next one quickly{*},
then there is no need to throw an error here.A warning is well enough.
> fix up watchdog timeout error msg in common.sh(e2e test)
> --
>
> Key: FLINK-32668
> URL: https://issues.apache.org/jira/browse/FLINK-32668
> Project: Flink
> Issue Type: Bug
> Components: Build System / CI
>Affects Versions: 1.16.2, 1.18.0, 1.17.1
>Reporter: Hongshun Wang
>Assignee: Hongshun Wang
>Priority: Minor
> Attachments: image-2023-07-25-15-27-37-441.png
>
>
> When run e2e test, an error like this occrurs:
> !image-2023-07-25-15-27-37-441.png|width=733,height=115!
>
> The corresponding code:
> {code:java}
> kill_test_watchdog() {
> local watchdog_pid=$(cat $TEST_DATA_DIR/job_watchdog.pid)
> echo "Stopping job timeout watchdog (with pid=$watchdog_pid)"
> kill $watchdog_pid
> }
> internal_run_with_timeout() {
> local timeout_in_seconds="$1"
> local on_failure="$2"
> local command_label="$3"
> local command="${@:4}"
> on_exit kill_test_watchdog
> (
> command_pid=$BASHPID
> (sleep "${timeout_in_seconds}" # set a timeout for this command
> echo "${command_label:-"The command '${command}'"} (pid:
> $command_pid) did not finish after $timeout_in_seconds seconds."
> eval "${on_failure}"
> kill "$command_pid") & watchdog_pid=$!
> echo $watchdog_pid > $TEST_DATA_DIR/job_watchdog.pid
> # invoke
> $command
> )
> }{code}
>
> When {{$command}} completes before the timeout, the watchdog process is
> killed successfully. However, when {{$command}} times out, the watchdog
> process kills {{$command}} and then exits itself, leaving behind an error
> message when trying to kill its own process ID with {{{}kill
> $watchdog_pid{}}}.This error msg "no such process" is hard to understand.
>
> So, I will modify like this with better error message:
>
> {code:java}
> kill_test_watchdog() {
> local watchdog_pid=$(cat $TEST_DATA_DIR/job_watchdog.pid)
> if kill -0 $watchdog_pid > /dev/null 2>&1; then
> echo "Stopping job timeout watchdog (with pid=$watchdog_pid)"
> kill $watchdog_pid
> else
> echo "[ERROR] Test is timeout"
> exit 1
> fi
> } {code}
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Comment Edited] (FLINK-32668) fix up watchdog timeout error msg in common.sh(e2e test)
[
https://issues.apache.org/jira/browse/FLINK-32668?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17747240#comment-17747240
]
Hongshun Wang edited comment on FLINK-32668 at 7/26/23 3:15 AM:
[~mapohl]
In fact, it's also hard for me to decide whether to throw an error, which
depends on whether a timeout test case meets our requirements.
* If we think test case needs to meet the required time performance
requirements, we need to throw an error here.
* If we think that even if the timeout is met, it's still ok. The reason why
kills this test is that {*}we just want to execute the next one quickly{*},
then there is no need to throw an error here.A warning is well enough.
was (Author: JIRAUSER298968):
[~mapohl]
In fact, it's also hard for me to decide whether to throw an error, which
depends on whether a timeout test case meets our requirements.
* If we think test case needs to meet the required time performance
requirements, we need to throw an error here.
* If we think that even if the timeout is met, it's still ok. The reason why
kills this test is that {*}we just want to execute the next one quickly{*},
then there is no need to throw an error here.
> fix up watchdog timeout error msg in common.sh(e2e test)
> --
>
> Key: FLINK-32668
> URL: https://issues.apache.org/jira/browse/FLINK-32668
> Project: Flink
> Issue Type: Bug
> Components: Build System / CI
>Affects Versions: 1.16.2, 1.18.0, 1.17.1
>Reporter: Hongshun Wang
>Assignee: Hongshun Wang
>Priority: Minor
> Attachments: image-2023-07-25-15-27-37-441.png
>
>
> When run e2e test, an error like this occrurs:
> !image-2023-07-25-15-27-37-441.png|width=733,height=115!
>
> The corresponding code:
> {code:java}
> kill_test_watchdog() {
> local watchdog_pid=$(cat $TEST_DATA_DIR/job_watchdog.pid)
> echo "Stopping job timeout watchdog (with pid=$watchdog_pid)"
> kill $watchdog_pid
> }
> internal_run_with_timeout() {
> local timeout_in_seconds="$1"
> local on_failure="$2"
> local command_label="$3"
> local command="${@:4}"
> on_exit kill_test_watchdog
> (
> command_pid=$BASHPID
> (sleep "${timeout_in_seconds}" # set a timeout for this command
> echo "${command_label:-"The command '${command}'"} (pid:
> $command_pid) did not finish after $timeout_in_seconds seconds."
> eval "${on_failure}"
> kill "$command_pid") & watchdog_pid=$!
> echo $watchdog_pid > $TEST_DATA_DIR/job_watchdog.pid
> # invoke
> $command
> )
> }{code}
>
> When {{$command}} completes before the timeout, the watchdog process is
> killed successfully. However, when {{$command}} times out, the watchdog
> process kills {{$command}} and then exits itself, leaving behind an error
> message when trying to kill its own process ID with {{{}kill
> $watchdog_pid{}}}.This error msg "no such process" is hard to understand.
>
> So, I will modify like this with better error message:
>
> {code:java}
> kill_test_watchdog() {
> local watchdog_pid=$(cat $TEST_DATA_DIR/job_watchdog.pid)
> if kill -0 $watchdog_pid > /dev/null 2>&1; then
> echo "Stopping job timeout watchdog (with pid=$watchdog_pid)"
> kill $watchdog_pid
> else
> echo "[ERROR] Test is timeout"
> exit 1
> fi
> } {code}
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (FLINK-32668) fix up watchdog timeout error msg in common.sh(e2e test)
[
https://issues.apache.org/jira/browse/FLINK-32668?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17747240#comment-17747240
]
Hongshun Wang commented on FLINK-32668:
---
[~mapohl]
In fact, it's also hard for me to decide whether to throw an error, which
depends on whether a timeout test case meets our requirements.
* If we think test case needs to meet the required time performance
requirements, we need to throw an error here.
* If we think that even if the timeout is met, it's still ok. The reason why
kills this test is that {*}we just want to execute the next one quickly{*},
then there is no need to throw an error here.
> fix up watchdog timeout error msg in common.sh(e2e test)
> --
>
> Key: FLINK-32668
> URL: https://issues.apache.org/jira/browse/FLINK-32668
> Project: Flink
> Issue Type: Bug
> Components: Build System / CI
>Affects Versions: 1.16.2, 1.18.0, 1.17.1
>Reporter: Hongshun Wang
>Assignee: Hongshun Wang
>Priority: Minor
> Attachments: image-2023-07-25-15-27-37-441.png
>
>
> When run e2e test, an error like this occrurs:
> !image-2023-07-25-15-27-37-441.png|width=733,height=115!
>
> The corresponding code:
> {code:java}
> kill_test_watchdog() {
> local watchdog_pid=$(cat $TEST_DATA_DIR/job_watchdog.pid)
> echo "Stopping job timeout watchdog (with pid=$watchdog_pid)"
> kill $watchdog_pid
> }
> internal_run_with_timeout() {
> local timeout_in_seconds="$1"
> local on_failure="$2"
> local command_label="$3"
> local command="${@:4}"
> on_exit kill_test_watchdog
> (
> command_pid=$BASHPID
> (sleep "${timeout_in_seconds}" # set a timeout for this command
> echo "${command_label:-"The command '${command}'"} (pid:
> $command_pid) did not finish after $timeout_in_seconds seconds."
> eval "${on_failure}"
> kill "$command_pid") & watchdog_pid=$!
> echo $watchdog_pid > $TEST_DATA_DIR/job_watchdog.pid
> # invoke
> $command
> )
> }{code}
>
> When {{$command}} completes before the timeout, the watchdog process is
> killed successfully. However, when {{$command}} times out, the watchdog
> process kills {{$command}} and then exits itself, leaving behind an error
> message when trying to kill its own process ID with {{{}kill
> $watchdog_pid{}}}.This error msg "no such process" is hard to understand.
>
> So, I will modify like this with better error message:
>
> {code:java}
> kill_test_watchdog() {
> local watchdog_pid=$(cat $TEST_DATA_DIR/job_watchdog.pid)
> if kill -0 $watchdog_pid > /dev/null 2>&1; then
> echo "Stopping job timeout watchdog (with pid=$watchdog_pid)"
> kill $watchdog_pid
> else
> echo "[ERROR] Test is timeout"
> exit 1
> fi
> } {code}
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[GitHub] [flink] Tartarus0zm commented on a diff in pull request #23060: [FLINK-32519][docs] Add doc for [CREATE OR] REPLACE TABLE AS statement
Tartarus0zm commented on code in PR #23060:
URL: https://github.com/apache/flink/pull/23060#discussion_r1274313857
##
docs/content.zh/docs/dev/table/sql/create.md:
##
@@ -33,6 +33,7 @@ CREATE 语句用于向当前或指定的 [Catalog]({{< ref "docs/dev/table/catal
目前 Flink SQL 支持下列 CREATE 语句：
- CREATE TABLE
+- CREATE TABLE [USING]
Review Comment:
`CREATE TABLE [USING]` is a reference to the[ databricks spark
docs](https://docs.databricks.com/sql/language-manual/sql-ref-syntax-ddl-create-table-using.html),
I'll adjust it to `[CREATE OR] REPLACE TABLE` to be more intuitive
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Commented] (FLINK-27054) Elasticsearch SQL connector SSL issue
[ https://issues.apache.org/jira/browse/FLINK-27054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17747239#comment-17747239 ] Kelu Tao commented on FLINK-27054: -- Hi, [~martijnvisser] and [~alexanderpreuss] , for this issue, do we have any plan to support it. I think we can define a option in connector, such as properties.ssl.path.xxx, to find the ssl certificate path to construct the es client. Do you have any concern for this proposal? Thanks. > Elasticsearch SQL connector SSL issue > - > > Key: FLINK-27054 > URL: https://issues.apache.org/jira/browse/FLINK-27054 > Project: Flink > Issue Type: Bug > Components: Connectors / ElasticSearch >Reporter: ricardo >Priority: Major > > The current Flink ElasticSearch SQL connector > https://nightlies.apache.org/flink/flink-docs-master/docs/connectors/table/elasticsearch/ > is missing SSL options, can't connect to ES clusters which require SSL > certificate. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (FLINK-31634) FLIP-301: Hybrid Shuffle supports Remote Storage
[ https://issues.apache.org/jira/browse/FLINK-31634?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17747238#comment-17747238 ] Yuxin Tan commented on FLINK-31634: --- [~knaufk] Thanks for helping check the doc. I have created a task https://issues.apache.org/jira/browse/FLINK-32675 , and I will do it ASAP. > FLIP-301: Hybrid Shuffle supports Remote Storage > > > Key: FLINK-31634 > URL: https://issues.apache.org/jira/browse/FLINK-31634 > Project: Flink > Issue Type: New Feature > Components: Runtime / Network >Affects Versions: 1.18.0 >Reporter: Yuxin Tan >Assignee: Yuxin Tan >Priority: Major > Labels: Umbrella > > This is an umbrella ticket for > [FLIP-301|https://cwiki.apache.org/confluence/display/FLINK/FLIP-301%3A+Hybrid+Shuffle+supports+Remote+Storage]. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (FLINK-32675) Add doc for the tiered storage of hybrid shuffle
[ https://issues.apache.org/jira/browse/FLINK-32675?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuxin Tan updated FLINK-32675: -- Description: The new Hybrid Shuffle mode supporting remote storage (https://issues.apache.org/jira/browse/FLINK-31634) has finished, we should also update the Flink doc of Hybrid Shuffle. (was: The new Hybrid Shuffle mode supporting remote storage (https://issues.apache.org/jira/browse/FLINK-30469) has finished, we should also update the Flink doc of Hybrid Shuffle.) > Add doc for the tiered storage of hybrid shuffle > > > Key: FLINK-32675 > URL: https://issues.apache.org/jira/browse/FLINK-32675 > Project: Flink > Issue Type: Improvement > Components: Documentation, Runtime / Network >Affects Versions: 1.18.0 >Reporter: Yuxin Tan >Assignee: Yuxin Tan >Priority: Major > > The new Hybrid Shuffle mode supporting remote storage > (https://issues.apache.org/jira/browse/FLINK-31634) has finished, we should > also update the Flink doc of Hybrid Shuffle. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (FLINK-32676) Add doc for catalog modification listener
Fang Yong created FLINK-32676: - Summary: Add doc for catalog modification listener Key: FLINK-32676 URL: https://issues.apache.org/jira/browse/FLINK-32676 Project: Flink Issue Type: Improvement Components: Documentation, Table SQL / Runtime Affects Versions: 1.18.0 Reporter: Fang Yong Add doc for catalog modification listener -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (FLINK-32402) FLIP-294: Support Customized Catalog Modification Listener
[ https://issues.apache.org/jira/browse/FLINK-32402?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17747235#comment-17747235 ] Fang Yong commented on FLINK-32402: --- Thanks [~knaufk], I'll create an issue to add doc for this feature > FLIP-294: Support Customized Catalog Modification Listener > -- > > Key: FLINK-32402 > URL: https://issues.apache.org/jira/browse/FLINK-32402 > Project: Flink > Issue Type: Improvement > Components: Table SQL / Ecosystem >Affects Versions: 1.18.0 >Reporter: Fang Yong >Assignee: Fang Yong >Priority: Major > > Issue for > https://cwiki.apache.org/confluence/display/FLINK/FLIP-294%3A+Support+Customized+Catalog+Modification+Listener -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Comment Edited] (FLINK-32655) RecreateOnResetOperatorCoordinator did not forward notifyCheckpointAborted to the real OperatorCoordinator
[
https://issues.apache.org/jira/browse/FLINK-32655?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17746871#comment-17746871
]
Rui Fan edited comment on FLINK-32655 at 7/26/23 2:07 AM:
--
Thanks [~Ming Li] for the fix, merged via:
e8a8b05b18f5bf5362d88a548e5c861097f7fa8f
1.17 8a9c943661c227ee58b2d744dee2c92f0f61a51b
1.16 849523ed2f76865e6caf106db76f3a6a4910e3ac
was (Author: fanrui):
Merged via:
e8a8b05b18f5bf5362d88a548e5c861097f7fa8f
1.17 8a9c943661c227ee58b2d744dee2c92f0f61a51b
1.16 849523ed2f76865e6caf106db76f3a6a4910e3ac
> RecreateOnResetOperatorCoordinator did not forward notifyCheckpointAborted to
> the real OperatorCoordinator
> --
>
> Key: FLINK-32655
> URL: https://issues.apache.org/jira/browse/FLINK-32655
> Project: Flink
> Issue Type: Bug
> Components: Runtime / Checkpointing
>Affects Versions: 1.17.1
>Reporter: Ming Li
>Assignee: Ming Li
>Priority: Major
> Labels: pull-request-available
>
> {{[RecreateOnResetOperatorCoordinator|https://github.com/apache/flink/blob/master/flink-runtime/src/main/java/org/apache/flink/runtime/operators/coordination/RecreateOnResetOperatorCoordinator.java#L115]}}
> does not override {{{}notifyCheckpointAborted{}}}, which causes the
> {{SplitEnumerator}} in {{SourceCoordinator}} can not receive the checkpoint
> abort message.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Resolved] (FLINK-32655) RecreateOnResetOperatorCoordinator did not forward notifyCheckpointAborted to the real OperatorCoordinator
[
https://issues.apache.org/jira/browse/FLINK-32655?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rui Fan resolved FLINK-32655.
-
Fix Version/s: 1.18.0
1.16.3
1.17.2
Resolution: Fixed
> RecreateOnResetOperatorCoordinator did not forward notifyCheckpointAborted to
> the real OperatorCoordinator
> --
>
> Key: FLINK-32655
> URL: https://issues.apache.org/jira/browse/FLINK-32655
> Project: Flink
> Issue Type: Bug
> Components: Runtime / Checkpointing
>Affects Versions: 1.17.1
>Reporter: Ming Li
>Assignee: Ming Li
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.18.0, 1.16.3, 1.17.2
>
>
> {{[RecreateOnResetOperatorCoordinator|https://github.com/apache/flink/blob/master/flink-runtime/src/main/java/org/apache/flink/runtime/operators/coordination/RecreateOnResetOperatorCoordinator.java#L115]}}
> does not override {{{}notifyCheckpointAborted{}}}, which causes the
> {{SplitEnumerator}} in {{SourceCoordinator}} can not receive the checkpoint
> abort message.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Comment Edited] (FLINK-32655) RecreateOnResetOperatorCoordinator did not forward notifyCheckpointAborted to the real OperatorCoordinator
[
https://issues.apache.org/jira/browse/FLINK-32655?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17746871#comment-17746871
]
Rui Fan edited comment on FLINK-32655 at 7/26/23 2:07 AM:
--
Merged via:
e8a8b05b18f5bf5362d88a548e5c861097f7fa8f
1.17 8a9c943661c227ee58b2d744dee2c92f0f61a51b
1.16 849523ed2f76865e6caf106db76f3a6a4910e3ac
was (Author: fanrui):
Merged via:
e8a8b05b18f5bf5362d88a548e5c861097f7fa8f
1.17
1.16
> RecreateOnResetOperatorCoordinator did not forward notifyCheckpointAborted to
> the real OperatorCoordinator
> --
>
> Key: FLINK-32655
> URL: https://issues.apache.org/jira/browse/FLINK-32655
> Project: Flink
> Issue Type: Bug
> Components: Runtime / Checkpointing
>Affects Versions: 1.17.1
>Reporter: Ming Li
>Assignee: Ming Li
>Priority: Major
> Labels: pull-request-available
>
> {{[RecreateOnResetOperatorCoordinator|https://github.com/apache/flink/blob/master/flink-runtime/src/main/java/org/apache/flink/runtime/operators/coordination/RecreateOnResetOperatorCoordinator.java#L115]}}
> does not override {{{}notifyCheckpointAborted{}}}, which causes the
> {{SplitEnumerator}} in {{SourceCoordinator}} can not receive the checkpoint
> abort message.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[GitHub] [flink] 1996fanrui merged pull request #23071: [FLINK-32655][runtime] Fix checkpoint aborted message being swallowed by RecreateOnResetOperatorCoordinator.
1996fanrui merged PR #23071: URL: https://github.com/apache/flink/pull/23071 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] 1996fanrui merged pull request #23070: [FLINK-32655][runtime] Fix checkpoint aborted message being swallowed by RecreateOnResetOperatorCoordinator.
1996fanrui merged PR #23070: URL: https://github.com/apache/flink/pull/23070 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot commented on pull request #23076: [FLINK-32460][doc] Add doc for show procedures statement
flinkbot commented on PR #23076: URL: https://github.com/apache/flink/pull/23076#issuecomment-1650855772 ## CI report: * 7bef28d54b12f28b3cfeaeb08e2a9afd3d4d3afc UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot commented on pull request #23075: [FLINK-32674][doc] Add documentation for the new getTargetColumns in DynamicTableSink
flinkbot commented on PR #23075: URL: https://github.com/apache/flink/pull/23075#issuecomment-1650851240 ## CI report: * 806494851d14980c87aa484d2e96fb1429584e79 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (FLINK-32460) Add doc for list procedures
[ https://issues.apache.org/jira/browse/FLINK-32460?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated FLINK-32460: --- Labels: pull-request-available (was: ) > Add doc for list procedures > --- > > Key: FLINK-32460 > URL: https://issues.apache.org/jira/browse/FLINK-32460 > Project: Flink > Issue Type: Sub-task > Components: Documentation >Reporter: luoyuxia >Priority: Major > Labels: pull-request-available > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [flink] luoyuxia opened a new pull request, #23076: [FLINK-32460][doc] Add doc for show procedures statement
luoyuxia opened a new pull request, #23076: URL: https://github.com/apache/flink/pull/23076 ## What is the purpose of the change Add doc for show procedures statement. ## Brief change log Add doc for show procedures statement. ## Verifying this change ## Does this pull request potentially affect one of the following parts: - Dependencies (does it add or upgrade a dependency): (yes / **no**) - The public API, i.e., is any changed class annotated with `@Public(Evolving)`: (yes / **no**) - The serializers: (yes / **no** / don't know) - The runtime per-record code paths (performance sensitive): (yes / **no** / don't know) - Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: (yes / **no** / don't know) - The S3 file system connector: (yes / **no** / don't know) ## Documentation - Does this pull request introduce a new feature? (yes / **no**) - If yes, how is the feature documented? (not applicable / **docs** / JavaDocs / not documented) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (FLINK-32674) Add documentation for the new Context.getTargetColumns
[ https://issues.apache.org/jira/browse/FLINK-32674?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated FLINK-32674: --- Labels: pull-request-available (was: ) > Add documentation for the new Context.getTargetColumns > -- > > Key: FLINK-32674 > URL: https://issues.apache.org/jira/browse/FLINK-32674 > Project: Flink > Issue Type: Sub-task > Components: Documentation >Affects Versions: 1.18.0 >Reporter: lincoln lee >Assignee: lincoln lee >Priority: Major > Labels: pull-request-available > Fix For: 1.18.0 > > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (FLINK-32675) Add doc for the tiered storage of hybrid shuffle
Yuxin Tan created FLINK-32675: - Summary: Add doc for the tiered storage of hybrid shuffle Key: FLINK-32675 URL: https://issues.apache.org/jira/browse/FLINK-32675 Project: Flink Issue Type: Improvement Components: Documentation, Runtime / Network Affects Versions: 1.18.0 Reporter: Yuxin Tan Assignee: Yuxin Tan The new Hybrid Shuffle mode supporting remote storage (https://issues.apache.org/jira/browse/FLINK-30469) has finished, we should also update the Flink doc of Hybrid Shuffle. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (FLINK-31487) Add targetColumns to DynamicTableSink#Context
[ https://issues.apache.org/jira/browse/FLINK-31487?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17747225#comment-17747225 ] lincoln lee commented on FLINK-31487: - [~knaufk] Yes, only the javadoc is currently available. It may be beneficial for connector developers to add appropriate notes to the current documentation. I've created a subtask and updated the doc. > Add targetColumns to DynamicTableSink#Context > - > > Key: FLINK-31487 > URL: https://issues.apache.org/jira/browse/FLINK-31487 > Project: Flink > Issue Type: Improvement > Components: Table SQL / API >Reporter: lincoln lee >Assignee: lincoln lee >Priority: Major > Labels: pull-request-available > Fix For: 1.18.0 > > > FLIP-300: Add targetColumns to DynamicTableSink#Context to solve the null > overwrite problem of partial-insert > https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=240885081 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [flink] RanJinh commented on pull request #23000: [FLINK-32594][runtime] Use blocking ResultPartitionType if operator only outputs records on EOF
RanJinh commented on PR #23000: URL: https://github.com/apache/flink/pull/23000#issuecomment-1650824543 @flinkbot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (FLINK-32674) Add documentation for the new Context.getTargetColumns
lincoln lee created FLINK-32674: --- Summary: Add documentation for the new Context.getTargetColumns Key: FLINK-32674 URL: https://issues.apache.org/jira/browse/FLINK-32674 Project: Flink Issue Type: Sub-task Components: Documentation Affects Versions: 1.18.0 Reporter: lincoln lee Assignee: lincoln lee Fix For: 1.18.0 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Closed] (FLINK-32468) Replace Akka by Pekko
[ https://issues.apache.org/jira/browse/FLINK-32468?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chesnay Schepler closed FLINK-32468. Resolution: Fixed master: c8ae39d4ac73f81873e1d8ac37e17c29ae330b23 > Replace Akka by Pekko > - > > Key: FLINK-32468 > URL: https://issues.apache.org/jira/browse/FLINK-32468 > Project: Flink > Issue Type: Improvement > Components: Runtime / Coordination >Reporter: Konstantin Knauf >Assignee: Chesnay Schepler >Priority: Critical > Labels: pull-request-available > Fix For: 1.18.0 > > > Akka 2.6.x will not receive security fixes from September 2023 onwards (see > https://discuss.lightbend.com/t/2-6-x-maintenance-proposal/9949). > A mid-term plan to replace Akka is described in FLINK-29281. In the meantime, > we suggest to replace Akka by Apache Pekko (incubating), which is a fork of > Akka 2.6.x under the Apache 2.0 license. This way - if needed - we at least > have the ability to release security fixes ourselves in collaboration with > the Pekko community. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [flink-jira-bot] dependabot[bot] closed pull request #25: Bump certifi from 2020.12.5 to 2022.12.7
dependabot[bot] closed pull request #25: Bump certifi from 2020.12.5 to 2022.12.7 URL: https://github.com/apache/flink-jira-bot/pull/25 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink-jira-bot] dependabot[bot] opened a new pull request, #27: Bump certifi from 2020.12.5 to 2023.7.22
dependabot[bot] opened a new pull request, #27: URL: https://github.com/apache/flink-jira-bot/pull/27 Bumps [certifi](https://github.com/certifi/python-certifi) from 2020.12.5 to 2023.7.22. Commits https://github.com/certifi/python-certifi/commit/8fb96ed81f71e7097ed11bc4d9b19afd7ea5c909";>8fb96ed 2023.07.22 https://github.com/certifi/python-certifi/commit/afe77220e0eaa722593fc5d294213ff5275d1b40";>afe7722 Bump actions/setup-python from 4.6.1 to 4.7.0 (https://redirect.github.com/certifi/python-certifi/issues/230";>#230) https://github.com/certifi/python-certifi/commit/2038739ad56abec7aaddfa90ad2ce6b3ed7f5c7b";>2038739 Bump dessant/lock-threads from 3.0.0 to 4.0.1 (https://redirect.github.com/certifi/python-certifi/issues/229";>#229) https://github.com/certifi/python-certifi/commit/44df761f4c09d19f32b3cc09208a739043a5e25b";>44df761 Hash pin Actions and enable dependabot (https://redirect.github.com/certifi/python-certifi/issues/228";>#228) https://github.com/certifi/python-certifi/commit/8b3d7bae85bbc87c9181cc1d39548db3d31627f0";>8b3d7ba 2023.05.07 https://github.com/certifi/python-certifi/commit/53da2405b1af430f6bafa21ba45d8dd8dfc726b8";>53da240 ci: Add Python 3.12-dev to the testing (https://redirect.github.com/certifi/python-certifi/issues/224";>#224) https://github.com/certifi/python-certifi/commit/c2fc3b1f64d6946f1057971ee897ea828ae848d8";>c2fc3b1 Create a Security Policy (https://redirect.github.com/certifi/python-certifi/issues/222";>#222) https://github.com/certifi/python-certifi/commit/c211ef482a01aff5f1bc92c4128bfa0c955f4a01";>c211ef4 Set up permissions to github workflows (https://redirect.github.com/certifi/python-certifi/issues/218";>#218) https://github.com/certifi/python-certifi/commit/2087de5d0aa1d472145fc1dbdfece3fe652bbac5";>2087de5 Don't let deprecation warning fail CI (https://redirect.github.com/certifi/python-certifi/issues/219";>#219) https://github.com/certifi/python-certifi/commit/e0b9fc5c8f52ac8c300da502e5760ce3d41429ec";>e0b9fc5 remove paragraphs about 1024-bit roots from README Additional commits viewable in https://github.com/certifi/python-certifi/compare/2020.12.05...2023.07.22";>compare view [](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- Dependabot commands and options You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) You can disable automated security fix PRs for this repo from the [Security Alerts page](https://github.com/apache/flink-jira-bot/network/alerts). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] zentol commented on pull request #22996: [FLINK-32468][rpc] Switch from Akka to Pekko
zentol commented on PR #22996: URL: https://github.com/apache/flink/pull/22996#issuecomment-1650191304 We can still bump it in the following days if needed. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Closed] (FLINK-32168) Log required/available resources in RM
[ https://issues.apache.org/jira/browse/FLINK-32168?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chesnay Schepler closed FLINK-32168. Resolution: Fixed master: dc08df6132921dec0083ebf5a54a66f8447aa2c8 > Log required/available resources in RM > -- > > Key: FLINK-32168 > URL: https://issues.apache.org/jira/browse/FLINK-32168 > Project: Flink > Issue Type: Sub-task > Components: Runtime / Coordination >Reporter: Chesnay Schepler >Assignee: Chesnay Schepler >Priority: Major > Labels: pull-request-available > Fix For: 1.18.0 > > > When matching requirements against available resource the RM currently > doesn't log anything apart from whether it could fulfill the resources or not. > We can make the system easier to audit by logging the current requirements, > available resources, and how many resources are left after the matching. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Closed] (FLINK-32165) Improve observability of fine-grained resource management
[ https://issues.apache.org/jira/browse/FLINK-32165?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chesnay Schepler closed FLINK-32165. Resolution: Fixed > Improve observability of fine-grained resource management > - > > Key: FLINK-32165 > URL: https://issues.apache.org/jira/browse/FLINK-32165 > Project: Flink > Issue Type: Improvement > Components: Runtime / Coordination, Runtime / Web Frontend >Reporter: Chesnay Schepler >Assignee: Chesnay Schepler >Priority: Major > Fix For: 1.18.0 > > > Right now fine-grained resource management is way too much of a black-box, > with the only source of information being the taskmanager rest endpoints. > While this is fine-ish for services built around it the developer experience > is suffering greatly and it becomes impossible to reason about the system > afterwards (because we don't even log anything). -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [flink] zentol merged pull request #23064: [FLINK-32168] RM logs missing and current resources
zentol merged PR #23064: URL: https://github.com/apache/flink/pull/23064 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (FLINK-32673) Migrage Google PubSub connector to V2
[ https://issues.apache.org/jira/browse/FLINK-32673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17747062#comment-17747062 ] Ryan Skraba commented on FLINK-32673: - Hello! There's been some work on this: https://github.com/apache/flink-connector-gcp-pubsub/pull/2 (adapted from previous work on FLINK-20625), but it's stalled due to lack of reviewers with Pub/Sub expertise :/ Any ideas? > Migrage Google PubSub connector to V2 > - > > Key: FLINK-32673 > URL: https://issues.apache.org/jira/browse/FLINK-32673 > Project: Flink > Issue Type: Sub-task >Reporter: Alexander Fedulov >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [flink-kubernetes-operator] afedulov commented on a diff in pull request #637: [FLINK-29634] Support periodic checkpoint triggering
afedulov commented on code in PR #637:
URL:
https://github.com/apache/flink-kubernetes-operator/pull/637#discussion_r1273800142
##
flink-kubernetes-operator/src/test/java/org/apache/flink/kubernetes/operator/reconciler/sessionjob/SessionJobReconcilerTest.java:
##
@@ -409,24 +417,143 @@ public void testTriggerSavepoint() throws Exception {
sp1SessionJob, JobState.RUNNING, RECONCILING.name(), null,
flinkService.listJobs());
sp1SessionJob.getStatus().getJobStatus().getSavepointInfo().resetTrigger();
-ReconciliationUtils.updateLastReconciledSavepointTriggerNonce(
-sp1SessionJob.getStatus().getJobStatus().getSavepointInfo(),
sp1SessionJob);
+ReconciliationUtils.updateLastReconciledSnapshotTriggerNonce(
+sp1SessionJob.getStatus().getJobStatus().getSavepointInfo(),
+sp1SessionJob,
+SAVEPOINT);
// trigger when new nonce is defined
sp1SessionJob.getSpec().getJob().setSavepointTriggerNonce(4L);
reconciler.reconcile(sp1SessionJob, readyContext);
assertEquals(
-"trigger_1",
+"savepoint_trigger_1",
sp1SessionJob.getStatus().getJobStatus().getSavepointInfo().getTriggerId());
sp1SessionJob.getStatus().getJobStatus().getSavepointInfo().resetTrigger();
-ReconciliationUtils.updateLastReconciledSavepointTriggerNonce(
-sp1SessionJob.getStatus().getJobStatus().getSavepointInfo(),
sp1SessionJob);
+ReconciliationUtils.updateLastReconciledSnapshotTriggerNonce(
+sp1SessionJob.getStatus().getJobStatus().getSavepointInfo(),
+sp1SessionJob,
+SAVEPOINT);
// don't trigger when nonce is cleared
sp1SessionJob.getSpec().getJob().setSavepointTriggerNonce(null);
reconciler.reconcile(sp1SessionJob, readyContext);
-
assertFalse(SavepointUtils.savepointInProgress(sp1SessionJob.getStatus().getJobStatus()));
+
assertFalse(SnapshotUtils.savepointInProgress(sp1SessionJob.getStatus().getJobStatus()));
+}
+
+@Test
+public void testTriggerCheckpoint() throws Exception {
+FlinkSessionJob sessionJob = TestUtils.buildSessionJob();
+
assertFalse(SnapshotUtils.checkpointInProgress(getJobStatus(sessionJob)));
+
+var readyContext = TestUtils.createContextWithReadyFlinkDeployment();
+reconciler.reconcile(sessionJob, readyContext);
+verifyAndSetRunningJobsToStatus(
+sessionJob, JobState.RUNNING, RECONCILING.name(), null,
flinkService.listJobs());
+
+
assertFalse(SnapshotUtils.checkpointInProgress(getJobStatus(sessionJob)));
+
+// trigger checkpoint
+var sp1SessionJob = ReconciliationUtils.clone(sessionJob);
+
+// do not trigger checkpoint if nonce is null
+reconciler.reconcile(sp1SessionJob, readyContext);
+
assertFalse(SnapshotUtils.checkpointInProgress(getJobStatus(sp1SessionJob)));
+
+getJobSpec(sp1SessionJob).setCheckpointTriggerNonce(2L);
+getJobStatus(sp1SessionJob).setState(CREATED.name());
+reconciler.reconcile(sp1SessionJob, readyContext);
+// do not trigger checkpoint if job is not running
+
assertFalse(SnapshotUtils.checkpointInProgress(getJobStatus(sp1SessionJob)));
+
+getJobStatus(sp1SessionJob).setState(RUNNING.name());
+
+reconciler.reconcile(sp1SessionJob, readyContext);
+
assertTrue(SnapshotUtils.checkpointInProgress(getJobStatus(sp1SessionJob)));
+
+// the last reconcile nonce updated
+
assertNull(getReconciledJobSpec(sp1SessionJob).getCheckpointTriggerNonce());
+
+// don't trigger new checkpoint when checkpoint is in progress
+getJobSpec(sp1SessionJob).setCheckpointTriggerNonce(3L);
+reconciler.reconcile(sp1SessionJob, readyContext);
+assertEquals("checkpoint_trigger_0",
getCheckpointInfo(sp1SessionJob).getTriggerId());
+/*
+TODO: this section needs to be reintroduced in case the LAST_STATE
optimization gets
+ added
+
+// don't trigger upgrade when checkpoint is in progress
+assertEquals(
+1,
+sp1SessionJob
+.getStatus()
+.getReconciliationStatus()
+.deserializeLastReconciledSpec()
+.getJob()
+.getParallelism());
+getJobSpec(sp1SessionJob).setParallelism(100);
+reconciler.reconcile(sp1SessionJob, readyContext);
+assertEquals(
+"checkpoint_trigger_0",
+getCheckpointInfo(sp1SessionJob).getTriggerId());
+assertEquals(
+SnapshotTriggerType.MANUAL,
+getCheckpointInfo(sp1SessionJob).getTriggerType());
+
+
[GitHub] [flink-kubernetes-operator] afedulov commented on a diff in pull request #637: [FLINK-29634] Support periodic checkpoint triggering
afedulov commented on code in PR #637:
URL:
https://github.com/apache/flink-kubernetes-operator/pull/637#discussion_r1273795909
##
flink-kubernetes-operator/src/main/java/org/apache/flink/kubernetes/operator/config/KubernetesOperatorConfigOptions.java:
##
@@ -369,6 +395,13 @@ public static String operatorConfigKey(String key) {
.withDescription(
"Type of the binary format in which a savepoint
should be taken.");
+@Documentation.Section(SECTION_DYNAMIC)
+public static final ConfigOption OPERATOR_CHECKPOINT_TYPE =
+operatorConfig("checkpoint.type")
+.enumType(CheckpointType.class)
+.defaultValue(CheckpointType.FULL)
+.withDescription("Type of checkpoint.");
Review Comment:
Added automatically-generated description to the docs.
##
flink-kubernetes-operator-api/src/main/java/org/apache/flink/kubernetes/operator/api/status/Snapshot.java:
##
@@ -0,0 +1,21 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.kubernetes.operator.api.status;
+
+/** A common interface for {@link Checkpoint} and {@link Savepoint}. */
+public interface Snapshot {}
Review Comment:
Added getTimesStamp and getTriggerType .
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [flink-kubernetes-operator] afedulov commented on a diff in pull request #637: [FLINK-29634] Support periodic checkpoint triggering
afedulov commented on code in PR #637:
URL:
https://github.com/apache/flink-kubernetes-operator/pull/637#discussion_r1273795126
##
flink-kubernetes-operator/src/main/java/org/apache/flink/kubernetes/operator/observer/SnapshotObserver.java:
##
@@ -74,8 +80,32 @@ public void observeSavepointStatus(FlinkResourceContext
ctx) {
cleanupSavepointHistory(ctx, savepointInfo);
}
+public void observeCheckpointStatus(FlinkResourceContext ctx) {
+if (!isCheckpointsTriggeringSupported(ctx.getObserveConfig())) {
+return;
+}
+var resource = ctx.getResource();
+var jobStatus = resource.getStatus().getJobStatus();
+var checkpointInfo = jobStatus.getSavepointInfo();
Review Comment:
Removed, it is currently not used at all.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [flink-kubernetes-operator] afedulov commented on a diff in pull request #637: [FLINK-29634] Support periodic checkpoint triggering
afedulov commented on code in PR #637:
URL:
https://github.com/apache/flink-kubernetes-operator/pull/637#discussion_r1273792943
##
flink-kubernetes-operator/src/main/java/org/apache/flink/kubernetes/operator/observer/SnapshotObserver.java:
##
@@ -124,14 +159,78 @@ private void
observeTriggeredSavepoint(FlinkResourceContext ctx, String jobI
savepointFetchResult.getLocation(),
savepointInfo.getTriggerType(),
savepointInfo.getFormatType(),
-SavepointTriggerType.MANUAL ==
savepointInfo.getTriggerType()
+SnapshotTriggerType.MANUAL ==
savepointInfo.getTriggerType()
?
resource.getSpec().getJob().getSavepointTriggerNonce()
: null);
-
ReconciliationUtils.updateLastReconciledSavepointTriggerNonce(savepointInfo,
resource);
+ReconciliationUtils.updateLastReconciledSnapshotTriggerNonce(
+savepointInfo, resource, SAVEPOINT);
savepointInfo.updateLastSavepoint(savepoint);
}
+/**
+ * Observe the status of triggered checkpoints.
+ *
+ * @param ctx Resource context.
+ * @param jobID the jobID of the observed job.
+ */
+private void observeTriggeredCheckpoint(FlinkResourceContext ctx,
String jobID) {
Review Comment:
Introduced for consistency with observeTriggeredSavepoint. Drop it there too?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [flink-kubernetes-operator] afedulov commented on a diff in pull request #637: [FLINK-29634] Support periodic checkpoint triggering
afedulov commented on code in PR #637:
URL:
https://github.com/apache/flink-kubernetes-operator/pull/637#discussion_r1273789422
##
flink-kubernetes-operator/src/main/java/org/apache/flink/kubernetes/operator/observer/SnapshotObserver.java:
##
@@ -74,8 +80,32 @@ public void observeSavepointStatus(FlinkResourceContext
ctx) {
cleanupSavepointHistory(ctx, savepointInfo);
}
+public void observeCheckpointStatus(FlinkResourceContext ctx) {
+if (!isCheckpointsTriggeringSupported(ctx.getObserveConfig())) {
+return;
+}
+var resource = ctx.getResource();
+var jobStatus = resource.getStatus().getJobStatus();
+var checkpointInfo = jobStatus.getSavepointInfo();
+var jobId = jobStatus.getJobId();
+
+// If any manual or periodic savepoint is in progress, observe it
+if (SnapshotUtils.checkpointInProgress(jobStatus)) {
+observeTriggeredCheckpoint(ctx, jobId);
+}
+
+// REVIEW: clarify if this is relevant for checkpoints.
+/*
+// If job is in globally terminal state, observe last savepoint
+if (ReconciliationUtils.isJobInTerminalState(resource.getStatus())) {
+observeLatestCheckpoint(
+ctx.getFlinkService(), checkpointInfo, jobId,
ctx.getObserveConfig());
+}
+*/
Review Comment:
@gyfora there is something I do not quite get in the current logic: the
symmetric `observeLatestSavepoint` uses `getLastCheckpoint` that does not
explicitly make a distinction between savepoints and checkpoints. IIUC, if a
checkpoint is stored to an externally-addressable location, it will be
returned, pretending to be a Savepoint, and potentially used for recovery. Is
that desirable? I guess in this case we do not need to introduce
`observeLatestCheckpoint `.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [flink-kubernetes-operator] afedulov commented on a diff in pull request #637: [FLINK-29634] Support periodic checkpoint triggering
afedulov commented on code in PR #637:
URL:
https://github.com/apache/flink-kubernetes-operator/pull/637#discussion_r1273789422
##
flink-kubernetes-operator/src/main/java/org/apache/flink/kubernetes/operator/observer/SnapshotObserver.java:
##
@@ -74,8 +80,32 @@ public void observeSavepointStatus(FlinkResourceContext
ctx) {
cleanupSavepointHistory(ctx, savepointInfo);
}
+public void observeCheckpointStatus(FlinkResourceContext ctx) {
+if (!isCheckpointsTriggeringSupported(ctx.getObserveConfig())) {
+return;
+}
+var resource = ctx.getResource();
+var jobStatus = resource.getStatus().getJobStatus();
+var checkpointInfo = jobStatus.getSavepointInfo();
+var jobId = jobStatus.getJobId();
+
+// If any manual or periodic savepoint is in progress, observe it
+if (SnapshotUtils.checkpointInProgress(jobStatus)) {
+observeTriggeredCheckpoint(ctx, jobId);
+}
+
+// REVIEW: clarify if this is relevant for checkpoints.
+/*
+// If job is in globally terminal state, observe last savepoint
+if (ReconciliationUtils.isJobInTerminalState(resource.getStatus())) {
+observeLatestCheckpoint(
+ctx.getFlinkService(), checkpointInfo, jobId,
ctx.getObserveConfig());
+}
+*/
Review Comment:
@gyfora there is something I do not quite get in the current logic: the
symmetric `observeLatestSavepoint` uses `getLastCheckpoint` that does not
explicitly make a distinction between savepoints and checkpoints. IIUC, if a
checkpoint is stored to an externally-addressable location, it will be returned
and potentially used for recovery. Is that desirable? I guess in this case we
do not need to necessarily need to introduce `observeLatestCheckpoint `.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [flink-kubernetes-operator] afedulov commented on a diff in pull request #637: [FLINK-29634] Support periodic checkpoint triggering
afedulov commented on code in PR #637:
URL:
https://github.com/apache/flink-kubernetes-operator/pull/637#discussion_r1273789422
##
flink-kubernetes-operator/src/main/java/org/apache/flink/kubernetes/operator/observer/SnapshotObserver.java:
##
@@ -74,8 +80,32 @@ public void observeSavepointStatus(FlinkResourceContext
ctx) {
cleanupSavepointHistory(ctx, savepointInfo);
}
+public void observeCheckpointStatus(FlinkResourceContext ctx) {
+if (!isCheckpointsTriggeringSupported(ctx.getObserveConfig())) {
+return;
+}
+var resource = ctx.getResource();
+var jobStatus = resource.getStatus().getJobStatus();
+var checkpointInfo = jobStatus.getSavepointInfo();
+var jobId = jobStatus.getJobId();
+
+// If any manual or periodic savepoint is in progress, observe it
+if (SnapshotUtils.checkpointInProgress(jobStatus)) {
+observeTriggeredCheckpoint(ctx, jobId);
+}
+
+// REVIEW: clarify if this is relevant for checkpoints.
+/*
+// If job is in globally terminal state, observe last savepoint
+if (ReconciliationUtils.isJobInTerminalState(resource.getStatus())) {
+observeLatestCheckpoint(
+ctx.getFlinkService(), checkpointInfo, jobId,
ctx.getObserveConfig());
+}
+*/
Review Comment:
@gyfora there is something I do not quite get in the current logic: the
symmetric `observeLatestSavepoint` uses `getLastCheckpoint` that does not
explicitly make a distinction between savepoints and checkpoints. IIUC, if a
checkpoint is stored to an externally-addressable location, it will be
returned, pretending to be a Savepoint, and potentially used for recovery. Is
that desirable? I guess in this case we do not need to necessarily need to
introduce `observeLatestCheckpoint `.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [flink] XComp commented on a diff in pull request #22996: [FLINK-32468][rpc] Switch from Akka to Pekko
XComp commented on code in PR #22996: URL: https://github.com/apache/flink/pull/22996#discussion_r1273769632 ## flink-core/src/main/java/org/apache/flink/configuration/AkkaOptions.java: ## Review Comment: AkkaOptions is left like that because it's labeled as `@PublicEvolving`. Deprecating this one requires a discussion which we would do in a separate step. But thanks for looking over it. :+1: -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] mdedetrich commented on a diff in pull request #22996: [FLINK-32468][rpc] Switch from Akka to Pekko
mdedetrich commented on code in PR #22996: URL: https://github.com/apache/flink/pull/22996#discussion_r1273759360 ## flink-core/src/main/java/org/apache/flink/configuration/AkkaOptions.java: ## Review Comment: This file is still called `AkkaOptions.java`, shouldn't it be renamed to `PekkoOptions.scala`? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] XComp commented on pull request #22996: [FLINK-32468][rpc] Switch from Akka to Pekko
XComp commented on PR #22996: URL: https://github.com/apache/flink/pull/22996#issuecomment-1650088706 @mdedetrich pointed out today that there is a Pekko 1.0.1 release on the way. It includes a fix in `pekko-remote` which we use. It's not 100% clear whether the issue actually affects Flink users (it's about Scala class names with `$`). But considering that we have to do another CI run, anyway (due to the `runtime-web` test instability), it might be easy enough to bump the version to 1.0.1 right away. They triggered the publishing of the jars today ([ML thread](https://lists.apache.org/thread/kqnm6rhbxp7x67x7x0fgjkr3w1vncmvy)) and expect it to be done by tomorrow morning latest. WDYT? On the topic of merging the PR: We got the verbal :+1: to merge this issue by the release managers in today's release sync. I was waiting for a summary of the meeting in the ML which didn't happen, yet. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink-kubernetes-operator] JTaky opened a new pull request, #638: Update doc to ensure that users keep in mind how rollback feature works
JTaky opened a new pull request, #638: URL: https://github.com/apache/flink-kubernetes-operator/pull/638 ## What is the purpose of the change Update documentation of the rollback features. Flink devs who is not aware about internals of the operator's CRDs is not obvious why rollbacks of incompatible Helm chart changes will not lead to the job running. ## Brief change log Documentation of the corner case to reduce miss-understanding of the roll back feature behaviour ## Verifying this change This change is a trivial rework / code cleanup without any test coverage. ## Does this pull request potentially affect one of the following parts: - Dependencies (does it add or upgrade a dependency): no - The public API, i.e., is any changes to the `CustomResourceDescriptors`: no - Core observer or reconciler logic that is regularly executed: no ## Documentation - Does this pull request introduce a new feature? no - If yes, how is the feature documented? not applicable -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (FLINK-28046) Annotate SourceFunction as deprecated
[ https://issues.apache.org/jira/browse/FLINK-28046?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17747031#comment-17747031 ] Leonard Xu commented on FLINK-28046: [~chesnay] We've discussed with [~afedulov], Konstantin offline and have posted the consensus to dev mail list. > You ran into issues on the Paimon/CDC side, where are the tickets to fix the > problems? Flavio ran into issues, where's the ticket for that? We have walked around but have some initial ideas, I think we can propose a FLIP to enhance new Source in the future. > Annotate SourceFunction as deprecated > - > > Key: FLINK-28046 > URL: https://issues.apache.org/jira/browse/FLINK-28046 > Project: Flink > Issue Type: Sub-task > Components: API / DataStream >Affects Versions: 1.15.3 >Reporter: Alexander Fedulov >Assignee: Alexander Fedulov >Priority: Major > Labels: pull-request-available, stale-assigned > Fix For: 1.18.0 > > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [flink-kubernetes-operator] gyfora commented on a diff in pull request #637: [FLINK-29634] Support periodic checkpoint triggering
gyfora commented on code in PR #637: URL: https://github.com/apache/flink-kubernetes-operator/pull/637#discussion_r1273650143 ## flink-kubernetes-operator-api/src/main/java/org/apache/flink/kubernetes/operator/api/status/SnapshotType.java: ## @@ -0,0 +1,24 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.flink.kubernetes.operator.api.status; Review Comment: observer/reconciler package? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink-kubernetes-operator] afedulov commented on a diff in pull request #637: [FLINK-29634] Support periodic checkpoint triggering
afedulov commented on code in PR #637:
URL:
https://github.com/apache/flink-kubernetes-operator/pull/637#discussion_r1273647391
##
flink-kubernetes-operator/src/test/java/org/apache/flink/kubernetes/operator/reconciler/sessionjob/SessionJobReconcilerTest.java:
##
@@ -409,24 +417,143 @@ public void testTriggerSavepoint() throws Exception {
sp1SessionJob, JobState.RUNNING, RECONCILING.name(), null,
flinkService.listJobs());
sp1SessionJob.getStatus().getJobStatus().getSavepointInfo().resetTrigger();
-ReconciliationUtils.updateLastReconciledSavepointTriggerNonce(
-sp1SessionJob.getStatus().getJobStatus().getSavepointInfo(),
sp1SessionJob);
+ReconciliationUtils.updateLastReconciledSnapshotTriggerNonce(
+sp1SessionJob.getStatus().getJobStatus().getSavepointInfo(),
+sp1SessionJob,
+SAVEPOINT);
// trigger when new nonce is defined
sp1SessionJob.getSpec().getJob().setSavepointTriggerNonce(4L);
reconciler.reconcile(sp1SessionJob, readyContext);
assertEquals(
-"trigger_1",
+"savepoint_trigger_1",
sp1SessionJob.getStatus().getJobStatus().getSavepointInfo().getTriggerId());
sp1SessionJob.getStatus().getJobStatus().getSavepointInfo().resetTrigger();
-ReconciliationUtils.updateLastReconciledSavepointTriggerNonce(
-sp1SessionJob.getStatus().getJobStatus().getSavepointInfo(),
sp1SessionJob);
+ReconciliationUtils.updateLastReconciledSnapshotTriggerNonce(
+sp1SessionJob.getStatus().getJobStatus().getSavepointInfo(),
+sp1SessionJob,
+SAVEPOINT);
// don't trigger when nonce is cleared
sp1SessionJob.getSpec().getJob().setSavepointTriggerNonce(null);
reconciler.reconcile(sp1SessionJob, readyContext);
-
assertFalse(SavepointUtils.savepointInProgress(sp1SessionJob.getStatus().getJobStatus()));
+
assertFalse(SnapshotUtils.savepointInProgress(sp1SessionJob.getStatus().getJobStatus()));
+}
+
+@Test
+public void testTriggerCheckpoint() throws Exception {
+FlinkSessionJob sessionJob = TestUtils.buildSessionJob();
+
assertFalse(SnapshotUtils.checkpointInProgress(getJobStatus(sessionJob)));
+
+var readyContext = TestUtils.createContextWithReadyFlinkDeployment();
+reconciler.reconcile(sessionJob, readyContext);
+verifyAndSetRunningJobsToStatus(
+sessionJob, JobState.RUNNING, RECONCILING.name(), null,
flinkService.listJobs());
+
+
assertFalse(SnapshotUtils.checkpointInProgress(getJobStatus(sessionJob)));
+
+// trigger checkpoint
+var sp1SessionJob = ReconciliationUtils.clone(sessionJob);
+
+// do not trigger checkpoint if nonce is null
+reconciler.reconcile(sp1SessionJob, readyContext);
+
assertFalse(SnapshotUtils.checkpointInProgress(getJobStatus(sp1SessionJob)));
+
+getJobSpec(sp1SessionJob).setCheckpointTriggerNonce(2L);
+getJobStatus(sp1SessionJob).setState(CREATED.name());
+reconciler.reconcile(sp1SessionJob, readyContext);
+// do not trigger checkpoint if job is not running
+
assertFalse(SnapshotUtils.checkpointInProgress(getJobStatus(sp1SessionJob)));
+
+getJobStatus(sp1SessionJob).setState(RUNNING.name());
+
+reconciler.reconcile(sp1SessionJob, readyContext);
+
assertTrue(SnapshotUtils.checkpointInProgress(getJobStatus(sp1SessionJob)));
+
+// the last reconcile nonce updated
+
assertNull(getReconciledJobSpec(sp1SessionJob).getCheckpointTriggerNonce());
+
+// don't trigger new checkpoint when checkpoint is in progress
+getJobSpec(sp1SessionJob).setCheckpointTriggerNonce(3L);
+reconciler.reconcile(sp1SessionJob, readyContext);
+assertEquals("checkpoint_trigger_0",
getCheckpointInfo(sp1SessionJob).getTriggerId());
+/*
+TODO: this section needs to be reintroduced in case the LAST_STATE
optimization gets
+ added
Review Comment:
I mostly left it for the review discussion. @gyfora what do you think, is is
a worthy optimization in your opinion?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [flink-kubernetes-operator] afedulov commented on a diff in pull request #637: [FLINK-29634] Support periodic checkpoint triggering
afedulov commented on code in PR #637: URL: https://github.com/apache/flink-kubernetes-operator/pull/637#discussion_r1273645546 ## flink-kubernetes-operator-api/src/main/java/org/apache/flink/kubernetes/operator/api/status/SnapshotType.java: ## @@ -0,0 +1,24 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.flink.kubernetes.operator.api.status; Review Comment: What is the better place for it in your opinion? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Assigned] (FLINK-32673) Migrage Google PubSub connector to V2
[ https://issues.apache.org/jira/browse/FLINK-32673?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Leonard Xu reassigned FLINK-32673: -- Assignee: (was: Alex Chaffee) > Migrage Google PubSub connector to V2 > - > > Key: FLINK-32673 > URL: https://issues.apache.org/jira/browse/FLINK-32673 > Project: Flink > Issue Type: Sub-task >Reporter: Alexander Fedulov >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (FLINK-32673) Migrage Google PubSub connector to V2
[ https://issues.apache.org/jira/browse/FLINK-32673?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Leonard Xu reassigned FLINK-32673: -- Assignee: Alex Chaffee > Migrage Google PubSub connector to V2 > - > > Key: FLINK-32673 > URL: https://issues.apache.org/jira/browse/FLINK-32673 > Project: Flink > Issue Type: Sub-task >Reporter: Alexander Fedulov >Assignee: Alex Chaffee >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [flink] RyanSkraba commented on pull request #23006: [FLINK-32560][Scala] Deprecate all Scala API
RyanSkraba commented on PR #23006: URL: https://github.com/apache/flink/pull/23006#issuecomment-1649909051 Just to summarize (the main discussion is on the JIRA): * Rebased and squashed all commits * Only top level classes are `@deprecated` to ensure that the dev has the info about removing these classes and modules in 2.0 (but the entrypoints to the Scala API have been deprecated since 1.17.0) * The scala `@deprecation` is preferred, I didn't search and replace existing `@Deprecation` annotations. These are largely equivalent. The original (overly annotated) PR is temporarily available at https://github.com/apache/flink/compare/master...RyanSkraba:flink:rskraba/FLINK-32560-deprecate-scala-original -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (FLINK-32673) Migrage Google PubSub connector to V2
Alexander Fedulov created FLINK-32673: - Summary: Migrage Google PubSub connector to V2 Key: FLINK-32673 URL: https://issues.apache.org/jira/browse/FLINK-32673 Project: Flink Issue Type: Sub-task Reporter: Alexander Fedulov -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (FLINK-32672) Migrate RabbitMQ connector to Source V2 API
Alexander Fedulov created FLINK-32672: - Summary: Migrate RabbitMQ connector to Source V2 API Key: FLINK-32672 URL: https://issues.apache.org/jira/browse/FLINK-32672 Project: Flink Issue Type: Sub-task Components: Connectors/ RabbitMQ Reporter: Alexander Fedulov -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [flink] liming30 commented on pull request #23071: [FLINK-32655][runtime] Fix checkpoint aborted message being swallowed by RecreateOnResetOperatorCoordinator.
liming30 commented on PR #23071: URL: https://github.com/apache/flink/pull/23071#issuecomment-1649855073 @1996fanrui CI has passed, please merge it at your convenience. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] liming30 commented on pull request #23070: [FLINK-32655][runtime] Fix checkpoint aborted message being swallowed by RecreateOnResetOperatorCoordinator.
liming30 commented on PR #23070: URL: https://github.com/apache/flink/pull/23070#issuecomment-1649854603 @1996fanrui CI has passed, please merge it at your convenience. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Comment Edited] (FLINK-32548) Make watermark alignment ready for production use
[ https://issues.apache.org/jira/browse/FLINK-32548?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17746998#comment-17746998 ] Rui Fan edited comment on FLINK-32548 at 7/25/23 1:24 PM: -- Hi [~knaufk] , thanks for the asking. I think the beta can be removed at flink 1.18. And I want to here more thoughts from core contributors of watermark alignment. cc [~pnowojski] [~dwysakowicz] I can remove it if most of us think the beta can be removed. was (Author: fanrui): Hi [~knaufk] , thanks for the asking. I think the beta can be removed at flink 1.18. And I want to here more thoughts from core contributors of watermark alignment. cc [~pnowojski] [~dwysakowicz] > Make watermark alignment ready for production use > - > > Key: FLINK-32548 > URL: https://issues.apache.org/jira/browse/FLINK-32548 > Project: Flink > Issue Type: Bug >Affects Versions: 1.16.2, 1.17.1 >Reporter: Rui Fan >Assignee: Rui Fan >Priority: Major > Fix For: 1.18.0, 1.16.3, 1.17.2 > > > We found a series of watermark alignment bugs and performance issues and hope > to reach production availability in 1.18.0. > And fixes all bugs found in 1.16.3 and 1.17.2. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (FLINK-32548) Make watermark alignment ready for production use
[ https://issues.apache.org/jira/browse/FLINK-32548?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17746998#comment-17746998 ] Rui Fan commented on FLINK-32548: - Hi [~knaufk] , thanks for the asking. I think the beta can be removed at flink 1.18. And I want to here more thoughts from core contributors of watermark alignment. cc [~pnowojski] [~dwysakowicz] > Make watermark alignment ready for production use > - > > Key: FLINK-32548 > URL: https://issues.apache.org/jira/browse/FLINK-32548 > Project: Flink > Issue Type: Bug >Affects Versions: 1.16.2, 1.17.1 >Reporter: Rui Fan >Assignee: Rui Fan >Priority: Major > Fix For: 1.18.0, 1.16.3, 1.17.2 > > > We found a series of watermark alignment bugs and performance issues and hope > to reach production availability in 1.18.0. > And fixes all bugs found in 1.16.3 and 1.17.2. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (FLINK-31889) Add documentation for implementing/loading enrichers
[ https://issues.apache.org/jira/browse/FLINK-31889?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Konstantin Knauf reassigned FLINK-31889: Assignee: Konstantin Knauf (was: Panagiotis Garefalakis) > Add documentation for implementing/loading enrichers > > > Key: FLINK-31889 > URL: https://issues.apache.org/jira/browse/FLINK-31889 > Project: Flink > Issue Type: Sub-task >Reporter: Panagiotis Garefalakis >Assignee: Konstantin Knauf >Priority: Major > Labels: pull-request-available > > Describe how enrichers can be implemented and loaded to Flink as part of > documentation -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (FLINK-31889) Add documentation for implementing/loading enrichers
[ https://issues.apache.org/jira/browse/FLINK-31889?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Konstantin Knauf reassigned FLINK-31889: Assignee: Panagiotis Garefalakis (was: Konstantin Knauf) > Add documentation for implementing/loading enrichers > > > Key: FLINK-31889 > URL: https://issues.apache.org/jira/browse/FLINK-31889 > Project: Flink > Issue Type: Sub-task >Reporter: Panagiotis Garefalakis >Assignee: Panagiotis Garefalakis >Priority: Major > Labels: pull-request-available > > Describe how enrichers can be implemented and loaded to Flink as part of > documentation -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (FLINK-31889) Add documentation for implementing/loading enrichers
[ https://issues.apache.org/jira/browse/FLINK-31889?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Konstantin Knauf reassigned FLINK-31889: Assignee: Panagiotis Garefalakis > Add documentation for implementing/loading enrichers > > > Key: FLINK-31889 > URL: https://issues.apache.org/jira/browse/FLINK-31889 > Project: Flink > Issue Type: Sub-task >Reporter: Panagiotis Garefalakis >Assignee: Panagiotis Garefalakis >Priority: Major > Labels: pull-request-available > > Describe how enrichers can be implemented and loaded to Flink as part of > documentation -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [flink] flinkbot commented on pull request #23074: [BP-1.17][FLINK-30629][Client/Job Submission] Increase clientHeartbeatTimeout …
flinkbot commented on PR #23074: URL: https://github.com/apache/flink/pull/23074#issuecomment-1649809503 ## CI report: * 41c9d47a036569d7e65147089ae2a72858ddfb77 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] luoyuxia commented on a diff in pull request #23060: [FLINK-32519][docs] Add doc for [CREATE OR] REPLACE TABLE AS statement
luoyuxia commented on code in PR #23060:
URL: https://github.com/apache/flink/pull/23060#discussion_r1273468797
##
docs/content.zh/docs/dev/table/sql/create.md:
##
@@ -557,6 +558,58 @@ INSERT INTO my_ctas_table SELECT id, name, age FROM
source_table WHERE mod(id, 1
{{< top >}}
+## CREATE TABLE [USING]
Review Comment:
dito
##
docs/content/docs/dev/table/sql/create.md:
##
@@ -557,6 +558,58 @@ INSERT INTO my_ctas_table SELECT id, name, age FROM
source_table WHERE mod(id, 1
{{< top >}}
+## CREATE TABLE [USING]
+```sql
+[CREATE OR] REPLACE TABLE [catalog_name.][db_name.]table_name
+[COMMENT table_comment]
+WITH (key1=val1, key2=val2, ...)
+AS select_query
+```
+Tables can also be replaced(or created) and populated by the results of a
query in one replace-table-as-select (RTAS) statement. RTAS is the simplest
and fastest way to replace and insert data into a table with a single command.
+
+There are two parts in RTAS, the SELECT part can be any [SELECT query]({{< ref
"docs/dev/table/sql/queries/overview" >}}) supported by Flink SQL. The `[CREATE
OR] REPLACE` part takes the resulting schema from the `SELECT` part and replace
the target table. Similar to `CREATE TABLE` and `CTAS`, RTAS requires the
required options of the target table must be specified in WITH clause.
+
+Consider the example statement below:
+
+```sql
+REPLACE TABLE my_rtas_table
+WITH (
+'connector' = 'kafka',
+...
+)
+AS SELECT id, name, age FROM source_table WHERE mod(id, 10) = 0;
+```
+
+The resulting table `my_rtas_table` is equivalent to first drop the table,
then create the table and insert the data with the following statement:
Review Comment:
Should be the statement is is equivalent to instead of the resulting table.
WDYT?
##
docs/content.zh/docs/dev/table/sql/create.md:
##
@@ -33,6 +33,7 @@ CREATE 语句用于向当前或指定的 [Catalog]({{< ref "docs/dev/table/catal
目前 Flink SQL 支持下列 CREATE 语句：
- CREATE TABLE
+- CREATE TABLE [USING]
Review Comment:
Why `CREATE TABLE [USING]`?
##
docs/content/docs/dev/table/sql/create.md:
##
@@ -557,6 +558,58 @@ INSERT INTO my_ctas_table SELECT id, name, age FROM
source_table WHERE mod(id, 1
{{< top >}}
+## CREATE TABLE [USING]
+```sql
+[CREATE OR] REPLACE TABLE [catalog_name.][db_name.]table_name
+[COMMENT table_comment]
+WITH (key1=val1, key2=val2, ...)
+AS select_query
+```
+Tables can also be replaced(or created) and populated by the results of a
query in one replace-table-as-select (RTAS) statement. RTAS is the simplest
and fastest way to replace and insert data into a table with a single command.
+
+There are two parts in RTAS, the SELECT part can be any [SELECT query]({{< ref
"docs/dev/table/sql/queries/overview" >}}) supported by Flink SQL. The `[CREATE
OR] REPLACE` part takes the resulting schema from the `SELECT` part and replace
the target table. Similar to `CREATE TABLE` and `CTAS`, RTAS requires the
required options of the target table must be specified in WITH clause.
+
+Consider the example statement below:
+
+```sql
+REPLACE TABLE my_rtas_table
+WITH (
+'connector' = 'kafka',
+...
+)
+AS SELECT id, name, age FROM source_table WHERE mod(id, 10) = 0;
+```
+
+The resulting table `my_rtas_table` is equivalent to first drop the table,
then create the table and insert the data with the following statement:
+```sql
+DROP TABLE my_rtas_table;
+
+CREATE TABLE my_rtas_table (
+id BIGINT,
+name STRING,
+age INT
+) WITH (
+'connector' = 'kafka',
+...
+);
+
+INSERT INTO my_rtas_table SELECT id, name, age FROM source_table WHERE mod(id,
10) = 0;
+```
+
+**Note** RTAS has the following semantic:
+* REPLACE TABLE AS SELECT statement, the target table to be replaced must
exist or an exception is thrown.
+* CREATE OR REPLACE TABLE AS SELECT statement, the target table to be replaced
is created if it does not exist; if it does exist, it is replaced.
+
+**Note** RTAS has these restrictions:
+
+Does not support replacing a temporary table yet.
Review Comment:
`* Does not support creating a temporary table yet.`.
##
docs/content/docs/dev/table/sql/create.md:
##
@@ -557,6 +558,58 @@ INSERT INTO my_ctas_table SELECT id, name, age FROM
source_table WHERE mod(id, 1
{{< top >}}
+## CREATE TABLE [USING]
+```sql
+[CREATE OR] REPLACE TABLE [catalog_name.][db_name.]table_name
+[COMMENT table_comment]
+WITH (key1=val1, key2=val2, ...)
+AS select_query
+```
+Tables can also be replaced(or created) and populated by the results of a
query in one replace-table-as-select (RTAS) statement. RTAS is the simplest
and fastest way to replace and insert data into a table with a single command.
+
+There are two parts in RTAS, the SELECT part can be any [SELECT query]({{< ref
"docs/dev/table/sql/queries/overview" >}}) supported by Flink SQL. The `[CREATE
OR] REPLACE` part takes the resulting schema from the `SELECT` part and replace
the target table.
[GitHub] [flink] snuyanzin opened a new pull request, #23074: [FLINK-30629][Client/Job Submission] Increase clientHeartbeatTimeout …
snuyanzin opened a new pull request, #23074: URL: https://github.com/apache/flink/pull/23074 …to 1 second (#22742) ## What is the purpose of the change *(For example: This pull request makes task deployment go through the blob server, rather than through RPC. That way we avoid re-transferring them on each deployment (during recovery).)* ## Brief change log *(for example:)* - *The TaskInfo is stored in the blob store on job creation time as a persistent artifact* - *Deployments RPC transmits only the blob storage reference* - *TaskManagers retrieve the TaskInfo from the blob cache* ## Verifying this change Please make sure both new and modified tests in this PR follows the conventions defined in our code quality guide: https://flink.apache.org/contributing/code-style-and-quality-common.html#testing *(Please pick either of the following options)* This change is a trivial rework / code cleanup without any test coverage. *(or)* This change is already covered by existing tests, such as *(please describe tests)*. *(or)* This change added tests and can be verified as follows: *(example:)* - *Added integration tests for end-to-end deployment with large payloads (100MB)* - *Extended integration test for recovery after master (JobManager) failure* - *Added test that validates that TaskInfo is transferred only once across recoveries* - *Manually verified the change by running a 4 node cluster with 2 JobManagers and 4 TaskManagers, a stateful streaming program, and killing one JobManager and two TaskManagers during the execution, verifying that recovery happens correctly.* ## Does this pull request potentially affect one of the following parts: - Dependencies (does it add or upgrade a dependency): (yes / no) - The public API, i.e., is any changed class annotated with `@Public(Evolving)`: (yes / no) - The serializers: (yes / no / don't know) - The runtime per-record code paths (performance sensitive): (yes / no / don't know) - Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: (yes / no / don't know) - The S3 file system connector: (yes / no / don't know) ## Documentation - Does this pull request introduce a new feature? (yes / no) - If yes, how is the feature documented? (not applicable / docs / JavaDocs / not documented) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (FLINK-32574) RescalingITCase.testSavepointRescalingInPartitionedOperatorState fails on AZP
[
https://issues.apache.org/jira/browse/FLINK-32574?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17746971#comment-17746971
]
Yanfei Lei commented on FLINK-32574:
[~renqs] I can't reproduce this bug locally, but by observing the logs of the
two failures, I found that they both enabled unaligned checkpoint, I'm trying
to reproduce it with `PseudoRandomValueSelector` disabled.
> RescalingITCase.testSavepointRescalingInPartitionedOperatorState fails on AZP
> -
>
> Key: FLINK-32574
> URL: https://issues.apache.org/jira/browse/FLINK-32574
> Project: Flink
> Issue Type: Bug
> Components: Tests
>Affects Versions: 1.18.0
>Reporter: Sergey Nuyanzin
>Assignee: Yanfei Lei
>Priority: Critical
>
> This build
> https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=51129&view=logs&j=8fd9202e-fd17-5b26-353c-ac1ff76c8f28&t=ea7cf968-e585-52cb-e0fc-f48de023a7ca&l=8308
> fails with
> {noformat}
> Jul 09 01:23:15 01:23:15.355 [ERROR] Tests run: 48, Failures: 1, Errors: 0,
> Skipped: 0, Time elapsed: 247.495 s <<< FAILURE! - in
> org.apache.flink.test.checkpointing.RescalingITCase
> Jul 09 01:23:15 01:23:15.355 [ERROR]
> RescalingITCase.testSavepointRescalingInPartitionedOperatorState Time
> elapsed: 2.037 s <<< FAILURE!
> Jul 09 01:23:15 java.lang.AssertionError: expected:<524> but was:<24>
> Jul 09 01:23:15 at org.junit.Assert.fail(Assert.java:89)
> Jul 09 01:23:15 at org.junit.Assert.failNotEquals(Assert.java:835)
> Jul 09 01:23:15 at org.junit.Assert.assertEquals(Assert.java:647)
> Jul 09 01:23:15 at org.junit.Assert.assertEquals(Assert.java:633)
> Jul 09 01:23:15 at
> org.apache.flink.test.checkpointing.RescalingITCase.testSavepointRescalingPartitionedOperatorState(RescalingITCase.java:621)
> Jul 09 01:23:15 at
> org.apache.flink.test.checkpointing.RescalingITCase.testSavepointRescalingInPartitionedOperatorState(RescalingITCase.java:484)
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Closed] (FLINK-32574) RescalingITCase.testSavepointRescalingInPartitionedOperatorState fails on AZP
[
https://issues.apache.org/jira/browse/FLINK-32574?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yanfei Lei closed FLINK-32574.
--
Resolution: Duplicate
> RescalingITCase.testSavepointRescalingInPartitionedOperatorState fails on AZP
> -
>
> Key: FLINK-32574
> URL: https://issues.apache.org/jira/browse/FLINK-32574
> Project: Flink
> Issue Type: Bug
> Components: Tests
>Affects Versions: 1.18.0
>Reporter: Sergey Nuyanzin
>Assignee: Yanfei Lei
>Priority: Critical
>
> This build
> https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=51129&view=logs&j=8fd9202e-fd17-5b26-353c-ac1ff76c8f28&t=ea7cf968-e585-52cb-e0fc-f48de023a7ca&l=8308
> fails with
> {noformat}
> Jul 09 01:23:15 01:23:15.355 [ERROR] Tests run: 48, Failures: 1, Errors: 0,
> Skipped: 0, Time elapsed: 247.495 s <<< FAILURE! - in
> org.apache.flink.test.checkpointing.RescalingITCase
> Jul 09 01:23:15 01:23:15.355 [ERROR]
> RescalingITCase.testSavepointRescalingInPartitionedOperatorState Time
> elapsed: 2.037 s <<< FAILURE!
> Jul 09 01:23:15 java.lang.AssertionError: expected:<524> but was:<24>
> Jul 09 01:23:15 at org.junit.Assert.fail(Assert.java:89)
> Jul 09 01:23:15 at org.junit.Assert.failNotEquals(Assert.java:835)
> Jul 09 01:23:15 at org.junit.Assert.assertEquals(Assert.java:647)
> Jul 09 01:23:15 at org.junit.Assert.assertEquals(Assert.java:633)
> Jul 09 01:23:15 at
> org.apache.flink.test.checkpointing.RescalingITCase.testSavepointRescalingPartitionedOperatorState(RescalingITCase.java:621)
> Jul 09 01:23:15 at
> org.apache.flink.test.checkpointing.RescalingITCase.testSavepointRescalingInPartitionedOperatorState(RescalingITCase.java:484)
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (FLINK-32663) RescalingITCase.testSavepointRescalingInPartitionedOperatorStateList fails on AZP
[
https://issues.apache.org/jira/browse/FLINK-32663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17746972#comment-17746972
]
Yanfei Lei commented on FLINK-32663:
[~mapohl] I think they are duplicate, I'll close FLINK-32574 and marked it as
a duplicate.
> RescalingITCase.testSavepointRescalingInPartitionedOperatorStateList fails on
> AZP
> -
>
> Key: FLINK-32663
> URL: https://issues.apache.org/jira/browse/FLINK-32663
> Project: Flink
> Issue Type: Bug
> Components: Runtime / Coordination
>Affects Versions: 1.18.0
>Reporter: Sergey Nuyanzin
>Assignee: Matthias Pohl
>Priority: Critical
> Labels: test-stability
>
> This build
> https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=51501&view=logs&j=8fd9202e-fd17-5b26-353c-ac1ff76c8f28&t=ea7cf968-e585-52cb-e0fc-f48de023a7ca&l=8665
> fails as
> {noformat}
> Jul 21 01:24:54 01:24:54.146 [ERROR]
> RescalingITCase.testSavepointRescalingInPartitionedOperatorStateList Time
> elapsed: 1.485 s <<< FAILURE!
> Jul 21 01:24:54 java.lang.AssertionError: expected:<530> but was:<30>
> Jul 21 01:24:54 at org.junit.Assert.fail(Assert.java:89)
> Jul 21 01:24:54 at org.junit.Assert.failNotEquals(Assert.java:835)
> Jul 21 01:24:54 at org.junit.Assert.assertEquals(Assert.java:647)
> Jul 21 01:24:54 at org.junit.Assert.assertEquals(Assert.java:633)
> Jul 21 01:24:54 at
> org.apache.flink.test.checkpointing.RescalingITCase.testSavepointRescalingPartitionedOperatorState(RescalingITCase.java:621)
> Jul 21 01:24:54 at
> org.apache.flink.test.checkpointing.RescalingITCase.testSavepointRescalingInPartitionedOperatorStateList(RescalingITCase.java:508)
> Jul 21 01:24:54 at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
> Method)
> Jul 21 01:24:54 at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> Jul 21 01:24:54 at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:4
> ...
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (FLINK-32670) Annotate interfaces that inherit from SourceFunction as deprecated
[ https://issues.apache.org/jira/browse/FLINK-32670?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexander Fedulov updated FLINK-32670: -- Description: ParallelSourceFunction, RichParallelSourceFunction, ExternallyInducedSource (was: ProcessFunction, RichParallelSourceFunction, ExternallyInducedSource) > Annotate interfaces that inherit from SourceFunction as deprecated > --- > > Key: FLINK-32670 > URL: https://issues.apache.org/jira/browse/FLINK-32670 > Project: Flink > Issue Type: Sub-task >Reporter: Alexander Fedulov >Priority: Major > > ParallelSourceFunction, RichParallelSourceFunction, ExternallyInducedSource -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [flink] XComp commented on a diff in pull request #22987: [FLINK-32583][rest] Fix deadlock in RestClient
XComp commented on code in PR #22987:
URL: https://github.com/apache/flink/pull/22987#discussion_r1272076683
##
flink-runtime/src/test/java/org/apache/flink/runtime/rest/RestClientTest.java:
##
@@ -207,6 +210,40 @@ public void testRestClientClosedHandling() throws
Exception {
}
}
+/**
+ * Tests that the futures returned by {@link RestClient} fail immediately
if the client is
+ * already closed.
+ *
+ * See FLINK-32583
+ */
+@Test
+public void testCloseClientBeforeRequest() throws Exception {
Review Comment:
I looked into the multi-threading of netty a bit more and found the
`SelectStrategy` being a viable tool to control the state of the request. I
came up with the following test:
```
@Test
public void testCloseClientWhileProcessingRequest() throws Exception {
final OneShotLatch connectTriggered = new OneShotLatch();
final OneShotLatch closeTriggered = new OneShotLatch();
final SelectStrategy fallbackSelectStrategy =
DefaultSelectStrategyFactory.INSTANCE.newSelectStrategy();
final SelectStrategy selectStrategy =
(selectSupplier, hasTasks) -> {
connectTriggered.trigger();
closeTriggered.awaitQuietly();
return
fallbackSelectStrategy.calculateStrategy(selectSupplier, hasTasks);
};
// Note that the executor passed to the RestClient constructor is
not the same as the
// executor used by Netty
try (final RestClient restClient =
new RestClient(
new Configuration(), Executors.directExecutor(), ()
-> selectStrategy)) {
final CompletableFuture requestFuture =
restClient.sendRequest(
unroutableIp,
80,
new TestMessageHeaders(),
EmptyMessageParameters.getInstance(),
EmptyRequestBody.getInstance());
connectTriggered.await();
final CompletableFuture closeFuture =
restClient.closeAsync();
closeTriggered.trigger();
// close should complete successfully
closeFuture.get();
final Throwable cause =
assertThrows(
ExecutionException.class,
() -> requestFuture.get(0,
TimeUnit.SECONDS))
.getCause();
assertThat(cause, instanceOf(IllegalStateException.class));
}
}
```
It would require an extension of the `RestClient` adding a package-private
constructor:
```
// ...
public RestClient(Configuration configuration, Executor executor)
throws ConfigurationException {
this(configuration, executor, DefaultSelectStrategyFactory.INSTANCE);
}
@VisibleForTesting
RestClient(
Configuration configuration,
Executor executor,
SelectStrategyFactory selectStrategyFactory)
//...
NioEventLoopGroup group =
new NioEventLoopGroup(
1,
new ExecutorThreadFactory("flink-rest-client-netty"),
SelectorProvider.provider(),
selectStrategyFactory);
// ...
```
WDYT? That should test the `isRunning.get()` code path.
##
flink-runtime/src/test/java/org/apache/flink/runtime/rest/RestClientTest.java:
##
@@ -207,6 +210,40 @@ public void testRestClientClosedHandling() throws
Exception {
}
}
+/**
+ * Tests that the futures returned by {@link RestClient} fail immediately
if the client is
+ * already closed.
+ *
+ * See FLINK-32583
+ */
+@Test
+public void testCloseClientBeforeRequest() throws Exception {
Review Comment:
I looked into the multi-threading of netty a bit more and found the
`SelectStrategy` being a viable tool to control the state of the request. I
came up with the following test:
```
@Test
public void testCloseClientWhileProcessingRequest() throws Exception {
final OneShotLatch connectTriggered = new OneShotLatch();
final OneShotLatch closeTriggered = new OneShotLatch();
final SelectStrategy fallbackSelectStrategy =
DefaultSelectStrategyFactory.INSTANCE.newSelectStrategy();
final SelectStrategy selectStrategy =
(selectSupplier, hasTasks) -> {
connectTriggered.trigger();
closeTriggered.awaitQuietly();
return
fallbackSelectStrategy.calculateStrategy(sel
[jira] [Commented] (FLINK-32472) FLIP-308: Support Time Travel
[ https://issues.apache.org/jira/browse/FLINK-32472?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17746953#comment-17746953 ] Feng Jin commented on FLINK-32472: -- [~knaufk] Sure, the current task only requires completing the documentation, and I will do so as soon as possible. > FLIP-308: Support Time Travel > - > > Key: FLINK-32472 > URL: https://issues.apache.org/jira/browse/FLINK-32472 > Project: Flink > Issue Type: New Feature > Components: Table SQL / API >Reporter: Feng Jin >Assignee: Feng Jin >Priority: Major > Fix For: 1.18.0 > > > Umbrella issue for > https://cwiki.apache.org/confluence/display/FLINK/FLIP-308%3A+Support+Time+Travel -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (FLINK-15736) Support Java 17 (LTS)
[ https://issues.apache.org/jira/browse/FLINK-15736?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17746950#comment-17746950 ] Konstantin Knauf commented on FLINK-15736: -- [~chesnay]Do we need any user-facing documentation for this? I guess it would make sense somewhere, no? > Support Java 17 (LTS) > - > > Key: FLINK-15736 > URL: https://issues.apache.org/jira/browse/FLINK-15736 > Project: Flink > Issue Type: New Feature > Components: Build System >Reporter: Chesnay Schepler >Assignee: Chesnay Schepler >Priority: Major > Labels: auto-deprioritized-major, pull-request-available, > stale-assigned > Fix For: 1.18.0 > > > Long-term issue for preparing Flink for Java 17. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (FLINK-29344) Make Adaptive Scheduler supports Fine-Grained Resource Management
[
https://issues.apache.org/jira/browse/FLINK-29344?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Konstantin Knauf updated FLINK-29344:
-
Fix Version/s: (was: 1.18.0)
> Make Adaptive Scheduler supports Fine-Grained Resource Management
> -
>
> Key: FLINK-29344
> URL: https://issues.apache.org/jira/browse/FLINK-29344
> Project: Flink
> Issue Type: Improvement
> Components: Runtime / Coordination
>Reporter: Xintong Song
>Assignee: Chesnay Schepler
>Priority: Major
>
> This ticket is a reflection of the following Slack discussion:
> {quote}
> Donatien Schmitz
> Adaptive Scheduler thread:
> Hey all, it seems like the Adaptive Scheduler does not support fine grain
> resource management. I have fixed it and would like to know if you would be
> interested in a PR or if it was purposely designed to not support Fine grain
> resource management.
> rmetzger
> @Donatien Schmitz: I’m concerned that we don’t have a lot of review capacity
> right now, and I’m now aware of any users asking for it.
> rmetzger
> I couldn’t find a ticket for adding this feature, did you find one?
> If not, can you add one? This will allow us to at least making this feature
> show up on google, and people might comment on it, if they need it.
> rmetzger
> If the change is fairly self-contained, is unlikely to cause instabilities,
> then we can also consider merging it
> rmetzger
> @Xintong Song what do you think?
> Xintong Song
> @rmetzger, thanks for involving me.
> @Donatien Schmitz, thanks for bringing this up, and for volunteering on
> fixing this. Could you explain a bit more about how do you plan to fix this?
> Fine-grained resource management is not yet supported by adaptive scheduler,
> because there’s an issue that we haven’t find a good solution for. Namely, if
> only part of the resource requirements can be fulfilled, how do we decide
> which requirements should be fulfilled. E.g., say the job declares it needs
> 10 slots with resource 1 for map tasks, and another 10 slots with resource 2
> for reduce tasks. If there’s not enough resources (say only 10 slots can be
> allocated for simplicity), how many slots for map / reduce tasks should be
> allocated? Obviously, <10 map, 0 reduce> & <0 map, 10 reduce> would not work.
> For this example, a proportional scale-down (<5 map, 5 reduce>) seems
> reasonable. However, a proportional scale-down is not always easy (e.g.,
> requirements is <100 map, 1 reduce>), and the issue grows more complicated if
> you take lots of stages and the differences of slot sizes into consideration.
> I’d like to see adaptive scheduler also supports fine-grained resource
> management. If there’s a good solution to the above issue, I’d love to help
> review the effort.
> Donatien Schmitz
> Dear Robert and Xintong, thanks for reading and reacting to my message! I'll
> reply tomorrow (GTM +1 time) if that's quite alright with you. Best, Donatien
> Schmitz
> Donatien Schmitz
> @Xintong Song
> * We are working on fine-grain scheduling for resource optimisation of long
> running or periodic jobs. One of the feature we are experiencing is a
> "rescheduling plan", a mapping of operators and Resource Profiles that can be
> dynamically applied to a running job. This rescheduling would be triggered by
> policies about some metrics (focus on RocksDB in our case).
> * While developing this new feature, we decided to implement it on the
> Adpative Scheduler instead of the Base Scheduler because the logic brought by
> the state machine already present made it more logical: transitions from
> states Executing -> Cancelling -> Rescheduling -> Waiting for Resources ->
> Creating -> Executing
> * In our case we are working on a POC and thus focusing on a real simple job
> with a // of 1. The issue you brought is indeed something we have faced while
> raising the // of the job.
> * If you create a Jira Ticket we can discuss it over there if you'd like!
> Donatien Schmitz
> @rmetzger The changes do not break the default resource management but does
> not fix the issue brought out by Xintong.
> {quote}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (FLINK-29344) Make Adaptive Scheduler supports Fine-Grained Resource Management
[
https://issues.apache.org/jira/browse/FLINK-29344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17746947#comment-17746947
]
Konstantin Knauf commented on FLINK-29344:
--
Since the feature freeze has passed, I will mark this as Won't Do for Flink
1.18 in the Wiki and remove the fixVersion. Thanks, Konstantin (one of the
release managers for Flink 1.18).
> Make Adaptive Scheduler supports Fine-Grained Resource Management
> -
>
> Key: FLINK-29344
> URL: https://issues.apache.org/jira/browse/FLINK-29344
> Project: Flink
> Issue Type: Improvement
> Components: Runtime / Coordination
>Reporter: Xintong Song
>Assignee: Chesnay Schepler
>Priority: Major
> Fix For: 1.18.0
>
>
> This ticket is a reflection of the following Slack discussion:
> {quote}
> Donatien Schmitz
> Adaptive Scheduler thread:
> Hey all, it seems like the Adaptive Scheduler does not support fine grain
> resource management. I have fixed it and would like to know if you would be
> interested in a PR or if it was purposely designed to not support Fine grain
> resource management.
> rmetzger
> @Donatien Schmitz: I’m concerned that we don’t have a lot of review capacity
> right now, and I’m now aware of any users asking for it.
> rmetzger
> I couldn’t find a ticket for adding this feature, did you find one?
> If not, can you add one? This will allow us to at least making this feature
> show up on google, and people might comment on it, if they need it.
> rmetzger
> If the change is fairly self-contained, is unlikely to cause instabilities,
> then we can also consider merging it
> rmetzger
> @Xintong Song what do you think?
> Xintong Song
> @rmetzger, thanks for involving me.
> @Donatien Schmitz, thanks for bringing this up, and for volunteering on
> fixing this. Could you explain a bit more about how do you plan to fix this?
> Fine-grained resource management is not yet supported by adaptive scheduler,
> because there’s an issue that we haven’t find a good solution for. Namely, if
> only part of the resource requirements can be fulfilled, how do we decide
> which requirements should be fulfilled. E.g., say the job declares it needs
> 10 slots with resource 1 for map tasks, and another 10 slots with resource 2
> for reduce tasks. If there’s not enough resources (say only 10 slots can be
> allocated for simplicity), how many slots for map / reduce tasks should be
> allocated? Obviously, <10 map, 0 reduce> & <0 map, 10 reduce> would not work.
> For this example, a proportional scale-down (<5 map, 5 reduce>) seems
> reasonable. However, a proportional scale-down is not always easy (e.g.,
> requirements is <100 map, 1 reduce>), and the issue grows more complicated if
> you take lots of stages and the differences of slot sizes into consideration.
> I’d like to see adaptive scheduler also supports fine-grained resource
> management. If there’s a good solution to the above issue, I’d love to help
> review the effort.
> Donatien Schmitz
> Dear Robert and Xintong, thanks for reading and reacting to my message! I'll
> reply tomorrow (GTM +1 time) if that's quite alright with you. Best, Donatien
> Schmitz
> Donatien Schmitz
> @Xintong Song
> * We are working on fine-grain scheduling for resource optimisation of long
> running or periodic jobs. One of the feature we are experiencing is a
> "rescheduling plan", a mapping of operators and Resource Profiles that can be
> dynamically applied to a running job. This rescheduling would be triggered by
> policies about some metrics (focus on RocksDB in our case).
> * While developing this new feature, we decided to implement it on the
> Adpative Scheduler instead of the Base Scheduler because the logic brought by
> the state machine already present made it more logical: transitions from
> states Executing -> Cancelling -> Rescheduling -> Waiting for Resources ->
> Creating -> Executing
> * In our case we are working on a POC and thus focusing on a real simple job
> with a // of 1. The issue you brought is indeed something we have faced while
> raising the // of the job.
> * If you create a Jira Ticket we can discuss it over there if you'd like!
> Donatien Schmitz
> @rmetzger The changes do not break the default resource management but does
> not fix the issue brought out by Xintong.
> {quote}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (FLINK-32671) Document Externalized Declarative Resource Management
Konstantin Knauf created FLINK-32671: Summary: Document Externalized Declarative Resource Management Key: FLINK-32671 URL: https://issues.apache.org/jira/browse/FLINK-32671 Project: Flink Issue Type: Sub-task Reporter: Konstantin Knauf -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (FLINK-31316) FLIP-291: Externalized Declarative Resource Management
[ https://issues.apache.org/jira/browse/FLINK-31316?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Konstantin Knauf reassigned FLINK-31316: Assignee: David Morávek (was: Konstantin Knauf) > FLIP-291: Externalized Declarative Resource Management > -- > > Key: FLINK-31316 > URL: https://issues.apache.org/jira/browse/FLINK-31316 > Project: Flink > Issue Type: New Feature > Components: Runtime / Coordination, Runtime / REST >Reporter: David Morávek >Assignee: David Morávek >Priority: Major > Fix For: 1.18.0 > > > This is an umbrella ticket for > [FLIP-291|https://cwiki.apache.org/confluence/display/FLINK/FLIP-291%3A+Externalized+Declarative+Resource+Management]. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (FLINK-31316) FLIP-291: Externalized Declarative Resource Management
[ https://issues.apache.org/jira/browse/FLINK-31316?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Konstantin Knauf reassigned FLINK-31316: Assignee: Konstantin Knauf (was: David Morávek) > FLIP-291: Externalized Declarative Resource Management > -- > > Key: FLINK-31316 > URL: https://issues.apache.org/jira/browse/FLINK-31316 > Project: Flink > Issue Type: New Feature > Components: Runtime / Coordination, Runtime / REST >Reporter: David Morávek >Assignee: Konstantin Knauf >Priority: Major > Fix For: 1.18.0 > > > This is an umbrella ticket for > [FLIP-291|https://cwiki.apache.org/confluence/display/FLINK/FLIP-291%3A+Externalized+Declarative+Resource+Management]. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (FLINK-32486) FLIP-324: Introduce Runtime Filter for Flink Batch Jobs
[ https://issues.apache.org/jira/browse/FLINK-32486?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17746942#comment-17746942 ] Konstantin Knauf commented on FLINK-32486: -- [~wanglijie] Tan Is there user facing documentation for this already? If not, is there a sub-task that tracks this? > FLIP-324: Introduce Runtime Filter for Flink Batch Jobs > --- > > Key: FLINK-32486 > URL: https://issues.apache.org/jira/browse/FLINK-32486 > Project: Flink > Issue Type: Improvement > Components: Table SQL / Planner, Table SQL / Runtime >Reporter: Lijie Wang >Assignee: Lijie Wang >Priority: Major > Fix For: 1.18.0 > > > This is an umbrella ticket for > [FLIP-324|https://cwiki.apache.org/confluence/display/FLINK/FLIP-324%3A+Introduce+Runtime+Filter+for+Flink+Batch+Jobs] -- This message was sent by Atlassian Jira (v8.20.10#820010)
