[jira] [Closed] (FLINK-36401) CI is not triggered for PRs
[ https://issues.apache.org/jira/browse/FLINK-36401?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xintong Song closed FLINK-36401. Resolution: Fixed > CI is not triggered for PRs > --- > > Key: FLINK-36401 > URL: https://issues.apache.org/jira/browse/FLINK-36401 > Project: Flink > Issue Type: Bug >Reporter: xuhuang >Assignee: Jing Ge >Priority: Blocker > > The PR CI has failed and does not trigger a CI when a new commit is pushed. > The failure may have begun on the afternoon of 24/09/27. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (FLINK-36401) CI is not triggered for PRs
[ https://issues.apache.org/jira/browse/FLINK-36401?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xintong Song reassigned FLINK-36401: Assignee: Jing Ge > CI is not triggered for PRs > --- > > Key: FLINK-36401 > URL: https://issues.apache.org/jira/browse/FLINK-36401 > Project: Flink > Issue Type: Bug >Reporter: xuhuang >Assignee: Jing Ge >Priority: Blocker > > The PR CI has failed and does not trigger a CI when a new commit is pushed. > The failure may have begun on the afternoon of 24/09/27. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (FLINK-36401) CI is not triggered for PRs
[ https://issues.apache.org/jira/browse/FLINK-36401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17885696#comment-17885696 ] Xintong Song commented on FLINK-36401: -- [~jingge] has helped re-started the ci-bot. It should work now. > CI is not triggered for PRs > --- > > Key: FLINK-36401 > URL: https://issues.apache.org/jira/browse/FLINK-36401 > Project: Flink > Issue Type: Bug >Reporter: xuhuang >Assignee: Jing Ge >Priority: Blocker > > The PR CI has failed and does not trigger a CI when a new commit is pushed. > The failure may have begun on the afternoon of 24/09/27. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (FLINK-36369) Move deprecated user-visible classes in table modules to the legacy package to make it easier to delete them later
[ https://issues.apache.org/jira/browse/FLINK-36369?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17885686#comment-17885686 ] Xintong Song commented on FLINK-36369: -- remove annotation in master: 74864b0b376cf22b6b85c68f0ea102809dd5ca99 > Move deprecated user-visible classes in table modules to the legacy package > to make it easier to delete them later > -- > > Key: FLINK-36369 > URL: https://issues.apache.org/jira/browse/FLINK-36369 > Project: Flink > Issue Type: Technical Debt > Components: Table SQL / API >Reporter: xuyang >Assignee: xuyang >Priority: Major > Labels: pull-request-available > Fix For: 2.0-preview > > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (FLINK-36401) CI is not triggered for PRs
[ https://issues.apache.org/jira/browse/FLINK-36401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17885652#comment-17885652 ] Xintong Song commented on FLINK-36401: -- It looks like the ci-bot has stopped mirroring new PRs to the flink-ci/flink repo. [~jingge], could you please help check the status of machine where the ci-bot runs? > CI is not triggered for PRs > --- > > Key: FLINK-36401 > URL: https://issues.apache.org/jira/browse/FLINK-36401 > Project: Flink > Issue Type: Bug >Reporter: xuhuang >Priority: Blocker > > The PR CI has failed and does not trigger a CI when a new commit is pushed. > The failure may have begun on the afternoon of 24/09/27. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (FLINK-36401) CI is not triggered for PRs
[ https://issues.apache.org/jira/browse/FLINK-36401?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xintong Song updated FLINK-36401: - Summary: CI is not triggered for PRs (was: Pull Request CI not trigger) > CI is not triggered for PRs > --- > > Key: FLINK-36401 > URL: https://issues.apache.org/jira/browse/FLINK-36401 > Project: Flink > Issue Type: Bug >Reporter: xuhuang >Priority: Blocker > > The PR CI has failed and does not trigger a CI when a new commit is pushed. > The failure may have begun on the afternoon of 24/09/27. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (FLINK-36360) Prepare release process and Scripts for the preview release
[ https://issues.apache.org/jira/browse/FLINK-36360?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xintong Song updated FLINK-36360: - Description: Flink Repo || ||Branch||Version||Tag (if any)|| |Regular|master|2.0-SNAPSHOT| | | |release-1.20|1.20-SNAPSHOT| | | |release-1.20-rc1|1.20.0|release-1.20.0| |Preview|master|2.0-SNAPSHOT| | | |2.0-preview1-rc1|2.0-preview1|release-2.0-preview1| Docs || ||Doc Version||Pointing Branch||Notes|| |Regular|1.20.X|release-1.20| | |Preview|2.0-previewX|2.0-preview1-rc1 (branch of the most recent preview & rc)|Should be removed once 2.0.0 is out| Docker ||Heading 1||Version||Branch||Notes|| |Regular|1.20.X|dev-1.20| | |Preview|2.0-previewX|dev-2.0|2.0.x should use the same branch| was: Flink Repo || ||Branch||Version||Tag (if any)|| |Regular|master|2.0-SNAPSHOT| | |release-1.20|1.20-SNAPSHOT| | |release-1.20-rc1|1.20.0|release-1.20.0| |Preview|master|2.0-SNAPSHOT| | |2.0-preview1-rc1|2.0-preview1|release-2.0-preview1| Docs || ||Doc Version||Pointing Branch||Notes|| |Regular|1.20.X|release-1.20| | |Preview|2.0-previewX|2.0-preview1-rc1 (branch of the most recent preview & rc)|Should be removed once 2.0.0 is out| Docker ||Heading 1||Version||Branch||Notes|| |Regular|1.20.X|dev-1.20| | |Preview|2.0-previewX|dev-2.0|2.0.x should use the same branch| > Prepare release process and Scripts for the preview release > --- > > Key: FLINK-36360 > URL: https://issues.apache.org/jira/browse/FLINK-36360 > Project: Flink > Issue Type: New Feature > Components: Release System >Reporter: Xintong Song >Assignee: Xintong Song >Priority: Blocker > Labels: pull-request-available > Fix For: 2.0-preview > > > Flink Repo > || ||Branch||Version||Tag (if any)|| > |Regular|master|2.0-SNAPSHOT| | > | |release-1.20|1.20-SNAPSHOT| | > | |release-1.20-rc1|1.20.0|release-1.20.0| > |Preview|master|2.0-SNAPSHOT| | > | |2.0-preview1-rc1|2.0-preview1|release-2.0-preview1| > > Docs > || ||Doc Version||Pointing Branch||Notes|| > |Regular|1.20.X|release-1.20| | > |Preview|2.0-previewX|2.0-preview1-rc1 (branch of the most recent preview & > rc)|Should be removed once 2.0.0 is out| > > Docker > ||Heading 1||Version||Branch||Notes|| > |Regular|1.20.X|dev-1.20| | > |Preview|2.0-previewX|dev-2.0|2.0.x should use the same branch| -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (FLINK-36360) Prepare release process and Scripts for the preview release
[ https://issues.apache.org/jira/browse/FLINK-36360?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xintong Song updated FLINK-36360: - Description: Flink Repo || ||Branch||Version||Tag (if any)|| |Regular|master|2.0-SNAPSHOT| | |release-1.20|1.20-SNAPSHOT| | |release-1.20-rc1|1.20.0|release-1.20.0| |Preview|master|2.0-SNAPSHOT| | |2.0-preview1-rc1|2.0-preview1|release-2.0-preview1| Docs || ||Doc Version||Pointing Branch||Notes|| |Regular|1.20.X|release-1.20| | |Preview|2.0-previewX|2.0-preview1-rc1 (branch of the most recent preview & rc)|Should be removed once 2.0.0 is out| Docker ||Heading 1||Version||Branch||Notes|| |Regular|1.20.X|dev-1.20| | |Preview|2.0-previewX|dev-2.0|2.0.x should use the same branch| was: Flink Repo || ||Branch||Version||Tag (if any)|| |Regular|master|2.0-SNAPSHOT| | |release-1.20|1.20-SNAPSHOT| | |release-1.20-rc1|1.20.0|release-1.20.0| |Preview|master|2.0-SNAPSHOT| | |2.0-preview1-rc1|2.0-preview1|release-2.0-preview1| Docs || ||Doc Version||Pointing Branch||Notes|| |Regular|1.20.X|release-1.20| | |Preview|2.0-previewX|2.0-preview1-rc1 (branch of the most recent preview & rc)|Should be removed once 2.0.0 is out| Docker ||Heading 1||Version||Branch||Notes|| |Regular|1.20.X|dev-1.20| | |Preview|2.0-previewX|dev-2.0|2.0.x should use the same branch| > Prepare release process and Scripts for the preview release > --- > > Key: FLINK-36360 > URL: https://issues.apache.org/jira/browse/FLINK-36360 > Project: Flink > Issue Type: New Feature > Components: Release System >Reporter: Xintong Song >Assignee: Xintong Song >Priority: Blocker > Fix For: 2.0-preview > > > Flink Repo > || ||Branch||Version||Tag (if any)|| > |Regular|master|2.0-SNAPSHOT| | > |release-1.20|1.20-SNAPSHOT| | > |release-1.20-rc1|1.20.0|release-1.20.0| > |Preview|master|2.0-SNAPSHOT| | > |2.0-preview1-rc1|2.0-preview1|release-2.0-preview1| > > Docs > || ||Doc Version||Pointing Branch||Notes|| > |Regular|1.20.X|release-1.20| | > |Preview|2.0-previewX|2.0-preview1-rc1 (branch of the most recent preview & > rc)|Should be removed once 2.0.0 is out| > > Docker > ||Heading 1||Version||Branch||Notes|| > |Regular|1.20.X|dev-1.20| | > |Preview|2.0-previewX|dev-2.0|2.0.x should use the same branch| -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (FLINK-36360) Prepare release process and Scripts for the preview release
[ https://issues.apache.org/jira/browse/FLINK-36360?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xintong Song updated FLINK-36360: - Description: Flink Repo || ||Branch||Version||Tag (if any)|| |Regular|master|2.0-SNAPSHOT| | |release-1.20|1.20-SNAPSHOT| | |release-1.20-rc1|1.20.0|release-1.20.0| |Preview|master|2.0-SNAPSHOT| | |2.0-preview1-rc1|2.0-preview1|release-2.0-preview1| Docs || ||Doc Version||Pointing Branch||Notes|| |Regular|1.20.X|release-1.20| | |Preview|2.0-previewX|2.0-preview1-rc1 (branch of the most recent preview & rc)|Should be removed once 2.0.0 is out| Docker ||Heading 1||Version||Branch||Notes|| |Regular|1.20.X|dev-1.20| | |Preview|2.0-previewX|dev-2.0|2.0.x should use the same branch| > Prepare release process and Scripts for the preview release > --- > > Key: FLINK-36360 > URL: https://issues.apache.org/jira/browse/FLINK-36360 > Project: Flink > Issue Type: New Feature > Components: Release System >Reporter: Xintong Song >Assignee: Xintong Song >Priority: Blocker > Fix For: 2.0-preview > > > Flink Repo > > || ||Branch||Version||Tag (if any)|| > |Regular|master|2.0-SNAPSHOT| | > |release-1.20|1.20-SNAPSHOT| | > |release-1.20-rc1|1.20.0|release-1.20.0| > |Preview|master|2.0-SNAPSHOT| | > |2.0-preview1-rc1|2.0-preview1|release-2.0-preview1| > > Docs > > || ||Doc Version||Pointing Branch||Notes|| > |Regular|1.20.X|release-1.20| | > |Preview|2.0-previewX|2.0-preview1-rc1 (branch of the most recent preview & > rc)|Should be removed once 2.0.0 is out| > Docker > ||Heading 1||Version||Branch||Notes|| > |Regular|1.20.X|dev-1.20| | > |Preview|2.0-previewX|dev-2.0|2.0.x should use the same branch| -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (FLINK-36360) Prepare release process and Scripts for the preview release
Xintong Song created FLINK-36360: Summary: Prepare release process and Scripts for the preview release Key: FLINK-36360 URL: https://issues.apache.org/jira/browse/FLINK-36360 Project: Flink Issue Type: New Feature Components: Release System Reporter: Xintong Song Assignee: Xintong Song Fix For: 2.0-preview -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (FLINK-36112) Add Support for CreateFlag.NO_LOCAL_WRITE in FLINK on YARN's File Creation to Manage Disk Space and Network Load in Labeled YARN Nodes
[
https://issues.apache.org/jira/browse/FLINK-36112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xintong Song updated FLINK-36112:
-
Component/s: Connectors / FileSystem
> Add Support for CreateFlag.NO_LOCAL_WRITE in FLINK on YARN's File Creation to
> Manage Disk Space and Network Load in Labeled YARN Nodes
> --
>
> Key: FLINK-36112
> URL: https://issues.apache.org/jira/browse/FLINK-36112
> Project: Flink
> Issue Type: Improvement
> Components: Connectors / FileSystem
>Reporter: liang yu
>Assignee: liang yu
>Priority: Major
> Labels: pull-request-available
> Fix For: 2.0-preview
>
> Attachments: image-2024-08-20-18-51-11-864.png
>
>
> {*}Description{*}: I am currently using Apache Flink to write files into
> Hadoop. The Flink application runs on a labeled YARN queue. During operation,
> it has been observed that the local disks on these labeled nodes get filled
> up quickly, and the network load is significantly high. This issue arises
> because Hadoop prioritizes writing files to the local node first, and the
> number of these labeled nodes is quite limited.
>
> {*}Problem{*}: The current behavior leads to inefficient disk space
> utilization and high network traffic on these few labeled nodes, which could
> potentially affect the performance and reliability of the application. As
> shown in the picture, the host I circled have a average net_bytes_sent speed
> 1.2GB/s while the others are just 50MB/s, this imbalance in network and disk
> space nearly destroyed the whole cluster.
>
> !image-2024-08-20-18-51-11-864.png|width=1056,height=301!
>
> {*}Implementation{*}: The implementation would involve adding a method of
> FileSystem.class to support the {{CreateFlag.NO_LOCAL_WRITE}} when we try to
> create a new file through HadoopFileSystem.create() API. What's more, I
> modify the code of FileSink class so that we can choose to enable
> no_local_write or disable this feature. This will provide flexibility to
> Flink running in labeled Yarn queues to opt for non-local writes when
> necessary.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Closed] (FLINK-36112) Add Support for CreateFlag.NO_LOCAL_WRITE in FLINK on YARN's File Creation to Manage Disk Space and Network Load in Labeled YARN Nodes
[
https://issues.apache.org/jira/browse/FLINK-36112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xintong Song closed FLINK-36112.
Fix Version/s: 2.0-preview
Assignee: liang yu
Resolution: Done
master (2.0): 5559126733bbb404536db29ed872decf58937e6e
> Add Support for CreateFlag.NO_LOCAL_WRITE in FLINK on YARN's File Creation to
> Manage Disk Space and Network Load in Labeled YARN Nodes
> --
>
> Key: FLINK-36112
> URL: https://issues.apache.org/jira/browse/FLINK-36112
> Project: Flink
> Issue Type: Improvement
>Reporter: liang yu
>Assignee: liang yu
>Priority: Major
> Labels: pull-request-available
> Fix For: 2.0-preview
>
> Attachments: image-2024-08-20-18-51-11-864.png
>
>
> {*}Description{*}: I am currently using Apache Flink to write files into
> Hadoop. The Flink application runs on a labeled YARN queue. During operation,
> it has been observed that the local disks on these labeled nodes get filled
> up quickly, and the network load is significantly high. This issue arises
> because Hadoop prioritizes writing files to the local node first, and the
> number of these labeled nodes is quite limited.
>
> {*}Problem{*}: The current behavior leads to inefficient disk space
> utilization and high network traffic on these few labeled nodes, which could
> potentially affect the performance and reliability of the application. As
> shown in the picture, the host I circled have a average net_bytes_sent speed
> 1.2GB/s while the others are just 50MB/s, this imbalance in network and disk
> space nearly destroyed the whole cluster.
>
> !image-2024-08-20-18-51-11-864.png|width=1056,height=301!
>
> {*}Implementation{*}: The implementation would involve adding a method of
> FileSystem.class to support the {{CreateFlag.NO_LOCAL_WRITE}} when we try to
> create a new file through HadoopFileSystem.create() API. What's more, I
> modify the code of FileSink class so that we can choose to enable
> no_local_write or disable this feature. This will provide flexibility to
> Flink running in labeled Yarn queues to opt for non-local writes when
> necessary.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (FLINK-36261) Revisit all breaking changes before Flink 2.0 Preview
Xintong Song created FLINK-36261: Summary: Revisit all breaking changes before Flink 2.0 Preview Key: FLINK-36261 URL: https://issues.apache.org/jira/browse/FLINK-36261 Project: Flink Issue Type: Technical Debt Reporter: Xintong Song Fix For: 2.0-preview Japicmp for @Deprecated APIs is disabled in FLINK-36207. We need to check whether there's any unexpected breaking changes right before the Flink 2.0 Preview release, by re-enable the checking locally. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (FLINK-36217) Remove powermock usage
[ https://issues.apache.org/jira/browse/FLINK-36217?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xintong Song updated FLINK-36217: - Fix Version/s: 2.0-preview (was: 2.0.0) > Remove powermock usage > -- > > Key: FLINK-36217 > URL: https://issues.apache.org/jira/browse/FLINK-36217 > Project: Flink > Issue Type: Technical Debt > Components: Tests >Reporter: Sergey Nuyanzin >Assignee: Sergey Nuyanzin >Priority: Major > Labels: pull-request-available > Fix For: 2.0-preview > > > Most of the tests are either moved to a different repo like connectors or > rewritten in powermock free way. > Powermock itself became unmaintained (latest release was in 2020 > https://github.com/powermock/powermock/releases/tag/powermock-2.0.9) > and latest commit 2 years ago https://github.com/powermock/powermock > also there is no support for junit5 (the request to support it and even PR > from junit5 maintainers is ready for review since Feb 2023 > https://github.com/powermock/powermock/pull/1146, however still no feedback > from maintainers...) -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (FLINK-36234) Add 1.20 to PreviousDocs list
[ https://issues.apache.org/jira/browse/FLINK-36234?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xintong Song updated FLINK-36234: - Fix Version/s: 2.0-preview (was: 2.0.0) > Add 1.20 to PreviousDocs list > - > > Key: FLINK-36234 > URL: https://issues.apache.org/jira/browse/FLINK-36234 > Project: Flink > Issue Type: Technical Debt > Components: Documentation >Affects Versions: 2.0.0 >Reporter: Aleksandr Pilipenko >Assignee: Aleksandr Pilipenko >Priority: Major > Labels: pull-request-available > Fix For: 2.0-preview > > > Documentation for 2.0-SNAPSHOT is missing 1.20 from all versions list, as > well as from version picker. > [https://nightlies.apache.org/flink/flink-docs-master/versions/] > > Reported in mailing list: > https://lists.apache.org/thread/8g0hwk5lxly38vpcqwhd1hcy6djv9rq6 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (FLINK-36201) StateLocalitySlotAssigner should be only used when local recovery is enabled for Adaptive Scheduler
[ https://issues.apache.org/jira/browse/FLINK-36201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17879439#comment-17879439 ] Xintong Song commented on FLINK-36201: -- Hi [~dmvk], for code changes merged before the preview release, please mark the FixVersion as 2.0-preview. > StateLocalitySlotAssigner should be only used when local recovery is enabled > for Adaptive Scheduler > --- > > Key: FLINK-36201 > URL: https://issues.apache.org/jira/browse/FLINK-36201 > Project: Flink > Issue Type: Bug > Components: Runtime / Coordination >Reporter: Rui Fan >Assignee: Rui Fan >Priority: Major > Labels: pull-request-available > Fix For: 2.0-preview > > > SlotSharingSlotAllocator created the StateLocalitySlotAssigner[1] instead of > DefaultSlotAssigner whenever failover happens. > I'm curious why we use StateLocalitySlotAssigner when local recovery is > disable. > As I understand, the local recovery doesn't take effect if flink doesn't > backup state on the TM local disk. So StateLocalitySlotAssigner should be > only used when local recovery is enabled. > > [1] > [https://github.com/apache/flink/blob/c869326d089705475481c2c2ea42a6efabb8c828/flink-runtime/src/main/java/org/apache/flink/runtime/scheduler/adaptive/allocator/SlotSharingSlotAllocator.java#L136] -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (FLINK-36201) StateLocalitySlotAssigner should be only used when local recovery is enabled for Adaptive Scheduler
[ https://issues.apache.org/jira/browse/FLINK-36201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xintong Song updated FLINK-36201: - Fix Version/s: 2.0-preview (was: 2.0.0) > StateLocalitySlotAssigner should be only used when local recovery is enabled > for Adaptive Scheduler > --- > > Key: FLINK-36201 > URL: https://issues.apache.org/jira/browse/FLINK-36201 > Project: Flink > Issue Type: Bug > Components: Runtime / Coordination >Reporter: Rui Fan >Assignee: Rui Fan >Priority: Major > Labels: pull-request-available > Fix For: 2.0-preview > > > SlotSharingSlotAllocator created the StateLocalitySlotAssigner[1] instead of > DefaultSlotAssigner whenever failover happens. > I'm curious why we use StateLocalitySlotAssigner when local recovery is > disable. > As I understand, the local recovery doesn't take effect if flink doesn't > backup state on the TM local disk. So StateLocalitySlotAssigner should be > only used when local recovery is enabled. > > [1] > [https://github.com/apache/flink/blob/c869326d089705475481c2c2ea42a6efabb8c828/flink-runtime/src/main/java/org/apache/flink/runtime/scheduler/adaptive/allocator/SlotSharingSlotAllocator.java#L136] -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (FLINK-36011) Generalize RescaleManager to become StateTransitionManager
[ https://issues.apache.org/jira/browse/FLINK-36011?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17878741#comment-17878741 ] Xintong Song commented on FLINK-36011: -- hi [~mapohl], for tickets resolved between Flink 1.20 and the preview version of Flink 2.0, please mark them as fixed in 2.0-preview. > Generalize RescaleManager to become StateTransitionManager > -- > > Key: FLINK-36011 > URL: https://issues.apache.org/jira/browse/FLINK-36011 > Project: Flink > Issue Type: Sub-task > Components: Runtime / Coordination >Reporter: Zdenek Tison >Assignee: Zdenek Tison >Priority: Major > Labels: pull-request-available > Fix For: 2.0-preview > > > The goal is to change the RescaleManager component to one with a broader > responsibility that will manage the adaptive scheduler's state transitions. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (FLINK-36011) Generalize RescaleManager to become StateTransitionManager
[ https://issues.apache.org/jira/browse/FLINK-36011?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xintong Song updated FLINK-36011: - Fix Version/s: 2.0-preview (was: 2.0.0) > Generalize RescaleManager to become StateTransitionManager > -- > > Key: FLINK-36011 > URL: https://issues.apache.org/jira/browse/FLINK-36011 > Project: Flink > Issue Type: Sub-task > Components: Runtime / Coordination >Reporter: Zdenek Tison >Assignee: Zdenek Tison >Priority: Major > Labels: pull-request-available > Fix For: 2.0-preview > > > The goal is to change the RescaleManager component to one with a broader > responsibility that will manage the adaptive scheduler's state transitions. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Closed] (FLINK-32378) 2.0 Breaking Metric system changes
[ https://issues.apache.org/jira/browse/FLINK-32378?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xintong Song closed FLINK-32378. Fix Version/s: (was: 2.0.0) Resolution: Invalid All subtasks are migrated to 3.0.0 > 2.0 Breaking Metric system changes > -- > > Key: FLINK-32378 > URL: https://issues.apache.org/jira/browse/FLINK-32378 > Project: Flink > Issue Type: Technical Debt > Components: Runtime / Metrics >Reporter: Chesnay Schepler >Assignee: Chesnay Schepler >Priority: Major > > Umbrella issue for all breaking changes to the metric system -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (FLINK-24078) Remove brackets around variables
[
https://issues.apache.org/jira/browse/FLINK-24078?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xintong Song updated FLINK-24078:
-
Labels: (was: 2.0-related)
> Remove brackets around variables
>
>
> Key: FLINK-24078
> URL: https://issues.apache.org/jira/browse/FLINK-24078
> Project: Flink
> Issue Type: Sub-task
> Components: Runtime / Metrics
>Reporter: Chesnay Schepler
>Assignee: Chesnay Schepler
>Priority: Major
> Fix For: 3.0.0
>
>
> Internally, variable keys are stored with brackets, e.g., {{}}.
> In practice all reporters will filter these characters in one way or the
> other, and it overall is a subtle trap that we keep running into without
> providing any benefit.
> We should get rid of them.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (FLINK-24078) Remove brackets around variables
[
https://issues.apache.org/jira/browse/FLINK-24078?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xintong Song updated FLINK-24078:
-
Parent Issue: FLINK-36200 (was: FLINK-32378)
> Remove brackets around variables
>
>
> Key: FLINK-24078
> URL: https://issues.apache.org/jira/browse/FLINK-24078
> Project: Flink
> Issue Type: Sub-task
> Components: Runtime / Metrics
>Reporter: Chesnay Schepler
>Assignee: Chesnay Schepler
>Priority: Major
> Fix For: 3.0.0
>
>
> Internally, variable keys are stored with brackets, e.g., {{}}.
> In practice all reporters will filter these characters in one way or the
> other, and it overall is a subtle trap that we keep running into without
> providing any benefit.
> We should get rid of them.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (FLINK-24078) Remove brackets around variables
[
https://issues.apache.org/jira/browse/FLINK-24078?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xintong Song updated FLINK-24078:
-
Fix Version/s: 3.0.0
(was: 2.0-preview)
> Remove brackets around variables
>
>
> Key: FLINK-24078
> URL: https://issues.apache.org/jira/browse/FLINK-24078
> Project: Flink
> Issue Type: Sub-task
> Components: Runtime / Metrics
>Reporter: Chesnay Schepler
>Assignee: Chesnay Schepler
>Priority: Major
> Labels: 2.0-related
> Fix For: 3.0.0
>
>
> Internally, variable keys are stored with brackets, e.g., {{}}.
> In practice all reporters will filter these characters in one way or the
> other, and it overall is a subtle trap that we keep running into without
> providing any benefit.
> We should get rid of them.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (FLINK-6053) Gauge should only take subclasses of Number, rather than everything
[
https://issues.apache.org/jira/browse/FLINK-6053?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xintong Song updated FLINK-6053:
Parent Issue: FLINK-36200 (was: FLINK-32378)
> Gauge should only take subclasses of Number, rather than everything
> --
>
> Key: FLINK-6053
> URL: https://issues.apache.org/jira/browse/FLINK-6053
> Project: Flink
> Issue Type: Sub-task
> Components: Runtime / Metrics
>Affects Versions: 1.2.0
>Reporter: Bowen Li
>Priority: Minor
> Labels: pull-request-available
> Fix For: 3.0.0
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Currently, Flink's Gauge is defined as
> ```java
> public interface Gauge extends Metric {
> T getValue();
> }
> ```
> But it doesn't make sense to have Gauge take generic types other than Number.
> And it blocks I from finishing FLINK-6013, because I cannot assume Gauge is
> only about Number. So the class should be like
> ```java
> public interface Gauge extends Metric {
> T getValue();
> }
> ```
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (FLINK-6053) Gauge should only take subclasses of Number, rather than everything
[
https://issues.apache.org/jira/browse/FLINK-6053?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xintong Song updated FLINK-6053:
Labels: pull-request-available (was: 2.0-related pull-request-available)
> Gauge should only take subclasses of Number, rather than everything
> --
>
> Key: FLINK-6053
> URL: https://issues.apache.org/jira/browse/FLINK-6053
> Project: Flink
> Issue Type: Sub-task
> Components: Runtime / Metrics
>Affects Versions: 1.2.0
>Reporter: Bowen Li
>Priority: Minor
> Labels: pull-request-available
> Fix For: 3.0.0
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Currently, Flink's Gauge is defined as
> ```java
> public interface Gauge extends Metric {
> T getValue();
> }
> ```
> But it doesn't make sense to have Gauge take generic types other than Number.
> And it blocks I from finishing FLINK-6013, because I cannot assume Gauge is
> only about Number. So the class should be like
> ```java
> public interface Gauge extends Metric {
> T getValue();
> }
> ```
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (FLINK-6053) Gauge should only take subclasses of Number, rather than everything
[
https://issues.apache.org/jira/browse/FLINK-6053?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xintong Song updated FLINK-6053:
Fix Version/s: 3.0.0
(was: 2.0.0)
> Gauge should only take subclasses of Number, rather than everything
> --
>
> Key: FLINK-6053
> URL: https://issues.apache.org/jira/browse/FLINK-6053
> Project: Flink
> Issue Type: Sub-task
> Components: Runtime / Metrics
>Affects Versions: 1.2.0
>Reporter: Bowen Li
>Priority: Minor
> Labels: 2.0-related, pull-request-available
> Fix For: 3.0.0
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Currently, Flink's Gauge is defined as
> ```java
> public interface Gauge extends Metric {
> T getValue();
> }
> ```
> But it doesn't make sense to have Gauge take generic types other than Number.
> And it blocks I from finishing FLINK-6013, because I cannot assume Gauge is
> only about Number. So the class should be like
> ```java
> public interface Gauge extends Metric {
> T getValue();
> }
> ```
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (FLINK-7957) Add MetricGroup#getLogicalScope
[
https://issues.apache.org/jira/browse/FLINK-7957?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xintong Song updated FLINK-7957:
Labels: auto-deprioritized-minor auto-unassigned (was: 2.0-related
auto-deprioritized-minor auto-unassigned)
> Add MetricGroup#getLogicalScope
> ---
>
> Key: FLINK-7957
> URL: https://issues.apache.org/jira/browse/FLINK-7957
> Project: Flink
> Issue Type: Sub-task
> Components: Runtime / Metrics
>Affects Versions: 1.4.0
>Reporter: Chesnay Schepler
>Priority: Not a Priority
> Labels: auto-deprioritized-minor, auto-unassigned
> Fix For: 3.0.0
>
>
> Various reporters make use of a generalized scope string (e.g.
> "taskmanager.job.task."). This string can be generated with
> {{AbstractMetricGroup#getLogicalScope}}, which is however an internal API. As
> a result, the access pattern currently looks like this:
> {code}
> ((FrontMetricGroup>)group).getLogicalScope(CHARACTER_FILTER,
> '.')
> {code}
> Given the wide-spread of this kind of scope i propose to move it into the
> MetricGroup interface.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (FLINK-7957) Add MetricGroup#getLogicalScope
[
https://issues.apache.org/jira/browse/FLINK-7957?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xintong Song updated FLINK-7957:
Parent Issue: FLINK-36200 (was: FLINK-32378)
> Add MetricGroup#getLogicalScope
> ---
>
> Key: FLINK-7957
> URL: https://issues.apache.org/jira/browse/FLINK-7957
> Project: Flink
> Issue Type: Sub-task
> Components: Runtime / Metrics
>Affects Versions: 1.4.0
>Reporter: Chesnay Schepler
>Priority: Not a Priority
> Labels: 2.0-related, auto-deprioritized-minor, auto-unassigned
> Fix For: 3.0.0
>
>
> Various reporters make use of a generalized scope string (e.g.
> "taskmanager.job.task."). This string can be generated with
> {{AbstractMetricGroup#getLogicalScope}}, which is however an internal API. As
> a result, the access pattern currently looks like this:
> {code}
> ((FrontMetricGroup>)group).getLogicalScope(CHARACTER_FILTER,
> '.')
> {code}
> Given the wide-spread of this kind of scope i propose to move it into the
> MetricGroup interface.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (FLINK-7957) Add MetricGroup#getLogicalScope
[
https://issues.apache.org/jira/browse/FLINK-7957?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xintong Song updated FLINK-7957:
Fix Version/s: 3.0.0
> Add MetricGroup#getLogicalScope
> ---
>
> Key: FLINK-7957
> URL: https://issues.apache.org/jira/browse/FLINK-7957
> Project: Flink
> Issue Type: Sub-task
> Components: Runtime / Metrics
>Affects Versions: 1.4.0
>Reporter: Chesnay Schepler
>Priority: Not a Priority
> Labels: 2.0-related, auto-deprioritized-minor, auto-unassigned
> Fix For: 3.0.0
>
>
> Various reporters make use of a generalized scope string (e.g.
> "taskmanager.job.task."). This string can be generated with
> {{AbstractMetricGroup#getLogicalScope}}, which is however an internal API. As
> a result, the access pattern currently looks like this:
> {code}
> ((FrontMetricGroup>)group).getLogicalScope(CHARACTER_FILTER,
> '.')
> {code}
> Given the wide-spread of this kind of scope i propose to move it into the
> MetricGroup interface.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (FLINK-14350) Introduce dedicated MetricScope
[ https://issues.apache.org/jira/browse/FLINK-14350?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xintong Song updated FLINK-14350: - Parent Issue: FLINK-36200 (was: FLINK-32378) > Introduce dedicated MetricScope > --- > > Key: FLINK-14350 > URL: https://issues.apache.org/jira/browse/FLINK-14350 > Project: Flink > Issue Type: Sub-task > Components: Runtime / Metrics >Reporter: Chesnay Schepler >Priority: Not a Priority > Labels: auto-deprioritized-major, auto-deprioritized-minor, > auto-unassigned, pull-request-available > Fix For: 3.0.0 > > Time Spent: 10m > Remaining Estimate: 0h > > The MetricGroup interface is currently overloaded, being used both for > registering groups/metrics (typically called from user-functions) and > accessing scope information (typically called from reporters). > Due to effectively having 2 different audiences with different use-cases it > makes sense to move the scope-related methods into a separate interface. > This should make it a lot easier to extend these interfaces, as significantly > less tests have to be adjusted. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (FLINK-14350) Introduce dedicated MetricScope
[ https://issues.apache.org/jira/browse/FLINK-14350?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xintong Song updated FLINK-14350: - Fix Version/s: 3.0.0 > Introduce dedicated MetricScope > --- > > Key: FLINK-14350 > URL: https://issues.apache.org/jira/browse/FLINK-14350 > Project: Flink > Issue Type: Sub-task > Components: Runtime / Metrics >Reporter: Chesnay Schepler >Priority: Not a Priority > Labels: 2.0-related, auto-deprioritized-major, > auto-deprioritized-minor, auto-unassigned, pull-request-available > Fix For: 3.0.0 > > Time Spent: 10m > Remaining Estimate: 0h > > The MetricGroup interface is currently overloaded, being used both for > registering groups/metrics (typically called from user-functions) and > accessing scope information (typically called from reporters). > Due to effectively having 2 different audiences with different use-cases it > makes sense to move the scope-related methods into a separate interface. > This should make it a lot easier to extend these interfaces, as significantly > less tests have to be adjusted. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (FLINK-14350) Introduce dedicated MetricScope
[ https://issues.apache.org/jira/browse/FLINK-14350?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xintong Song updated FLINK-14350: - Labels: auto-deprioritized-major auto-deprioritized-minor auto-unassigned pull-request-available (was: 2.0-related auto-deprioritized-major auto-deprioritized-minor auto-unassigned pull-request-available) > Introduce dedicated MetricScope > --- > > Key: FLINK-14350 > URL: https://issues.apache.org/jira/browse/FLINK-14350 > Project: Flink > Issue Type: Sub-task > Components: Runtime / Metrics >Reporter: Chesnay Schepler >Priority: Not a Priority > Labels: auto-deprioritized-major, auto-deprioritized-minor, > auto-unassigned, pull-request-available > Fix For: 3.0.0 > > Time Spent: 10m > Remaining Estimate: 0h > > The MetricGroup interface is currently overloaded, being used both for > registering groups/metrics (typically called from user-functions) and > accessing scope information (typically called from reporters). > Due to effectively having 2 different audiences with different use-cases it > makes sense to move the scope-related methods into a separate interface. > This should make it a lot easier to extend these interfaces, as significantly > less tests have to be adjusted. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (FLINK-32597) Drop Yarn specific get rest endpoints
[ https://issues.apache.org/jira/browse/FLINK-32597?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xintong Song updated FLINK-32597: - Parent: FLINK-36200 Issue Type: Sub-task (was: Improvement) > Drop Yarn specific get rest endpoints > -- > > Key: FLINK-32597 > URL: https://issues.apache.org/jira/browse/FLINK-32597 > Project: Flink > Issue Type: Sub-task > Components: Deployment / YARN >Affects Versions: 1.18.0 >Reporter: Zhenqiu Huang >Assignee: Zhenqiu Huang >Priority: Minor > Fix For: 3.0.0 > > > As listed in in the 2.0 release, we need to Drop YARN-specific mutating GET > REST endpoints (yarn-cancel, yarn-stop) > We shouldn't continue having such hacks in our APIs to work around YARN > deficiencies. > https://cwiki.apache.org/confluence/display/FLINK/2.0+Release -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (FLINK-32597) Drop Yarn specific get rest endpoints
[ https://issues.apache.org/jira/browse/FLINK-32597?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xintong Song updated FLINK-32597: - Fix Version/s: 3.0.0 > Drop Yarn specific get rest endpoints > -- > > Key: FLINK-32597 > URL: https://issues.apache.org/jira/browse/FLINK-32597 > Project: Flink > Issue Type: Improvement > Components: Deployment / YARN >Affects Versions: 1.18.0 >Reporter: Zhenqiu Huang >Assignee: Zhenqiu Huang >Priority: Minor > Fix For: 3.0.0 > > > As listed in in the 2.0 release, we need to Drop YARN-specific mutating GET > REST endpoints (yarn-cancel, yarn-stop) > We shouldn't continue having such hacks in our APIs to work around YARN > deficiencies. > https://cwiki.apache.org/confluence/display/FLINK/2.0+Release -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (FLINK-32597) Drop Yarn specific get rest endpoints
[ https://issues.apache.org/jira/browse/FLINK-32597?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xintong Song updated FLINK-32597: - Labels: (was: 2.0-related) > Drop Yarn specific get rest endpoints > -- > > Key: FLINK-32597 > URL: https://issues.apache.org/jira/browse/FLINK-32597 > Project: Flink > Issue Type: Improvement > Components: Deployment / YARN >Affects Versions: 1.18.0 >Reporter: Zhenqiu Huang >Assignee: Zhenqiu Huang >Priority: Minor > > As listed in in the 2.0 release, we need to Drop YARN-specific mutating GET > REST endpoints (yarn-cancel, yarn-stop) > We shouldn't continue having such hacks in our APIs to work around YARN > deficiencies. > https://cwiki.apache.org/confluence/display/FLINK/2.0+Release -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (FLINK-32688) Remove deprecated exception history fields
[ https://issues.apache.org/jira/browse/FLINK-32688?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xintong Song updated FLINK-32688: - Priority: Blocker (was: Major) > Remove deprecated exception history fields > -- > > Key: FLINK-32688 > URL: https://issues.apache.org/jira/browse/FLINK-32688 > Project: Flink > Issue Type: Sub-task > Components: Runtime / REST >Affects Versions: 1.18.0 >Reporter: Matthias Pohl >Assignee: Matthias Pohl >Priority: Blocker > Labels: 2.0-related, pull-request-available > Fix For: 2.0-preview > > > The fields were already marked as deprecated (see > [JobExceptionInfo|https://github.com/apache/flink/blob/a49f1aaec6239401cc8b1dac731d290e95290caf/flink-runtime/src/main/java/org/apache/flink/runtime/rest/messages/JobExceptionsInfo.java#L35]) > but were not discussed as part of a FLIP. Working on this issue would > require creating a FLIP to cover the REST API change. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (FLINK-32377) 2.0 Breaking REST API changes
[ https://issues.apache.org/jira/browse/FLINK-32377?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xintong Song updated FLINK-32377: - Priority: Blocker (was: Major) > 2.0 Breaking REST API changes > - > > Key: FLINK-32377 > URL: https://issues.apache.org/jira/browse/FLINK-32377 > Project: Flink > Issue Type: Technical Debt > Components: Runtime / REST >Reporter: Chesnay Schepler >Assignee: Chesnay Schepler >Priority: Blocker > Fix For: 2.0.0 > > > Umbrella issue for all breaking changes to the REST API. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (FLINK-32377) 2.0 Breaking REST API changes
[ https://issues.apache.org/jira/browse/FLINK-32377?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xintong Song updated FLINK-32377: - Fix Version/s: 2.0-preview (was: 2.0.0) > 2.0 Breaking REST API changes > - > > Key: FLINK-32377 > URL: https://issues.apache.org/jira/browse/FLINK-32377 > Project: Flink > Issue Type: Technical Debt > Components: Runtime / REST >Reporter: Chesnay Schepler >Assignee: Chesnay Schepler >Priority: Blocker > Fix For: 2.0-preview > > > Umbrella issue for all breaking changes to the REST API. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (FLINK-32198) Enforce single maxExceptions query parameter
[ https://issues.apache.org/jira/browse/FLINK-32198?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xintong Song updated FLINK-32198: - Parent Issue: FLINK-36200 (was: FLINK-32377) > Enforce single maxExceptions query parameter > > > Key: FLINK-32198 > URL: https://issues.apache.org/jira/browse/FLINK-32198 > Project: Flink > Issue Type: Sub-task > Components: Runtime / REST >Reporter: Panagiotis Garefalakis >Assignee: Panagiotis Garefalakis >Priority: Major > Labels: pull-request-available > Fix For: 3.0.0 > > > While working on FLINK-31894 I realized that `UpperLimitExceptionParameter` > allows multiple values to be collected as a comma separated list even though > JobExceptionsHandler is only using the first one > [https://github.com/apache/flink/blob/1293958652053c0d163fde28e8dfefb5ee8f6101/flink-runtime/src/main/java/org/apache/flink/runtime/rest/handler/job/JobExceptionsHandler.java#L101-L104] > A better approach would be to deny multiple `maxExceptions` params and let > the users know. > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (FLINK-15787) Upgrade REST API response to remove '-' from key names.
[
https://issues.apache.org/jira/browse/FLINK-15787?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xintong Song updated FLINK-15787:
-
Parent Issue: FLINK-36200 (was: FLINK-32377)
> Upgrade REST API response to remove '-' from key names.
> ---
>
> Key: FLINK-15787
> URL: https://issues.apache.org/jira/browse/FLINK-15787
> Project: Flink
> Issue Type: Sub-task
> Components: Runtime / REST
>Affects Versions: 1.10.0
>Reporter: Daryl Roberts
>Priority: Not a Priority
> Labels: auto-deprioritized-major, auto-deprioritized-minor,
> auto-unassigned
> Fix For: 3.0.0
>
>
> There are some REST API responses that include keys with hyphens in them.
> This results in the frontend having to use string-lookups to access those
> values and we lose the type information when doing that.
> Example from {{/jobs//vertices//backpressure}}
> {{export interface JobBackpressureInterface {}}
> {{ status: string;}}
> {{ 'backpressure-level': string;}}
> {{ 'end-timestamp': number;}}
> {{ subtasks: JobBackpressureSubtaskInterface[];}}
> {{}}}
> I would like to update all of these to use {{_}} instead so we can maintain
> the type information we have in the web-runtime.
> My suggestion to do this with out a version bump to the API is to just make
> an addition to all the enpoints that include the _ versions as well. Then
> after the web-runtime has completely switched over to using the new keys, you
> can deprecate and remove the old hypenated keys at your own pace.
> [~chesnay] [~trohrmann]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (FLINK-29863) Properly handle NaN/Infinity in OpenAPI spec
[ https://issues.apache.org/jira/browse/FLINK-29863?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xintong Song updated FLINK-29863: - Parent Issue: FLINK-36200 (was: FLINK-32377) > Properly handle NaN/Infinity in OpenAPI spec > > > Key: FLINK-29863 > URL: https://issues.apache.org/jira/browse/FLINK-29863 > Project: Flink > Issue Type: Sub-task > Components: Documentation, Runtime / REST >Affects Versions: 1.15.0 >Reporter: Chesnay Schepler >Priority: Major > Fix For: 3.0.0 > > > Our OpenAPI spec maps all float/double fields to float64, but we at times > also return NaN/infinity which can't be represented as such since the JSON > spec doesn't support it. > One alternative could be to document it as an either type, returning either a > float64 or a string. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (FLINK-32198) Enforce single maxExceptions query parameter
[ https://issues.apache.org/jira/browse/FLINK-32198?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xintong Song updated FLINK-32198: - Labels: pull-request-available (was: 2.0-related pull-request-available) > Enforce single maxExceptions query parameter > > > Key: FLINK-32198 > URL: https://issues.apache.org/jira/browse/FLINK-32198 > Project: Flink > Issue Type: Sub-task > Components: Runtime / REST >Reporter: Panagiotis Garefalakis >Assignee: Panagiotis Garefalakis >Priority: Major > Labels: pull-request-available > Fix For: 3.0.0 > > > While working on FLINK-31894 I realized that `UpperLimitExceptionParameter` > allows multiple values to be collected as a comma separated list even though > JobExceptionsHandler is only using the first one > [https://github.com/apache/flink/blob/1293958652053c0d163fde28e8dfefb5ee8f6101/flink-runtime/src/main/java/org/apache/flink/runtime/rest/handler/job/JobExceptionsHandler.java#L101-L104] > A better approach would be to deny multiple `maxExceptions` params and let > the users know. > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (FLINK-32198) Enforce single maxExceptions query parameter
[ https://issues.apache.org/jira/browse/FLINK-32198?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xintong Song updated FLINK-32198: - Fix Version/s: 3.0.0 (was: 2.0.0) > Enforce single maxExceptions query parameter > > > Key: FLINK-32198 > URL: https://issues.apache.org/jira/browse/FLINK-32198 > Project: Flink > Issue Type: Sub-task > Components: Runtime / REST >Reporter: Panagiotis Garefalakis >Assignee: Panagiotis Garefalakis >Priority: Major > Labels: 2.0-related, pull-request-available > Fix For: 3.0.0 > > > While working on FLINK-31894 I realized that `UpperLimitExceptionParameter` > allows multiple values to be collected as a comma separated list even though > JobExceptionsHandler is only using the first one > [https://github.com/apache/flink/blob/1293958652053c0d163fde28e8dfefb5ee8f6101/flink-runtime/src/main/java/org/apache/flink/runtime/rest/handler/job/JobExceptionsHandler.java#L101-L104] > A better approach would be to deny multiple `maxExceptions` params and let > the users know. > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (FLINK-29863) Properly handle NaN/Infinity in OpenAPI spec
[ https://issues.apache.org/jira/browse/FLINK-29863?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xintong Song updated FLINK-29863: - Labels: (was: 2.0-related) > Properly handle NaN/Infinity in OpenAPI spec > > > Key: FLINK-29863 > URL: https://issues.apache.org/jira/browse/FLINK-29863 > Project: Flink > Issue Type: Sub-task > Components: Documentation, Runtime / REST >Affects Versions: 1.15.0 >Reporter: Chesnay Schepler >Priority: Major > Fix For: 3.0.0 > > > Our OpenAPI spec maps all float/double fields to float64, but we at times > also return NaN/infinity which can't be represented as such since the JSON > spec doesn't support it. > One alternative could be to document it as an either type, returning either a > float64 or a string. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (FLINK-29863) Properly handle NaN/Infinity in OpenAPI spec
[ https://issues.apache.org/jira/browse/FLINK-29863?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xintong Song updated FLINK-29863: - Fix Version/s: 3.0.0 (was: 2.0.0) > Properly handle NaN/Infinity in OpenAPI spec > > > Key: FLINK-29863 > URL: https://issues.apache.org/jira/browse/FLINK-29863 > Project: Flink > Issue Type: Sub-task > Components: Documentation, Runtime / REST >Affects Versions: 1.15.0 >Reporter: Chesnay Schepler >Priority: Major > Labels: 2.0-related > Fix For: 3.0.0 > > > Our OpenAPI spec maps all float/double fields to float64, but we at times > also return NaN/infinity which can't be represented as such since the JSON > spec doesn't support it. > One alternative could be to document it as an either type, returning either a > float64 or a string. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (FLINK-15787) Upgrade REST API response to remove '-' from key names.
[
https://issues.apache.org/jira/browse/FLINK-15787?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xintong Song updated FLINK-15787:
-
Fix Version/s: 3.0.0
> Upgrade REST API response to remove '-' from key names.
> ---
>
> Key: FLINK-15787
> URL: https://issues.apache.org/jira/browse/FLINK-15787
> Project: Flink
> Issue Type: Sub-task
> Components: Runtime / REST
>Affects Versions: 1.10.0
>Reporter: Daryl Roberts
>Priority: Not a Priority
> Labels: auto-deprioritized-major, auto-deprioritized-minor,
> auto-unassigned
> Fix For: 3.0.0
>
>
> There are some REST API responses that include keys with hyphens in them.
> This results in the frontend having to use string-lookups to access those
> values and we lose the type information when doing that.
> Example from {{/jobs//vertices//backpressure}}
> {{export interface JobBackpressureInterface {}}
> {{ status: string;}}
> {{ 'backpressure-level': string;}}
> {{ 'end-timestamp': number;}}
> {{ subtasks: JobBackpressureSubtaskInterface[];}}
> {{}}}
> I would like to update all of these to use {{_}} instead so we can maintain
> the type information we have in the web-runtime.
> My suggestion to do this with out a version bump to the API is to just make
> an addition to all the enpoints that include the _ versions as well. Then
> after the web-runtime has completely switched over to using the new keys, you
> can deprecate and remove the old hypenated keys at your own pace.
> [~chesnay] [~trohrmann]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (FLINK-15787) Upgrade REST API response to remove '-' from key names.
[
https://issues.apache.org/jira/browse/FLINK-15787?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xintong Song updated FLINK-15787:
-
Labels: auto-deprioritized-major auto-deprioritized-minor auto-unassigned
(was: 2.0-related auto-deprioritized-major auto-deprioritized-minor
auto-unassigned)
> Upgrade REST API response to remove '-' from key names.
> ---
>
> Key: FLINK-15787
> URL: https://issues.apache.org/jira/browse/FLINK-15787
> Project: Flink
> Issue Type: Sub-task
> Components: Runtime / REST
>Affects Versions: 1.10.0
>Reporter: Daryl Roberts
>Priority: Not a Priority
> Labels: auto-deprioritized-major, auto-deprioritized-minor,
> auto-unassigned
>
> There are some REST API responses that include keys with hyphens in them.
> This results in the frontend having to use string-lookups to access those
> values and we lose the type information when doing that.
> Example from {{/jobs//vertices//backpressure}}
> {{export interface JobBackpressureInterface {}}
> {{ status: string;}}
> {{ 'backpressure-level': string;}}
> {{ 'end-timestamp': number;}}
> {{ subtasks: JobBackpressureSubtaskInterface[];}}
> {{}}}
> I would like to update all of these to use {{_}} instead so we can maintain
> the type information we have in the web-runtime.
> My suggestion to do this with out a version bump to the API is to just make
> an addition to all the enpoints that include the _ versions as well. Then
> after the web-runtime has completely switched over to using the new keys, you
> can deprecate and remove the old hypenated keys at your own pace.
> [~chesnay] [~trohrmann]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (FLINK-36200) Breaking changes for Flink 3.0
Xintong Song created FLINK-36200: Summary: Breaking changes for Flink 3.0 Key: FLINK-36200 URL: https://issues.apache.org/jira/browse/FLINK-36200 Project: Flink Issue Type: Technical Debt Reporter: Xintong Song Fix For: 3.0.0 This is an umbrella for tracking planed breaking changes that are targeting for the next major version bump. Please be aware that, in order for the breaking changes to be included in the next major version bump, the following preparations need to be done in advance: * The old APIs need to be deprecated, 2 minor releases before the major version bump for @Public APIs, and 1 minor release for @PublicEvolving APIs * All internal usages of the deprecated APIs should be either migrated to replacing new APIs (if any) or deprecated / removed. This ensure that we can quickly remove the APIs during major version bump without need for extra migration efforts. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (FLINK-24078) Remove brackets around variables
[
https://issues.apache.org/jira/browse/FLINK-24078?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xintong Song updated FLINK-24078:
-
Fix Version/s: 2.0-preview
(was: 2.0.0)
> Remove brackets around variables
>
>
> Key: FLINK-24078
> URL: https://issues.apache.org/jira/browse/FLINK-24078
> Project: Flink
> Issue Type: Sub-task
> Components: Runtime / Metrics
>Reporter: Chesnay Schepler
>Assignee: Chesnay Schepler
>Priority: Major
> Labels: 2.0-related
> Fix For: 2.0-preview
>
>
> Internally, variable keys are stored with brackets, e.g., {{}}.
> In practice all reporters will filter these characters in one way or the
> other, and it overall is a subtle trap that we keep running into without
> providing any benefit.
> We should get rid of them.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (FLINK-19034) Remove deprecated StreamExecutionEnvironment#set/getNumberOfExecutionRetries
[
https://issues.apache.org/jira/browse/FLINK-19034?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xintong Song updated FLINK-19034:
-
Fix Version/s: 2.0-preview
(was: 2.0.0)
> Remove deprecated StreamExecutionEnvironment#set/getNumberOfExecutionRetries
>
>
> Key: FLINK-19034
> URL: https://issues.apache.org/jira/browse/FLINK-19034
> Project: Flink
> Issue Type: Sub-task
> Components: API / DataStream
>Reporter: Dawid Wysakowicz
>Assignee: Daisy Tsang
>Priority: Major
> Labels: auto-unassigned, stale-assigned
> Fix For: 2.0-preview
>
>
> Remove deprecated
> {code}
> StreamExecutionEnvironment#setNumberOfExecutionRetries/getNumberOfExecutionRetries
> {code}
> The corresponding settings in {{ExecutionConfig}} will be removed in a
> separate issue, as they are {{Public}}.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (FLINK-13926) `ProcessingTimeSessionWindows` and `EventTimeSessionWindows` should be generic
[
https://issues.apache.org/jira/browse/FLINK-13926?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xintong Song updated FLINK-13926:
-
Fix Version/s: 2.0-preview
(was: 2.0.0)
> `ProcessingTimeSessionWindows` and `EventTimeSessionWindows` should be
> generic
> ---
>
> Key: FLINK-13926
> URL: https://issues.apache.org/jira/browse/FLINK-13926
> Project: Flink
> Issue Type: Sub-task
> Components: API / DataStream
>Affects Versions: 1.9.0
>Reporter: zhihao zhang
>Priority: Major
> Labels: pull-request-available, windows
> Fix For: 2.0-preview
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> `ProcessingTimeSessionWindows` and `EventTimeSessionWindows` should be
> generic just like `DynamicEventTimeSessionWindows` and
> `DynamicProcessingTimeSessionWindows`.
> now:
>
> {code:java}
> public class ProcessingTimeSessionWindows extends
> MergingWindowAssigner {}
> {code}
> proposal:
>
> {code:java}
> public class ProcessingTimeSessionWindows extends MergingWindowAssigner TimeWindow> {}
> {code}
> If this ticket is ok to go, I would like to take it.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (FLINK-6375) Fix LongValue hashCode
[
https://issues.apache.org/jira/browse/FLINK-6375?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xintong Song updated FLINK-6375:
Fix Version/s: 2.0-preview
> Fix LongValue hashCode
> --
>
> Key: FLINK-6375
> URL: https://issues.apache.org/jira/browse/FLINK-6375
> Project: Flink
> Issue Type: Sub-task
> Components: API / Type Serialization System
>Affects Versions: 2.0.0
>Reporter: Greg Hogan
>Priority: Minor
> Labels: auto-unassigned
> Fix For: 2.0-preview
>
>
> Match {{LongValue.hashCode}} to {{Long.hashCode}} (and the other numeric
> types) by simply adding the high and low words rather than shifting the hash
> by adding 43.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (FLINK-9529) Do not expose keyed operations on ProcessFunction.Context
[
https://issues.apache.org/jira/browse/FLINK-9529?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xintong Song updated FLINK-9529:
Fix Version/s: 2.0-preview
> Do not expose keyed operations on ProcessFunction.Context
> -
>
> Key: FLINK-9529
> URL: https://issues.apache.org/jira/browse/FLINK-9529
> Project: Flink
> Issue Type: Sub-task
> Components: API / DataStream
>Affects Versions: 1.5.0
>Reporter: Till Rohrmann
>Priority: Major
> Labels: auto-unassigned
> Fix For: 2.0-preview
>
>
> Currently, the {{ProcessFunction}} provides a {{Context}} object from which
> on can access the {{TimerService}}. The {{TimerService}} allows to register
> timers if one operates on a keyed stream. If it is not a keyed stream, then
> this operation fails. Recently, we added a {{KeyedProcessFunction}} which
> captures this semantic difference. I would propose to clean up the
> {{Context}} interface to give only access to supported operations when
> operating on a non-keyed and keyed state respectively.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (FLINK-14658) Drop ".returns()" for TypeInformation in the DataStream API
[
https://issues.apache.org/jira/browse/FLINK-14658?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xintong Song updated FLINK-14658:
-
Fix Version/s: 2.0-preview
(was: 2.0.0)
> Drop ".returns()" for TypeInformation in the DataStream API
> ---
>
> Key: FLINK-14658
> URL: https://issues.apache.org/jira/browse/FLINK-14658
> Project: Flink
> Issue Type: Sub-task
> Components: API / DataStream
>Reporter: Stephan Ewen
>Priority: Major
> Fix For: 2.0-preview
>
>
> The pattern to use {{.map(function).returns(type)}} to override the
> automatically extracted type is flawed and should not be used.
> Instead, each transformation method should be overloaded to have a variant
> that accepts the return type as the second argument, if in case we want to
> override the type extractor.
> See [FLINK-14380] for good example of why the {{.returns(type)}} pattern is
> broken.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (FLINK-9798) Drop canEqual() from TypeInformation, TypeSerializer, etc.
[
https://issues.apache.org/jira/browse/FLINK-9798?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xintong Song updated FLINK-9798:
Fix Version/s: 2.0-preview
> Drop canEqual() from TypeInformation, TypeSerializer, etc.
> --
>
> Key: FLINK-9798
> URL: https://issues.apache.org/jira/browse/FLINK-9798
> Project: Flink
> Issue Type: Sub-task
> Components: API / Type Serialization System
>Reporter: Aljoscha Krettek
>Priority: Major
> Labels: auto-unassigned
> Fix For: 2.0-preview
>
>
> See discussion from
> https://lists.apache.org/thread.html/7cc6cfd66e96e8d33c768629b55481b6c951c68128f10256abb328fe@%3Cdev.flink.apache.org%3E
> {quote}
> Hi all!
> As part of an attempt to simplify some code in the TypeInfo and
> TypeSerializer area, I would like to drop the "canEqual" methods for the
> following reason:
> "canEqual()" is necessary to make proper equality checks across hierarchies
> of types. This is for example useful in a collection API, stating for
> example whether a List can be equal to a Collection if they have the same
> contents. We don't have that here.
> A certain type information (and serializer) is equal to another one if they
> describe the same type, strictly. There is no necessity for cross hierarchy
> checks.
> This has also let to the situation that most type infos and serializers
> implement just a dummy/default version of "canEqual". Many "equals()"
> methods do not even call the other object's "canEqual", etc.
> As a first step, we could simply deprecate the method and implement an
> empty default, and remove all calls to that method.
> Best,
> Stephan
> {quote}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (FLINK-5875) Use TypeComparator.hash() instead of Object.hashCode() for keying in DataStream API
[ https://issues.apache.org/jira/browse/FLINK-5875?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xintong Song updated FLINK-5875: Fix Version/s: 2.0-preview (was: 2.0.0) > Use TypeComparator.hash() instead of Object.hashCode() for keying in > DataStream API > --- > > Key: FLINK-5875 > URL: https://issues.apache.org/jira/browse/FLINK-5875 > Project: Flink > Issue Type: Sub-task > Components: API / DataStream >Reporter: Robert Metzger >Priority: Major > Fix For: 2.0-preview > > > See FLINK-5874 for details. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (FLINK-5346) Remove all ad-hoc config loading via GlobalConfiguration
[
https://issues.apache.org/jira/browse/FLINK-5346?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xintong Song updated FLINK-5346:
Fix Version/s: 2.0-preview
(was: 2.0.0)
> Remove all ad-hoc config loading via GlobalConfiguration
>
>
> Key: FLINK-5346
> URL: https://issues.apache.org/jira/browse/FLINK-5346
> Project: Flink
> Issue Type: Sub-task
> Components: Runtime / Configuration
>Reporter: Stephan Ewen
>Priority: Major
> Fix For: 2.0-preview
>
>
> I think we should get rid of the static calls to {{GlobalConfiguration}} that
> load configuration ad hoc. It will not properly work anyways because
> different setups (standalone / Yarn / Mesos / etc) store and access the
> configuration at different places.
> The only point where the configuration should be loaded is in the entry
> points of the Processes (TaskManager, JobManager, ApplicationMaster, etc)
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (FLINK-6912) Consider changing the RichFunction#open method signature to take no arguments.
[ https://issues.apache.org/jira/browse/FLINK-6912?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xintong Song updated FLINK-6912: Fix Version/s: 2.0-preview (was: 2.0.0) > Consider changing the RichFunction#open method signature to take no arguments. > -- > > Key: FLINK-6912 > URL: https://issues.apache.org/jira/browse/FLINK-6912 > Project: Flink > Issue Type: Sub-task > Components: API / DataSet, API / DataStream >Affects Versions: 1.3.0 >Reporter: Mikhail Pryakhin >Priority: Not a Priority > Labels: pull-request-available > Fix For: 2.0-preview > > > RichFunction#open(org.apache.flink.configuration.Configuration) method takes > a Configuration instance as an argument which is always [passed as a new > instance|https://github.com/apache/flink/blob/master/flink-streaming-java/src/main/java/org/apache/flink/streaming/api/operators/AbstractUdfStreamOperator.java#L111] > bearing no configuration parameters. As I figured out it is a remnant of the > past since that method signature originates from the Record API. Consider > changing the RichFunction#open method signature to take no arguments as well > as actualizing java docs. > You can find the complete discussion > [here|http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/RichMapFunction-setup-method-td13696.html] -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (FLINK-7691) Remove ClassTag in Scala DataSet API
[
https://issues.apache.org/jira/browse/FLINK-7691?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xintong Song updated FLINK-7691:
Fix Version/s: 2.0-preview
(was: 2.0.0)
> Remove ClassTag in Scala DataSet API
>
>
> Key: FLINK-7691
> URL: https://issues.apache.org/jira/browse/FLINK-7691
> Project: Flink
> Issue Type: Sub-task
> Components: API / DataSet
>Reporter: Timo Walther
>Priority: Major
> Fix For: 2.0-preview
>
>
> In the DataStream API a {{ClassTag}} is not required, which allows to pass
> {{TypeInformation}} manually if required. In the DataSet API most methods
> look like:
> {code}
> // DataSet API
> def fromElements[T: ClassTag : TypeInformation](data: T*): DataSet[T]
> // DataStream API
> def fromElements[T: TypeInformation](data: T*): DataStream[T]
> {code}
> I would propose to remove the ClassTag, if possible. This would make it
> easier e.g. to supply TypeInformation for the {{Row}} type. Or is there an
> easier way in Scala that I don't know?
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (FLINK-5336) Make Path immutable
[
https://issues.apache.org/jira/browse/FLINK-5336?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xintong Song updated FLINK-5336:
Fix Version/s: 2.0-preview
(was: 2.0.0)
> Make Path immutable
> ---
>
> Key: FLINK-5336
> URL: https://issues.apache.org/jira/browse/FLINK-5336
> Project: Flink
> Issue Type: Sub-task
> Components: API / DataSet
>Reporter: Stephan Ewen
>Priority: Major
> Labels: pull-request-available
> Fix For: 2.0-preview
>
>
> The {{Path}} class is currently mutable to support the {{IOReadableWritable}}
> serialization. Since that serialization is not used any more, I suggest to
> drop that interface from Path and make the Path's URI final.
> Being immutable, we can store configures paths properly without the chance of
> them being mutated as side effects.
> Many parts of the code make the assumption that the Path is immutable, being
> susceptible to subtle errors.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (FLINK-5337) Introduce backwards compatible state to task assignment
[ https://issues.apache.org/jira/browse/FLINK-5337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xintong Song updated FLINK-5337: Fix Version/s: 2.0-preview (was: 2.0.0) > Introduce backwards compatible state to task assignment > --- > > Key: FLINK-5337 > URL: https://issues.apache.org/jira/browse/FLINK-5337 > Project: Flink > Issue Type: Sub-task > Components: Runtime / State Backends >Reporter: Stefan Richter >Priority: Minor > Fix For: 2.0-preview > > > Flink assigns state to tasks through either a) user defined uids or b) uids > generated through hashing certain properties of the StreamingJobGraph. > While method a) is backwards compatible, it can be tedious for the user. > Method b) easily breaks w.r.t. backwards compatibility under any changes to > graph generation, e.g. when removing chaining. > We should think about a backwards compatible way assigning state to tasks, > that hopefully works without explicit uid assignment by the user. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (FLINK-4971) Unify Stream Sinks and OutputFormats
[ https://issues.apache.org/jira/browse/FLINK-4971?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xintong Song updated FLINK-4971: Fix Version/s: 2.0-preview (was: 2.0.0) > Unify Stream Sinks and OutputFormats > > > Key: FLINK-4971 > URL: https://issues.apache.org/jira/browse/FLINK-4971 > Project: Flink > Issue Type: Sub-task > Components: API / DataSet, API / DataStream >Reporter: Stephan Ewen >Priority: Major > Fix For: 2.0-preview > > > Right now, many output connectors exist in two variants: OutputFormat and > StreamSink. > Wrapping an output format to a stream sink is not quite possible, because > output formats cannot be cleanly integrated into checkpointing. For example, > they have no hook to ensure that data is written/persisted at the time of a > checkpoint. > We should try and see if we can make both one, or at least make the > OutputFormats such that they can be wrapped into a stream sink and behave > well with respect to checkpointing. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (FLINK-4758) Remove IOReadableWritable from classes where not needed
[
https://issues.apache.org/jira/browse/FLINK-4758?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xintong Song updated FLINK-4758:
Fix Version/s: 2.0-preview
(was: 2.0.0)
> Remove IOReadableWritable from classes where not needed
> ---
>
> Key: FLINK-4758
> URL: https://issues.apache.org/jira/browse/FLINK-4758
> Project: Flink
> Issue Type: Sub-task
> Components: API / DataSet
>Affects Versions: 1.2.0
>Reporter: Stephan Ewen
>Priority: Major
> Fix For: 2.0-preview
>
>
> Many classes implement for historic reasons the {{IOReadableWritable}}
> interface, where it is not needed any more.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (FLINK-4757) Unify "GlobalJobParameters" and "Configuration"
[ https://issues.apache.org/jira/browse/FLINK-4757?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xintong Song updated FLINK-4757: Fix Version/s: 2.0-preview (was: 2.0.0) > Unify "GlobalJobParameters" and "Configuration" > --- > > Key: FLINK-4757 > URL: https://issues.apache.org/jira/browse/FLINK-4757 > Project: Flink > Issue Type: Sub-task > Components: API / DataSet, API / DataStream >Reporter: Stephan Ewen >Priority: Major > Fix For: 2.0-preview > > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (FLINK-4503) Remove with method from CoGroupedStream and JoinedStream, and change apply method return type
[
https://issues.apache.org/jira/browse/FLINK-4503?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xintong Song updated FLINK-4503:
Fix Version/s: 2.0-preview
(was: 2.0.0)
> Remove with method from CoGroupedStream and JoinedStream, and change apply
> method return type
> -
>
> Key: FLINK-4503
> URL: https://issues.apache.org/jira/browse/FLINK-4503
> Project: Flink
> Issue Type: Sub-task
> Components: API / DataStream
>Reporter: Jark Wu
>Priority: Major
> Fix For: 2.0-preview
>
>
> We introduced (and immediately deprecated) the with(...) method in
> FLINK-4271. It is a temporary workaround for setting parallelism after
> co-group and join operator and not breaking binary compatibility. The
> with(...) method only differs in the return type and calls apply(...),
> casting the returned value.
> So we need to remove the {{with(...)}} method in Flink 2.0. And change the
> apply method return type.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (FLINK-4675) Remove Parameter from WindowAssigner.getDefaultTrigger()
[
https://issues.apache.org/jira/browse/FLINK-4675?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xintong Song updated FLINK-4675:
Fix Version/s: 2.0-preview
(was: 2.0.0)
> Remove Parameter from WindowAssigner.getDefaultTrigger()
>
>
> Key: FLINK-4675
> URL: https://issues.apache.org/jira/browse/FLINK-4675
> Project: Flink
> Issue Type: Sub-task
> Components: API / DataStream
>Reporter: Aljoscha Krettek
>Priority: Major
> Labels: pull-request-available
> Fix For: 2.0-preview
>
>
> For legacy reasons the method has {{StreamExecutionEnvironment}} as a
> parameter. This is not needed anymore.
> [~StephanEwen] do you think we should break this now? {{WindowAssigner}} is
> {{PublicEvolving}} but I wanted to play it conservative for now.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (FLINK-5130) Remove Deprecated Methods from WindowedStream
[ https://issues.apache.org/jira/browse/FLINK-5130?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xintong Song updated FLINK-5130: Fix Version/s: 2.0-preview (was: 2.0.0) > Remove Deprecated Methods from WindowedStream > - > > Key: FLINK-5130 > URL: https://issues.apache.org/jira/browse/FLINK-5130 > Project: Flink > Issue Type: Sub-task > Components: API / DataStream >Reporter: Aljoscha Krettek >Priority: Major > Fix For: 2.0-preview > > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (FLINK-5126) Remove Checked Exceptions from State Interfaces
[
https://issues.apache.org/jira/browse/FLINK-5126?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xintong Song updated FLINK-5126:
Fix Version/s: 2.0-preview
(was: 2.0.0)
> Remove Checked Exceptions from State Interfaces
> ---
>
> Key: FLINK-5126
> URL: https://issues.apache.org/jira/browse/FLINK-5126
> Project: Flink
> Issue Type: Sub-task
> Components: API / DataStream
>Reporter: Aljoscha Krettek
>Priority: Major
> Fix For: 2.0-preview
>
>
> Most of the methods can throw {{Exception}} or {{IOException}} but the user
> cannot really do anything to handle them. The exceptions just pollute any
> code a user implements.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (FLINK-4367) Offer separate API for watermark generation and timestamp extraction
[
https://issues.apache.org/jira/browse/FLINK-4367?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xintong Song updated FLINK-4367:
Fix Version/s: 2.0-preview
(was: 2.0.0)
> Offer separate API for watermark generation and timestamp extraction
>
>
> Key: FLINK-4367
> URL: https://issues.apache.org/jira/browse/FLINK-4367
> Project: Flink
> Issue Type: Sub-task
> Components: API / DataStream
>Affects Versions: 1.1.0
>Reporter: Robert Metzger
>Priority: Major
> Fix For: 2.0-preview
>
>
> Right now, the {{AssignerWithPunctuatedWatermarks}} and
> {{AssignerWithPeriodicWatermarks}} interfaces also require implementing a
> {{TimestampAssigner}}.
> For cases where the source emits records with timestamps, its not necessary
> to extract timestamps again from the records, we just want to generate
> watermarks.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (FLINK-3992) Remove Key interface
[ https://issues.apache.org/jira/browse/FLINK-3992?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xintong Song updated FLINK-3992: Fix Version/s: 2.0-preview (was: 2.0.0) > Remove Key interface > > > Key: FLINK-3992 > URL: https://issues.apache.org/jira/browse/FLINK-3992 > Project: Flink > Issue Type: Sub-task > Components: API / DataSet >Affects Versions: 1.0.0 >Reporter: Chesnay Schepler >Priority: Major > Fix For: 2.0-preview > > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (FLINK-3986) Rename readFileStream from the StreamExecutionEnvironment
[ https://issues.apache.org/jira/browse/FLINK-3986?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xintong Song updated FLINK-3986: Fix Version/s: 2.0-preview (was: 2.0.0) > Rename readFileStream from the StreamExecutionEnvironment > - > > Key: FLINK-3986 > URL: https://issues.apache.org/jira/browse/FLINK-3986 > Project: Flink > Issue Type: Sub-task > Components: API / DataStream >Reporter: Kostas Kloudas >Priority: Major > Fix For: 2.0-preview > > > The readFileStream(String filePath, long intervalMillis, WatchType watchType) > has to be renamed to readFile match the naming conventions of the rest of the > methods, or even removed. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (FLINK-3959) Remove implicit sinks
[ https://issues.apache.org/jira/browse/FLINK-3959?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xintong Song updated FLINK-3959: Fix Version/s: 2.0-preview (was: 2.0.0) > Remove implicit sinks > - > > Key: FLINK-3959 > URL: https://issues.apache.org/jira/browse/FLINK-3959 > Project: Flink > Issue Type: Sub-task > Components: API / DataStream >Reporter: Stephan Ewen >Priority: Major > Fix For: 2.0-preview > > > Right now, streaming programs need not specify a sink. All transformations > that are not consumed are implicitly sinks. > That behavior makes it hard to lazily construct the stream graph on > execution, and prevents interactive sessions on streams. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (FLINK-3991) Remove deprecated configuration keys from ConfigConstants
[ https://issues.apache.org/jira/browse/FLINK-3991?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xintong Song updated FLINK-3991: Fix Version/s: 2.0-preview (was: 2.0.0) > Remove deprecated configuration keys from ConfigConstants > - > > Key: FLINK-3991 > URL: https://issues.apache.org/jira/browse/FLINK-3991 > Project: Flink > Issue Type: Sub-task > Components: Runtime / Configuration >Reporter: Robert Metzger >Priority: Major > Labels: pull-request-available > Fix For: 2.0-preview > > Time Spent: 20m > Remaining Estimate: 0h > > In > https://github.com/apache/flink/commit/b0acd97935cd21843bac3b9b5afa3662b52bb95d#diff-40616c4678c3fbfe07c0701505ce0567 > I deprecated some configuration keys. > They are unused and need to be removed with the 2.0 release. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (FLINK-4330) Consider removing min()/minBy()/max()/maxBy()/sum() utility methods from the DataStream API
[ https://issues.apache.org/jira/browse/FLINK-4330?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xintong Song updated FLINK-4330: Fix Version/s: 2.0-preview (was: 2.0.0) > Consider removing min()/minBy()/max()/maxBy()/sum() utility methods from the > DataStream API > --- > > Key: FLINK-4330 > URL: https://issues.apache.org/jira/browse/FLINK-4330 > Project: Flink > Issue Type: Sub-task > Components: API / DataStream >Reporter: Robert Metzger >Priority: Major > Fix For: 2.0-preview > > > I think we should consider removing the min()/minBy()/max()/maxBy()/sum() > utility methods from the DataStream API. They make the maintenance of the > code unnecessary complicated, and don't add enough value for the users (who > can not access the window metadata). > If we are keeping the methods, we should consolidate the min/minBy methods: > the difference is subtle, and minBy can subsume the min method. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (FLINK-4147) Consider moving the file sources from the StreamExecutionEnvironment to the flink-connector-filesystem
[ https://issues.apache.org/jira/browse/FLINK-4147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xintong Song updated FLINK-4147: Fix Version/s: 2.0-preview (was: 2.0.0) > Consider moving the file sources from the StreamExecutionEnvironment to the > flink-connector-filesystem > -- > > Key: FLINK-4147 > URL: https://issues.apache.org/jira/browse/FLINK-4147 > Project: Flink > Issue Type: Sub-task > Components: API / DataStream >Reporter: Robert Metzger >Priority: Major > Fix For: 2.0-preview > > > The StreamExecutionEnvironment has many convenience methods for reading > static files. > However, it seems that this makes it hard to maintain Flink's APIs and that > not so many users are using the file sources. > The filesystem connector is the logical location for persistent file reading. > I suggest to move those sources there. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (FLINK-3957) Breaking changes for Flink 2.0
[ https://issues.apache.org/jira/browse/FLINK-3957?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xintong Song updated FLINK-3957: Fix Version/s: 2.0-preview (was: 2.0.0) > Breaking changes for Flink 2.0 > -- > > Key: FLINK-3957 > URL: https://issues.apache.org/jira/browse/FLINK-3957 > Project: Flink > Issue Type: New Feature > Components: API / DataSet, API / DataStream, Build System >Affects Versions: 1.0.0 >Reporter: Robert Metzger >Priority: Minor > Labels: 2.0-related, auto-deprioritized-major > Fix For: 2.0-preview > > > From time to time, we find APIs in Flink (1.x.y) marked as stable, even > though we would like to change them at some point. > This JIRA is to track all planned breaking API changes. > I would suggest to add subtasks to this one. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (FLINK-33749) Remove deprecated getter method in Configuration.
[ https://issues.apache.org/jira/browse/FLINK-33749?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xintong Song updated FLINK-33749: - Fix Version/s: 2.0-preview > Remove deprecated getter method in Configuration. > - > > Key: FLINK-33749 > URL: https://issues.apache.org/jira/browse/FLINK-33749 > Project: Flink > Issue Type: Sub-task > Components: API / Core >Reporter: Junrui Li >Assignee: Junrui Li >Priority: Major > Labels: 2.0-related > Fix For: 2.0-preview > > > Currently, the Configuration has several getter methods for retrieving config > values based on a String key, such as getString(String key, String > defaultValue), and all of which have been deprecated. > We should remove these getter methods in FLINK-2.0. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (FLINK-32383) 2.0 Breaking configuration changes
[ https://issues.apache.org/jira/browse/FLINK-32383?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xintong Song updated FLINK-32383: - Fix Version/s: 2.0-preview (was: 2.0.0) > 2.0 Breaking configuration changes > -- > > Key: FLINK-32383 > URL: https://issues.apache.org/jira/browse/FLINK-32383 > Project: Flink > Issue Type: Technical Debt > Components: Runtime / Configuration >Reporter: Zhu Zhu >Priority: Blocker > Labels: 2.0-related > Fix For: 2.0-preview > > > Umbrella issue for all breaking changes to Flink configuration. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (FLINK-33678) Remove configuration getters/setters that return/set complex Java objects
[ https://issues.apache.org/jira/browse/FLINK-33678?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xintong Song updated FLINK-33678: - Fix Version/s: 2.0-preview > Remove configuration getters/setters that return/set complex Java objects > - > > Key: FLINK-33678 > URL: https://issues.apache.org/jira/browse/FLINK-33678 > Project: Flink > Issue Type: Sub-task > Components: API / Core >Reporter: Junrui Li >Priority: Major > Labels: 2.0-related, pull-request-available > Fix For: 2.0-preview > > > FLINK-33581/FLIP-381 Deprecate configuration getters/setters that return/set > complex Java objects. > In Flink 2.0 we should remove these deprecated method and fields. This change > will prevent users from configuring their jobs by passing complex Java > objects, encouraging them to use ConfigOption instead. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (FLINK-33677) Remove flink-conf.yaml from flink dist
[ https://issues.apache.org/jira/browse/FLINK-33677?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xintong Song updated FLINK-33677: - Fix Version/s: 2.0-preview > Remove flink-conf.yaml from flink dist > -- > > Key: FLINK-33677 > URL: https://issues.apache.org/jira/browse/FLINK-33677 > Project: Flink > Issue Type: Sub-task > Components: Runtime / Configuration >Reporter: Zhu Zhu >Assignee: Junrui Li >Priority: Major > Labels: 2.0-related, pull-request-available > Fix For: 2.0-preview > > > FLINK-33297/FLIP-366 supports parsing standard YAML files for Flink > configuration. A new configuration file config.yaml, which should be a > standard YAML file, is introduced. > To ensure compatibility, in Flink 1.x, the old configuration parser will > still be used if the old configuration file flink-conf.yaml exists. Only if > it does not exist, the new configuration file will be used. > In Flink 2.0, we should remove the old configuration file from flink dist, as > well as the old configuration parser. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (FLINK-36185) Remove RuntimeContext#getExecutionConfig
[ https://issues.apache.org/jira/browse/FLINK-36185?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xintong Song updated FLINK-36185: - Fix Version/s: 2.0-preview > Remove RuntimeContext#getExecutionConfig > > > Key: FLINK-36185 > URL: https://issues.apache.org/jira/browse/FLINK-36185 > Project: Flink > Issue Type: Sub-task > Components: API / Core >Reporter: Junrui Li >Priority: Major > Labels: pull-request-available > Fix For: 2.0-preview > > > FLINK-33713/FLIP-391 Deprecate RuntimeContext#getExecutionConfig > In Flink 2.0 we should remove these deprecated method. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (FLINK-32384) Remove deprecated configuration keys which violate YAML spec
[ https://issues.apache.org/jira/browse/FLINK-32384?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xintong Song updated FLINK-32384: - Fix Version/s: 2.0-preview (was: 2.0.0) > Remove deprecated configuration keys which violate YAML spec > > > Key: FLINK-32384 > URL: https://issues.apache.org/jira/browse/FLINK-32384 > Project: Flink > Issue Type: Sub-task > Components: Runtime / Configuration >Reporter: Zhu Zhu >Priority: Major > Labels: 2.0-related > Fix For: 2.0-preview > > > In FLINK-29372, key that violate YAML spec are renamed to a valid form and > the old names are deprecated. > In Flink 2.0 we should remove these deprecated keys. This will prevent users > (unintentionally) to create invalid YAML form flink-conf.yaml. > Then with the work of FLINK-23620, we can remove the non-standard YAML > parsing logic and enforce standard YAML validation in CI. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (FLINK-32688) Remove deprecated exception history fields
[ https://issues.apache.org/jira/browse/FLINK-32688?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xintong Song updated FLINK-32688: - Fix Version/s: 2.0-preview (was: 2.0.0) > Remove deprecated exception history fields > -- > > Key: FLINK-32688 > URL: https://issues.apache.org/jira/browse/FLINK-32688 > Project: Flink > Issue Type: Sub-task > Components: Runtime / REST >Affects Versions: 1.18.0 >Reporter: Matthias Pohl >Assignee: Matthias Pohl >Priority: Major > Labels: 2.0-related, pull-request-available > Fix For: 2.0-preview > > > The fields were already marked as deprecated (see > [JobExceptionInfo|https://github.com/apache/flink/blob/a49f1aaec6239401cc8b1dac731d290e95290caf/flink-runtime/src/main/java/org/apache/flink/runtime/rest/messages/JobExceptionsInfo.java#L35]) > but were not discussed as part of a FLIP. Working on this issue would > require creating a FLIP to cover the REST API change. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (FLINK-34037) FLIP-398: Improve Serialization Configuration And Usage In Flink
[ https://issues.apache.org/jira/browse/FLINK-34037?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xintong Song updated FLINK-34037: - Fix Version/s: 2.0-preview > FLIP-398: Improve Serialization Configuration And Usage In Flink > > > Key: FLINK-34037 > URL: https://issues.apache.org/jira/browse/FLINK-34037 > Project: Flink > Issue Type: Improvement > Components: API / Type Serialization System, Runtime / Configuration >Affects Versions: 1.19.0 >Reporter: Fang Yong >Assignee: Zhanghao Chen >Priority: Major > Labels: 2.0-related > Fix For: 2.0-preview > > > Improve serialization in > https://cwiki.apache.org/confluence/display/FLINK/FLIP-398%3A+Improve+Serialization+Configuration+And+Usage+In+Flink -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (FLINK-4602) Move RocksDB backend to proper package
[
https://issues.apache.org/jira/browse/FLINK-4602?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xintong Song updated FLINK-4602:
Fix Version/s: 2.0-preview
(was: 2.0.0)
> Move RocksDB backend to proper package
> --
>
> Key: FLINK-4602
> URL: https://issues.apache.org/jira/browse/FLINK-4602
> Project: Flink
> Issue Type: Sub-task
> Components: Runtime / State Backends
>Reporter: Aljoscha Krettek
>Priority: Major
> Labels: 2.0-related, auto-unassigned
> Fix For: 2.0-preview
>
>
> Right now the package is {{org.apache.flink.contrib.streaming.state}}, it
> should probably be in {{org.apache.flink.runtime.state.rocksdb}}.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (FLINK-23620) Introduce proper YAML parsing to Flink's configuration
[
https://issues.apache.org/jira/browse/FLINK-23620?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xintong Song updated FLINK-23620:
-
Fix Version/s: 2.0-preview
(was: 2.0.0)
> Introduce proper YAML parsing to Flink's configuration
> --
>
> Key: FLINK-23620
> URL: https://issues.apache.org/jira/browse/FLINK-23620
> Project: Flink
> Issue Type: Improvement
> Components: Runtime / Configuration
>Reporter: Mika Naylor
>Assignee: Junrui Li
>Priority: Minor
> Labels: 2.0-related
> Fix For: 2.0-preview
>
>
> At the moment, the YAML parsing for Flink's configuration file
> ({{conf/flink-conf.yaml)}} is pretty basic. It only supports basic key value
> pairs, such as:
> {code:java}
> a.b.c: a value
> a.b.d: another value{code}
> As well as supporting some invalid YAML syntax, such as:
> {code:java}
> a: b: value{code}
>
> Introducing proper YAML parsing to the configuration component would let
> Flink users use features such as nested keys, such as:
> {code:java}
> a:
> b:
> c: a value
> d: another value{code}
> as well as make it easier to integrate configuration tools/languages that
> compile to YAML, such as Dhall.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (FLINK-35573) [FLIP-447] Upgrade FRocksDB from 6.20.3 to 8.10.0
[ https://issues.apache.org/jira/browse/FLINK-35573?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xintong Song updated FLINK-35573: - Fix Version/s: 2.0-preview (was: 2.0.0) > [FLIP-447] Upgrade FRocksDB from 6.20.3 to 8.10.0 > - > > Key: FLINK-35573 > URL: https://issues.apache.org/jira/browse/FLINK-35573 > Project: Flink > Issue Type: Improvement > Components: Runtime / State Backends >Affects Versions: 2.0.0 >Reporter: Yue Ma >Assignee: Yue Ma >Priority: Major > Fix For: 2.0-preview > > > The FLIP: > [https://cwiki.apache.org/confluence/display/FLINK/FLIP-447%3A+Upgrade+FRocksDB+from+6.20.3++to+8.10.0] > > RocksDBStateBackend is widely used by Flink users in large state > scenarios.The last upgrade of FRocksDB was in version Flink-1.14, which > mainly supported features such as support arm platform, deleteRange API, > period compaction, etc. It has been a long time since then, and RocksDB has > now been released to version 8.x. The main motivation for this upgrade is to > leverage the features of higher versions of Rocksdb to make Flink > RocksDBStateBackend more powerful. While RocksDB is also continuously > optimizing and bug fixing, we hope to keep FRocksDB more or less in sync with > RocksDB and upgrade it periodically. > The plan is as follows > *Release Frocksdb-8.10.0 official products* > # Prepare the compiled frocksdb branch > # Support Disable PERF-CONTEXT in compilation > # Cherry pick IngestDB requires commit > # Setup the CI environment for FRocksDB-8.10 > # Release Frocksdb-8.10.0 official products > # Update the dependency of FRocksDB in pom file of Flink-RocksDB-Statebackend > *Make ingestDB available* > # Flink ingestDB code related bug fixes > # Remove comments from the code related to ingestDB and mark the > functionality of ingestDB as available (perhaps we can mark ingestDB as an > experimental feature first) > # Using ingestDB as the default recovery mode for rescaling > （已编辑） -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (FLINK-36134) Remove the classes associated with TableSource and TableSink as part of the optimization process.
[ https://issues.apache.org/jira/browse/FLINK-36134?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xintong Song updated FLINK-36134: - Fix Version/s: 2.0-preview (was: 2.0.0) > Remove the classes associated with TableSource and TableSink as part of the > optimization process. > - > > Key: FLINK-36134 > URL: https://issues.apache.org/jira/browse/FLINK-36134 > Project: Flink > Issue Type: Technical Debt > Components: Table SQL / API, Table SQL / Planner >Reporter: xuyang >Assignee: xuyang >Priority: Major > Labels: 2.0-related > Fix For: 2.0-preview > > > This Jira is a sub-task in https://issues.apache.org/jira/browse/FLINK-33748 > Remove related rules and nodes -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (FLINK-33750) Remove deprecated config options.
[ https://issues.apache.org/jira/browse/FLINK-33750?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xintong Song updated FLINK-33750: - Fix Version/s: 2.0-preview > Remove deprecated config options. > - > > Key: FLINK-33750 > URL: https://issues.apache.org/jira/browse/FLINK-33750 > Project: Flink > Issue Type: Sub-task > Components: API / Core >Reporter: Junrui Li >Assignee: Junrui Li >Priority: Major > Labels: 2.0-related > Fix For: 2.0-preview > > > Remove deprecated config options in FLINK-2.0. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (FLINK-36142) Remove TestTableSourceWithTime and its subclasses to prepare to remove TableEnvironmentInternal#registerTableSourceInternal
[ https://issues.apache.org/jira/browse/FLINK-36142?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xintong Song updated FLINK-36142: - Fix Version/s: 2.0-preview (was: 2.0.0) > Remove TestTableSourceWithTime and its subclasses to prepare to remove > TableEnvironmentInternal#registerTableSourceInternal > --- > > Key: FLINK-36142 > URL: https://issues.apache.org/jira/browse/FLINK-36142 > Project: Flink > Issue Type: Sub-task > Components: Table SQL / Planner, Table SQL / Runtime >Reporter: xuyang >Assignee: xuyang >Priority: Major > Labels: 2.0-related, pull-request-available > Fix For: 2.0-preview > > > This work will involve removing the existing UT and IT tests for > LegacyTableSource and LegacyTableSink. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (FLINK-33748) Remove legacy TableSource/TableSink API in 2.0
[
https://issues.apache.org/jira/browse/FLINK-33748?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xintong Song updated FLINK-33748:
-
Fix Version/s: 2.0-preview
(was: 2.0.0)
> Remove legacy TableSource/TableSink API in 2.0
> --
>
> Key: FLINK-33748
> URL: https://issues.apache.org/jira/browse/FLINK-33748
> Project: Flink
> Issue Type: Technical Debt
> Components: Table SQL / API
>Reporter: Weijie Guo
>Assignee: xuyang
>Priority: Major
> Labels: 2.0-related
> Fix For: 2.0-preview
>
>
> {{TableSource}} and {{TableSink}} already marked as deprecated in
> FLINK-19453, and can be removed in 2.0.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (FLINK-36136) Remove TableSource & TableSink impl and utils, docs and etc.
[ https://issues.apache.org/jira/browse/FLINK-36136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xintong Song updated FLINK-36136: - Fix Version/s: 2.0-preview (was: 2.0.0) > Remove TableSource & TableSink impl and utils, docs and etc. > > > Key: FLINK-36136 > URL: https://issues.apache.org/jira/browse/FLINK-36136 > Project: Flink > Issue Type: Technical Debt > Components: Table SQL / API, Table SQL / Planner >Reporter: xuyang >Assignee: xuyang >Priority: Major > Labels: 2.0-related > Fix For: 2.0-preview > > > This Jira is a sub-task in https://issues.apache.org/jira/browse/FLINK-33748 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (FLINK-33747) Remove Sink V1 API in 2.0
[ https://issues.apache.org/jira/browse/FLINK-33747?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xintong Song updated FLINK-33747: - Fix Version/s: 2.0-preview (was: 2.0.0) > Remove Sink V1 API in 2.0 > - > > Key: FLINK-33747 > URL: https://issues.apache.org/jira/browse/FLINK-33747 > Project: Flink > Issue Type: Technical Debt > Components: Connectors / Common >Reporter: Weijie Guo >Priority: Major > Labels: 2.0-related > Fix For: 2.0-preview > > > We have already mark sink v1 related API to `@deprecated` in 1.15. They can > be removed in Flink 2.0. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (FLINK-36135) Remove the TableSource and TableSink ecosystem APIs like TableFactory, DefinedFieldMapping, FilterableTableSource, etc.
[ https://issues.apache.org/jira/browse/FLINK-36135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xintong Song updated FLINK-36135: - Fix Version/s: 2.0-preview (was: 2.0.0) > Remove the TableSource and TableSink ecosystem APIs like TableFactory, > DefinedFieldMapping, FilterableTableSource, etc. > --- > > Key: FLINK-36135 > URL: https://issues.apache.org/jira/browse/FLINK-36135 > Project: Flink > Issue Type: Technical Debt > Components: Table SQL / API, Table SQL / Planner >Reporter: xuyang >Assignee: xuyang >Priority: Major > Labels: 2.0-related > Fix For: 2.0-preview > > > This Jira is a sub-task in https://issues.apache.org/jira/browse/FLINK-33748 > Remove apis such as > * DefinedRowtimeAttributes > * ProjectableTableSource > * FilterableTableSource > * ... -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (FLINK-36175) Remove `register_table_source` and `register_table_sink` in python module
[ https://issues.apache.org/jira/browse/FLINK-36175?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xintong Song updated FLINK-36175: - Fix Version/s: 2.0-preview (was: 2.0.0) > Remove `register_table_source` and `register_table_sink` in python module > - > > Key: FLINK-36175 > URL: https://issues.apache.org/jira/browse/FLINK-36175 > Project: Flink > Issue Type: Sub-task > Components: API / Python >Reporter: xuyang >Priority: Major > Labels: pull-request-available > Fix For: 2.0-preview > > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (FLINK-36132) Remove codes that references TableSource and TableSink directly at the API level.
[ https://issues.apache.org/jira/browse/FLINK-36132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xintong Song updated FLINK-36132: - Fix Version/s: 2.0-preview (was: 2.0.0) > Remove codes that references TableSource and TableSink directly at the API > level. > - > > Key: FLINK-36132 > URL: https://issues.apache.org/jira/browse/FLINK-36132 > Project: Flink > Issue Type: Technical Debt > Components: Table SQL / API, Table SQL / Planner >Reporter: xuyang >Assignee: xuyang >Priority: Major > Labels: 2.0-related > Fix For: 2.0-preview > > > This Jira is a sub-task in https://issues.apache.org/jira/browse/FLINK-33748 > This Jira is responsible for the following removal: > ● Remove APIs related to TableEnvironmentInternal > ○ #fromTableSource > ○ #registerTableSourceInternal > ○ #registerTableSinkInternal > ● Remove APIs related to StreamTableEnvironment > ○ #toAppendStream > ○ #toRetractStream > ● Update subclasses of Operation > ○ Remove TableSourceQueryOperation > ○ Update QueryOperationCatalogViewTable > ○ Remove UnregisteredSinkModifyOperation > ○ Remove RichTableSourceQueryOperation (heavily tested, consider a > separate PR) > ○ Remove OutputConversionModifyOperation (this part is pending, depends > on Python) > ... -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (FLINK-36133) Remove subclass implementations of TableSource and TableSink, along with their corresponding Factory classes.
[ https://issues.apache.org/jira/browse/FLINK-36133?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xintong Song updated FLINK-36133: - Fix Version/s: 2.0-preview (was: 2.0.0) > Remove subclass implementations of TableSource and TableSink, along with > their corresponding Factory classes. > - > > Key: FLINK-36133 > URL: https://issues.apache.org/jira/browse/FLINK-36133 > Project: Flink > Issue Type: Sub-task > Components: Table SQL / API, Table SQL / Planner >Reporter: xuyang >Assignee: xuyang >Priority: Major > Labels: 2.0-related > Fix For: 2.0-preview > > > This Jira is a sub-task in https://issues.apache.org/jira/browse/FLINK-33748 > Remove subclasses such as > * ArrowTableSource > * CsvTableSource > * ... -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (FLINK-29741) [FLIP-265] Remove all Scala APIs
[ https://issues.apache.org/jira/browse/FLINK-29741?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xintong Song updated FLINK-29741: - Fix Version/s: 2.0-preview (was: 2.0.0) > [FLIP-265] Remove all Scala APIs > > > Key: FLINK-29741 > URL: https://issues.apache.org/jira/browse/FLINK-29741 > Project: Flink > Issue Type: Sub-task > Components: API / Scala >Reporter: Martijn Visser >Priority: Major > Labels: 2.0-related > Fix For: 2.0-preview > > > - Remove all @Public, @PublicEvolving and @Experimental Scala APIs (which > should have been marked as @Deprecated in FLINK-29740) > - Remove all Scala API documentation -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (FLINK-35939) Do not set empty config values via ConfigUtils#encodeCollectionToConfig
[
https://issues.apache.org/jira/browse/FLINK-35939?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xintong Song updated FLINK-35939:
-
Fix Version/s: (was: 2.0.0)
> Do not set empty config values via ConfigUtils#encodeCollectionToConfig
> ---
>
> Key: FLINK-35939
> URL: https://issues.apache.org/jira/browse/FLINK-35939
> Project: Flink
> Issue Type: Improvement
>Affects Versions: 1.20.0
>Reporter: Ferenc Csaky
>Assignee: Ferenc Csaky
>Priority: Minor
> Labels: pull-request-available
> Fix For: 2.0-preview
>
>
> The {{ConfigUtils#encodeCollectionToConfig}} function only skips to set a
> given {{ConfigOption}} value, if that value is null. If the passed collection
> is empty, it will set that empty collection.
> I think this behavior is less logical and can cause more undesired
> situations, when we only set a value if it is not empty AND not null.
> Furthermore, the method's
> [javadoc|https://github.com/apache/flink/blob/82b628d4730eef32b2f7a022e3b73cb18f950e6e/flink-core/src/main/java/org/apache/flink/configuration/ConfigUtils.java#L73]
> describes the logic I just mentioned above, which is in conflict with the
> actual implementation and tests, which sets an empty collection.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (FLINK-35227) Remove execution-mode in ExecutionConfigInfo
[ https://issues.apache.org/jira/browse/FLINK-35227?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xintong Song updated FLINK-35227: - Fix Version/s: (was: 2.0.0) > Remove execution-mode in ExecutionConfigInfo > > > Key: FLINK-35227 > URL: https://issues.apache.org/jira/browse/FLINK-35227 > Project: Flink > Issue Type: Sub-task > Components: Runtime / REST >Reporter: Rui Fan >Assignee: Rui Fan >Priority: Major > Labels: pull-request-available > Fix For: 2.0-preview > > -- This message was sent by Atlassian Jira (v8.20.10#820010)
