Re: [ANNOUNCE] Progress updates for Apache Flink 1.9.0 release
Hi Gordon, Thanks for the following updates of current progress. In addition, it might be better to also cover the fix of network resource leak in jira ticket [1] which would be merged soon I think. [1] FLINK-13245: This fixes the leak of releasing reader/view with partition in network stack. Best, Zhijiang -- From:Tzu-Li (Gordon) Tai Send Time:2019年7月27日(星期六) 10:41 To:dev Subject:Re: [ANNOUNCE] Progress updates for Apache Flink 1.9.0 release Hi all, It's been a while since our last update for the release testing of 1.9.0, so I want to bring attention to the current status of the release. We are approaching RC1 soon, waiting on the following specific last ongoing threads to be closed: - FLINK-13241: This fixes a problem where when using YARN, slot allocation requests may be ignored [1] - FLINK-13371: Potential partitions resource leak in case of producer restarts [2] - FLINK-13350: Distinguish between temporary tables and persisted tables [3]. Strictly speaking this would be a new feature, but there was a discussion here [4] to include a workaround for now in 1.9.0, and a proper solution later on in 1.10.x. - FLINK-12858: Potential distributed deadlock in case of synchronous savepoint failure [5] The above is the critical path for moving forward with an RC1 for official voting. All of them have PRs already, and are currently being reviewed or close to being merged. Cheers, Gordon [1] https://issues.apache.org/jira/browse/FLINK-13241 [2] https://issues.apache.org/jira/browse/FLINK-13371 [3] https://issues.apache.org/jira/browse/FLINK-13350 [4] http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Support-temporary-tables-in-SQL-API-td30831.html [5] https://issues.apache.org/jira/browse/FLINK-12858 On Tue, Jul 16, 2019 at 5:26 AM Tzu-Li (Gordon) Tai wrote: > Update: RC0 for 1.9.0 has been created. Please see [1] for the preview > source / binary releases and Maven artifacts. > > Cheers, > Gordon > > [1] > http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/PREVIEW-Apache-Flink-1-9-0-release-candidate-0-td30583.html > > On Mon, Jul 15, 2019 at 6:39 PM Tzu-Li (Gordon) Tai > wrote: > >> Hi Flink devs, >> >> As previously announced by Kurt [1], the release branch for 1.9.0 has >> been cut [2] and we've now started the testing phase for this release, as >> well as resolving remaining blockers. >> >> I want to quickly provide an overview of our progress here. >> Also, over the course of the testing phase, we will update this mail >> thread every 2-3 days with the overall progress of the release to keep you >> updated. >> >> *1. Remaining blockers and critical issues* >> You can find a link here [3] for a release Kanban board that provides an >> overview of the remaining blockers and critical issues for releasing 1.9.0. >> The issues listed there are high priority for the release, so any help >> with reviewing or fixing them is highly appreciated! >> If you do assign yourself to any unassigned issue and start working on >> it, please make sure to pull it to the "In Progress" column to let others >> be aware of this. >> >> *2. Creating RC 0 for 1.9.0* >> We will create RC0 now to drive forward the testing efforts. >> This should be ready by tomorrow morning (July 16, 8am CET). >> Note that we will not have an official vote for RC0, as this is mainly to >> drive testing efforts. >> RC1 with an official vote will be created once the blockers listed in [3] >> are resolved. >> >> Cheers, >> Gordon >> >> [1] >> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/ANNOUNCE-Flink-1-9-release-branch-has-been-created-td30500.html >> [2] >> https://gitbox.apache.org/repos/asf?p=flink.git;a=shortlog;h=refs/heads/release-1.9 >> [3] >> https://issues.apache.org/jira/secure/RapidBoard.jspa?projectKey=FLINK=328 >> >
[jira] [Created] (FLINK-13447) Change default planner to legacy planner instead of any one
Jark Wu created FLINK-13447:
---
Summary: Change default planner to legacy planner instead of any
one
Key: FLINK-13447
URL: https://issues.apache.org/jira/browse/FLINK-13447
Project: Flink
Issue Type: Task
Components: Table SQL / API
Reporter: Jark Wu
Assignee: Jark Wu
Fix For: 1.9.0, 1.10.0
As discussed in FLINK-13399, we will change the default behavior of the
{{EnvironmentSettings}} to use old planner instead of any planner. This will
enable us to have both planner in the classpath. This will also enable
users/connectors to have both planner in dependency and without using
{{EnvironmentSettings}}.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Created] (FLINK-13446) Row count sliding window outputs incorrectly in blink planner
Hequn Cheng created FLINK-13446:
---
Summary: Row count sliding window outputs incorrectly in blink
planner
Key: FLINK-13446
URL: https://issues.apache.org/jira/browse/FLINK-13446
Project: Flink
Issue Type: Bug
Components: Table SQL / Runtime
Affects Versions: 1.9.0
Reporter: Hequn Cheng
For blink planner, the Row count sliding window outputs incorrectly. The window
assigner assigns less window than what expected. This means the window outputs
fewer data. The bug can be reproduced by the following test:
{code:java}
@Test
def testGroupWindowWithoutKeyInProjection(): Unit = {
val data = List(
(1L, 1, "Hi", 1, 1),
(2L, 2, "Hello", 2, 2),
(4L, 2, "Hello", 2, 2),
(8L, 3, "Hello world", 3, 3),
(16L, 3, "Hello world", 3, 3))
val stream = failingDataSource(data)
val table = stream.toTable(tEnv, 'long, 'int, 'string, 'int2, 'int3,
'proctime.proctime)
val weightAvgFun = new WeightedAvg
val countDistinct = new CountDistinct
val windowedTable = table
.window(Slide over 2.rows every 1.rows on 'proctime as 'w)
.groupBy('w, 'int2, 'int3, 'string)
.select(weightAvgFun('long, 'int), countDistinct('long))
val sink = new TestingAppendSink
windowedTable.toAppendStream[Row].addSink(sink)
env.execute()
val expected = Seq("12,2", "8,1", "2,1", "3,2", "1,1")
assertEquals(expected.sorted, sink.getAppendResults.sorted)
}
{code}
The expected output is Seq("12,2", "8,1", "2,1", "3,2", "1,1") while the actual
output is Seq("12,2", "3,2")
To fix the problem, we can correct the assign logic in
CountSlidingWindowAssigner.assignWindows.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Created] (FLINK-13445) Distinguishing Memory Configuration for TaskManager and JobManager
madong created FLINK-13445: -- Summary: Distinguishing Memory Configuration for TaskManager and JobManager Key: FLINK-13445 URL: https://issues.apache.org/jira/browse/FLINK-13445 Project: Flink Issue Type: New Feature Components: Runtime / Configuration Affects Versions: 1.8.1 Reporter: madong we use flink to run some job build in non-java language, so we increase the value of `containerized.heap-cutoff-ratio` to reserve more memory for non-java process , which would affect memory allocation for jobManager. Considering the different behaviors of taskManager and jobManager, should we use this configuration separately？ -- This message was sent by Atlassian JIRA (v7.6.14#76016)
