[jira] [Created] (FLINK-16231) Hive connector is missing jdk.tools exclusion against Hive 2.x.x
Chesnay Schepler created FLINK-16231: Summary: Hive connector is missing jdk.tools exclusion against Hive 2.x.x Key: FLINK-16231 URL: https://issues.apache.org/jira/browse/FLINK-16231 Project: Flink Issue Type: Bug Components: Build System, Connectors / Hive Affects Versions: 1.10.0 Reporter: Chesnay Schepler Assignee: Chesnay Schepler Fix For: 1.10.1, 1.11.0 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (FLINK-16232) Remove redundant log4j2 exclusions from hive connector
Chesnay Schepler created FLINK-16232: Summary: Remove redundant log4j2 exclusions from hive connector Key: FLINK-16232 URL: https://issues.apache.org/jira/browse/FLINK-16232 Project: Flink Issue Type: Improvement Components: Build System, Connectors / Hive Reporter: Chesnay Schepler Assignee: Chesnay Schepler Fix For: 1.11.0 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (FLINK-16233) Hive connector missing log4j1 exclusions against certain hive versions
Chesnay Schepler created FLINK-16233: Summary: Hive connector missing log4j1 exclusions against certain hive versions Key: FLINK-16233 URL: https://issues.apache.org/jira/browse/FLINK-16233 Project: Flink Issue Type: Bug Components: Build System, Connectors / Hive Affects Versions: 1.11.0 Reporter: Chesnay Schepler Assignee: Chesnay Schepler Fix For: 1.11.0 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (FLINK-16234) Use LinkedHashSet in StreamGraph.java
cpugputpu created FLINK-16234: - Summary: Use LinkedHashSet in StreamGraph.java Key: FLINK-16234 URL: https://issues.apache.org/jira/browse/FLINK-16234 Project: Flink Issue Type: Bug Reporter: cpugputpu The test in _org.apache.flink.streaming.api.graph.StreamingJobGraphGeneratorTest#testSlotSharingOnAllVerticesInSameSlotSharingGroupByDefaultDisabled_ will cause the following failure: java.lang.AssertionError: expected: but was: at org.apache.flink.streaming.api.graph.StreamingJobGraphGeneratorTest.assertSameSlotSharingGroup(StreamingJobGraphGeneratorTest.java:843) at org.apache.flink.streaming.api.graph.StreamingJobGraphGeneratorTest.testSlotSharingOnAllVerticesInSameSlotSharingGroupByDefaultDisabled(StreamingJobGraphGeneratorTest.java:814) I analyze the assertion failure and find that the root cause of it lies in the clear() method in StreamGraph.java, where the variable _sources_ is initialized as a HashSet. Because the iteration order of HashSet is non-deterministic, so the test becomes flaky. The fix is to change _HashSet_ to _LinkedHashSet_ and then the failure above is removed. The stacktrace information is presented as follows for your reference: java.util.HashSet.iterator(HashSet.java:173) org.apache.flink.streaming.api.graph.StreamingJobGraphGenerator.setChaining(StreamingJobGraphGenerator.java:251) org.apache.flink.streaming.api.graph.StreamingJobGraphGenerator.createJobGraph(StreamingJobGraphGenerator.java:166) org.apache.flink.streaming.api.graph.StreamingJobGraphGenerator.createJobGraph(StreamingJobGraphGenerator.java:104) org.apache.flink.streaming.api.graph.StreamingJobGraphGenerator.createJobGraph(StreamingJobGraphGenerator.java:100) org.apache.flink.streaming.api.graph.StreamingJobGraphGeneratorTest.testSlotSharingOnAllVerticesInSameSlotSharingGroupByDefaultDisabled(StreamingJobGraphGeneratorTest.java:803) -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (FLINK-16235) Use LinkedHashSet in StreamGraph.java
cpugputpu created FLINK-16235: - Summary: Use LinkedHashSet in StreamGraph.java Key: FLINK-16235 URL: https://issues.apache.org/jira/browse/FLINK-16235 Project: Flink Issue Type: Bug Reporter: cpugputpu The test in _org.apache.flink.streaming.api.graph.StreamingJobGraphGeneratorTest#testSlotSharingOnAllVerticesInSameSlotSharingGroupByDefaultDisabled_ will cause the following failure: java.lang.AssertionError: expected: but was: at org.apache.flink.streaming.api.graph.StreamingJobGraphGeneratorTest.assertSameSlotSharingGroup(StreamingJobGraphGeneratorTest.java:843) at org.apache.flink.streaming.api.graph.StreamingJobGraphGeneratorTest.testSlotSharingOnAllVerticesInSameSlotSharingGroupByDefaultDisabled(StreamingJobGraphGeneratorTest.java:814) I analyze the assertion failure and find that the root cause of it lies in the clear() method in StreamGraph.java, where the variable _sources_ is initialized as a HashSet. Because the iteration order of HashSet is non-deterministic, so the test becomes flaky. The fix is to change _HashSet_ to _LinkedHashSet_ and then the failure above is removed. The stacktrace information is presented as follows for your reference: java.util.HashSet.iterator(HashSet.java:173) org.apache.flink.streaming.api.graph.StreamingJobGraphGenerator.setChaining(StreamingJobGraphGenerator.java:251) org.apache.flink.streaming.api.graph.StreamingJobGraphGenerator.createJobGraph(StreamingJobGraphGenerator.java:166) org.apache.flink.streaming.api.graph.StreamingJobGraphGenerator.createJobGraph(StreamingJobGraphGenerator.java:104) org.apache.flink.streaming.api.graph.StreamingJobGraphGenerator.createJobGraph(StreamingJobGraphGenerator.java:100) org.apache.flink.streaming.api.graph.StreamingJobGraphGeneratorTest.testSlotSharingOnAllVerticesInSameSlotSharingGroupByDefaultDisabled(StreamingJobGraphGeneratorTest.java:803) -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (FLINK-16236) Fix YARNSessionFIFOSecuredITCase not loading the correct security context factory
Rong Rong created FLINK-16236: - Summary: Fix YARNSessionFIFOSecuredITCase not loading the correct security context factory Key: FLINK-16236 URL: https://issues.apache.org/jira/browse/FLINK-16236 Project: Flink Issue Type: Sub-task Components: Deployment / YARN Reporter: Rong Rong Assignee: Rong Rong Follow up on FLINK-11589. Currently due to the override of the TestHadoopModuleFactory, it is not loading the HadoopContextFactory due to the compatibility checker. -- This message was sent by Atlassian Jira (v8.3.4#803005)
Re: [ANNOUNCE] Apache Flink Python API(PyFlink) 1.9.2 released
It’s an interesting idea to use the Beam SDK to execute PyFlink (i.e. non-Beam) UDFs. Thanks for the info. On Thu, Feb 20, 2020 at 5:41 PM Xingbo Huang wrote: > Hi Chad, > Beam Portability module solves the management of the Python runtime > environment and the communication between the Python runtime environment > Python VM and the Java runtime environment JVM in PyFlink. For details, you > can refer to FLIP-58[1] and the Blog[2]. > > Hope to resolve your doubts. > > Best, > Xingbo > > [1] > https://cwiki.apache.org/confluence/display/FLINK/FLIP-58%3A+Flink+Python+User-Defined+Stateless+Function+for+Table > [2] > https://enjoyment.cool/2020/02/19/Deep-dive-how-to-support-Python-UDF-in-Apache-Flink-1-10/ > > Chad Dombrova 于2020年2月21日周五 上午12:16写道: > >> Hi, >> Very cool. I’m curious about the relationship between this feature and >> Apache Beam. What parts of Beam are used and for what? Does this have any >> impact on existing Beam users like myself who use the Beam python API on >> top of Flink? Can someone give me a brief overview or point me at the >> right documentation? >> >> Thanks! >> -chad >> >> >> On Thu, Feb 20, 2020 at 5:39 AM Xingbo Huang wrote: >> >>> Thanks a lot for the release. >>> Great Work, Jincheng! >>> Also thanks to participants who contribute to this release. >>> >>> Best, >>> Xingbo >>> >>> >>> Till Rohrmann 于2020年2月18日周二 下午11:40写道: >>> Thanks for updating the 1.9.2 release wrt Flink's Python API Jincheng! Cheers, Till On Thu, Feb 13, 2020 at 12:25 PM Hequn Cheng wrote: > Thanks a lot for the release, Jincheng! > Also thanks to everyone that make this release possible! > > Best, > Hequn > > On Thu, Feb 13, 2020 at 2:18 PM Dian Fu wrote: > > > Thanks for the great work, Jincheng. > > > > Regards, > > Dian > > > > 在 2020年2月13日,下午1:32,jincheng sun 写道: > > > > Hi everyone, > > > > The Apache Flink community is very happy to announce the release of > Apache > > Flink Python API(PyFlink) 1.9.2, which is the first release to PyPI > for the > > Apache Flink Python API 1.9 series. > > > > Apache Flink® is an open-source stream processing framework for > > distributed, high-performing, always-available, and accurate data > streaming > > applications. > > > > The release is available for download at: > > > > https://pypi.org/project/apache-flink/1.9.2/#files > > > > Or installed using pip command: > > > > pip install apache-flink==1.9.2 > > > > We would like to thank all contributors of the Apache Flink > community who > > helped to verify this release and made this release possible! > > > > Best, > > Jincheng > > > > > > >
[jira] [Created] (FLINK-16237) Wordcount on Docker test (custom fs plugin) fails.
Yangze Guo created FLINK-16237: -- Summary: Wordcount on Docker test (custom fs plugin) fails. Key: FLINK-16237 URL: https://issues.apache.org/jira/browse/FLINK-16237 Project: Flink Issue Type: Bug Components: Tests Reporter: Yangze Guo Wordcount on Docker test (custom fs plugin) fails on Travis. {code:bash} pass WordCount Checking for errors... Found error in log files: Attaching to docker_job-cluster_1 job-cluster_1 | Starting the job-cluster job-cluster_1 | Starting standalonejob as a console application on host 042c3c490edc. job-cluster_1 | ERROR StatusLogger No Log4j 2 configuration file found. Using default configuration (logging only errors to the console), or user programmatically provided configurations. Set system property 'log4j2.debug' to show Log4j 2 internal initialization logging. See https://logging.apache.org/log4j/2.x/manual/configuration.html for instructions on how to configure Log4j 2 Attaching to docker_taskmanager_1 taskmanager_1 | Starting the task-manager taskmanager_1 | Starting taskexecutor as a console application on host 059bd37e4232. taskmanager_1 | ERROR StatusLogger No Log4j 2 configuration file found. Using default configuration (logging only errors to the console), or user programmatically provided configurations. Set system property 'log4j2.debug' to show Log4j 2 internal initialization logging. See https://logging.apache.org/log4j/2.x/manual/configuration.html for instructions on how to configure Log4j 2 Checking for exceptions... {code} -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (FLINK-16238) Rename class name of Fabric8ClientTest to Fabric8FlinkKubeClient
Canbin Zheng created FLINK-16238: Summary: Rename class name of Fabric8ClientTest to Fabric8FlinkKubeClient Key: FLINK-16238 URL: https://issues.apache.org/jira/browse/FLINK-16238 Project: Flink Issue Type: Improvement Components: Deployment / Kubernetes Affects Versions: 1.10.0 Reporter: Canbin Zheng Fix For: 1.11.0 It's a minor change to alignment the test class name of \{{org.apache.flink.kubernetes.kubeclient.Fabric8FlinkKubeClient}} -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (FLINK-16239) Port KubernetesSessionCliTest to the right package
Canbin Zheng created FLINK-16239: Summary: Port KubernetesSessionCliTest to the right package Key: FLINK-16239 URL: https://issues.apache.org/jira/browse/FLINK-16239 Project: Flink Issue Type: Improvement Components: Deployment / Kubernetes Affects Versions: 1.10.0 Reporter: Canbin Zheng Fix For: 1.11.0 Port KubernetesSessionCliTest from {{org.apache.flink.kubernetes}} to {{org.apache.flink.kubernetes.cli}} -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (FLINK-16240) Port KubernetesUtilsTest to the right package
Canbin Zheng created FLINK-16240: Summary: Port KubernetesUtilsTest to the right package Key: FLINK-16240 URL: https://issues.apache.org/jira/browse/FLINK-16240 Project: Flink Issue Type: Improvement Components: Deployment / Kubernetes Affects Versions: 1.10.0 Reporter: Canbin Zheng Fix For: 1.11.0 Port {{KubernetesUtilsTest}} from {{org.apache.flink.kubernetes}} to {{org.apache.flink.kubernetes.utils.}} -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (FLINK-16241) Remove the license and notice file in flink-ml-lib module on release-1.10 branch
Hequn Cheng created FLINK-16241: --- Summary: Remove the license and notice file in flink-ml-lib module on release-1.10 branch Key: FLINK-16241 URL: https://issues.apache.org/jira/browse/FLINK-16241 Project: Flink Issue Type: Bug Components: Library / Machine Learning Affects Versions: 1.10.0 Reporter: Hequn Cheng Assignee: Hequn Cheng Fix For: 1.10.1 The jar of flink-ml-lib should not contain the license and notice file as it actually does not bundle the related dependencies. We should remove these file on branch release-1.10. BTW. The release-1.9 branch does not have this problem since the license and notice are added in 1.10. On master(1.11), we will bundle the dependencies, so the license and notice file should be kept, see FLINK-15847. -- This message was sent by Atlassian Jira (v8.3.4#803005)