[jira] [Created] (FLINK-16231) Hive connector is missing jdk.tools exclusion against Hive 2.x.x
Chesnay Schepler created FLINK-16231: Summary: Hive connector is missing jdk.tools exclusion against Hive 2.x.x Key: FLINK-16231 URL: https://issues.apache.org/jira/browse/FLINK-16231 Project: Flink Issue Type: Bug Components: Build System, Connectors / Hive Affects Versions: 1.10.0 Reporter: Chesnay Schepler Assignee: Chesnay Schepler Fix For: 1.10.1, 1.11.0 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (FLINK-16232) Remove redundant log4j2 exclusions from hive connector
Chesnay Schepler created FLINK-16232: Summary: Remove redundant log4j2 exclusions from hive connector Key: FLINK-16232 URL: https://issues.apache.org/jira/browse/FLINK-16232 Project: Flink Issue Type: Improvement Components: Build System, Connectors / Hive Reporter: Chesnay Schepler Assignee: Chesnay Schepler Fix For: 1.11.0 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (FLINK-16233) Hive connector missing log4j1 exclusions against certain hive versions
Chesnay Schepler created FLINK-16233: Summary: Hive connector missing log4j1 exclusions against certain hive versions Key: FLINK-16233 URL: https://issues.apache.org/jira/browse/FLINK-16233 Project: Flink Issue Type: Bug Components: Build System, Connectors / Hive Affects Versions: 1.11.0 Reporter: Chesnay Schepler Assignee: Chesnay Schepler Fix For: 1.11.0 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (FLINK-16234) Use LinkedHashSet in StreamGraph.java
cpugputpu created FLINK-16234: - Summary: Use LinkedHashSet in StreamGraph.java Key: FLINK-16234 URL: https://issues.apache.org/jira/browse/FLINK-16234 Project: Flink Issue Type: Bug Reporter: cpugputpu The test in _org.apache.flink.streaming.api.graph.StreamingJobGraphGeneratorTest#testSlotSharingOnAllVerticesInSameSlotSharingGroupByDefaultDisabled_ will cause the following failure: java.lang.AssertionError: expected: but was: at org.apache.flink.streaming.api.graph.StreamingJobGraphGeneratorTest.assertSameSlotSharingGroup(StreamingJobGraphGeneratorTest.java:843) at org.apache.flink.streaming.api.graph.StreamingJobGraphGeneratorTest.testSlotSharingOnAllVerticesInSameSlotSharingGroupByDefaultDisabled(StreamingJobGraphGeneratorTest.java:814) I analyze the assertion failure and find that the root cause of it lies in the clear() method in StreamGraph.java, where the variable _sources_ is initialized as a HashSet. Because the iteration order of HashSet is non-deterministic, so the test becomes flaky. The fix is to change _HashSet_ to _LinkedHashSet_ and then the failure above is removed. The stacktrace information is presented as follows for your reference: java.util.HashSet.iterator(HashSet.java:173) org.apache.flink.streaming.api.graph.StreamingJobGraphGenerator.setChaining(StreamingJobGraphGenerator.java:251) org.apache.flink.streaming.api.graph.StreamingJobGraphGenerator.createJobGraph(StreamingJobGraphGenerator.java:166) org.apache.flink.streaming.api.graph.StreamingJobGraphGenerator.createJobGraph(StreamingJobGraphGenerator.java:104) org.apache.flink.streaming.api.graph.StreamingJobGraphGenerator.createJobGraph(StreamingJobGraphGenerator.java:100) org.apache.flink.streaming.api.graph.StreamingJobGraphGeneratorTest.testSlotSharingOnAllVerticesInSameSlotSharingGroupByDefaultDisabled(StreamingJobGraphGeneratorTest.java:803) -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (FLINK-16235) Use LinkedHashSet in StreamGraph.java
cpugputpu created FLINK-16235: - Summary: Use LinkedHashSet in StreamGraph.java Key: FLINK-16235 URL: https://issues.apache.org/jira/browse/FLINK-16235 Project: Flink Issue Type: Bug Reporter: cpugputpu The test in _org.apache.flink.streaming.api.graph.StreamingJobGraphGeneratorTest#testSlotSharingOnAllVerticesInSameSlotSharingGroupByDefaultDisabled_ will cause the following failure: java.lang.AssertionError: expected: but was: at org.apache.flink.streaming.api.graph.StreamingJobGraphGeneratorTest.assertSameSlotSharingGroup(StreamingJobGraphGeneratorTest.java:843) at org.apache.flink.streaming.api.graph.StreamingJobGraphGeneratorTest.testSlotSharingOnAllVerticesInSameSlotSharingGroupByDefaultDisabled(StreamingJobGraphGeneratorTest.java:814) I analyze the assertion failure and find that the root cause of it lies in the clear() method in StreamGraph.java, where the variable _sources_ is initialized as a HashSet. Because the iteration order of HashSet is non-deterministic, so the test becomes flaky. The fix is to change _HashSet_ to _LinkedHashSet_ and then the failure above is removed. The stacktrace information is presented as follows for your reference: java.util.HashSet.iterator(HashSet.java:173) org.apache.flink.streaming.api.graph.StreamingJobGraphGenerator.setChaining(StreamingJobGraphGenerator.java:251) org.apache.flink.streaming.api.graph.StreamingJobGraphGenerator.createJobGraph(StreamingJobGraphGenerator.java:166) org.apache.flink.streaming.api.graph.StreamingJobGraphGenerator.createJobGraph(StreamingJobGraphGenerator.java:104) org.apache.flink.streaming.api.graph.StreamingJobGraphGenerator.createJobGraph(StreamingJobGraphGenerator.java:100) org.apache.flink.streaming.api.graph.StreamingJobGraphGeneratorTest.testSlotSharingOnAllVerticesInSameSlotSharingGroupByDefaultDisabled(StreamingJobGraphGeneratorTest.java:803) -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (FLINK-16236) Fix YARNSessionFIFOSecuredITCase not loading the correct security context factory
Rong Rong created FLINK-16236: - Summary: Fix YARNSessionFIFOSecuredITCase not loading the correct security context factory Key: FLINK-16236 URL: https://issues.apache.org/jira/browse/FLINK-16236 Project: Flink Issue Type: Sub-task Components: Deployment / YARN Reporter: Rong Rong Assignee: Rong Rong Follow up on FLINK-11589. Currently due to the override of the TestHadoopModuleFactory, it is not loading the HadoopContextFactory due to the compatibility checker. -- This message was sent by Atlassian Jira (v8.3.4#803005)
Re: [ANNOUNCE] Apache Flink Python API(PyFlink) 1.9.2 released
It’s an interesting idea to use the Beam SDK to execute PyFlink (i.e. non-Beam) UDFs. Thanks for the info. On Thu, Feb 20, 2020 at 5:41 PM Xingbo Huang wrote: > Hi Chad, > Beam Portability module solves the management of the Python runtime > environment and the communication between the Python runtime environment > Python VM and the Java runtime environment JVM in PyFlink. For details, you > can refer to FLIP-58[1] and the Blog[2]. > > Hope to resolve your doubts. > > Best, > Xingbo > > [1] > https://cwiki.apache.org/confluence/display/FLINK/FLIP-58%3A+Flink+Python+User-Defined+Stateless+Function+for+Table > [2] > https://enjoyment.cool/2020/02/19/Deep-dive-how-to-support-Python-UDF-in-Apache-Flink-1-10/ > > Chad Dombrova 于2020年2月21日周五 上午12:16写道： > >> Hi, >> Very cool. I’m curious about the relationship between this feature and >> Apache Beam. What parts of Beam are used and for what? Does this have any >> impact on existing Beam users like myself who use the Beam python API on >> top of Flink? Can someone give me a brief overview or point me at the >> right documentation? >> >> Thanks! >> -chad >> >> >> On Thu, Feb 20, 2020 at 5:39 AM Xingbo Huang wrote: >> >>> Thanks a lot for the release. >>> Great Work, Jincheng! >>> Also thanks to participants who contribute to this release. >>> >>> Best, >>> Xingbo >>> >>> >>> Till Rohrmann 于2020年2月18日周二 下午11:40写道： >>> Thanks for updating the 1.9.2 release wrt Flink's Python API Jincheng! Cheers, Till On Thu, Feb 13, 2020 at 12:25 PM Hequn Cheng wrote: > Thanks a lot for the release, Jincheng! > Also thanks to everyone that make this release possible! > > Best, > Hequn > > On Thu, Feb 13, 2020 at 2:18 PM Dian Fu wrote: > > > Thanks for the great work, Jincheng. > > > > Regards, > > Dian > > > > 在 2020年2月13日，下午1:32，jincheng sun 写道： > > > > Hi everyone, > > > > The Apache Flink community is very happy to announce the release of > Apache > > Flink Python API(PyFlink) 1.9.2, which is the first release to PyPI > for the > > Apache Flink Python API 1.9 series. > > > > Apache Flink® is an open-source stream processing framework for > > distributed, high-performing, always-available, and accurate data > streaming > > applications. > > > > The release is available for download at: > > > > https://pypi.org/project/apache-flink/1.9.2/#files > > > > Or installed using pip command: > > > > pip install apache-flink==1.9.2 > > > > We would like to thank all contributors of the Apache Flink > community who > > helped to verify this release and made this release possible! > > > > Best, > > Jincheng > > > > > > >
[jira] [Created] (FLINK-16237) Wordcount on Docker test (custom fs plugin) fails.
Yangze Guo created FLINK-16237:
--
Summary: Wordcount on Docker test (custom fs plugin) fails.
Key: FLINK-16237
URL: https://issues.apache.org/jira/browse/FLINK-16237
Project: Flink
Issue Type: Bug
Components: Tests
Reporter: Yangze Guo
Wordcount on Docker test (custom fs plugin) fails on Travis.
{code:bash}
pass WordCount
Checking for errors...
Found error in log files:
Attaching to docker_job-cluster_1
job-cluster_1 | Starting the job-cluster
job-cluster_1 | Starting standalonejob as a console application on host
042c3c490edc.
job-cluster_1 | ERROR StatusLogger No Log4j 2 configuration file found. Using
default configuration (logging only errors to the console), or user
programmatically provided configurations. Set system property 'log4j2.debug' to
show Log4j 2 internal initialization logging. See
https://logging.apache.org/log4j/2.x/manual/configuration.html for instructions
on how to configure Log4j 2
Attaching to docker_taskmanager_1
taskmanager_1 | Starting the task-manager
taskmanager_1 | Starting taskexecutor as a console application on host
059bd37e4232.
taskmanager_1 | ERROR StatusLogger No Log4j 2 configuration file found. Using
default configuration (logging only errors to the console), or user
programmatically provided configurations. Set system property 'log4j2.debug' to
show Log4j 2 internal initialization logging. See
https://logging.apache.org/log4j/2.x/manual/configuration.html for instructions
on how to configure Log4j 2
Checking for exceptions...
{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (FLINK-16238) Rename class name of Fabric8ClientTest to Fabric8FlinkKubeClient
Canbin Zheng created FLINK-16238:
Summary: Rename class name of Fabric8ClientTest to
Fabric8FlinkKubeClient
Key: FLINK-16238
URL: https://issues.apache.org/jira/browse/FLINK-16238
Project: Flink
Issue Type: Improvement
Components: Deployment / Kubernetes
Affects Versions: 1.10.0
Reporter: Canbin Zheng
Fix For: 1.11.0
It's a minor change to alignment the test class name of
\{{org.apache.flink.kubernetes.kubeclient.Fabric8FlinkKubeClient}}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (FLINK-16239) Port KubernetesSessionCliTest to the right package
Canbin Zheng created FLINK-16239:
Summary: Port KubernetesSessionCliTest to the right package
Key: FLINK-16239
URL: https://issues.apache.org/jira/browse/FLINK-16239
Project: Flink
Issue Type: Improvement
Components: Deployment / Kubernetes
Affects Versions: 1.10.0
Reporter: Canbin Zheng
Fix For: 1.11.0
Port KubernetesSessionCliTest from {{org.apache.flink.kubernetes}} to
{{org.apache.flink.kubernetes.cli}}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (FLINK-16240) Port KubernetesUtilsTest to the right package
Canbin Zheng created FLINK-16240:
Summary: Port KubernetesUtilsTest to the right package
Key: FLINK-16240
URL: https://issues.apache.org/jira/browse/FLINK-16240
Project: Flink
Issue Type: Improvement
Components: Deployment / Kubernetes
Affects Versions: 1.10.0
Reporter: Canbin Zheng
Fix For: 1.11.0
Port {{KubernetesUtilsTest}} from {{org.apache.flink.kubernetes}} to
{{org.apache.flink.kubernetes.utils.}}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (FLINK-16241) Remove the license and notice file in flink-ml-lib module on release-1.10 branch
Hequn Cheng created FLINK-16241: --- Summary: Remove the license and notice file in flink-ml-lib module on release-1.10 branch Key: FLINK-16241 URL: https://issues.apache.org/jira/browse/FLINK-16241 Project: Flink Issue Type: Bug Components: Library / Machine Learning Affects Versions: 1.10.0 Reporter: Hequn Cheng Assignee: Hequn Cheng Fix For: 1.10.1 The jar of flink-ml-lib should not contain the license and notice file as it actually does not bundle the related dependencies. We should remove these file on branch release-1.10. BTW. The release-1.9 branch does not have this problem since the license and notice are added in 1.10. On master(1.11), we will bundle the dependencies, so the license and notice file should be kept, see FLINK-15847. -- This message was sent by Atlassian Jira (v8.3.4#803005)
