[jira] [Commented] (FLINK-34892) Nightly AWS connectors build fails on running python tests
[
https://issues.apache.org/jira/browse/FLINK-34892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17829440#comment-17829440
]
Danny Cranmer commented on FLINK-34892:
---
Merged commit
[{{e8ba71e}}|https://github.com/apache/flink-connector-aws/commit/e8ba71ec3c27903c838701d536a8ae05bc5bb523]
into apache:main
> Nightly AWS connectors build fails on running python tests
> --
>
> Key: FLINK-34892
> URL: https://issues.apache.org/jira/browse/FLINK-34892
> Project: Flink
> Issue Type: Bug
> Components: Connectors / AWS
>Affects Versions: aws-connector-4.2.0
>Reporter: Aleksandr Pilipenko
>Priority: Major
> Labels: pull-request-available
>
> Build for externalized python connector code fails:
> [https://github.com/apache/flink-connector-aws/actions/runs/8351768294/job/22860710449]
> {code:java}
> 2024-03-20T00:14:35.5215863Z __
> FlinkKinesisTest.test_kinesis_streams_sink __
> 2024-03-20T00:14:35.5216781Z
> .tox/py310-cython/lib/python3.10/site-packages/pyflink/testing/test_case_utils.py:149:
> in setUp
> 2024-03-20T00:14:35.5217584Z self.env =
> StreamExecutionEnvironment.get_execution_environment()
> 2024-03-20T00:14:35.5218901Z
> .tox/py310-cython/lib/python3.10/site-packages/pyflink/datastream/stream_execution_environment.py:876:
> in get_execution_environment
> 2024-03-20T00:14:35.5219751Z gateway = get_gateway()
> 2024-03-20T00:14:35.5220635Z
> .tox/py310-cython/lib/python3.10/site-packages/pyflink/java_gateway.py:64: in
> get_gateway
> 2024-03-20T00:14:35.5221378Z _gateway = launch_gateway()
> 2024-03-20T00:14:35.5222111Z
> .tox/py310-cython/lib/python3.10/site-packages/pyflink/java_gateway.py:110:
> in launch_gateway
> 2024-03-20T00:14:35.5222956Z p = launch_gateway_server_process(env, args)
> 2024-03-20T00:14:35.5223854Z
> .tox/py310-cython/lib/python3.10/site-packages/pyflink/pyflink_gateway_server.py:262:
> in launch_gateway_server_process
> 2024-03-20T00:14:35.5224649Z java_executable = find_java_executable()
> 2024-03-20T00:14:35.5225583Z
> .tox/py310-cython/lib/python3.10/site-packages/pyflink/pyflink_gateway_server.py:75:
> in find_java_executable
> 2024-03-20T00:14:35.5226449Z java_home =
> read_from_config(KEY_ENV_JAVA_HOME, None, flink_conf_file)
> 2024-03-20T00:14:35.5227099Z _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
> 2024-03-20T00:14:35.5227450Z
> 2024-03-20T00:14:35.5227774Z key = 'env.java.home', default_value = None
> 2024-03-20T00:14:35.5228925Z flink_conf_file =
> '/home/runner/work/flink-connector-aws/flink-connector-aws/flink-python/.tox/py310-cython/lib/python3.10/site-packages/pyflink/conf/flink-conf.yaml'
> 2024-03-20T00:14:35.5229778Z
> 2024-03-20T00:14:35.5230010Z def read_from_config(key, default_value,
> flink_conf_file):
> 2024-03-20T00:14:35.5230581Z value = default_value
> 2024-03-20T00:14:35.5231236Z # get the realpath of tainted path value
> to avoid CWE22 problem that constructs a path or URI
> 2024-03-20T00:14:35.5232195Z # using the tainted value and might
> allow an attacker to access, modify, or test the existence
> 2024-03-20T00:14:35.5232940Z # of critical or sensitive files.
> 2024-03-20T00:14:35.5233417Z > with
> open(os.path.realpath(flink_conf_file), "r") as f:
> 2024-03-20T00:14:35.5234874Z E FileNotFoundError: [Errno 2] No such
> file or directory:
> '/home/runner/work/flink-connector-aws/flink-connector-aws/flink-python/.tox/py310-cython/lib/python3.10/site-packages/pyflink/conf/flink-conf.yaml'
> 2024-03-20T00:14:35.5235954Z
> 2024-03-20T00:14:35.5236484Z
> .tox/py310-cython/lib/python3.10/site-packages/pyflink/pyflink_gateway_server.py:58:
> FileNotFoundError {code}
> Failure started after the release of apache-flink python package for 1.19.0
> due to change of default config file provided within artifact.
>
>
> Issue comes from outdated copy of pyflink_gateway_server.py created as part
> of [https://github.com/apache/flink-connector-kafka/pull/69] (same change is
> duplicated in AWS connectors repository).
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (FLINK-34892) Nightly AWS connectors build fails on running python tests
[
https://issues.apache.org/jira/browse/FLINK-34892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17828922#comment-17828922
]
Aleksandr Pilipenko commented on FLINK-34892:
-
This issue should also be present for Kafka connector. However, currently there
is no python tests running as part of the weekly build there.
> Nightly AWS connectors build fails on running python tests
> --
>
> Key: FLINK-34892
> URL: https://issues.apache.org/jira/browse/FLINK-34892
> Project: Flink
> Issue Type: Bug
> Components: Connectors / AWS
>Affects Versions: aws-connector-4.2.0
>Reporter: Aleksandr Pilipenko
>Priority: Major
> Labels: pull-request-available
>
> Build for externalized python connector code fails:
> [https://github.com/apache/flink-connector-aws/actions/runs/8351768294/job/22860710449]
> {code:java}
> 2024-03-20T00:14:35.5215863Z __
> FlinkKinesisTest.test_kinesis_streams_sink __
> 2024-03-20T00:14:35.5216781Z
> .tox/py310-cython/lib/python3.10/site-packages/pyflink/testing/test_case_utils.py:149:
> in setUp
> 2024-03-20T00:14:35.5217584Z self.env =
> StreamExecutionEnvironment.get_execution_environment()
> 2024-03-20T00:14:35.5218901Z
> .tox/py310-cython/lib/python3.10/site-packages/pyflink/datastream/stream_execution_environment.py:876:
> in get_execution_environment
> 2024-03-20T00:14:35.5219751Z gateway = get_gateway()
> 2024-03-20T00:14:35.5220635Z
> .tox/py310-cython/lib/python3.10/site-packages/pyflink/java_gateway.py:64: in
> get_gateway
> 2024-03-20T00:14:35.5221378Z _gateway = launch_gateway()
> 2024-03-20T00:14:35.5222111Z
> .tox/py310-cython/lib/python3.10/site-packages/pyflink/java_gateway.py:110:
> in launch_gateway
> 2024-03-20T00:14:35.5222956Z p = launch_gateway_server_process(env, args)
> 2024-03-20T00:14:35.5223854Z
> .tox/py310-cython/lib/python3.10/site-packages/pyflink/pyflink_gateway_server.py:262:
> in launch_gateway_server_process
> 2024-03-20T00:14:35.5224649Z java_executable = find_java_executable()
> 2024-03-20T00:14:35.5225583Z
> .tox/py310-cython/lib/python3.10/site-packages/pyflink/pyflink_gateway_server.py:75:
> in find_java_executable
> 2024-03-20T00:14:35.5226449Z java_home =
> read_from_config(KEY_ENV_JAVA_HOME, None, flink_conf_file)
> 2024-03-20T00:14:35.5227099Z _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
> 2024-03-20T00:14:35.5227450Z
> 2024-03-20T00:14:35.5227774Z key = 'env.java.home', default_value = None
> 2024-03-20T00:14:35.5228925Z flink_conf_file =
> '/home/runner/work/flink-connector-aws/flink-connector-aws/flink-python/.tox/py310-cython/lib/python3.10/site-packages/pyflink/conf/flink-conf.yaml'
> 2024-03-20T00:14:35.5229778Z
> 2024-03-20T00:14:35.5230010Z def read_from_config(key, default_value,
> flink_conf_file):
> 2024-03-20T00:14:35.5230581Z value = default_value
> 2024-03-20T00:14:35.5231236Z # get the realpath of tainted path value
> to avoid CWE22 problem that constructs a path or URI
> 2024-03-20T00:14:35.5232195Z # using the tainted value and might
> allow an attacker to access, modify, or test the existence
> 2024-03-20T00:14:35.5232940Z # of critical or sensitive files.
> 2024-03-20T00:14:35.5233417Z > with
> open(os.path.realpath(flink_conf_file), "r") as f:
> 2024-03-20T00:14:35.5234874Z E FileNotFoundError: [Errno 2] No such
> file or directory:
> '/home/runner/work/flink-connector-aws/flink-connector-aws/flink-python/.tox/py310-cython/lib/python3.10/site-packages/pyflink/conf/flink-conf.yaml'
> 2024-03-20T00:14:35.5235954Z
> 2024-03-20T00:14:35.5236484Z
> .tox/py310-cython/lib/python3.10/site-packages/pyflink/pyflink_gateway_server.py:58:
> FileNotFoundError {code}
> Failure started after the release of apache-flink python package for 1.19.0
> due to change of default config file provided within artifact.
>
>
> Issue comes from outdated copy of pyflink_gateway_server.py created as part
> of [https://github.com/apache/flink-connector-kafka/pull/69] (same change is
> duplicated in AWS connectors repository).
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
