[
https://issues.apache.org/jira/browse/BEAM-9645?focusedWorklogId=415407&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-415407
]
ASF GitHub Bot logged work on BEAM-9645:
----------------------------------------
Author: ASF GitHub Bot
Created on: 03/Apr/20 09:37
Start Date: 03/Apr/20 09:37
Worklog Time Spent: 10m
Work Description: mxm commented on pull request #11303: [BEAM-9645] Fix
premature removal of Docker container and logs
URL: https://github.com/apache/beam/pull/11303
The default would lead to removing the Docker container on container exit
which would prevent log retrieval. This was a regression of
fbc84b61240a3d83d9c19f7ccc17ba22e5d7e2c9.
Post-Commit Tests Status (on master branch)
------------------------------------------------------------------------------------------------
Lang | SDK | Apex | Dataflow | Flink | Gearpump | Samza | Spark
--- | --- | --- | --- | --- | --- | --- | ---
Go | [](https://builds.apache.org/job/beam_PostCommit_Go/lastCompletedBuild/)
| --- | --- | [](https://builds.apache.org/job/beam_PostCommit_Go_VR_Flink/lastCompletedBuild/)
| --- | --- | [](https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/lastCompletedBuild/)
Java | [](https://builds.apache.org/job/beam_PostCommit_Java/lastCompletedBuild/)
| [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Apex/lastCompletedBuild/)
| [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_Java11/lastCompletedBuild/)
| [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Flink_Batch/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Flink_Streaming/lastCompletedBuild/)
| [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Gearpump/lastCompletedBuild/)
| [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Samza/lastCompletedBuild/)
| [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Spark_Batch/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_SparkStructuredStreaming/lastCompletedBuild/)
Python | [](https://builds.apache.org/job/beam_PostCommit_Python2/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Python35/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Python36/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Python37/lastCompletedBuild/)
| --- | [](https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow_V2/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Py_ValCont/lastCompletedBuild/)
| [](https://builds.apache.org/job/beam_PreCommit_Python2_PVR_Flink_Cron/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Python35_VR_Flink/lastCompletedBuild/)
| --- | --- | [](https://builds.apache.org/job/beam_PostCommit_Python_VR_Spark/lastCompletedBuild/)
XLang | --- | --- | --- | [](https://builds.apache.org/job/beam_PostCommit_XVR_Flink/lastCompletedBuild/)
| --- | --- | [](https://builds.apache.org/job/beam_PostCommit_XVR_Spark/lastCompletedBuild/)
Pre-Commit Tests Status (on master branch)
------------------------------------------------------------------------------------------------
--- |Java | Python | Go | Website
--- | --- | --- | --- | ---
Non-portable | [](https://builds.apache.org/job/beam_PreCommit_Java_Cron/lastCompletedBuild/)
| [](https://builds.apache.org/job/beam_PreCommit_Python_Cron/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PreCommit_PythonLint_Cron/lastCompletedBuild/)
| [](https://builds.apache.org/job/beam_PreCommit_Go_Cron/lastCompletedBuild/)
| [](https://builds.apache.org/job/beam_PreCommit_Website_Cron/lastCompletedBuild/)
Portable | --- | [](https://builds.apache.org/job/beam_PreCommit_Portable_Python_Cron/lastCompletedBuild/)
| --- | ---
See
[.test-infra/jenkins/README](https://github.com/apache/beam/blob/master/.test-infra/jenkins/README.md)
for trigger phrase, status and link of all Jenkins jobs.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
-------------------
Worklog Id: (was: 415407)
Remaining Estimate: 0h
Time Spent: 10m
> Python flinkrunner cannot inspect container
> -------------------------------------------
>
> Key: BEAM-9645
> URL: https://issues.apache.org/jira/browse/BEAM-9645
> Project: Beam
> Issue Type: Bug
> Components: runner-flink
> Affects Versions: 2.19.0
> Environment: dataproc cluster, running flink version 1.9
> Reporter: Léopold Boudard
> Priority: Major
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Hi,
> I'm trying to submit a python pipeline job as portable runner with
> FlinkRunner with a very simple setup file, though I can't see error logs
> since it fails retrieving logs/state from underlying container:
> {code:java}
> Caused by: java.io.IOException: Received exit code 1 for command 'docker
> inspect -f {{.State.Running}}
> 248d660be908ef58385eb962658cee831c4c3b1be9ea1f835e4563f53016363b'. stderr:
> Error: No such object:
> 248d660be908ef58385eb962658cee831c4c3b1be9ea1f835e4563f53016363bCaused by:
> java.io.IOException: Received exit code 1 for command 'docker inspect -f
> {{.State.Running}}
> 248d660be908ef58385eb962658cee831c4c3b1be9ea1f835e4563f53016363b'. stderr:
> Error: No such object:
> 248d660be908ef58385eb962658cee831c4c3b1be9ea1f835e4563f53016363b at
> org.apache.beam.runners.fnexecution.environment.DockerCommand.runShortCommand(DockerCommand.java:234)
> at
> org.apache.beam.runners.fnexecution.environment.DockerCommand.runShortCommand(DockerCommand.java:168)
> at
> org.apache.beam.runners.fnexecution.environment.DockerCommand.isContainerRunning(DockerCommand.java:112)
> at
> org.apache.beam.runners.fnexecution.environment.DockerEnvironmentFactory.createEnvironment(DockerEnvironmentFactory.java:165)
> at
> org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$1.load(DefaultJobBundleFactory.java:200)
> at
> org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$1.load(DefaultJobBundleFactory.java:184)
> at
> org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3528)
> at
> org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2277)
> at
> org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2154)
> at
> org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2044)
> at
> org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache.get(LocalCache.java:3952)
> at
> org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:3974)
> at
> org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:4958)
> at
> org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$LocalLoadingCache.getUnchecked(LocalCache.java:4964)
> ... 12 more Suppressed: java.io.IOException: Received exit code 1 for
> command 'docker kill
> 248d660be908ef58385eb962658cee831c4c3b1be9ea1f835e4563f53016363b'. stderr:
> Error response from daemon: Cannot kill container:
> 248d660be908ef58385eb962658cee831c4c3b1be9ea1f835e4563f53016363b: No such
> container: 248d660be908ef58385eb962658cee831c4c3b1be9ea1f835e4563f53016363b
> at
> org.apache.beam.runners.fnexecution.environment.DockerCommand.runShortCommand(DockerCommand.java:234)
> at
> org.apache.beam.runners.fnexecution.environment.DockerCommand.runShortCommand(DockerCommand.java:168)
> at
> org.apache.beam.runners.fnexecution.environment.DockerCommand.killContainer(DockerCommand.java:148)
> at
> org.apache.beam.runners.fnexecution.environment.DockerEnvironmentFactory.createEnvironment(DockerEnvironmentFactory.java:192)
> ... 22 moreERROR:root:java.io.IOException: Received exit code 1 for command
> 'docker inspect -f {{.State.Running}}
> 248d660be908ef58385eb962658cee831c4c3b1be9ea1f835e4563f53016363b'. stderr:
> Error: No such object:
> 248d660be908ef58385eb962658cee831c4c3b1be9ea1f835e4563f53016363b[flink-runner-job-invoker]
> INFO
> org.apache.beam.runners.fnexecution.artifact.AbstractArtifactRetrievalService
> - Manifest at
> /var/folders/s3/29yl1s8125j33_9vb5tczdxw0000gn/T/beam-tempiwx1szcz/artifacts6ptzmo6u/job_a340349c-cc95-4e32-9cbd-7f915c2d0407/MANIFEST
> has 1 artifact locations[flink-runner-job-invoker] INFO
> org.apache.beam.runners.fnexecution.artifact.BeamFileSystemArtifactStagingService
> - Removed dir
> /var/folders/s3/29yl1s8125j33_9vb5tczdxw0000gn/T/beam-tempiwx1szcz/artifacts6ptzmo6u/job_a340349c-cc95-4e32-9cbd-7f915c2d0407/Traceback
> (most recent call last): File "importer/test_runner.py", line 46, in
> <module> run() File "importer/test_runner.py", line 41, in run |
> 'write to file' >> WriteToText(known_args.output) File
> "/Users/leopold/.pyenv/versions/BenchmarkListingStreaming/lib/python3.6/site-packages/apache_beam/pipeline.py",
> line 481, in __exit__ self.run().wait_until_finish() File
> "/Users/leopold/.pyenv/versions/BenchmarkListingStreaming/lib/python3.6/site-packages/apache_beam/runners/portability/portable_runner.py",
> line 455, in wait_until_finish self._job_id, self._state,
> self._last_error_message()))
> {code}
>
> Job args
> --runner FlinkRunner --flink_master=\{flink_master} --flink_version 1.9
> --setup_file ./setup.py
> And corresponding setup file
> {code:java}
> import setuptoolsinstall_requires = ['networkx==2.4', 'PyYAML==5.3']
> setuptools.setup(
> name='benchmark_listing',
> version='0.1.0', #build_version('0.1.0'), author='MeilleursAgents',
> author_email='ge...@meilleursagents.com',
> description='Benchmark Listing Streaming',
> # long_description=long_description,
> # long_description_content_type='text/markdown',
> install_requires=install_requires,
> # packages=setuptools.find_packages(),
> # python_requires='>=3.7.6',
> # include_package_data=True,
> )
> {code}
> Could you advise on this issue please?
> Thanks!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
