[
https://issues.apache.org/jira/browse/BEAM-1924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15963254#comment-15963254
]
Sourabh Bajaj commented on BEAM-1924:
-------------------------------------
One improvement that comes to mind is that we should pass the subprocess.PIPE
options in this call similar to how we do it in the juliaset example
> pip download failed in post commit
> ----------------------------------
>
> Key: BEAM-1924
> URL: https://issues.apache.org/jira/browse/BEAM-1924
> Project: Beam
> Issue Type: Bug
> Components: sdk-py
> Reporter: Ahmet Altay
>
> A post commit failed with a pip failure:
> https://builds.apache.org/job/beam_PostCommit_Python_Verify/1801/consoleFull
> Captured output is not clear enough to tell what went wrong:
> ======================================================================
> ERROR: test_par_do_with_multiple_outputs_and_using_return
> (apache_beam.transforms.ptransform_test.PTransformTest)
> ----------------------------------------------------------------------
> Traceback (most recent call last):
> File
> "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python_Verify/sdks/python/apache_beam/transforms/ptransform_test.py",
> line 207, in test_par_do_with_multiple_outputs_and_using_return
> pipeline.run()
> File
> "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python_Verify/sdks/python/apache_beam/test_pipeline.py",
> line 91, in run
> result = super(TestPipeline, self).run()
> File
> "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python_Verify/sdks/python/apache_beam/pipeline.py",
> line 160, in run
> self.to_runner_api(), self.runner, self.options).run(False)
> File
> "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python_Verify/sdks/python/apache_beam/pipeline.py",
> line 169, in run
> return self.runner.run(self)
> File
> "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python_Verify/sdks/python/apache_beam/runners/dataflow/test_dataflow_runner.py",
> line 39, in run
> self.result = super(TestDataflowRunner, self).run(pipeline)
> File
> "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python_Verify/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",
> line 175, in run
> self.dataflow_client.create_job(self.job), self)
> File
> "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python_Verify/sdks/python/apache_beam/utils/retry.py",
> line 174, in wrapper
> return fun(*args, **kwargs)
> File
> "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python_Verify/sdks/python/apache_beam/runners/dataflow/internal/apiclient.py",
> line 425, in create_job
> self.create_job_description(job)
> File
> "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python_Verify/sdks/python/apache_beam/runners/dataflow/internal/apiclient.py",
> line 446, in create_job_description
> job.options, file_copy=self._gcs_file_copy)
> File
> "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python_Verify/sdks/python/apache_beam/runners/dataflow/internal/dependency.py",
> line 306, in stage_job_resources
> setup_options.requirements_file, requirements_cache_path)
> File
> "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python_Verify/sdks/python/apache_beam/runners/dataflow/internal/dependency.py",
> line 242, in _populate_requirements_cache
> processes.check_call(cmd_args)
> File
> "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python_Verify/sdks/python/apache_beam/utils/processes.py",
> line 40, in check_call
> return subprocess.check_call(*args, **kwargs)
> File "/usr/lib/python2.7/subprocess.py", line 540, in check_call
> raise CalledProcessError(retcode, cmd)
> CalledProcessError: Command
> '['/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python_Verify/sdks/python/bin/python',
> '-m', 'pip', 'install', '--download', '/tmp/dataflow-requirements-cache',
> '-r', 'postcommit_requirements.txt', '--no-binary', ':all:']' returned
> non-zero exit status 2
> Any ideas on how we can debug this failure or improve for the future? (cc:
> [~markflyhigh] [~sb2nov])
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
