[ https://issues.apache.org/jira/browse/BEAM-6683?focusedWorklogId=255027&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-255027 ]
ASF GitHub Bot logged work on BEAM-6683: ---------------------------------------- Author: ASF GitHub Bot Created on: 06/Jun/19 10:14 Start Date: 06/Jun/19 10:14 Worklog Time Spent: 10m Work Description: mxm commented on issue #8174: [BEAM-6683] add createCrossLanguageValidatesRunner task URL: https://github.com/apache/beam/pull/8174#issuecomment-499435604 Now I'm getting this when running the tests: ``` ====================================================================== ERROR: test_java_expansion_portable_runner (apache_beam.transforms.external_test.ExternalTransformTest) ---------------------------------------------------------------------- Traceback (most recent call last): File "/Users/max/Dev/beam/sdks/python/apache_beam/transforms/external_test.py", line 153, in test_java_expansion_portable_runner ExternalTransformTest.run_pipeline_with_portable_runner(None) File "/Users/max/Dev/beam/sdks/python/apache_beam/transforms/external_test.py", line 186, in run_pipeline_with_portable_runner pipeline_options, ExternalTransformTest.expansion_service_port, True) File "/Users/max/Dev/beam/sdks/python/apache_beam/transforms/external_test.py", line 219, in run_pipeline result = p.run() File "/Users/max/Dev/beam/sdks/python/apache_beam/testing/test_pipeline.py", line 107, in run else test_runner_api)) File "/Users/max/Dev/beam/sdks/python/apache_beam/pipeline.py", line 419, in run return self.runner.run_pipeline(self, self._options) File "/Users/max/Dev/beam/sdks/python/apache_beam/runners/portability/portable_runner.py", line 161, in run_pipeline self.init_dockerized_job_server() File "/Users/max/Dev/beam/sdks/python/apache_beam/runners/portability/portable_runner.py", line 148, in init_dockerized_job_server self._job_endpoint = docker.start() File "/Users/max/Dev/beam/sdks/python/apache_beam/runners/portability/job_server.py", line 56, in start docker_path = check_output(['which', 'docker']).strip() File "/usr/local/Cellar/python@2/2.7.15_1/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 216, in check_output process = Popen(stdout=PIPE, *popenargs, **kwargs) File "/usr/local/Cellar/python@2/2.7.15_1/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 394, in __init__ errread, errwrite) File "/usr/local/Cellar/python@2/2.7.15_1/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 1047, in _execute_child raise child_exception OSError: [Errno 9] Bad file descriptor ``` Strange because `which docker` returns the correct Docker executable on my machine. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking ------------------- Worklog Id: (was: 255027) Time Spent: 14h 20m (was: 14h 10m) > Add an integration test suite for cross-language transforms for Flink runner > ---------------------------------------------------------------------------- > > Key: BEAM-6683 > URL: https://issues.apache.org/jira/browse/BEAM-6683 > Project: Beam > Issue Type: Test > Components: testing > Reporter: Chamikara Jayalath > Assignee: Heejong Lee > Priority: Major > Time Spent: 14h 20m > Remaining Estimate: 0h > > We should add an integration test suite that covers following. > (1) Currently available Java IO connectors that do not use UDFs work for > Python SDK on Flink runner. > (2) Currently available Python IO connectors that do not use UDFs work for > Java SDK on Flink runner. > (3) Currently available Java/Python pipelines work in a scalable manner for > cross-language pipelines (for example, try 10GB, 100GB input for > textio/avroio for Java and Python). > -- This message was sent by Atlassian JIRA (v7.6.3#76005)