See <https://builds.apache.org/job/beam_PostCommit_Python37/1073/display/redirect?page=changes>
Changes: [chadrik] Make local job service accessible from external machines [kirillkozlov] MongoDb project push-down, needs tests [kirillkozlov] Add tests for MongoDb project push-down [kirillkozlov] Added cleanup for tests [kirillkozlov] rebase [chadrik] Provide methods to override bind and service addresses independently [chadrik] Fix lint [kirillkozlov] Check last executed query [lcwik] [BEAM-4287] Fix to use the residual instead of the current restriction ------------------------------------------ [...truncated 925.15 KB...] return filesystem.checksum(path) File "<https://builds.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/localfilesystem.py",> line 292, in checksum raise BeamIOError('Path does not exist: %s' % path) apache_beam.io.filesystem.BeamIOError: Path does not exist: /tmp/py-wordcount-direct-00000-of-00002 with exceptions None [while running 'write/Write/WriteImpl/FinalizeWrite'] with exceptions None [flink-runner-job-invoker] INFO org.apache.beam.runners.fnexecution.artifact.AbstractArtifactRetrievalService - Manifest at /tmp/beam-tempahnhzrn4/artifactsca6ap505/job_faad5977-29bb-4777-b933-50df3ae240e1/MANIFEST has 1 artifact locations [flink-runner-job-invoker] INFO org.apache.beam.runners.fnexecution.artifact.BeamFileSystemArtifactStagingService - Removed dir /tmp/beam-tempahnhzrn4/artifactsca6ap505/job_faad5977-29bb-4777-b933-50df3ae240e1/ INFO:apache_beam.runners.portability.portable_runner:Job state changed to FAILED Traceback (most recent call last): File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main "__main__", mod_spec) File "/usr/lib/python3.7/runpy.py", line 85, in _run_code exec(code, run_globals) File "<https://builds.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/examples/wordcount.py",> line 138, in <module> run() File "<https://builds.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/examples/wordcount.py",> line 117, in run result = p.run() File "<https://builds.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/pipeline.py",> line 416, in run self._options).run(False) File "<https://builds.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/pipeline.py",> line 429, in run return self.runner.run_pipeline(self, self._options) File "<https://builds.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/portability/flink_runner.py",> line 45, in run_pipeline return super(FlinkRunner, self).run_pipeline(pipeline, options) File "<https://builds.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/portability/portable_runner.py",> line 320, in run_pipeline result.wait_until_finish() File "<https://builds.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/portability/portable_runner.py",> line 438, in wait_until_finish self._job_id, self._state, self._last_error_message())) RuntimeError: Pipeline BeamApp-jenkins-1206005416-cca46770_99578f29-7496-413a-8d05-0d6215ddd000 failed in state FAILED: java.lang.RuntimeException: Error received from SDK harness for instruction 40: Traceback (most recent call last): File "<https://builds.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/common.py",> line 812, in process return self.do_fn_invoker.invoke_process(windowed_value) File "<https://builds.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/common.py",> line 610, in invoke_process windowed_value, additional_args, additional_kwargs, output_processor) File "<https://builds.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/common.py",> line 684, in _invoke_process_per_window windowed_value, self.process_method(*args_for_process)) File "<https://builds.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/common.py",> line 927, in process_outputs for result in results: File "<https://builds.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/iobase.py",> line 1090, in <genexpr> window.TimestampedValue(v, timestamp.MAX_TIMESTAMP) for v in outputs) File "<https://builds.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/filebasedsink.py",> line 276, in finalize_write self._check_state_for_finalize_write(writer_results, num_shards)) File "<https://builds.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/filebasedsink.py",> line 260, in _check_state_for_finalize_write FileSystems.checksum(src) == FileSystems.checksum(dst)): File "<https://builds.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/filesystems.py",> line 299, in checksum return filesystem.checksum(path) File "<https://builds.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/localfilesystem.py",> line 292, in checksum raise BeamIOError('Path does not exist: %s' % path) apache_beam.io.filesystem.BeamIOError: Path does not exist: /tmp/py-wordcount-direct-00000-of-00002 with exceptions None During handling of the above exception, another exception occurred: Traceback (most recent call last): File "<https://builds.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 136, in _execute response = task() File "<https://builds.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 186, in <lambda> lambda: self.create_worker().do_instruction(request), request) File "<https://builds.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 284, in do_instruction request.instruction_id) File "<https://builds.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 310, in process_bundle bundle_processor.process_bundle(instruction_id)) File "<https://builds.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/bundle_processor.py",> line 675, in process_bundle data.transform_id].process_encoded(data.data) File "<https://builds.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/bundle_processor.py",> line 146, in process_encoded self.output(decoded_value) File "<https://builds.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/operations.py",> line 259, in output cython.cast(Receiver, self.receivers[output_index]).receive(windowed_value) File "<https://builds.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/operations.py",> line 146, in receive self.consumer.process(windowed_value) File "<https://builds.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/operations.py",> line 597, in process delayed_application = self.dofn_receiver.receive(o) File "<https://builds.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/common.py",> line 808, in receive self.process(windowed_value) File "<https://builds.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/common.py",> line 814, in process self._reraise_augmented(exn) File "<https://builds.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/common.py",> line 881, in _reraise_augmented raise_with_traceback(new_exn) File "<https://builds.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/future/utils/__init__.py",> line 446, in raise_with_traceback raise exc.with_traceback(traceback) File "<https://builds.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/common.py",> line 812, in process return self.do_fn_invoker.invoke_process(windowed_value) File "<https://builds.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/common.py",> line 610, in invoke_process windowed_value, additional_args, additional_kwargs, output_processor) File "<https://builds.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/common.py",> line 684, in _invoke_process_per_window windowed_value, self.process_method(*args_for_process)) File "<https://builds.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/common.py",> line 927, in process_outputs for result in results: File "<https://builds.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/iobase.py",> line 1090, in <genexpr> window.TimestampedValue(v, timestamp.MAX_TIMESTAMP) for v in outputs) File "<https://builds.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/filebasedsink.py",> line 276, in finalize_write self._check_state_for_finalize_write(writer_results, num_shards)) File "<https://builds.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/filebasedsink.py",> line 260, in _check_state_for_finalize_write FileSystems.checksum(src) == FileSystems.checksum(dst)): File "<https://builds.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/filesystems.py",> line 299, in checksum return filesystem.checksum(path) File "<https://builds.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/localfilesystem.py",> line 292, in checksum raise BeamIOError('Path does not exist: %s' % path) apache_beam.io.filesystem.BeamIOError: Path does not exist: /tmp/py-wordcount-direct-00000-of-00002 with exceptions None [while running 'write/Write/WriteImpl/FinalizeWrite'] with exceptions None > Task > :sdks:python:test-suites:portable:py37:portableWordCountFlinkRunnerStreaming > FAILED > Task :sdks:python:test-suites:portable:py37:portableWordCountSparkRunnerBatch INFO:apache_beam.runners.portability.fn_api_runner_transforms:==================== <function lift_combiners at 0x7f6e835599d8> ==================== INFO:apache_beam.utils.subprocess_server:Using pre-built snapshot at <https://builds.apache.org/job/beam_PostCommit_Python37/ws/src/runners/spark/job-server/build/libs/beam-runners-spark-job-server-2.19.0-SNAPSHOT.jar> WARNING:apache_beam.utils.subprocess_server:Starting service with ['java' '-jar' '<https://builds.apache.org/job/beam_PostCommit_Python37/ws/src/runners/spark/job-server/build/libs/beam-runners-spark-job-server-2.19.0-SNAPSHOT.jar'> '--spark-master-url' 'local[4]' '--artifacts-dir' '/tmp/beam-temp773xn_dr/artifactsbcxe29ld' '--job-port' '46343' '--artifact-port' '0' '--expansion-port' '0'] 19/12/06 00:55:25 INFO org.apache.beam.runners.fnexecution.jobsubmission.JobServerDriver: ArtifactStagingService started on localhost:34173 19/12/06 00:55:25 INFO org.apache.beam.runners.fnexecution.jobsubmission.JobServerDriver: Java ExpansionService started on localhost:35329 19/12/06 00:55:26 INFO org.apache.beam.runners.fnexecution.jobsubmission.JobServerDriver: JobService started on localhost:46343 WARNING:root:Waiting for grpc channel to be ready at localhost:46343. WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['--parallelism=2', '--shutdown_sources_on_final_watermark'] 19/12/06 00:55:27 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job BeamApp-jenkins-1206005527-7e7b1a4_325d40ea-8f9a-4b47-95c3-386effb4858d 19/12/06 00:55:28 INFO org.apache.beam.runners.fnexecution.jobsubmission.JobInvocation: Starting job invocation BeamApp-jenkins-1206005527-7e7b1a4_325d40ea-8f9a-4b47-95c3-386effb4858d INFO:root:Waiting until the pipeline has finished because the environment "LOOPBACK" has started a component necessary for the execution. INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING 19/12/06 00:55:28 INFO org.apache.beam.runners.spark.SparkPipelineRunner: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath 19/12/06 00:55:28 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Will stage 1 files. (Enable logging at DEBUG level to see which files will be staged.) 19/12/06 00:55:28 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context. 19/12/06 00:55:30 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 19/12/06 00:55:32 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job BeamApp-jenkins-1206005527-7e7b1a4_325d40ea-8f9a-4b47-95c3-386effb4858d on Spark master local[4] 19/12/06 00:55:32 INFO org.apache.beam.runners.spark.aggregators.AggregatorsAccumulator: Instantiated aggregators accumulator: 19/12/06 00:55:32 INFO org.apache.beam.runners.spark.metrics.MetricsAccumulator: Instantiated metrics accumulator: MetricQueryResults() INFO:apache_beam.runners.worker.statecache:Creating state cache with size 0 INFO:apache_beam.runners.worker.sdk_worker:Creating insecure control channel for localhost:33575. INFO:apache_beam.runners.worker.sdk_worker:Control channel established. INFO:apache_beam.runners.worker.sdk_worker:Initializing SDKHarness with unbounded number of workers. 19/12/06 00:55:37 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 1-1 INFO:apache_beam.runners.worker.sdk_worker:Creating insecure state channel for localhost:39685. INFO:apache_beam.runners.worker.sdk_worker:State channel established. INFO:apache_beam.runners.worker.data_plane:Creating client data channel for localhost:46767 19/12/06 00:55:37 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected. 19/12/06 00:55:37 WARN org.apache.beam.runners.spark.translation.GroupNonMergingWindowsFunctions: Either coder LengthPrefixCoder(ByteArrayCoder) or GlobalWindow$Coder is not consistent with equals. That might cause issues on some runners. WARNING:apache_beam.io.filebasedsink:Deleting 4 existing files in target path matching: -*-of-%(num_shards)05d 19/12/06 00:56:06 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-1206005527-7e7b1a4_325d40ea-8f9a-4b47-95c3-386effb4858d: Pipeline translated successfully. Computing outputs INFO:apache_beam.io.filebasedsink:Starting finalize_write threads with num_shards: 4 (skipped: 0), batches: 4, num_threads: 4 INFO:apache_beam.io.filebasedsink:Renamed 4 shards in 0.14 seconds. 19/12/06 00:56:34 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-1206005527-7e7b1a4_325d40ea-8f9a-4b47-95c3-386effb4858d finished. 19/12/06 00:56:34 WARN org.apache.beam.runners.spark.SparkPipelineResult$BatchMode: Collecting monitoring infos is not implemented yet in Spark portable runner. 19/12/06 00:56:34 INFO org.apache.beam.runners.fnexecution.artifact.AbstractArtifactRetrievalService: Manifest at /tmp/beam-temp773xn_dr/artifactsbcxe29ld/job_c41d7c9f-31b5-4d94-919a-04980570779e/MANIFEST has 1 artifact locations 19/12/06 00:56:34 INFO org.apache.beam.runners.fnexecution.artifact.BeamFileSystemArtifactStagingService: Removed dir /tmp/beam-temp773xn_dr/artifactsbcxe29ld/job_c41d7c9f-31b5-4d94-919a-04980570779e/ INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE 19/12/06 00:56:34 INFO org.apache.beam.runners.fnexecution.jobsubmission.InMemoryJobService: Getting job metrics for BeamApp-jenkins-1206005527-7e7b1a4_325d40ea-8f9a-4b47-95c3-386effb4858d 19/12/06 00:56:34 INFO org.apache.beam.runners.fnexecution.jobsubmission.InMemoryJobService: Finished getting job metrics for BeamApp-jenkins-1206005527-7e7b1a4_325d40ea-8f9a-4b47-95c3-386effb4858d ERROR:apache_beam.runners.worker.data_plane:Failed to read inputs in the data plane. Traceback (most recent call last): File "<https://builds.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 272, in _read_inputs for elements in elements_iterator: File "<https://builds.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 392, in __next__ return self._next() File "<https://builds.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 561, in _next raise self grpc._channel._Rendezvous: <_Rendezvous of RPC that terminated with: status = StatusCode.UNAVAILABLE details = "Socket closed" debug_error_string = "{"created":"@1575593794.866558065","description":"Error received from peer ipv4:127.0.0.1:46767","file":"src/core/lib/surface/call.cc","file_line":1055,"grpc_message":"Socket closed","grpc_status":14}" > Exception in thread read_grpc_client_inputs: Traceback (most recent call last): File "/usr/lib/python3.7/threading.py", line 917, in _bootstrap_inner self.run() File "/usr/lib/python3.7/threading.py", line 865, in run self._target(*self._args, **self._kwargs) File "<https://builds.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 286, in <lambda> target=lambda: self._read_inputs(elements_iterator), File "<https://builds.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 272, in _read_inputs for elements in elements_iterator: File "<https://builds.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 392, in __next__ return self._next() File "<https://builds.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 561, in _next raise self grpc._channel._Rendezvous: <_Rendezvous of RPC that terminated with: status = StatusCode.UNAVAILABLE details = "Socket closed" debug_error_string = "{"created":"@1575593794.866558065","description":"Error received from peer ipv4:127.0.0.1:46767","file":"src/core/lib/surface/call.cc","file_line":1055,"grpc_message":"Socket closed","grpc_status":14}" > Exception in thread run_worker_1-1: Traceback (most recent call last): File "/usr/lib/python3.7/threading.py", line 917, in _bootstrap_inner self.run() File "/usr/lib/python3.7/threading.py", line 865, in run self._target(*self._args, **self._kwargs) File "<https://builds.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 111, in run for work_request in control_stub.Control(get_responses()): File "<https://builds.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 392, in __next__ return self._next() File "<https://builds.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 561, in _next raise self grpc._channel._Rendezvous: <_Rendezvous of RPC that terminated with: status = StatusCode.UNAVAILABLE details = "Socket closed" debug_error_string = "{"created":"@1575593794.866649843","description":"Error received from peer ipv4:127.0.0.1:33575","file":"src/core/lib/surface/call.cc","file_line":1055,"grpc_message":"Socket closed","grpc_status":14}" > Exception in thread read_state: Traceback (most recent call last): File "/usr/lib/python3.7/threading.py", line 917, in _bootstrap_inner self.run() File "/usr/lib/python3.7/threading.py", line 865, in run self._target(*self._args, **self._kwargs) File "<https://builds.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 530, in pull_responses for response in responses: File "<https://builds.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 392, in __next__ return self._next() File "<https://builds.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 561, in _next raise self grpc._channel._Rendezvous: <_Rendezvous of RPC that terminated with: status = StatusCode.UNAVAILABLE details = "Socket closed" debug_error_string = "{"created":"@1575593794.866637018","description":"Error received from peer ipv4:127.0.0.1:39685","file":"src/core/lib/surface/call.cc","file_line":1055,"grpc_message":"Socket closed","grpc_status":14}" > FAILURE: Build completed with 2 failures. 1: Task failed with an exception. ----------- * What went wrong: Execution failed for task ':sdks:python:test-suites:portable:py37:portableWordCountFlinkRunnerBatch'. > Process 'command 'sh'' finished with non-zero exit value 1 * Try: Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights. ============================================================================== 2: Task failed with an exception. ----------- * What went wrong: Execution failed for task ':sdks:python:test-suites:portable:py37:portableWordCountFlinkRunnerStreaming'. > Process 'command 'sh'' finished with non-zero exit value 1 * Try: Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights. ============================================================================== * Get more help at https://help.gradle.org Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0. Use '--warning-mode all' to show the individual deprecation warnings. See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings BUILD FAILED in 1h 21m 1s 84 actionable tasks: 64 executed, 20 from cache Publishing build scan... https://scans.gradle.com/s/ejbtuefvgpdew Build step 'Invoke Gradle script' changed build result to FAILURE Build step 'Invoke Gradle script' marked build as failure --------------------------------------------------------------------- To unsubscribe, e-mail: builds-unsubscr...@beam.apache.org For additional commands, e-mail: builds-h...@beam.apache.org