See <https://builds.apache.org/job/beam_PerformanceTests_TFRecordIOIT_HDFS/4/display/redirect?page=changes>
Changes: [alex] Correct BigQuery.write JavaDoc example [rober] Correct EventTime encoding. time.(Time).Unix() returns seconds not [tgroh] Enable Proto Round Trips in the DirectRunner Tests [lcwik] [BEAM-3249, BEAM-3457] Make Apache Rat a dependency of the Java and [sidhom] Add Java bounded read overrides [herohde] [BEAM-3817] Switch BQ write to not use side input [lcwik] [BEAM-3457] Exclude generated and data files part of the Go SDK. [sidhom] Add ExecutableStagePayload to make aid runner stage reconstruction [sidhom] Fix typo [coheigea] Replacing size() == 0 with isEmpty() [markliu] [BEAM-3841] Fix TestDataflowRunner.run to run_pipeline [coheigea] Put the String literal first when comparing it to an Object [ankurgoenka] Send error response for process bundle progress instead of throwing and [tgroh] Use InstructionRequestHandler in RemoteEnvironment [ekirpichov] BigQueryIO.read().fromQuery() improvements [szewinho] [BEAM-3060] Fixing mvn dependency issue when runnning filebasedIOIT [lukasz.gajowy] [BEAM-3798] Remove error check on dataflow when getting batch job state [tgroh] Add ExecutableStage -> ProcessBundleDescriptor [tgroh] Clean up GrpcFnServer Constructor Signature [tgroh] Add a RemoteExecutionTest [tgroh] Clean up a couple of SdkHarnessClient Signatures [tgroh] Separate RemoteInputDestinations from SdkHarnessClient [XuMingmin] [SQL] Add support for DOT expression (#4863) [herohde] Add TODO to revert Go IO to use side input [jb] [BEAM-3500] "Attach" JDBC connection to the bundle (improve the pooling) [jb] [BEAM-3500] Test if the user provides both withDataSourceConfiguration() [jb] [BEAM-3500] Wrap the datasource as a poolable datasource and expose [jb] [BEAM-3500] Add commons-pool2 dependency [jb] [BEAM-3500] Only expose max number of connections in the pool to the [jb] [BEAM-3500] Cleanup pool configuration parameters [jb] [BEAM-3500] Remove dataSourceFactory [jb] [BEAM-3500] Remove unecessary check on dataSourceConfiguration [szewinho] Jax-api added to dependencyManagement. [szewinho] Jaxb-api added to beam root pom.xml [szewinho] Removed empty line [Pablo] Fixing counter string name. [tgroh] Add Network Utility Methods [lcwik] Fix missing license header in test. [aaltay] Enable input subscription in Python streaming wordcount (#4864) [tgroh] fixup! Add Network Utility Methods [tgroh] Remove Future#set(null) in fn-execution [herohde] [BEAM-3869] Fix Go Dataflow break due to zero timestamps [axelmagn] Add a generic interface for the state service. [herohde] Remove WindowedValue on PCollections for Go SDK [cclauss] [BEAM-3872] Change Exception.message --> Exception.args[0] [alan.myrvold] [BEAM-3249] Publish java artifacts using gradle [herohde] CR: Fix comments to remove old windowing [lcwik] Fix minor checkstyle/findbugs errors [mariagh] Add support for streaming side inputs in the DirectRunner [andreas.ehrencrona] [BEAM-2264] Credentials were not being reused between GCS calls [ccy] Add support for PaneInfo in WindowedValues [axelmagn] Write unit tests for GrpcStateService. [XuMingmin] Bump calcite and avatica versions (#4887) [axelmagn] Fix StateRequestHandler interface to be idiomatic [herohde] Add Go support for universal runners, incl Flink [herohde] CR: Fixed comments for job service helper functions [iemejia] Add missing ASF license to ExecutableStageTranslation file [szewinho] Updated gradle build to use jaxb-api. Jaxb-api version set to 2.2.12 [mariand] Switched AvroIO default codec to snappyCodec(). [yifanzou] [BEAM-3840] Get python mobile-gaming automating on core runners [coheigea] Remove "i == numSplits" condition, which can never be reached as i is [sidhom] [BEAM-3565] Clean up ExecutableStage [wcn] Fix incorrect read of atomic counter. [tgroh] Add a suborder parameter to Networks#topologicalOrder [herohde] [BEAM-3893] Add fallback to unauthenticated access for GCS IO [robertwb] [BEAM-3865] Fix watermark hold handling bug. [robertwb] [BEAM-2927] Python support for dataflow portable side inputs over Fn API [Pablo] Fixing check for sideinput_io_metrics experiment flag. [herohde] CR: fix typo [herohde] [BEAM-3897] Add Go wordcount example with multi output DoFns [ccy] Replace side inputs when applying PTransformOverrides [tgroh] Add QueryablePipeline#forTransforms [herohde] Allow manual override of Go worker binary [aaltay] [BEAM-3861] Improve test infra in Python SDK for streaming end-to-end [iemejia] Remove testing package-info from main package for GCP IO [iemejia] Update maven failsafe/surefire plugin to version 2.21.0 [iemejia] [BEAM-3873] Update commons-compress to version 1.16.1 (fix [iemejia] Remove maven warnings [tgroh] Add Side Inputs to ExecutableStage [herohde] [BEAM-3866] Remove windowed value requirement for External [lcwik] [BEAM-3903] Build the Go SDK examples using gradle [markliu] Clean up terminal state check in TestDataflowRunner [robertwb] Add mapper microbenchmark. [szewinho] Scope removed from root pom.xml and jaxb-api dependency set to [Pablo] Updating dataflow API protocol buffers. [tgroh] Make RemoteEnvironment public [boyuanz] Add cython annotation to make DistributionAccumulator faster [tgroh] Move getId to the top-level pipeline node [tgroh] Add FusedPipeline#toPipeline [robertwb] Remove from docs generation. [wcn] BEAM-3894: Add Protocol Buffer support [altay] Convert big query writes to beam.io.WriteToBigQuery [aaltay] Improve test util for PubSub exist check (#4922) [XuMingmin] [BEAM-3851] Option to preserve element timestamp while publishing to [ankurgoenka] Send worker_id in all channels to runner Adding [wcn] Updating generated files. [ehudm] Reduce precommit test timeouts for Java and Go. [altay] More graceful fallback when grpc is not present. [markliu] [BEAM-3861] Complete streaming wordcount test in Python SDK [aaltay] [BEAM-3738] Enable py3 lint and cleanup tox.ini. (#4877) ------------------------------------------ Started by timer [EnvInject] - Loading node environment variables. Building remotely on beam2 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_TFRecordIOIT_HDFS/ws/> > git rev-parse --is-inside-work-tree # timeout=10 Fetching changes from the remote Git repository > git config remote.origin.url https://github.com/apache/beam.git # timeout=10 Fetching upstream changes from https://github.com/apache/beam.git > git --version # timeout=10 > git fetch --tags --progress https://github.com/apache/beam.git > +refs/heads/*:refs/remotes/origin/* > +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/* > git rev-parse origin/master^{commit} # timeout=10 Checking out Revision 8b72e5af513e5182c403f3b67ff6d16395c281db (origin/master) > git config core.sparsecheckout # timeout=10 > git checkout -f 8b72e5af513e5182c403f3b67ff6d16395c281db Commit message: "Merge pull request #4940: Update generated files" > git rev-list --no-walk 5b6ca47fec0b5b720ad5afb9274dc0d418545b43 # timeout=10 Cleaning workspace > git rev-parse --verify HEAD # timeout=10 Resetting working tree > git reset --hard # timeout=10 > git clean -fdx # timeout=10 [EnvInject] - Executing scripts and injecting environment variables after the SCM step. [EnvInject] - Injecting as environment variables the properties content SPARK_LOCAL_IP=127.0.0.1 [EnvInject] - Variables injected successfully. [beam_PerformanceTests_TFRecordIOIT_HDFS] $ /bin/bash -xe /tmp/jenkins8246568391384583897.sh + gcloud container clusters get-credentials io-datastores --zone=us-central1-a --verbosity=debug DEBUG: Running gcloud.container.clusters.get-credentials with Namespace(__calliope_internal_deepest_parser=ArgumentParser(prog='gcloud.container.clusters.get-credentials', usage=None, description='See https://cloud.google.com/container-engine/docs/kubectl for\nkubectl documentation.', version=None, formatter_class=<class 'argparse.HelpFormatter'>, conflict_handler='error', add_help=False), account=None, api_version=None, authority_selector=None, authorization_token_file=None, cmd_func=<bound method Command.Run of <googlecloudsdk.calliope.backend.Command object at 0x7f6fef241510>>, command_path=['gcloud', 'container', 'clusters', 'get-credentials'], configuration=None, credential_file_override=None, document=None, format=None, h=None, help=None, http_timeout=None, log_http=None, name='io-datastores', project=None, quiet=None, trace_email=None, trace_log=None, trace_token=None, user_output_enabled=None, verbosity='debug', version=None, zone='us-central1-a'). WARNING: Accessing a Container Engine cluster requires the kubernetes commandline client [kubectl]. To install, run $ gcloud components install kubectl Fetching cluster endpoint and auth data. DEBUG: Saved kubeconfig to /home/jenkins/.kube/config kubeconfig entry generated for io-datastores. INFO: Display format "default". [beam_PerformanceTests_TFRecordIOIT_HDFS] $ /bin/bash -xe /tmp/jenkins3974261244662825482.sh + cp /home/jenkins/.kube/config <https://builds.apache.org/job/beam_PerformanceTests_TFRecordIOIT_HDFS/ws/config-filebasedioithdfs-1521890089911> [beam_PerformanceTests_TFRecordIOIT_HDFS] $ /bin/bash -xe /tmp/jenkins5848965985188276448.sh + kubectl --kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_TFRecordIOIT_HDFS/ws/config-filebasedioithdfs-1521890089911> create namespace filebasedioithdfs-1521890089911 Error from server (AlreadyExists): namespaces "filebasedioithdfs-1521890089911" already exists Build step 'Execute shell' marked build as failure
