See <https://builds.apache.org/job/beam_PerformanceTests_Dataflow/227/display/redirect?page=changes>
Changes: [tgroh] Fix EmptyFlattenAsCreateFactory [tgroh] Explicitly duplicate input PCollections in Dataflow [tgroh] Reenable Flatten Tests in the Dataflow Runner ------------------------------------------ [...truncated 233.36 KB...] ! 79fcea3...21d4c8a refs/pull/2214/merge -> origin/pr/2214/merge (unable to update local ref) error: unable to resolve reference refs/remotes/origin/pr/2216/merge: No such file or directory ! f2fcfad...53b10b5 refs/pull/2216/merge -> origin/pr/2216/merge (unable to update local ref) error: unable to resolve reference refs/remotes/origin/pr/2243/merge: No such file or directory ! e928993...86f61c7 refs/pull/2243/merge -> origin/pr/2243/merge (unable to update local ref) error: unable to resolve reference refs/remotes/origin/pr/2245/merge: No such file or directory ! 2a2b532...c795e81 refs/pull/2245/merge -> origin/pr/2245/merge (unable to update local ref) error: unable to resolve reference refs/remotes/origin/pr/2255/merge: No such file or directory ! 1345d02...5d9ba26 refs/pull/2255/merge -> origin/pr/2255/merge (unable to update local ref) error: unable to resolve reference refs/remotes/origin/pr/2256/merge: No such file or directory ! 69f3ff7...5e70822 refs/pull/2256/merge -> origin/pr/2256/merge (unable to update local ref) error: unable to resolve reference refs/remotes/origin/pr/2257/merge: No such file or directory ! 72f462d...304973c refs/pull/2257/merge -> origin/pr/2257/merge (unable to update local ref) error: unable to resolve reference refs/remotes/origin/pr/2258/merge: No such file or directory ! cf36917...b9eaefa refs/pull/2258/merge -> origin/pr/2258/merge (unable to update local ref) error: unable to resolve reference refs/remotes/origin/pr/2260/merge: No such file or directory ! fa320f6...ef9ae02 refs/pull/2260/merge -> origin/pr/2260/merge (unable to update local ref) error: unable to resolve reference refs/remotes/origin/pr/2261/merge: No such file or directory ! 88e71b3...7279cf0 refs/pull/2261/merge -> origin/pr/2261/merge (unable to update local ref) error: unable to resolve reference refs/remotes/origin/pr/2262/merge: No such file or directory ! 2bd97bf...e3f23c3 refs/pull/2262/merge -> origin/pr/2262/merge (unable to update local ref) error: unable to resolve reference refs/remotes/origin/pr/2263/merge: No such file or directory ! d466932...53158e8 refs/pull/2263/merge -> origin/pr/2263/merge (unable to update local ref) error: unable to resolve reference refs/remotes/origin/pr/2264/merge: No such file or directory ! 01fcbfe...df44573 refs/pull/2264/merge -> origin/pr/2264/merge (unable to update local ref) error: unable to resolve reference refs/remotes/origin/pr/2266/merge: No such file or directory ! 78dd5bd...77cff73 refs/pull/2266/merge -> origin/pr/2266/merge (unable to update local ref) error: unable to resolve reference refs/remotes/origin/pr/2267/merge: No such file or directory ! 8b843e4...e8e818d refs/pull/2267/merge -> origin/pr/2267/merge (unable to update local ref) error: unable to resolve reference refs/remotes/origin/pr/2268/merge: No such file or directory ! ea38d4a...54ec413 refs/pull/2268/merge -> origin/pr/2268/merge (unable to update local ref) error: unable to resolve reference refs/remotes/origin/pr/2269/merge: No such file or directory ! 035265f...172852d refs/pull/2269/merge -> origin/pr/2269/merge (unable to update local ref) error: unable to resolve reference refs/remotes/origin/pr/2271/head: No such file or directory ! 0af2cf2...6868470 refs/pull/2271/head -> origin/pr/2271/head (unable to update local ref) error: unable to resolve reference refs/remotes/origin/pr/2271/merge: No such file or directory ! 91d4d90...b3c8aa9 refs/pull/2271/merge -> origin/pr/2271/merge (unable to update local ref) error: unable to resolve reference refs/remotes/origin/pr/2272/merge: No such file or directory ! 47dde47...e0ac186 refs/pull/2272/merge -> origin/pr/2272/merge (unable to update local ref) error: unable to resolve reference refs/remotes/origin/pr/2273/merge: No such file or directory ! 20f158c...0179c07 refs/pull/2273/merge -> origin/pr/2273/merge (unable to update local ref) error: unable to resolve reference refs/remotes/origin/pr/2275/merge: No such file or directory ! 44d6ecd...a9f73ad refs/pull/2275/merge -> origin/pr/2275/merge (unable to update local ref) error: unable to resolve reference refs/remotes/origin/pr/2277/merge: No such file or directory ! 6afc28d...bb651d2 refs/pull/2277/merge -> origin/pr/2277/merge (unable to update local ref) error: unable to resolve reference refs/remotes/origin/pr/2279/merge: No such file or directory ! 0d08531...61c32b1 refs/pull/2279/merge -> origin/pr/2279/merge (unable to update local ref) error: unable to resolve reference refs/remotes/origin/pr/2280/merge: No such file or directory ! 56737ca...95565f1 refs/pull/2280/merge -> origin/pr/2280/merge (unable to update local ref) error: unable to resolve reference refs/remotes/origin/pr/2282/merge: No such file or directory ! 2aaefd1...087b16c refs/pull/2282/merge -> origin/pr/2282/merge (unable to update local ref) * [new ref] refs/pull/2283/head -> origin/pr/2283/head * [new ref] refs/pull/2283/merge -> origin/pr/2283/merge * [new ref] refs/pull/2284/head -> origin/pr/2284/head * [new ref] refs/pull/2284/merge -> origin/pr/2284/merge * [new ref] refs/pull/2285/head -> origin/pr/2285/head * [new ref] refs/pull/2285/merge -> origin/pr/2285/merge * [new ref] refs/pull/2286/head -> origin/pr/2286/head * [new ref] refs/pull/2286/merge -> origin/pr/2286/merge * [new ref] refs/pull/2287/head -> origin/pr/2287/head * [new ref] refs/pull/2287/merge -> origin/pr/2287/merge * [new ref] refs/pull/2288/head -> origin/pr/2288/head * [new ref] refs/pull/2288/merge -> origin/pr/2288/merge * [new ref] refs/pull/2289/head -> origin/pr/2289/head * [new ref] refs/pull/2289/merge -> origin/pr/2289/merge * [new ref] refs/pull/2290/head -> origin/pr/2290/head * [new ref] refs/pull/2290/merge -> origin/pr/2290/merge * [new ref] refs/pull/2291/head -> origin/pr/2291/head * [new ref] refs/pull/2291/merge -> origin/pr/2291/merge * [new ref] refs/pull/2292/head -> origin/pr/2292/head * [new ref] refs/pull/2292/merge -> origin/pr/2292/merge * [new ref] refs/pull/2293/head -> origin/pr/2293/head * [new ref] refs/pull/2293/merge -> origin/pr/2293/merge * [new ref] refs/pull/2294/head -> origin/pr/2294/head * [new ref] refs/pull/2294/merge -> origin/pr/2294/merge * [new ref] refs/pull/2295/head -> origin/pr/2295/head * [new ref] refs/pull/2295/merge -> origin/pr/2295/merge * [new ref] refs/pull/2296/head -> origin/pr/2296/head * [new ref] refs/pull/2296/merge -> origin/pr/2296/merge * [new ref] refs/pull/2297/head -> origin/pr/2297/head * [new ref] refs/pull/2297/merge -> origin/pr/2297/merge * [new ref] refs/pull/2298/head -> origin/pr/2298/head * [new ref] refs/pull/2298/merge -> origin/pr/2298/merge * [new ref] refs/pull/2299/head -> origin/pr/2299/head * [new ref] refs/pull/2300/head -> origin/pr/2300/head * [new ref] refs/pull/2300/merge -> origin/pr/2300/merge * [new ref] refs/pull/2301/head -> origin/pr/2301/head * [new ref] refs/pull/2301/merge -> origin/pr/2301/merge * [new ref] refs/pull/2302/head -> origin/pr/2302/head * [new ref] refs/pull/2302/merge -> origin/pr/2302/merge * [new ref] refs/pull/2303/head -> origin/pr/2303/head * [new ref] refs/pull/2303/merge -> origin/pr/2303/merge * [new ref] refs/pull/2304/head -> origin/pr/2304/head * [new ref] refs/pull/2304/merge -> origin/pr/2304/merge * [new ref] refs/pull/2305/head -> origin/pr/2305/head * [new ref] refs/pull/2305/merge -> origin/pr/2305/merge * [new ref] refs/pull/2306/head -> origin/pr/2306/head * [new ref] refs/pull/2306/merge -> origin/pr/2306/merge * [new ref] refs/pull/2307/head -> origin/pr/2307/head * [new ref] refs/pull/2307/merge -> origin/pr/2307/merge * [new ref] refs/pull/2308/head -> origin/pr/2308/head * [new ref] refs/pull/2308/merge -> origin/pr/2308/merge * [new ref] refs/pull/2309/head -> origin/pr/2309/head * [new ref] refs/pull/2309/merge -> origin/pr/2309/merge * [new ref] refs/pull/2310/head -> origin/pr/2310/head * [new ref] refs/pull/2310/merge -> origin/pr/2310/merge * [new ref] refs/pull/2311/head -> origin/pr/2311/head * [new ref] refs/pull/2311/merge -> origin/pr/2311/merge * [new ref] refs/pull/2312/head -> origin/pr/2312/head * [new ref] refs/pull/2312/merge -> origin/pr/2312/merge * [new ref] refs/pull/2313/head -> origin/pr/2313/head * [new ref] refs/pull/2313/merge -> origin/pr/2313/merge * [new ref] refs/pull/2314/head -> origin/pr/2314/head * [new ref] refs/pull/2314/merge -> origin/pr/2314/merge * [new ref] refs/pull/2315/head -> origin/pr/2315/head * [new ref] refs/pull/2315/merge -> origin/pr/2315/merge * [new ref] refs/pull/2316/head -> origin/pr/2316/head * [new ref] refs/pull/2316/merge -> origin/pr/2316/merge * [new ref] refs/pull/2317/head -> origin/pr/2317/head * [new ref] refs/pull/2317/merge -> origin/pr/2317/merge * [new ref] refs/pull/2318/head -> origin/pr/2318/head * [new ref] refs/pull/2318/merge -> origin/pr/2318/merge * [new ref] refs/pull/2319/head -> origin/pr/2319/head * [new ref] refs/pull/2319/merge -> origin/pr/2319/merge * [new ref] refs/pull/2320/head -> origin/pr/2320/head * [new ref] refs/pull/2321/head -> origin/pr/2321/head * [new ref] refs/pull/2321/merge -> origin/pr/2321/merge * [new ref] refs/pull/2322/head -> origin/pr/2322/head * [new ref] refs/pull/2322/merge -> origin/pr/2322/merge * [new ref] refs/pull/2323/head -> origin/pr/2323/head * [new ref] refs/pull/2323/merge -> origin/pr/2323/merge * [new ref] refs/pull/2324/head -> origin/pr/2324/head * [new ref] refs/pull/2325/head -> origin/pr/2325/head * [new ref] refs/pull/2325/merge -> origin/pr/2325/merge error: unable to resolve reference refs/remotes/origin/pr/592/merge: No such file or directory ! ce53fff...05e1609 refs/pull/592/merge -> origin/pr/592/merge (unable to update local ref) at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:1793) at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandWithCredentials(CliGitAPIImpl.java:1519) at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.access$300(CliGitAPIImpl.java:64) at org.jenkinsci.plugins.gitclient.CliGitAPIImpl$1.execute(CliGitAPIImpl.java:315) at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler$1.call(RemoteGitImpl.java:153) at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler$1.call(RemoteGitImpl.java:146) at hudson.remoting.UserRequest.perform(UserRequest.java:153) at hudson.remoting.UserRequest.perform(UserRequest.java:50) at hudson.remoting.Request$2.run(Request.java:336) at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:68) at java.util.concurrent.FutureTask.run(FutureTask.java:262) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) at ......remote call to beam5(Native Method) at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1537) at hudson.remoting.UserResponse.retrieve(UserRequest.java:253) at hudson.remoting.Channel.call(Channel.java:822) at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler.execute(RemoteGitImpl.java:146) at sun.reflect.GeneratedMethodAccessor699.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler.invoke(RemoteGitImpl.java:132) at com.sun.proxy.$Proxy97.execute(Unknown Source) at hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:804) ... 11 more ERROR: null Retrying after 10 seconds > git rev-parse --is-inside-work-tree # timeout=10 Fetching changes from the remote Git repository > git config remote.origin.url https://github.com/apache/beam.git # timeout=10 Pruning obsolete local branches Fetching upstream changes from https://github.com/apache/beam.git > git --version # timeout=10 > git fetch --tags --progress https://github.com/apache/beam.git > +refs/heads/*:refs/remotes/origin/* +refs/pull/*:refs/remotes/origin/pr/* > --prune > git rev-parse origin/master^{commit} # timeout=10 Checking out Revision e5f1a6479c0a7c6f531c340fe371e4b66e7ea2a5 (origin/master) > git config core.sparsecheckout # timeout=10 > git checkout -f e5f1a6479c0a7c6f531c340fe371e4b66e7ea2a5 > git rev-list 2723d38a67cee15b974a61da8f9c0c5d5262a02f # timeout=10 Cleaning workspace > git rev-parse --verify HEAD # timeout=10 Resetting working tree > git reset --hard # timeout=10 > git clean -fdx # timeout=10 [EnvInject] - Executing scripts and injecting environment variables after the SCM step. [EnvInject] - Injecting as environment variables the properties content SPARK_LOCAL_IP=127.0.0.1 [EnvInject] - Variables injected successfully. [beam_PerformanceTests_Dataflow] $ /bin/bash -xe /tmp/hudson1938797388790989968.sh + rm -rf PerfKitBenchmarker [beam_PerformanceTests_Dataflow] $ /bin/bash -xe /tmp/hudson8781279967402653636.sh + git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git Cloning into 'PerfKitBenchmarker'... [beam_PerformanceTests_Dataflow] $ /bin/bash -xe /tmp/hudson9065991170757080236.sh + pip install --user -r PerfKitBenchmarker/requirements.txt Requirement already satisfied (use --upgrade to upgrade): python-gflags==3.0.4 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14)) Requirement already satisfied (use --upgrade to upgrade): jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15)) Requirement already satisfied (use --upgrade to upgrade): setuptools in /usr/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 16)) Requirement already satisfied (use --upgrade to upgrade): colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17)) Installing extra requirements: 'windows' Requirement already satisfied (use --upgrade to upgrade): blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18)) Requirement already satisfied (use --upgrade to upgrade): futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19)) Requirement already satisfied (use --upgrade to upgrade): PyYAML==3.11 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 20)) Requirement already satisfied (use --upgrade to upgrade): pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21)) Requirement already satisfied (use --upgrade to upgrade): numpy in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22)) Requirement already satisfied (use --upgrade to upgrade): functools32 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 23)) Requirement already satisfied (use --upgrade to upgrade): contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24)) Cleaning up... [beam_PerformanceTests_Dataflow] $ /bin/bash -xe /tmp/hudson2384171702631650966.sh + python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --official=true --benchmarks=dpb_wordcount_benchmark --dpb_dataflow_staging_location=gs://temp-storage-for-perf-tests/staging --dpb_wordcount_input=dataflow-samples/shakespeare/kinglear.txt --config_override=dpb_wordcount_benchmark.dpb_service.service_type=dataflow WARNING:root:File resource loader root perfkitbenchmarker/data/ycsb is not a directory. 2017-03-25 06:02:38,820 761cc1eb MainThread INFO Verbose logging to: /tmp/perfkitbenchmarker/runs/761cc1eb/pkb.log 2017-03-25 06:02:38,820 761cc1eb MainThread INFO PerfKitBenchmarker version: v1.11.0-23-gdc9ac3e 2017-03-25 06:02:38,821 761cc1eb MainThread INFO Flag values: --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --project=apache-beam-testing --bigquery_table=beam_performance.pkb_results --dpb_wordcount_input=dataflow-samples/shakespeare/kinglear.txt --dpb_log_level=INFO --official --benchmarks=dpb_wordcount_benchmark --config_override=dpb_wordcount_benchmark.dpb_service.service_type=dataflow --dpb_dataflow_staging_location=gs://temp-storage-for-perf-tests/staging 2017-03-25 06:02:39,077 761cc1eb MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo. 2017-03-25 06:02:39,156 761cc1eb MainThread dpb_wordcount_benchmark(1/1) INFO Provisioning resources for benchmark dpb_wordcount_benchmark 2017-03-25 06:02:39,159 761cc1eb MainThread dpb_wordcount_benchmark(1/1) INFO Preparing benchmark dpb_wordcount_benchmark 2017-03-25 06:02:39,159 761cc1eb MainThread dpb_wordcount_benchmark(1/1) INFO Running benchmark dpb_wordcount_benchmark 2017-03-25 06:02:39,165 761cc1eb MainThread dpb_wordcount_benchmark(1/1) ERROR Error during benchmark dpb_wordcount_benchmark Traceback (most recent call last): File "<https://builds.apache.org/job/beam_PerformanceTests_Dataflow/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 510, in RunBenchmark DoRunPhase(spec, collector, detailed_timer) File "<https://builds.apache.org/job/beam_PerformanceTests_Dataflow/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 426, in DoRunPhase samples = spec.BenchmarkRun(spec) File "<https://builds.apache.org/job/beam_PerformanceTests_Dataflow/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/dpb_wordcount_benchmark.py",> line 162, in Run job_type=job_type) File "<https://builds.apache.org/job/beam_PerformanceTests_Dataflow/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 115, in SubmitJob stdout, _, _ = vm_util.IssueCommand(cmd) File "<https://builds.apache.org/job/beam_PerformanceTests_Dataflow/ws/PerfKitBenchmarker/perfkitbenchmarker/vm_util.py",> line 287, in IssueCommand full_cmd = ' '.join(cmd) TypeError: sequence item 2: expected string, NoneType found 2017-03-25 06:02:39,167 761cc1eb MainThread dpb_wordcount_benchmark(1/1) ERROR Benchmark 1/1 dpb_wordcount_benchmark (UID: dpb_wordcount_benchmark0) failed. Execution will continue. 2017-03-25 06:02:39,198 761cc1eb MainThread INFO Benchmark run statuses: --------------------------------------------------------- Name UID Status --------------------------------------------------------- dpb_wordcount_benchmark dpb_wordcount_benchmark0 FAILED --------------------------------------------------------- Success rate: 0.00% (0/1) 2017-03-25 06:02:39,198 761cc1eb MainThread INFO Complete logs can be found at: /tmp/perfkitbenchmarker/runs/761cc1eb/pkb.log Build step 'Execute shell' marked build as failure