[jira] [Commented] (BEAM-3040) Python precommit timed out after 150 minutes

2018-10-10 Thread Valentyn Tymofieiev (JIRA) 


[ 
https://issues.apache.org/jira/browse/BEAM-3040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16645492#comment-16645492
 ] 

Valentyn Tymofieiev commented on BEAM-3040:
---

~30 tests are skipped in 
https://github.com/apache/beam/blob/dc4830d03525586e42d3cc893bccfe23959196f0/sdks/python/apache_beam/runners/portability/portable_runner_test.py
 
due to this issue, which is now marked fixed. Not sure if it's safe to unskip 
them.

> Python precommit timed out after 150 minutes
> 
>
> Key: BEAM-3040
> URL: https://issues.apache.org/jira/browse/BEAM-3040
> Project: Beam
>  Issue Type: Bug
>  Components: sdk-py-core
>Reporter: Kenneth Knowles
>Assignee: Ahmet Altay
>Priority: Major
> Fix For: Not applicable
>
>
> https://builds.apache.org/job/beam_PreCommit_Python_MavenInstall/143/consoleFull
> Within about 10 minutes it reaches this point:
> {code}
> ...
> 2017-10-10T03:33:33.591 [INFO] --- findbugs-maven-plugin:3.0.4:check 
> (default) @ beam-sdks-python ---
> 2017-10-10T03:33:33.702 [INFO] 
> 2017-10-10T03:33:33.702 [INFO] --- exec-maven-plugin:1.5.0:exec 
> (setuptools-test) @ beam-sdks-python ---
> {code}
> and the final output is like this:
> {code}
> ...
> 2017-10-10T03:33:33.702 [INFO] --- exec-maven-plugin:1.5.0:exec 
> (setuptools-test) @ beam-sdks-python ---
> docs create: 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/target/.tox/docs
> GLOB sdist-make: 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/setup.py
> lint create: 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/target/.tox/lint
> py27 create: 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/target/.tox/py27
> py27cython create: 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/target/.tox/py27cython
> py27cython installdeps: nose==1.3.7, grpcio-tools==1.3.5, cython==0.25.2
> docs installdeps: nose==1.3.7, grpcio-tools==1.3.5, Sphinx==1.5.5, 
> sphinx_rtd_theme==0.2.4
> lint installdeps: nose==1.3.7, pycodestyle==2.3.1, pylint==1.7.1
> py27 installdeps: nose==1.3.7, grpcio-tools==1.3.5
> lint inst: 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/target/.tox/dist/apache-beam-2.3.0.dev.zip
> py27 inst: 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/target/.tox/dist/apache-beam-2.3.0.dev.zip
> py27cython inst: 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/target/.tox/dist/apache-beam-2.3.0.dev.zip
> py27 runtests: PYTHONHASHSEED='2225684666'
> py27 runtests: commands[0] | python --version
> py27 runtests: commands[1] | - find apache_beam -type f -name *.pyc -delete
> py27 runtests: commands[2] | pip install -e .[test]
> lint runtests: PYTHONHASHSEED='2225684666'
> lint runtests: commands[0] | time pip install -e .[test]
> docs inst: 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/target/.tox/dist/apache-beam-2.3.0.dev.zip
> py27 runtests: commands[3] | python 
> apache_beam/examples/complete/autocomplete_test.py
> lint runtests: commands[1] | time 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/run_pylint.sh
> py27 runtests: commands[4] | python setup.py test
> docs runtests: PYTHONHASHSEED='2225684666'
> docs runtests: commands[0] | time pip install -e .[test,gcp,docs]
> docs runtests: commands[1] | time 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/generate_pydoc.sh
> py27gcp create: 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/target/.tox/py27gcp
> py27gcp installdeps: nose==1.3.7
> py27cython runtests: PYTHONHASHSEED='2225684666'
> py27cython runtests: commands[0] | python --version
> py27cython runtests: commands[1] | - find apache_beam -type f -name *.pyc 
> -delete
> py27cython runtests: commands[2] | - find apache_beam -type f -name *.c 
> -delete
> py27cython runtests: commands[3] | - find apache_beam -type f -name *.so 
> -delete
> py27cython runtests: commands[4] | - find target/build -type f -name *.c 
> -delete
> py27cython runtests: commands[5] | - find target/build -type f -name *.so 
> -delete
> py27cython runtests: commands[6] | time pip install -e .[test]
> py27gcp inst: 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/target/.tox/dist/apache-beam-2.3.0.dev.zip
> py27gcp runtests: PYTHONHASHSEED='2225684666'
> py27gcp runtests: commands[0] | pip install -e .[test,gcp]
> py27gcp runtests: commands[1] | python --version
> py27gcp runtests: commands[2] | - find apache_beam -type f -name *.pyc

[jira] [Commented] (BEAM-3040) Python precommit timed out after 150 minutes

2018-03-01 Thread Ahmet Altay (JIRA) 

[ 
https://issues.apache.org/jira/browse/BEAM-3040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16382419#comment-16382419
 ] 

Ahmet Altay commented on BEAM-3040:
---

Closing this, this is no longer an issue.

> Python precommit timed out after 150 minutes
> 
>
> Key: BEAM-3040
> URL: https://issues.apache.org/jira/browse/BEAM-3040
> Project: Beam
>  Issue Type: Bug
>  Components: sdk-py-core
>Reporter: Kenneth Knowles
>Assignee: Ahmet Altay
>Priority: Major
> Fix For: Not applicable
>
>
> https://builds.apache.org/job/beam_PreCommit_Python_MavenInstall/143/consoleFull
> Within about 10 minutes it reaches this point:
> {code}
> ...
> 2017-10-10T03:33:33.591 [INFO] --- findbugs-maven-plugin:3.0.4:check 
> (default) @ beam-sdks-python ---
> 2017-10-10T03:33:33.702 [INFO] 
> 2017-10-10T03:33:33.702 [INFO] --- exec-maven-plugin:1.5.0:exec 
> (setuptools-test) @ beam-sdks-python ---
> {code}
> and the final output is like this:
> {code}
> ...
> 2017-10-10T03:33:33.702 [INFO] --- exec-maven-plugin:1.5.0:exec 
> (setuptools-test) @ beam-sdks-python ---
> docs create: 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/target/.tox/docs
> GLOB sdist-make: 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/setup.py
> lint create: 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/target/.tox/lint
> py27 create: 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/target/.tox/py27
> py27cython create: 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/target/.tox/py27cython
> py27cython installdeps: nose==1.3.7, grpcio-tools==1.3.5, cython==0.25.2
> docs installdeps: nose==1.3.7, grpcio-tools==1.3.5, Sphinx==1.5.5, 
> sphinx_rtd_theme==0.2.4
> lint installdeps: nose==1.3.7, pycodestyle==2.3.1, pylint==1.7.1
> py27 installdeps: nose==1.3.7, grpcio-tools==1.3.5
> lint inst: 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/target/.tox/dist/apache-beam-2.3.0.dev.zip
> py27 inst: 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/target/.tox/dist/apache-beam-2.3.0.dev.zip
> py27cython inst: 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/target/.tox/dist/apache-beam-2.3.0.dev.zip
> py27 runtests: PYTHONHASHSEED='2225684666'
> py27 runtests: commands[0] | python --version
> py27 runtests: commands[1] | - find apache_beam -type f -name *.pyc -delete
> py27 runtests: commands[2] | pip install -e .[test]
> lint runtests: PYTHONHASHSEED='2225684666'
> lint runtests: commands[0] | time pip install -e .[test]
> docs inst: 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/target/.tox/dist/apache-beam-2.3.0.dev.zip
> py27 runtests: commands[3] | python 
> apache_beam/examples/complete/autocomplete_test.py
> lint runtests: commands[1] | time 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/run_pylint.sh
> py27 runtests: commands[4] | python setup.py test
> docs runtests: PYTHONHASHSEED='2225684666'
> docs runtests: commands[0] | time pip install -e .[test,gcp,docs]
> docs runtests: commands[1] | time 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/generate_pydoc.sh
> py27gcp create: 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/target/.tox/py27gcp
> py27gcp installdeps: nose==1.3.7
> py27cython runtests: PYTHONHASHSEED='2225684666'
> py27cython runtests: commands[0] | python --version
> py27cython runtests: commands[1] | - find apache_beam -type f -name *.pyc 
> -delete
> py27cython runtests: commands[2] | - find apache_beam -type f -name *.c 
> -delete
> py27cython runtests: commands[3] | - find apache_beam -type f -name *.so 
> -delete
> py27cython runtests: commands[4] | - find target/build -type f -name *.c 
> -delete
> py27cython runtests: commands[5] | - find target/build -type f -name *.so 
> -delete
> py27cython runtests: commands[6] | time pip install -e .[test]
> py27gcp inst: 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/target/.tox/dist/apache-beam-2.3.0.dev.zip
> py27gcp runtests: PYTHONHASHSEED='2225684666'
> py27gcp runtests: commands[0] | pip install -e .[test,gcp]
> py27gcp runtests: commands[1] | python --version
> py27gcp runtests: commands[2] | - find apache_beam -type f -name *.pyc -delete
> py27gcp runtests: commands[3] | python 
> apache_beam/examples/complete/autocomplete_test.py
> py27gcp runtests: commands[4] | python setup.py test
> py27cython runtests: commands[7] | python 
>

[jira] [Commented] (BEAM-3040) Python precommit timed out after 150 minutes

2017-12-05 Thread Valentyn Tymofieiev (JIRA) 

[ 
https://issues.apache.org/jira/browse/BEAM-3040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16279681#comment-16279681
 ] 

Valentyn Tymofieiev commented on BEAM-3040:
---

cc: [~robertwb]
Another recent error:
test_errors 
(apache_beam.runners.portability.universal_local_runner_test.UniversalLocalRunnerTest)
 ... Traceback (most recent call last):
  File 
"/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/src/sdks/python/apache_beam/runners/portability/universal_local_runner.py",
 line 243, in run
).run_via_runner_api(self._pipeline_proto)
  File 
"/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/src/sdks/python/apache_beam/runners/portability/fn_api_runner.py",
 line 204, in run_via_runner_api
return self.run_stages(*self.create_stages(pipeline_proto))
  File 
"/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/src/sdks/python/apache_beam/runners/portability/fn_api_runner.py",
 line 670, in run_stages
pcoll_buffers, safe_coders).process_bundle.metrics
  File 
"/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/src/sdks/python/apache_beam/runners/portability/fn_api_runner.py",
 line 763, in run_stage
controller.control_handler.push(process_bundle)
  File 
"/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/src/sdks/python/apache_beam/runners/portability/fn_api_runner.py",
 line 871, in push
response = self.worker.do_instruction(request)
  File 
"/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",
 line 126, in do_instruction
getattr(request, request_type), request.instruction_id)
  File 
"/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",
 line 144, in process_bundle
processor.process_bundle(instruction_id)
  File 
"/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/src/sdks/python/apache_beam/runners/worker/bundle_processor.py",
 line 293, in process_bundle
op.start()
  File 
"/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/src/sdks/python/apache_beam/runners/worker/operations.py",
 line 233, in start
self.output(windowed_value)
  File 
"/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/src/sdks/python/apache_beam/runners/worker/operations.py",
 line 156, in output
cython.cast(Receiver, self.receivers[output_index]).receive(windowed_value)
  File 
"/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/src/sdks/python/apache_beam/runners/worker/operations.py",
 line 87, in receive
cython.cast(Operation, consumer).process(windowed_value)
  File 
"/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/src/sdks/python/apache_beam/runners/worker/operations.py",
 line 371, in process
self.dofn_receiver.receive(o)
  File 
"/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/src/sdks/python/apache_beam/runners/common.py",
 line 382, in receive
self.process(windowed_value)
  File 
"/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/src/sdks/python/apache_beam/runners/common.py",
 line 390, in process
self._reraise_augmented(exn)
  File 
"/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/src/sdks/python/apache_beam/runners/common.py",
 line 388, in process
self.do_fn_invoker.invoke_process(windowed_value)
  File 
"/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/src/sdks/python/apache_beam/runners/common.py",
 line 190, in invoke_process
windowed_value, self.process_method(windowed_value.value))
  File 
"/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/src/sdks/python/apache_beam/runners/common.py",
 line 480, in process_outputs
self.main_receivers.receive(windowed_value)
  File 
"/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/src/sdks/python/apache_beam/runners/worker/operations.py",
 line 87, in receive
cython.cast(Operation, consumer).process(windowed_value)
  File 
"/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/src/sdks/python/apache_beam/runners/worker/operations.py",
 line 371, in process
self.dofn_receiver.receive(o)
  File 
"/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/src/sdks/python/apache_beam/runners/common.py",
 line 382, in receive
self.process(windowed_value)
  File 
"/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/src/sdks/python/apache_beam/runners/common.py",
 line 390, in process
self._reraise_augmented(exn)
  File 
"/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/src/sdks/python/apache_beam/runners/common.py",
 line 388, in

[jira] [Commented] (BEAM-3040) Python precommit timed out after 150 minutes

2017-11-21 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/BEAM-3040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16261528#comment-16261528
 ] 

ASF GitHub Bot commented on BEAM-3040:
--

Github user asfgit closed the pull request at:

https://github.com/apache/beam/pull/4159


> Python precommit timed out after 150 minutes
> 
>
> Key: BEAM-3040
> URL: https://issues.apache.org/jira/browse/BEAM-3040
> Project: Beam
>  Issue Type: Bug
>  Components: sdk-py-core
>Reporter: Kenneth Knowles
>Assignee: Ahmet Altay
>
> https://builds.apache.org/job/beam_PreCommit_Python_MavenInstall/143/consoleFull
> Within about 10 minutes it reaches this point:
> {code}
> ...
> 2017-10-10T03:33:33.591 [INFO] --- findbugs-maven-plugin:3.0.4:check 
> (default) @ beam-sdks-python ---
> 2017-10-10T03:33:33.702 [INFO] 
> 2017-10-10T03:33:33.702 [INFO] --- exec-maven-plugin:1.5.0:exec 
> (setuptools-test) @ beam-sdks-python ---
> {code}
> and the final output is like this:
> {code}
> ...
> 2017-10-10T03:33:33.702 [INFO] --- exec-maven-plugin:1.5.0:exec 
> (setuptools-test) @ beam-sdks-python ---
> docs create: 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/target/.tox/docs
> GLOB sdist-make: 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/setup.py
> lint create: 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/target/.tox/lint
> py27 create: 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/target/.tox/py27
> py27cython create: 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/target/.tox/py27cython
> py27cython installdeps: nose==1.3.7, grpcio-tools==1.3.5, cython==0.25.2
> docs installdeps: nose==1.3.7, grpcio-tools==1.3.5, Sphinx==1.5.5, 
> sphinx_rtd_theme==0.2.4
> lint installdeps: nose==1.3.7, pycodestyle==2.3.1, pylint==1.7.1
> py27 installdeps: nose==1.3.7, grpcio-tools==1.3.5
> lint inst: 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/target/.tox/dist/apache-beam-2.3.0.dev.zip
> py27 inst: 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/target/.tox/dist/apache-beam-2.3.0.dev.zip
> py27cython inst: 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/target/.tox/dist/apache-beam-2.3.0.dev.zip
> py27 runtests: PYTHONHASHSEED='2225684666'
> py27 runtests: commands[0] | python --version
> py27 runtests: commands[1] | - find apache_beam -type f -name *.pyc -delete
> py27 runtests: commands[2] | pip install -e .[test]
> lint runtests: PYTHONHASHSEED='2225684666'
> lint runtests: commands[0] | time pip install -e .[test]
> docs inst: 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/target/.tox/dist/apache-beam-2.3.0.dev.zip
> py27 runtests: commands[3] | python 
> apache_beam/examples/complete/autocomplete_test.py
> lint runtests: commands[1] | time 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/run_pylint.sh
> py27 runtests: commands[4] | python setup.py test
> docs runtests: PYTHONHASHSEED='2225684666'
> docs runtests: commands[0] | time pip install -e .[test,gcp,docs]
> docs runtests: commands[1] | time 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/generate_pydoc.sh
> py27gcp create: 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/target/.tox/py27gcp
> py27gcp installdeps: nose==1.3.7
> py27cython runtests: PYTHONHASHSEED='2225684666'
> py27cython runtests: commands[0] | python --version
> py27cython runtests: commands[1] | - find apache_beam -type f -name *.pyc 
> -delete
> py27cython runtests: commands[2] | - find apache_beam -type f -name *.c 
> -delete
> py27cython runtests: commands[3] | - find apache_beam -type f -name *.so 
> -delete
> py27cython runtests: commands[4] | - find target/build -type f -name *.c 
> -delete
> py27cython runtests: commands[5] | - find target/build -type f -name *.so 
> -delete
> py27cython runtests: commands[6] | time pip install -e .[test]
> py27gcp inst: 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/target/.tox/dist/apache-beam-2.3.0.dev.zip
> py27gcp runtests: PYTHONHASHSEED='2225684666'
> py27gcp runtests: commands[0] | pip install -e .[test,gcp]
> py27gcp runtests: commands[1] | python --version
> py27gcp runtests: commands[2] | - find apache_beam -type f -name *.pyc -delete
> py27gcp runtests: commands[3] | python 
> apache_beam/examples/complete/autocomplete_test.py
> py27gcp runtests: commands[4] | python setup.py test
> py27cython runtests: commands[7] | python 
>

[jira] [Commented] (BEAM-3040) Python precommit timed out after 150 minutes

2017-11-21 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/BEAM-3040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16261441#comment-16261441
 ] 

ASF GitHub Bot commented on BEAM-3040:
--

GitHub user robertwb opened a pull request:

https://github.com/apache/beam/pull/4159

[BEAM-3040] Add logging and test aborting for timeouts.

Follow this checklist to help us incorporate your contribution quickly and 
easily:

 - [ ] Make sure there is a [JIRA 
issue](https://issues.apache.org/jira/projects/BEAM/issues/) filed for the 
change (usually before you start working on it).  Trivial changes like typos do 
not require a JIRA issue.  Your pull request should address just this issue, 
without pulling in other changes.
 - [ ] Each commit in the pull request should have a meaningful subject 
line and body.
 - [ ] Format the pull request title like `[BEAM-XXX] Fixes bug in 
ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA 
issue.
 - [ ] Write a pull request description that is detailed enough to 
understand what the pull request does, how, and why.
 - [ ] Run `mvn clean verify` to make sure basic checks pass. A more 
thorough check will be performed on your pull request automatically.
 - [ ] If this contribution is large, please file an Apache [Individual 
Contributor License Agreement](https://www.apache.org/licenses/icla.pdf).

---


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/robertwb/incubator-beam debug-hang

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/beam/pull/4159.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #4159


commit 768c854f3723dd747e905aeb0d35024ae44e2cda
Author: Robert Bradshaw 
Date:   2017-11-21T20:25:18Z

Add logging and test aborting for timeouts.




> Python precommit timed out after 150 minutes
> 
>
> Key: BEAM-3040
> URL: https://issues.apache.org/jira/browse/BEAM-3040
> Project: Beam
>  Issue Type: Bug
>  Components: sdk-py-core
>Reporter: Kenneth Knowles
>Assignee: Ahmet Altay
>
> https://builds.apache.org/job/beam_PreCommit_Python_MavenInstall/143/consoleFull
> Within about 10 minutes it reaches this point:
> {code}
> ...
> 2017-10-10T03:33:33.591 [INFO] --- findbugs-maven-plugin:3.0.4:check 
> (default) @ beam-sdks-python ---
> 2017-10-10T03:33:33.702 [INFO] 
> 2017-10-10T03:33:33.702 [INFO] --- exec-maven-plugin:1.5.0:exec 
> (setuptools-test) @ beam-sdks-python ---
> {code}
> and the final output is like this:
> {code}
> ...
> 2017-10-10T03:33:33.702 [INFO] --- exec-maven-plugin:1.5.0:exec 
> (setuptools-test) @ beam-sdks-python ---
> docs create: 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/target/.tox/docs
> GLOB sdist-make: 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/setup.py
> lint create: 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/target/.tox/lint
> py27 create: 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/target/.tox/py27
> py27cython create: 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/target/.tox/py27cython
> py27cython installdeps: nose==1.3.7, grpcio-tools==1.3.5, cython==0.25.2
> docs installdeps: nose==1.3.7, grpcio-tools==1.3.5, Sphinx==1.5.5, 
> sphinx_rtd_theme==0.2.4
> lint installdeps: nose==1.3.7, pycodestyle==2.3.1, pylint==1.7.1
> py27 installdeps: nose==1.3.7, grpcio-tools==1.3.5
> lint inst: 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/target/.tox/dist/apache-beam-2.3.0.dev.zip
> py27 inst: 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/target/.tox/dist/apache-beam-2.3.0.dev.zip
> py27cython inst: 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/target/.tox/dist/apache-beam-2.3.0.dev.zip
> py27 runtests: PYTHONHASHSEED='2225684666'
> py27 runtests: commands[0] | python --version
> py27 runtests: commands[1] | - find apache_beam -type f -name *.pyc -delete
> py27 runtests: commands[2] | pip install -e .[test]
> lint runtests: PYTHONHASHSEED='2225684666'
> lint runtests: commands[0] | time pip install -e .[test]
> docs inst: 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/target/.tox/dist/apache-beam-2.3.0.dev.zip
> py27 runtests: commands[3] | python 
> apache_beam/examples/complete/autocomplete_test.py
> lint runtests: commands[1] | time 
>

[jira] [Commented] (BEAM-3040) Python precommit timed out after 150 minutes

2017-11-08 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/BEAM-3040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16244716#comment-16244716
 ] 

ASF GitHub Bot commented on BEAM-3040:
--

Github user asfgit closed the pull request at:

https://github.com/apache/beam/pull/4099


> Python precommit timed out after 150 minutes
> 
>
> Key: BEAM-3040
> URL: https://issues.apache.org/jira/browse/BEAM-3040
> Project: Beam
>  Issue Type: Bug
>  Components: sdk-py-core
>Reporter: Kenneth Knowles
>Assignee: Ahmet Altay
>
> https://builds.apache.org/job/beam_PreCommit_Python_MavenInstall/143/consoleFull
> Within about 10 minutes it reaches this point:
> {code}
> ...
> 2017-10-10T03:33:33.591 [INFO] --- findbugs-maven-plugin:3.0.4:check 
> (default) @ beam-sdks-python ---
> 2017-10-10T03:33:33.702 [INFO] 
> 2017-10-10T03:33:33.702 [INFO] --- exec-maven-plugin:1.5.0:exec 
> (setuptools-test) @ beam-sdks-python ---
> {code}
> and the final output is like this:
> {code}
> ...
> 2017-10-10T03:33:33.702 [INFO] --- exec-maven-plugin:1.5.0:exec 
> (setuptools-test) @ beam-sdks-python ---
> docs create: 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/target/.tox/docs
> GLOB sdist-make: 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/setup.py
> lint create: 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/target/.tox/lint
> py27 create: 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/target/.tox/py27
> py27cython create: 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/target/.tox/py27cython
> py27cython installdeps: nose==1.3.7, grpcio-tools==1.3.5, cython==0.25.2
> docs installdeps: nose==1.3.7, grpcio-tools==1.3.5, Sphinx==1.5.5, 
> sphinx_rtd_theme==0.2.4
> lint installdeps: nose==1.3.7, pycodestyle==2.3.1, pylint==1.7.1
> py27 installdeps: nose==1.3.7, grpcio-tools==1.3.5
> lint inst: 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/target/.tox/dist/apache-beam-2.3.0.dev.zip
> py27 inst: 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/target/.tox/dist/apache-beam-2.3.0.dev.zip
> py27cython inst: 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/target/.tox/dist/apache-beam-2.3.0.dev.zip
> py27 runtests: PYTHONHASHSEED='2225684666'
> py27 runtests: commands[0] | python --version
> py27 runtests: commands[1] | - find apache_beam -type f -name *.pyc -delete
> py27 runtests: commands[2] | pip install -e .[test]
> lint runtests: PYTHONHASHSEED='2225684666'
> lint runtests: commands[0] | time pip install -e .[test]
> docs inst: 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/target/.tox/dist/apache-beam-2.3.0.dev.zip
> py27 runtests: commands[3] | python 
> apache_beam/examples/complete/autocomplete_test.py
> lint runtests: commands[1] | time 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/run_pylint.sh
> py27 runtests: commands[4] | python setup.py test
> docs runtests: PYTHONHASHSEED='2225684666'
> docs runtests: commands[0] | time pip install -e .[test,gcp,docs]
> docs runtests: commands[1] | time 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/generate_pydoc.sh
> py27gcp create: 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/target/.tox/py27gcp
> py27gcp installdeps: nose==1.3.7
> py27cython runtests: PYTHONHASHSEED='2225684666'
> py27cython runtests: commands[0] | python --version
> py27cython runtests: commands[1] | - find apache_beam -type f -name *.pyc 
> -delete
> py27cython runtests: commands[2] | - find apache_beam -type f -name *.c 
> -delete
> py27cython runtests: commands[3] | - find apache_beam -type f -name *.so 
> -delete
> py27cython runtests: commands[4] | - find target/build -type f -name *.c 
> -delete
> py27cython runtests: commands[5] | - find target/build -type f -name *.so 
> -delete
> py27cython runtests: commands[6] | time pip install -e .[test]
> py27gcp inst: 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/target/.tox/dist/apache-beam-2.3.0.dev.zip
> py27gcp runtests: PYTHONHASHSEED='2225684666'
> py27gcp runtests: commands[0] | pip install -e .[test,gcp]
> py27gcp runtests: commands[1] | python --version
> py27gcp runtests: commands[2] | - find apache_beam -type f -name *.pyc -delete
> py27gcp runtests: commands[3] | python 
> apache_beam/examples/complete/autocomplete_test.py
> py27gcp runtests: commands[4] | python setup.py test
> py27cython runtests: commands[7] | python 
>

[jira] [Commented] (BEAM-3040) Python precommit timed out after 150 minutes

2017-11-08 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/BEAM-3040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16244548#comment-16244548
 ] 

ASF GitHub Bot commented on BEAM-3040:
--

GitHub user robertwb opened a pull request:

https://github.com/apache/beam/pull/4099

[BEAM-3040] Disable flaky subprocess variant of ULR test.

Follow this checklist to help us incorporate your contribution quickly and 
easily:

 - [ ] Make sure there is a [JIRA 
issue](https://issues.apache.org/jira/projects/BEAM/issues/) filed for the 
change (usually before you start working on it).  Trivial changes like typos do 
not require a JIRA issue.  Your pull request should address just this issue, 
without pulling in other changes.
 - [ ] Each commit in the pull request should have a meaningful subject 
line and body.
 - [ ] Format the pull request title like `[BEAM-XXX] Fixes bug in 
ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA 
issue.
 - [ ] Write a pull request description that is detailed enough to 
understand what the pull request does, how, and why.
 - [ ] Run `mvn clean verify` to make sure basic checks pass. A more 
thorough check will be performed on your pull request automatically.
 - [ ] If this contribution is large, please file an Apache [Individual 
Contributor License Agreement](https://www.apache.org/licenses/icla.pdf).

---


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/robertwb/incubator-beam beam-3040

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/beam/pull/4099.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #4099


commit 30740a53588194459e23e7b8ddc69a3ab506e2cb
Author: Robert Bradshaw 
Date:   2017-11-08T19:00:33Z

[BEAM-3040] Disable flaky subprocess variant of ULR test.




> Python precommit timed out after 150 minutes
> 
>
> Key: BEAM-3040
> URL: https://issues.apache.org/jira/browse/BEAM-3040
> Project: Beam
>  Issue Type: Bug
>  Components: sdk-py-core
>Reporter: Kenneth Knowles
>Assignee: Ahmet Altay
>
> https://builds.apache.org/job/beam_PreCommit_Python_MavenInstall/143/consoleFull
> Within about 10 minutes it reaches this point:
> {code}
> ...
> 2017-10-10T03:33:33.591 [INFO] --- findbugs-maven-plugin:3.0.4:check 
> (default) @ beam-sdks-python ---
> 2017-10-10T03:33:33.702 [INFO] 
> 2017-10-10T03:33:33.702 [INFO] --- exec-maven-plugin:1.5.0:exec 
> (setuptools-test) @ beam-sdks-python ---
> {code}
> and the final output is like this:
> {code}
> ...
> 2017-10-10T03:33:33.702 [INFO] --- exec-maven-plugin:1.5.0:exec 
> (setuptools-test) @ beam-sdks-python ---
> docs create: 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/target/.tox/docs
> GLOB sdist-make: 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/setup.py
> lint create: 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/target/.tox/lint
> py27 create: 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/target/.tox/py27
> py27cython create: 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/target/.tox/py27cython
> py27cython installdeps: nose==1.3.7, grpcio-tools==1.3.5, cython==0.25.2
> docs installdeps: nose==1.3.7, grpcio-tools==1.3.5, Sphinx==1.5.5, 
> sphinx_rtd_theme==0.2.4
> lint installdeps: nose==1.3.7, pycodestyle==2.3.1, pylint==1.7.1
> py27 installdeps: nose==1.3.7, grpcio-tools==1.3.5
> lint inst: 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/target/.tox/dist/apache-beam-2.3.0.dev.zip
> py27 inst: 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/target/.tox/dist/apache-beam-2.3.0.dev.zip
> py27cython inst: 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/target/.tox/dist/apache-beam-2.3.0.dev.zip
> py27 runtests: PYTHONHASHSEED='2225684666'
> py27 runtests: commands[0] | python --version
> py27 runtests: commands[1] | - find apache_beam -type f -name *.pyc -delete
> py27 runtests: commands[2] | pip install -e .[test]
> lint runtests: PYTHONHASHSEED='2225684666'
> lint runtests: commands[0] | time pip install -e .[test]
> docs inst: 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/target/.tox/dist/apache-beam-2.3.0.dev.zip
> py27 runtests: commands[3] | python 
> apache_beam/examples/complete/autocomplete_test.py
> lint runtests: commands[1] | time 
>

[jira] [Commented] (BEAM-3040) Python precommit timed out after 150 minutes

2017-10-13 Thread Jason Kuster (JIRA) 

[ 
https://issues.apache.org/jira/browse/BEAM-3040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16204318#comment-16204318
 ] 

Jason Kuster commented on BEAM-3040:


FWIW the aforementioned JIRA (INFRA-15195) had some activity recently -- 
assuming Puppet has run on all the slaves pip should be up to date.

> Python precommit timed out after 150 minutes
> 
>
> Key: BEAM-3040
> URL: https://issues.apache.org/jira/browse/BEAM-3040
> Project: Beam
>  Issue Type: Bug
>  Components: sdk-py-core
>Reporter: Kenneth Knowles
>Assignee: Ahmet Altay
>
> https://builds.apache.org/job/beam_PreCommit_Python_MavenInstall/143/consoleFull
> Within about 10 minutes it reaches this point:
> {code}
> ...
> 2017-10-10T03:33:33.591 [INFO] --- findbugs-maven-plugin:3.0.4:check 
> (default) @ beam-sdks-python ---
> 2017-10-10T03:33:33.702 [INFO] 
> 2017-10-10T03:33:33.702 [INFO] --- exec-maven-plugin:1.5.0:exec 
> (setuptools-test) @ beam-sdks-python ---
> {code}
> and the final output is like this:
> {code}
> ...
> 2017-10-10T03:33:33.702 [INFO] --- exec-maven-plugin:1.5.0:exec 
> (setuptools-test) @ beam-sdks-python ---
> docs create: 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/target/.tox/docs
> GLOB sdist-make: 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/setup.py
> lint create: 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/target/.tox/lint
> py27 create: 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/target/.tox/py27
> py27cython create: 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/target/.tox/py27cython
> py27cython installdeps: nose==1.3.7, grpcio-tools==1.3.5, cython==0.25.2
> docs installdeps: nose==1.3.7, grpcio-tools==1.3.5, Sphinx==1.5.5, 
> sphinx_rtd_theme==0.2.4
> lint installdeps: nose==1.3.7, pycodestyle==2.3.1, pylint==1.7.1
> py27 installdeps: nose==1.3.7, grpcio-tools==1.3.5
> lint inst: 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/target/.tox/dist/apache-beam-2.3.0.dev.zip
> py27 inst: 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/target/.tox/dist/apache-beam-2.3.0.dev.zip
> py27cython inst: 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/target/.tox/dist/apache-beam-2.3.0.dev.zip
> py27 runtests: PYTHONHASHSEED='2225684666'
> py27 runtests: commands[0] | python --version
> py27 runtests: commands[1] | - find apache_beam -type f -name *.pyc -delete
> py27 runtests: commands[2] | pip install -e .[test]
> lint runtests: PYTHONHASHSEED='2225684666'
> lint runtests: commands[0] | time pip install -e .[test]
> docs inst: 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/target/.tox/dist/apache-beam-2.3.0.dev.zip
> py27 runtests: commands[3] | python 
> apache_beam/examples/complete/autocomplete_test.py
> lint runtests: commands[1] | time 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/run_pylint.sh
> py27 runtests: commands[4] | python setup.py test
> docs runtests: PYTHONHASHSEED='2225684666'
> docs runtests: commands[0] | time pip install -e .[test,gcp,docs]
> docs runtests: commands[1] | time 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/generate_pydoc.sh
> py27gcp create: 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/target/.tox/py27gcp
> py27gcp installdeps: nose==1.3.7
> py27cython runtests: PYTHONHASHSEED='2225684666'
> py27cython runtests: commands[0] | python --version
> py27cython runtests: commands[1] | - find apache_beam -type f -name *.pyc 
> -delete
> py27cython runtests: commands[2] | - find apache_beam -type f -name *.c 
> -delete
> py27cython runtests: commands[3] | - find apache_beam -type f -name *.so 
> -delete
> py27cython runtests: commands[4] | - find target/build -type f -name *.c 
> -delete
> py27cython runtests: commands[5] | - find target/build -type f -name *.so 
> -delete
> py27cython runtests: commands[6] | time pip install -e .[test]
> py27gcp inst: 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/target/.tox/dist/apache-beam-2.3.0.dev.zip
> py27gcp runtests: PYTHONHASHSEED='2225684666'
> py27gcp runtests: commands[0] | pip install -e .[test,gcp]
> py27gcp runtests: commands[1] | python --version
> py27gcp runtests: commands[2] | - find apache_beam -type f -name *.pyc -delete
> py27gcp runtests: commands[3] | python 
> apache_beam/examples/complete/autocomplete_test.py
> py27gcp runtests: commands[4] | python setup.py test
> py27cython runtests: commands[7] |

[jira] [Commented] (BEAM-3040) Python precommit timed out after 150 minutes

2017-10-12 Thread Ahmet Altay (JIRA) 

[ 
https://issues.apache.org/jira/browse/BEAM-3040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16202649#comment-16202649
 ] 

Ahmet Altay commented on BEAM-3040:
---

150 minutes is reasonable. Successful jobs complete under an hour.

> Python precommit timed out after 150 minutes
> 
>
> Key: BEAM-3040
> URL: https://issues.apache.org/jira/browse/BEAM-3040
> Project: Beam
>  Issue Type: Bug
>  Components: sdk-py-core
>Reporter: Kenneth Knowles
>Assignee: Ahmet Altay
>
> https://builds.apache.org/job/beam_PreCommit_Python_MavenInstall/143/consoleFull
> Within about 10 minutes it reaches this point:
> {code}
> ...
> 2017-10-10T03:33:33.591 [INFO] --- findbugs-maven-plugin:3.0.4:check 
> (default) @ beam-sdks-python ---
> 2017-10-10T03:33:33.702 [INFO] 
> 2017-10-10T03:33:33.702 [INFO] --- exec-maven-plugin:1.5.0:exec 
> (setuptools-test) @ beam-sdks-python ---
> {code}
> and the final output is like this:
> {code}
> ...
> 2017-10-10T03:33:33.702 [INFO] --- exec-maven-plugin:1.5.0:exec 
> (setuptools-test) @ beam-sdks-python ---
> docs create: 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/target/.tox/docs
> GLOB sdist-make: 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/setup.py
> lint create: 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/target/.tox/lint
> py27 create: 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/target/.tox/py27
> py27cython create: 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/target/.tox/py27cython
> py27cython installdeps: nose==1.3.7, grpcio-tools==1.3.5, cython==0.25.2
> docs installdeps: nose==1.3.7, grpcio-tools==1.3.5, Sphinx==1.5.5, 
> sphinx_rtd_theme==0.2.4
> lint installdeps: nose==1.3.7, pycodestyle==2.3.1, pylint==1.7.1
> py27 installdeps: nose==1.3.7, grpcio-tools==1.3.5
> lint inst: 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/target/.tox/dist/apache-beam-2.3.0.dev.zip
> py27 inst: 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/target/.tox/dist/apache-beam-2.3.0.dev.zip
> py27cython inst: 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/target/.tox/dist/apache-beam-2.3.0.dev.zip
> py27 runtests: PYTHONHASHSEED='2225684666'
> py27 runtests: commands[0] | python --version
> py27 runtests: commands[1] | - find apache_beam -type f -name *.pyc -delete
> py27 runtests: commands[2] | pip install -e .[test]
> lint runtests: PYTHONHASHSEED='2225684666'
> lint runtests: commands[0] | time pip install -e .[test]
> docs inst: 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/target/.tox/dist/apache-beam-2.3.0.dev.zip
> py27 runtests: commands[3] | python 
> apache_beam/examples/complete/autocomplete_test.py
> lint runtests: commands[1] | time 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/run_pylint.sh
> py27 runtests: commands[4] | python setup.py test
> docs runtests: PYTHONHASHSEED='2225684666'
> docs runtests: commands[0] | time pip install -e .[test,gcp,docs]
> docs runtests: commands[1] | time 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/generate_pydoc.sh
> py27gcp create: 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/target/.tox/py27gcp
> py27gcp installdeps: nose==1.3.7
> py27cython runtests: PYTHONHASHSEED='2225684666'
> py27cython runtests: commands[0] | python --version
> py27cython runtests: commands[1] | - find apache_beam -type f -name *.pyc 
> -delete
> py27cython runtests: commands[2] | - find apache_beam -type f -name *.c 
> -delete
> py27cython runtests: commands[3] | - find apache_beam -type f -name *.so 
> -delete
> py27cython runtests: commands[4] | - find target/build -type f -name *.c 
> -delete
> py27cython runtests: commands[5] | - find target/build -type f -name *.so 
> -delete
> py27cython runtests: commands[6] | time pip install -e .[test]
> py27gcp inst: 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/target/.tox/dist/apache-beam-2.3.0.dev.zip
> py27gcp runtests: PYTHONHASHSEED='2225684666'
> py27gcp runtests: commands[0] | pip install -e .[test,gcp]
> py27gcp runtests: commands[1] | python --version
> py27gcp runtests: commands[2] | - find apache_beam -type f -name *.pyc -delete
> py27gcp runtests: commands[3] | python 
> apache_beam/examples/complete/autocomplete_test.py
> py27gcp runtests: commands[4] | python setup.py test
> py27cython runtests: commands[7] | python 
> apache_beam/examples/complete/autocomplete_test.py
> py27cython

[jira] [Commented] (BEAM-3040) Python precommit timed out after 150 minutes

2017-10-12 Thread Valentyn Tymofieiev (JIRA) 

[ 
https://issues.apache.org/jira/browse/BEAM-3040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16202643#comment-16202643
 ] 

Valentyn Tymofieiev commented on BEAM-3040:
---

Is 150 min a reasonable timeout for this test suite? 

> Python precommit timed out after 150 minutes
> 
>
> Key: BEAM-3040
> URL: https://issues.apache.org/jira/browse/BEAM-3040
> Project: Beam
>  Issue Type: Bug
>  Components: sdk-py-core
>Reporter: Kenneth Knowles
>Assignee: Ahmet Altay
>
> https://builds.apache.org/job/beam_PreCommit_Python_MavenInstall/143/consoleFull
> Within about 10 minutes it reaches this point:
> {code}
> ...
> 2017-10-10T03:33:33.591 [INFO] --- findbugs-maven-plugin:3.0.4:check 
> (default) @ beam-sdks-python ---
> 2017-10-10T03:33:33.702 [INFO] 
> 2017-10-10T03:33:33.702 [INFO] --- exec-maven-plugin:1.5.0:exec 
> (setuptools-test) @ beam-sdks-python ---
> {code}
> and the final output is like this:
> {code}
> ...
> 2017-10-10T03:33:33.702 [INFO] --- exec-maven-plugin:1.5.0:exec 
> (setuptools-test) @ beam-sdks-python ---
> docs create: 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/target/.tox/docs
> GLOB sdist-make: 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/setup.py
> lint create: 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/target/.tox/lint
> py27 create: 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/target/.tox/py27
> py27cython create: 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/target/.tox/py27cython
> py27cython installdeps: nose==1.3.7, grpcio-tools==1.3.5, cython==0.25.2
> docs installdeps: nose==1.3.7, grpcio-tools==1.3.5, Sphinx==1.5.5, 
> sphinx_rtd_theme==0.2.4
> lint installdeps: nose==1.3.7, pycodestyle==2.3.1, pylint==1.7.1
> py27 installdeps: nose==1.3.7, grpcio-tools==1.3.5
> lint inst: 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/target/.tox/dist/apache-beam-2.3.0.dev.zip
> py27 inst: 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/target/.tox/dist/apache-beam-2.3.0.dev.zip
> py27cython inst: 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/target/.tox/dist/apache-beam-2.3.0.dev.zip
> py27 runtests: PYTHONHASHSEED='2225684666'
> py27 runtests: commands[0] | python --version
> py27 runtests: commands[1] | - find apache_beam -type f -name *.pyc -delete
> py27 runtests: commands[2] | pip install -e .[test]
> lint runtests: PYTHONHASHSEED='2225684666'
> lint runtests: commands[0] | time pip install -e .[test]
> docs inst: 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/target/.tox/dist/apache-beam-2.3.0.dev.zip
> py27 runtests: commands[3] | python 
> apache_beam/examples/complete/autocomplete_test.py
> lint runtests: commands[1] | time 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/run_pylint.sh
> py27 runtests: commands[4] | python setup.py test
> docs runtests: PYTHONHASHSEED='2225684666'
> docs runtests: commands[0] | time pip install -e .[test,gcp,docs]
> docs runtests: commands[1] | time 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/generate_pydoc.sh
> py27gcp create: 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/target/.tox/py27gcp
> py27gcp installdeps: nose==1.3.7
> py27cython runtests: PYTHONHASHSEED='2225684666'
> py27cython runtests: commands[0] | python --version
> py27cython runtests: commands[1] | - find apache_beam -type f -name *.pyc 
> -delete
> py27cython runtests: commands[2] | - find apache_beam -type f -name *.c 
> -delete
> py27cython runtests: commands[3] | - find apache_beam -type f -name *.so 
> -delete
> py27cython runtests: commands[4] | - find target/build -type f -name *.c 
> -delete
> py27cython runtests: commands[5] | - find target/build -type f -name *.so 
> -delete
> py27cython runtests: commands[6] | time pip install -e .[test]
> py27gcp inst: 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/target/.tox/dist/apache-beam-2.3.0.dev.zip
> py27gcp runtests: PYTHONHASHSEED='2225684666'
> py27gcp runtests: commands[0] | pip install -e .[test,gcp]
> py27gcp runtests: commands[1] | python --version
> py27gcp runtests: commands[2] | - find apache_beam -type f -name *.pyc -delete
> py27gcp runtests: commands[3] | python 
> apache_beam/examples/complete/autocomplete_test.py
> py27gcp runtests: commands[4] | python setup.py test
> py27cython runtests: commands[7] | python 
> apache_beam/examples/complete/autocomplete_test.py
> py27cython

[jira] [Commented] (BEAM-3040) Python precommit timed out after 150 minutes

2017-10-12 Thread Ahmet Altay (JIRA) 

[ 
https://issues.apache.org/jira/browse/BEAM-3040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16202451#comment-16202451
 ] 

Ahmet Altay commented on BEAM-3040:
---

Yes we can do the virtualenv thing. Indeed virtualenv comes with its newer pip 
version. This should be already happening though, since tox will use virtualenv 
to run tests.

> Python precommit timed out after 150 minutes
> 
>
> Key: BEAM-3040
> URL: https://issues.apache.org/jira/browse/BEAM-3040
> Project: Beam
>  Issue Type: Bug
>  Components: sdk-py-core
>Reporter: Kenneth Knowles
>Assignee: Ahmet Altay
>
> https://builds.apache.org/job/beam_PreCommit_Python_MavenInstall/143/consoleFull
> Within about 10 minutes it reaches this point:
> {code}
> ...
> 2017-10-10T03:33:33.591 [INFO] --- findbugs-maven-plugin:3.0.4:check 
> (default) @ beam-sdks-python ---
> 2017-10-10T03:33:33.702 [INFO] 
> 2017-10-10T03:33:33.702 [INFO] --- exec-maven-plugin:1.5.0:exec 
> (setuptools-test) @ beam-sdks-python ---
> {code}
> and the final output is like this:
> {code}
> ...
> 2017-10-10T03:33:33.702 [INFO] --- exec-maven-plugin:1.5.0:exec 
> (setuptools-test) @ beam-sdks-python ---
> docs create: 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/target/.tox/docs
> GLOB sdist-make: 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/setup.py
> lint create: 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/target/.tox/lint
> py27 create: 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/target/.tox/py27
> py27cython create: 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/target/.tox/py27cython
> py27cython installdeps: nose==1.3.7, grpcio-tools==1.3.5, cython==0.25.2
> docs installdeps: nose==1.3.7, grpcio-tools==1.3.5, Sphinx==1.5.5, 
> sphinx_rtd_theme==0.2.4
> lint installdeps: nose==1.3.7, pycodestyle==2.3.1, pylint==1.7.1
> py27 installdeps: nose==1.3.7, grpcio-tools==1.3.5
> lint inst: 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/target/.tox/dist/apache-beam-2.3.0.dev.zip
> py27 inst: 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/target/.tox/dist/apache-beam-2.3.0.dev.zip
> py27cython inst: 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/target/.tox/dist/apache-beam-2.3.0.dev.zip
> py27 runtests: PYTHONHASHSEED='2225684666'
> py27 runtests: commands[0] | python --version
> py27 runtests: commands[1] | - find apache_beam -type f -name *.pyc -delete
> py27 runtests: commands[2] | pip install -e .[test]
> lint runtests: PYTHONHASHSEED='2225684666'
> lint runtests: commands[0] | time pip install -e .[test]
> docs inst: 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/target/.tox/dist/apache-beam-2.3.0.dev.zip
> py27 runtests: commands[3] | python 
> apache_beam/examples/complete/autocomplete_test.py
> lint runtests: commands[1] | time 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/run_pylint.sh
> py27 runtests: commands[4] | python setup.py test
> docs runtests: PYTHONHASHSEED='2225684666'
> docs runtests: commands[0] | time pip install -e .[test,gcp,docs]
> docs runtests: commands[1] | time 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/generate_pydoc.sh
> py27gcp create: 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/target/.tox/py27gcp
> py27gcp installdeps: nose==1.3.7
> py27cython runtests: PYTHONHASHSEED='2225684666'
> py27cython runtests: commands[0] | python --version
> py27cython runtests: commands[1] | - find apache_beam -type f -name *.pyc 
> -delete
> py27cython runtests: commands[2] | - find apache_beam -type f -name *.c 
> -delete
> py27cython runtests: commands[3] | - find apache_beam -type f -name *.so 
> -delete
> py27cython runtests: commands[4] | - find target/build -type f -name *.c 
> -delete
> py27cython runtests: commands[5] | - find target/build -type f -name *.so 
> -delete
> py27cython runtests: commands[6] | time pip install -e .[test]
> py27gcp inst: 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/target/.tox/dist/apache-beam-2.3.0.dev.zip
> py27gcp runtests: PYTHONHASHSEED='2225684666'
> py27gcp runtests: commands[0] | pip install -e .[test,gcp]
> py27gcp runtests: commands[1] | python --version
> py27gcp runtests: commands[2] | - find apache_beam -type f -name *.pyc -delete
> py27gcp runtests: commands[3] | python 
> apache_beam/examples/complete/autocomplete_test.py
> py27gcp runtests: commands[4] | python setup.py test
>

[jira] [Commented] (BEAM-3040) Python precommit timed out after 150 minutes

2017-10-12 Thread Kenneth Knowles (JIRA) 

[ 
https://issues.apache.org/jira/browse/BEAM-3040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16202434#comment-16202434
 ] 

Kenneth Knowles commented on BEAM-3040:
---

Can we just use their pip to get a virtualenv and put in a recent pip?

> Python precommit timed out after 150 minutes
> 
>
> Key: BEAM-3040
> URL: https://issues.apache.org/jira/browse/BEAM-3040
> Project: Beam
>  Issue Type: Bug
>  Components: sdk-py-core
>Reporter: Kenneth Knowles
>Assignee: Ahmet Altay
>
> https://builds.apache.org/job/beam_PreCommit_Python_MavenInstall/143/consoleFull
> Within about 10 minutes it reaches this point:
> {code}
> ...
> 2017-10-10T03:33:33.591 [INFO] --- findbugs-maven-plugin:3.0.4:check 
> (default) @ beam-sdks-python ---
> 2017-10-10T03:33:33.702 [INFO] 
> 2017-10-10T03:33:33.702 [INFO] --- exec-maven-plugin:1.5.0:exec 
> (setuptools-test) @ beam-sdks-python ---
> {code}
> and the final output is like this:
> {code}
> ...
> 2017-10-10T03:33:33.702 [INFO] --- exec-maven-plugin:1.5.0:exec 
> (setuptools-test) @ beam-sdks-python ---
> docs create: 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/target/.tox/docs
> GLOB sdist-make: 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/setup.py
> lint create: 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/target/.tox/lint
> py27 create: 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/target/.tox/py27
> py27cython create: 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/target/.tox/py27cython
> py27cython installdeps: nose==1.3.7, grpcio-tools==1.3.5, cython==0.25.2
> docs installdeps: nose==1.3.7, grpcio-tools==1.3.5, Sphinx==1.5.5, 
> sphinx_rtd_theme==0.2.4
> lint installdeps: nose==1.3.7, pycodestyle==2.3.1, pylint==1.7.1
> py27 installdeps: nose==1.3.7, grpcio-tools==1.3.5
> lint inst: 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/target/.tox/dist/apache-beam-2.3.0.dev.zip
> py27 inst: 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/target/.tox/dist/apache-beam-2.3.0.dev.zip
> py27cython inst: 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/target/.tox/dist/apache-beam-2.3.0.dev.zip
> py27 runtests: PYTHONHASHSEED='2225684666'
> py27 runtests: commands[0] | python --version
> py27 runtests: commands[1] | - find apache_beam -type f -name *.pyc -delete
> py27 runtests: commands[2] | pip install -e .[test]
> lint runtests: PYTHONHASHSEED='2225684666'
> lint runtests: commands[0] | time pip install -e .[test]
> docs inst: 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/target/.tox/dist/apache-beam-2.3.0.dev.zip
> py27 runtests: commands[3] | python 
> apache_beam/examples/complete/autocomplete_test.py
> lint runtests: commands[1] | time 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/run_pylint.sh
> py27 runtests: commands[4] | python setup.py test
> docs runtests: PYTHONHASHSEED='2225684666'
> docs runtests: commands[0] | time pip install -e .[test,gcp,docs]
> docs runtests: commands[1] | time 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/generate_pydoc.sh
> py27gcp create: 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/target/.tox/py27gcp
> py27gcp installdeps: nose==1.3.7
> py27cython runtests: PYTHONHASHSEED='2225684666'
> py27cython runtests: commands[0] | python --version
> py27cython runtests: commands[1] | - find apache_beam -type f -name *.pyc 
> -delete
> py27cython runtests: commands[2] | - find apache_beam -type f -name *.c 
> -delete
> py27cython runtests: commands[3] | - find apache_beam -type f -name *.so 
> -delete
> py27cython runtests: commands[4] | - find target/build -type f -name *.c 
> -delete
> py27cython runtests: commands[5] | - find target/build -type f -name *.so 
> -delete
> py27cython runtests: commands[6] | time pip install -e .[test]
> py27gcp inst: 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/target/.tox/dist/apache-beam-2.3.0.dev.zip
> py27gcp runtests: PYTHONHASHSEED='2225684666'
> py27gcp runtests: commands[0] | pip install -e .[test,gcp]
> py27gcp runtests: commands[1] | python --version
> py27gcp runtests: commands[2] | - find apache_beam -type f -name *.pyc -delete
> py27gcp runtests: commands[3] | python 
> apache_beam/examples/complete/autocomplete_test.py
> py27gcp runtests: commands[4] | python setup.py test
> py27cython runtests: commands[7] | python 
> apache_beam/examples/complete/autocomplete_test.py
>

[jira] [Commented] (BEAM-3040) Python precommit timed out after 150 minutes

2017-10-12 Thread Jason Kuster (JIRA) 

[ 
https://issues.apache.org/jira/browse/BEAM-3040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16202371#comment-16202371
 ] 

Jason Kuster commented on BEAM-3040:


When I was chatting with [~robertwb] he also mentioned that the version of pip 
on the executors was very old and may be causing build time problems. I've 
opened https://issues.apache.org/jira/browse/INFRA-15195 with Infra, but as can 
be seen it's over SLO, so they're probably swamped and may not get to it on a 
timeframe which is useful to us here.

> Python precommit timed out after 150 minutes
> 
>
> Key: BEAM-3040
> URL: https://issues.apache.org/jira/browse/BEAM-3040
> Project: Beam
>  Issue Type: Bug
>  Components: sdk-py-core
>Reporter: Kenneth Knowles
>Assignee: Ahmet Altay
>
> https://builds.apache.org/job/beam_PreCommit_Python_MavenInstall/143/consoleFull
> Within about 10 minutes it reaches this point:
> {code}
> ...
> 2017-10-10T03:33:33.591 [INFO] --- findbugs-maven-plugin:3.0.4:check 
> (default) @ beam-sdks-python ---
> 2017-10-10T03:33:33.702 [INFO] 
> 2017-10-10T03:33:33.702 [INFO] --- exec-maven-plugin:1.5.0:exec 
> (setuptools-test) @ beam-sdks-python ---
> {code}
> and the final output is like this:
> {code}
> ...
> 2017-10-10T03:33:33.702 [INFO] --- exec-maven-plugin:1.5.0:exec 
> (setuptools-test) @ beam-sdks-python ---
> docs create: 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/target/.tox/docs
> GLOB sdist-make: 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/setup.py
> lint create: 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/target/.tox/lint
> py27 create: 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/target/.tox/py27
> py27cython create: 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/target/.tox/py27cython
> py27cython installdeps: nose==1.3.7, grpcio-tools==1.3.5, cython==0.25.2
> docs installdeps: nose==1.3.7, grpcio-tools==1.3.5, Sphinx==1.5.5, 
> sphinx_rtd_theme==0.2.4
> lint installdeps: nose==1.3.7, pycodestyle==2.3.1, pylint==1.7.1
> py27 installdeps: nose==1.3.7, grpcio-tools==1.3.5
> lint inst: 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/target/.tox/dist/apache-beam-2.3.0.dev.zip
> py27 inst: 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/target/.tox/dist/apache-beam-2.3.0.dev.zip
> py27cython inst: 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/target/.tox/dist/apache-beam-2.3.0.dev.zip
> py27 runtests: PYTHONHASHSEED='2225684666'
> py27 runtests: commands[0] | python --version
> py27 runtests: commands[1] | - find apache_beam -type f -name *.pyc -delete
> py27 runtests: commands[2] | pip install -e .[test]
> lint runtests: PYTHONHASHSEED='2225684666'
> lint runtests: commands[0] | time pip install -e .[test]
> docs inst: 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/target/.tox/dist/apache-beam-2.3.0.dev.zip
> py27 runtests: commands[3] | python 
> apache_beam/examples/complete/autocomplete_test.py
> lint runtests: commands[1] | time 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/run_pylint.sh
> py27 runtests: commands[4] | python setup.py test
> docs runtests: PYTHONHASHSEED='2225684666'
> docs runtests: commands[0] | time pip install -e .[test,gcp,docs]
> docs runtests: commands[1] | time 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/generate_pydoc.sh
> py27gcp create: 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/target/.tox/py27gcp
> py27gcp installdeps: nose==1.3.7
> py27cython runtests: PYTHONHASHSEED='2225684666'
> py27cython runtests: commands[0] | python --version
> py27cython runtests: commands[1] | - find apache_beam -type f -name *.pyc 
> -delete
> py27cython runtests: commands[2] | - find apache_beam -type f -name *.c 
> -delete
> py27cython runtests: commands[3] | - find apache_beam -type f -name *.so 
> -delete
> py27cython runtests: commands[4] | - find target/build -type f -name *.c 
> -delete
> py27cython runtests: commands[5] | - find target/build -type f -name *.so 
> -delete
> py27cython runtests: commands[6] | time pip install -e .[test]
> py27gcp inst: 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/target/.tox/dist/apache-beam-2.3.0.dev.zip
> py27gcp runtests: PYTHONHASHSEED='2225684666'
> py27gcp runtests: commands[0] | pip install -e .[test,gcp]
> py27gcp runtests: commands[1] | python --version
> py27gcp runtests: commands[2] | - find apache_beam -type f

[jira] [Commented] (BEAM-3040) Python precommit timed out after 150 minutes

2017-10-12 Thread Ahmet Altay (JIRA) 

[ 
https://issues.apache.org/jira/browse/BEAM-3040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16202364#comment-16202364
 ] 

Ahmet Altay commented on BEAM-3040:
---

The last thing in the above test log is the following:

{code}
test_read 
(apache_beam.runners.portability.universal_local_runner_test.UniversalLocalRunnerTestWithSubprocesses)
 ... apache_beam/coders/typecoders.py:134: UserWarning: Using fallback coder 
for typehint: .
  warnings.warn('Using fallback coder for typehint: %r.' % typehint)
Exception in thread Thread-6 (most likely raised during interpreter shutdown):
Traceback (most recent call last):
  File "/usr/lib/python2.7/threading.py", line 810, in __bootstrap_inner
  File "/usr/lib/python2.7/threading.py", line 763, in run
  File 
"/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/target/.tox/py27cython/lib/python2.7/site-packages/grpc/_channel.py",
 line 838, in _poll_connectivity
  File "/usr/lib/python2.7/threading.py", line 167, in acquire
: 'NoneType' object is not callable
Build timed out (after 150 minutes). Marking the build as aborted.
Build was aborted
{code}

This might be indicative of test being stuck somewhere in that test. Similarly 
the timeouts from 202 and 203 also terminate with log statements from 
universal_local_runner_test

> Python precommit timed out after 150 minutes
> 
>
> Key: BEAM-3040
> URL: https://issues.apache.org/jira/browse/BEAM-3040
> Project: Beam
>  Issue Type: Bug
>  Components: sdk-py-core
>Reporter: Kenneth Knowles
>Assignee: Ahmet Altay
>
> https://builds.apache.org/job/beam_PreCommit_Python_MavenInstall/143/consoleFull
> Within about 10 minutes it reaches this point:
> {code}
> ...
> 2017-10-10T03:33:33.591 [INFO] --- findbugs-maven-plugin:3.0.4:check 
> (default) @ beam-sdks-python ---
> 2017-10-10T03:33:33.702 [INFO] 
> 2017-10-10T03:33:33.702 [INFO] --- exec-maven-plugin:1.5.0:exec 
> (setuptools-test) @ beam-sdks-python ---
> {code}
> and the final output is like this:
> {code}
> ...
> 2017-10-10T03:33:33.702 [INFO] --- exec-maven-plugin:1.5.0:exec 
> (setuptools-test) @ beam-sdks-python ---
> docs create: 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/target/.tox/docs
> GLOB sdist-make: 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/setup.py
> lint create: 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/target/.tox/lint
> py27 create: 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/target/.tox/py27
> py27cython create: 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/target/.tox/py27cython
> py27cython installdeps: nose==1.3.7, grpcio-tools==1.3.5, cython==0.25.2
> docs installdeps: nose==1.3.7, grpcio-tools==1.3.5, Sphinx==1.5.5, 
> sphinx_rtd_theme==0.2.4
> lint installdeps: nose==1.3.7, pycodestyle==2.3.1, pylint==1.7.1
> py27 installdeps: nose==1.3.7, grpcio-tools==1.3.5
> lint inst: 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/target/.tox/dist/apache-beam-2.3.0.dev.zip
> py27 inst: 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/target/.tox/dist/apache-beam-2.3.0.dev.zip
> py27cython inst: 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/target/.tox/dist/apache-beam-2.3.0.dev.zip
> py27 runtests: PYTHONHASHSEED='2225684666'
> py27 runtests: commands[0] | python --version
> py27 runtests: commands[1] | - find apache_beam -type f -name *.pyc -delete
> py27 runtests: commands[2] | pip install -e .[test]
> lint runtests: PYTHONHASHSEED='2225684666'
> lint runtests: commands[0] | time pip install -e .[test]
> docs inst: 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/target/.tox/dist/apache-beam-2.3.0.dev.zip
> py27 runtests: commands[3] | python 
> apache_beam/examples/complete/autocomplete_test.py
> lint runtests: commands[1] | time 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/run_pylint.sh
> py27 runtests: commands[4] | python setup.py test
> docs runtests: PYTHONHASHSEED='2225684666'
> docs runtests: commands[0] | time pip install -e .[test,gcp,docs]
> docs runtests: commands[1] | time 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/generate_pydoc.sh
> py27gcp create: 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/target/.tox/py27gcp
> py27gcp installdeps: nose==1.3.7
> py27cython runtests: PYTHONHASHSEED='2225684666'
> py27cython runtests: commands[0] | python --version
> py27cython runtests: commands[1] | - find apache_beam -type f

[jira] [Commented] (BEAM-3040) Python precommit timed out after 150 minutes

2017-10-12 Thread Ahmet Altay (JIRA) 

[ 
https://issues.apache.org/jira/browse/BEAM-3040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16202301#comment-16202301
 ] 

Ahmet Altay commented on BEAM-3040:
---

Tests are still timing out (e.g. 
https://builds.apache.org/job/beam_PreCommit_Python_MavenInstall/208/consoleFull).


> Python precommit timed out after 150 minutes
> 
>
> Key: BEAM-3040
> URL: https://issues.apache.org/jira/browse/BEAM-3040
> Project: Beam
>  Issue Type: Bug
>  Components: sdk-py-core
>Reporter: Kenneth Knowles
>Assignee: Ahmet Altay
>
> https://builds.apache.org/job/beam_PreCommit_Python_MavenInstall/143/consoleFull
> Within about 10 minutes it reaches this point:
> {code}
> ...
> 2017-10-10T03:33:33.591 [INFO] --- findbugs-maven-plugin:3.0.4:check 
> (default) @ beam-sdks-python ---
> 2017-10-10T03:33:33.702 [INFO] 
> 2017-10-10T03:33:33.702 [INFO] --- exec-maven-plugin:1.5.0:exec 
> (setuptools-test) @ beam-sdks-python ---
> {code}
> and the final output is like this:
> {code}
> ...
> 2017-10-10T03:33:33.702 [INFO] --- exec-maven-plugin:1.5.0:exec 
> (setuptools-test) @ beam-sdks-python ---
> docs create: 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/target/.tox/docs
> GLOB sdist-make: 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/setup.py
> lint create: 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/target/.tox/lint
> py27 create: 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/target/.tox/py27
> py27cython create: 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/target/.tox/py27cython
> py27cython installdeps: nose==1.3.7, grpcio-tools==1.3.5, cython==0.25.2
> docs installdeps: nose==1.3.7, grpcio-tools==1.3.5, Sphinx==1.5.5, 
> sphinx_rtd_theme==0.2.4
> lint installdeps: nose==1.3.7, pycodestyle==2.3.1, pylint==1.7.1
> py27 installdeps: nose==1.3.7, grpcio-tools==1.3.5
> lint inst: 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/target/.tox/dist/apache-beam-2.3.0.dev.zip
> py27 inst: 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/target/.tox/dist/apache-beam-2.3.0.dev.zip
> py27cython inst: 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/target/.tox/dist/apache-beam-2.3.0.dev.zip
> py27 runtests: PYTHONHASHSEED='2225684666'
> py27 runtests: commands[0] | python --version
> py27 runtests: commands[1] | - find apache_beam -type f -name *.pyc -delete
> py27 runtests: commands[2] | pip install -e .[test]
> lint runtests: PYTHONHASHSEED='2225684666'
> lint runtests: commands[0] | time pip install -e .[test]
> docs inst: 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/target/.tox/dist/apache-beam-2.3.0.dev.zip
> py27 runtests: commands[3] | python 
> apache_beam/examples/complete/autocomplete_test.py
> lint runtests: commands[1] | time 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/run_pylint.sh
> py27 runtests: commands[4] | python setup.py test
> docs runtests: PYTHONHASHSEED='2225684666'
> docs runtests: commands[0] | time pip install -e .[test,gcp,docs]
> docs runtests: commands[1] | time 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/generate_pydoc.sh
> py27gcp create: 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/target/.tox/py27gcp
> py27gcp installdeps: nose==1.3.7
> py27cython runtests: PYTHONHASHSEED='2225684666'
> py27cython runtests: commands[0] | python --version
> py27cython runtests: commands[1] | - find apache_beam -type f -name *.pyc 
> -delete
> py27cython runtests: commands[2] | - find apache_beam -type f -name *.c 
> -delete
> py27cython runtests: commands[3] | - find apache_beam -type f -name *.so 
> -delete
> py27cython runtests: commands[4] | - find target/build -type f -name *.c 
> -delete
> py27cython runtests: commands[5] | - find target/build -type f -name *.so 
> -delete
> py27cython runtests: commands[6] | time pip install -e .[test]
> py27gcp inst: 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/target/.tox/dist/apache-beam-2.3.0.dev.zip
> py27gcp runtests: PYTHONHASHSEED='2225684666'
> py27gcp runtests: commands[0] | pip install -e .[test,gcp]
> py27gcp runtests: commands[1] | python --version
> py27gcp runtests: commands[2] | - find apache_beam -type f -name *.pyc -delete
> py27gcp runtests: commands[3] | python 
> apache_beam/examples/complete/autocomplete_test.py
> py27gcp runtests: commands[4] | python setup.py test
> py27cython runtests: commands[7] | python 
>

[jira] [Commented] (BEAM-3040) Python precommit timed out after 150 minutes

2017-10-10 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/BEAM-3040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16199578#comment-16199578
 ] 

ASF GitHub Bot commented on BEAM-3040:
--

Github user asfgit closed the pull request at:

https://github.com/apache/beam/pull/3974


> Python precommit timed out after 150 minutes
> 
>
> Key: BEAM-3040
> URL: https://issues.apache.org/jira/browse/BEAM-3040
> Project: Beam
>  Issue Type: Bug
>  Components: sdk-py-core
>Reporter: Kenneth Knowles
>Assignee: Ahmet Altay
>
> https://builds.apache.org/job/beam_PreCommit_Python_MavenInstall/143/consoleFull
> Within about 10 minutes it reaches this point:
> {code}
> ...
> 2017-10-10T03:33:33.591 [INFO] --- findbugs-maven-plugin:3.0.4:check 
> (default) @ beam-sdks-python ---
> 2017-10-10T03:33:33.702 [INFO] 
> 2017-10-10T03:33:33.702 [INFO] --- exec-maven-plugin:1.5.0:exec 
> (setuptools-test) @ beam-sdks-python ---
> {code}
> and the final output is like this:
> {code}
> ...
> 2017-10-10T03:33:33.702 [INFO] --- exec-maven-plugin:1.5.0:exec 
> (setuptools-test) @ beam-sdks-python ---
> docs create: 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/target/.tox/docs
> GLOB sdist-make: 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/setup.py
> lint create: 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/target/.tox/lint
> py27 create: 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/target/.tox/py27
> py27cython create: 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/target/.tox/py27cython
> py27cython installdeps: nose==1.3.7, grpcio-tools==1.3.5, cython==0.25.2
> docs installdeps: nose==1.3.7, grpcio-tools==1.3.5, Sphinx==1.5.5, 
> sphinx_rtd_theme==0.2.4
> lint installdeps: nose==1.3.7, pycodestyle==2.3.1, pylint==1.7.1
> py27 installdeps: nose==1.3.7, grpcio-tools==1.3.5
> lint inst: 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/target/.tox/dist/apache-beam-2.3.0.dev.zip
> py27 inst: 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/target/.tox/dist/apache-beam-2.3.0.dev.zip
> py27cython inst: 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/target/.tox/dist/apache-beam-2.3.0.dev.zip
> py27 runtests: PYTHONHASHSEED='2225684666'
> py27 runtests: commands[0] | python --version
> py27 runtests: commands[1] | - find apache_beam -type f -name *.pyc -delete
> py27 runtests: commands[2] | pip install -e .[test]
> lint runtests: PYTHONHASHSEED='2225684666'
> lint runtests: commands[0] | time pip install -e .[test]
> docs inst: 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/target/.tox/dist/apache-beam-2.3.0.dev.zip
> py27 runtests: commands[3] | python 
> apache_beam/examples/complete/autocomplete_test.py
> lint runtests: commands[1] | time 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/run_pylint.sh
> py27 runtests: commands[4] | python setup.py test
> docs runtests: PYTHONHASHSEED='2225684666'
> docs runtests: commands[0] | time pip install -e .[test,gcp,docs]
> docs runtests: commands[1] | time 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/generate_pydoc.sh
> py27gcp create: 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/target/.tox/py27gcp
> py27gcp installdeps: nose==1.3.7
> py27cython runtests: PYTHONHASHSEED='2225684666'
> py27cython runtests: commands[0] | python --version
> py27cython runtests: commands[1] | - find apache_beam -type f -name *.pyc 
> -delete
> py27cython runtests: commands[2] | - find apache_beam -type f -name *.c 
> -delete
> py27cython runtests: commands[3] | - find apache_beam -type f -name *.so 
> -delete
> py27cython runtests: commands[4] | - find target/build -type f -name *.c 
> -delete
> py27cython runtests: commands[5] | - find target/build -type f -name *.so 
> -delete
> py27cython runtests: commands[6] | time pip install -e .[test]
> py27gcp inst: 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/target/.tox/dist/apache-beam-2.3.0.dev.zip
> py27gcp runtests: PYTHONHASHSEED='2225684666'
> py27gcp runtests: commands[0] | pip install -e .[test,gcp]
> py27gcp runtests: commands[1] | python --version
> py27gcp runtests: commands[2] | - find apache_beam -type f -name *.pyc -delete
> py27gcp runtests: commands[3] | python 
> apache_beam/examples/complete/autocomplete_test.py
> py27gcp runtests: commands[4] | python setup.py test
> py27cython runtests: commands[7] | python 
>

[jira] [Commented] (BEAM-3040) Python precommit timed out after 150 minutes

2017-10-10 Thread Robert Bradshaw (JIRA) 

[ 
https://issues.apache.org/jira/browse/BEAM-3040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16199470#comment-16199470
 ] 

Robert Bradshaw commented on BEAM-3040:
---

Yes, Jason and I were discussing this last week. +1 to reverting detox for the 
time being.

> Python precommit timed out after 150 minutes
> 
>
> Key: BEAM-3040
> URL: https://issues.apache.org/jira/browse/BEAM-3040
> Project: Beam
>  Issue Type: Bug
>  Components: sdk-py-core
>Reporter: Kenneth Knowles
>Assignee: Ahmet Altay
>
> https://builds.apache.org/job/beam_PreCommit_Python_MavenInstall/143/consoleFull
> Within about 10 minutes it reaches this point:
> {code}
> ...
> 2017-10-10T03:33:33.591 [INFO] --- findbugs-maven-plugin:3.0.4:check 
> (default) @ beam-sdks-python ---
> 2017-10-10T03:33:33.702 [INFO] 
> 2017-10-10T03:33:33.702 [INFO] --- exec-maven-plugin:1.5.0:exec 
> (setuptools-test) @ beam-sdks-python ---
> {code}
> and the final output is like this:
> {code}
> ...
> 2017-10-10T03:33:33.702 [INFO] --- exec-maven-plugin:1.5.0:exec 
> (setuptools-test) @ beam-sdks-python ---
> docs create: 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/target/.tox/docs
> GLOB sdist-make: 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/setup.py
> lint create: 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/target/.tox/lint
> py27 create: 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/target/.tox/py27
> py27cython create: 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/target/.tox/py27cython
> py27cython installdeps: nose==1.3.7, grpcio-tools==1.3.5, cython==0.25.2
> docs installdeps: nose==1.3.7, grpcio-tools==1.3.5, Sphinx==1.5.5, 
> sphinx_rtd_theme==0.2.4
> lint installdeps: nose==1.3.7, pycodestyle==2.3.1, pylint==1.7.1
> py27 installdeps: nose==1.3.7, grpcio-tools==1.3.5
> lint inst: 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/target/.tox/dist/apache-beam-2.3.0.dev.zip
> py27 inst: 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/target/.tox/dist/apache-beam-2.3.0.dev.zip
> py27cython inst: 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/target/.tox/dist/apache-beam-2.3.0.dev.zip
> py27 runtests: PYTHONHASHSEED='2225684666'
> py27 runtests: commands[0] | python --version
> py27 runtests: commands[1] | - find apache_beam -type f -name *.pyc -delete
> py27 runtests: commands[2] | pip install -e .[test]
> lint runtests: PYTHONHASHSEED='2225684666'
> lint runtests: commands[0] | time pip install -e .[test]
> docs inst: 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/target/.tox/dist/apache-beam-2.3.0.dev.zip
> py27 runtests: commands[3] | python 
> apache_beam/examples/complete/autocomplete_test.py
> lint runtests: commands[1] | time 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/run_pylint.sh
> py27 runtests: commands[4] | python setup.py test
> docs runtests: PYTHONHASHSEED='2225684666'
> docs runtests: commands[0] | time pip install -e .[test,gcp,docs]
> docs runtests: commands[1] | time 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/generate_pydoc.sh
> py27gcp create: 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/target/.tox/py27gcp
> py27gcp installdeps: nose==1.3.7
> py27cython runtests: PYTHONHASHSEED='2225684666'
> py27cython runtests: commands[0] | python --version
> py27cython runtests: commands[1] | - find apache_beam -type f -name *.pyc 
> -delete
> py27cython runtests: commands[2] | - find apache_beam -type f -name *.c 
> -delete
> py27cython runtests: commands[3] | - find apache_beam -type f -name *.so 
> -delete
> py27cython runtests: commands[4] | - find target/build -type f -name *.c 
> -delete
> py27cython runtests: commands[5] | - find target/build -type f -name *.so 
> -delete
> py27cython runtests: commands[6] | time pip install -e .[test]
> py27gcp inst: 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/target/.tox/dist/apache-beam-2.3.0.dev.zip
> py27gcp runtests: PYTHONHASHSEED='2225684666'
> py27gcp runtests: commands[0] | pip install -e .[test,gcp]
> py27gcp runtests: commands[1] | python --version
> py27gcp runtests: commands[2] | - find apache_beam -type f -name *.pyc -delete
> py27gcp runtests: commands[3] | python 
> apache_beam/examples/complete/autocomplete_test.py
> py27gcp runtests: commands[4] | python setup.py test
> py27cython runtests: commands[7] | python 
>

[jira] [Commented] (BEAM-3040) Python precommit timed out after 150 minutes

2017-10-10 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/BEAM-3040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16199468#comment-16199468
 ] 

ASF GitHub Bot commented on BEAM-3040:
--

GitHub user aaltay opened a pull request:

https://github.com/apache/beam/pull/3974

[BEAM-3040] Use tox instead of detox



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/aaltay/beam testt

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/beam/pull/3974.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3974


commit 7532035a8f100a83c94ac1bd33d718a0c8e4c877
Author: Ahmet Altay 
Date:   2017-10-10T22:20:22Z

Use tox instead of detox




> Python precommit timed out after 150 minutes
> 
>
> Key: BEAM-3040
> URL: https://issues.apache.org/jira/browse/BEAM-3040
> Project: Beam
>  Issue Type: Bug
>  Components: sdk-py-core
>Reporter: Kenneth Knowles
>Assignee: Ahmet Altay
>
> https://builds.apache.org/job/beam_PreCommit_Python_MavenInstall/143/consoleFull
> Within about 10 minutes it reaches this point:
> {code}
> ...
> 2017-10-10T03:33:33.591 [INFO] --- findbugs-maven-plugin:3.0.4:check 
> (default) @ beam-sdks-python ---
> 2017-10-10T03:33:33.702 [INFO] 
> 2017-10-10T03:33:33.702 [INFO] --- exec-maven-plugin:1.5.0:exec 
> (setuptools-test) @ beam-sdks-python ---
> {code}
> and the final output is like this:
> {code}
> ...
> 2017-10-10T03:33:33.702 [INFO] --- exec-maven-plugin:1.5.0:exec 
> (setuptools-test) @ beam-sdks-python ---
> docs create: 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/target/.tox/docs
> GLOB sdist-make: 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/setup.py
> lint create: 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/target/.tox/lint
> py27 create: 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/target/.tox/py27
> py27cython create: 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/target/.tox/py27cython
> py27cython installdeps: nose==1.3.7, grpcio-tools==1.3.5, cython==0.25.2
> docs installdeps: nose==1.3.7, grpcio-tools==1.3.5, Sphinx==1.5.5, 
> sphinx_rtd_theme==0.2.4
> lint installdeps: nose==1.3.7, pycodestyle==2.3.1, pylint==1.7.1
> py27 installdeps: nose==1.3.7, grpcio-tools==1.3.5
> lint inst: 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/target/.tox/dist/apache-beam-2.3.0.dev.zip
> py27 inst: 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/target/.tox/dist/apache-beam-2.3.0.dev.zip
> py27cython inst: 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/target/.tox/dist/apache-beam-2.3.0.dev.zip
> py27 runtests: PYTHONHASHSEED='2225684666'
> py27 runtests: commands[0] | python --version
> py27 runtests: commands[1] | - find apache_beam -type f -name *.pyc -delete
> py27 runtests: commands[2] | pip install -e .[test]
> lint runtests: PYTHONHASHSEED='2225684666'
> lint runtests: commands[0] | time pip install -e .[test]
> docs inst: 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/target/.tox/dist/apache-beam-2.3.0.dev.zip
> py27 runtests: commands[3] | python 
> apache_beam/examples/complete/autocomplete_test.py
> lint runtests: commands[1] | time 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/run_pylint.sh
> py27 runtests: commands[4] | python setup.py test
> docs runtests: PYTHONHASHSEED='2225684666'
> docs runtests: commands[0] | time pip install -e .[test,gcp,docs]
> docs runtests: commands[1] | time 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/generate_pydoc.sh
> py27gcp create: 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/target/.tox/py27gcp
> py27gcp installdeps: nose==1.3.7
> py27cython runtests: PYTHONHASHSEED='2225684666'
> py27cython runtests: commands[0] | python --version
> py27cython runtests: commands[1] | - find apache_beam -type f -name *.pyc 
> -delete
> py27cython runtests: commands[2] | - find apache_beam -type f -name *.c 
> -delete
> py27cython runtests: commands[3] | - find apache_beam -type f -name *.so 
> -delete
> py27cython runtests: commands[4] | - find target/build -type f -name *.c 
> -delete
> py27cython runtests: commands[5] | - find target/build -type f -name *.so 
> -delete
> py27cython runtests: commands[6] | time pip install -e .[test]
> py27gcp inst: 
>

[jira] [Commented] (BEAM-3040) Python precommit timed out after 150 minutes

2017-10-10 Thread Ahmet Altay (JIRA) 

[ 
https://issues.apache.org/jira/browse/BEAM-3040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16199463#comment-16199463
 ] 

Ahmet Altay commented on BEAM-3040:
---

cc: [~robertwb]

I am suspecting that use of {{detox}} to parallelize running multiple {{tox}} 
tests is the root cause of this. This change was made in 
https://github.com/apache/beam/pull/3829

Another issue is, cython compiled files are not properly isolated when multiple 
test targets run in parallel. This can lead to .so file being deleted 
prematurely. (Earlier discussion: 
https://lists.apache.org/thread.html/cf49c402c1537902aae5ddae7581ca17299c4345c35626199c2c99f7@%3Cdev.beam.apache.org%3E)

I will suggest switching back to {{tox}} to see if it helps with the random 
timeouts.

> Python precommit timed out after 150 minutes
> 
>
> Key: BEAM-3040
> URL: https://issues.apache.org/jira/browse/BEAM-3040
> Project: Beam
>  Issue Type: Bug
>  Components: sdk-py-core
>Reporter: Kenneth Knowles
>Assignee: Ahmet Altay
>
> https://builds.apache.org/job/beam_PreCommit_Python_MavenInstall/143/consoleFull
> Within about 10 minutes it reaches this point:
> {code}
> ...
> 2017-10-10T03:33:33.591 [INFO] --- findbugs-maven-plugin:3.0.4:check 
> (default) @ beam-sdks-python ---
> 2017-10-10T03:33:33.702 [INFO] 
> 2017-10-10T03:33:33.702 [INFO] --- exec-maven-plugin:1.5.0:exec 
> (setuptools-test) @ beam-sdks-python ---
> {code}
> and the final output is like this:
> {code}
> ...
> 2017-10-10T03:33:33.702 [INFO] --- exec-maven-plugin:1.5.0:exec 
> (setuptools-test) @ beam-sdks-python ---
> docs create: 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/target/.tox/docs
> GLOB sdist-make: 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/setup.py
> lint create: 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/target/.tox/lint
> py27 create: 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/target/.tox/py27
> py27cython create: 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/target/.tox/py27cython
> py27cython installdeps: nose==1.3.7, grpcio-tools==1.3.5, cython==0.25.2
> docs installdeps: nose==1.3.7, grpcio-tools==1.3.5, Sphinx==1.5.5, 
> sphinx_rtd_theme==0.2.4
> lint installdeps: nose==1.3.7, pycodestyle==2.3.1, pylint==1.7.1
> py27 installdeps: nose==1.3.7, grpcio-tools==1.3.5
> lint inst: 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/target/.tox/dist/apache-beam-2.3.0.dev.zip
> py27 inst: 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/target/.tox/dist/apache-beam-2.3.0.dev.zip
> py27cython inst: 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/target/.tox/dist/apache-beam-2.3.0.dev.zip
> py27 runtests: PYTHONHASHSEED='2225684666'
> py27 runtests: commands[0] | python --version
> py27 runtests: commands[1] | - find apache_beam -type f -name *.pyc -delete
> py27 runtests: commands[2] | pip install -e .[test]
> lint runtests: PYTHONHASHSEED='2225684666'
> lint runtests: commands[0] | time pip install -e .[test]
> docs inst: 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/target/.tox/dist/apache-beam-2.3.0.dev.zip
> py27 runtests: commands[3] | python 
> apache_beam/examples/complete/autocomplete_test.py
> lint runtests: commands[1] | time 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/run_pylint.sh
> py27 runtests: commands[4] | python setup.py test
> docs runtests: PYTHONHASHSEED='2225684666'
> docs runtests: commands[0] | time pip install -e .[test,gcp,docs]
> docs runtests: commands[1] | time 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/generate_pydoc.sh
> py27gcp create: 
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_MavenInstall/sdks/python/target/.tox/py27gcp
> py27gcp installdeps: nose==1.3.7
> py27cython runtests: PYTHONHASHSEED='2225684666'
> py27cython runtests: commands[0] | python --version
> py27cython runtests: commands[1] | - find apache_beam -type f -name *.pyc 
> -delete
> py27cython runtests: commands[2] | - find apache_beam -type f -name *.c 
> -delete
> py27cython runtests: commands[3] | - find apache_beam -type f -name *.so 
> -delete
> py27cython runtests: commands[4] | - find target/build -type f -name *.c 
> -delete
> py27cython runtests: commands[5] | - find target/build -type f -name *.so 
> -delete
> py27cython runtests: commands[6] | time pip install -e .[test]
> py27gcp inst: 
>