[jira] [Comment Edited] (BEAM-2587) Build fails due to python sdk

2017-07-31 Thread Ahmet Altay (JIRA)

[ 
https://issues.apache.org/jira/browse/BEAM-2587?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16107735#comment-16107735
 ] 

Ahmet Altay edited comment on BEAM-2587 at 7/31/17 7:50 PM:


[~jbonofre] I will suggest moving this out of the 2.1.0 blocker list. The 
reason is, it does not affect end users (i.e. published artifacts will already 
have the correct generated files), it does not affect daily development (i.e. 
works within a git tree), and even out of that conditions it fails based on 
pip/os versions that is not clearly defined yet. It will take some more time to 
reproduce and fix it and does not need to block the release.

However, this is a regression in the sense that at the time of 2.0.0 there was 
no code path for dynamically generating these files.


was (Author: altay):
[~jbonofre] I will suggest moving this out of the 2.1.0 blocker list. The 
reason is, it does not affect end users (i.e. published artifacts will already 
have the correct generated files), it does not affect daily development (i.e. 
works within a git tree), and event out of that conditions it fails based on 
pip/os versions that is not clearly defined yet. It will take some more time to 
reproduce and fix it and does not need to block the release.

However, this is a regression in the sense that at the time of 2.0.0 there was 
code path for dynamically generating these files.

> Build fails due to python sdk
> -
>
> Key: BEAM-2587
> URL: https://issues.apache.org/jira/browse/BEAM-2587
> Project: Beam
>  Issue Type: Bug
>  Components: sdk-py
>Affects Versions: 2.1.0
>Reporter: Ahmet Altay
>
> Build fails with the following errors when {{mvn clean package}} is used on a 
> clean Ubuntu 16.04 LTS machine with pip 8.x. The issue is resolved when pip 
> is upgraded to pip 9.x
> "RuntimeError: Not in apache git tree; unable to find proto definitions."
> "DistutilsOptionError: can't combine user with prefix, exec_prefix/home, or 
> install_(plat)base​"
> We need to understand the issue and maybe add a note about requiring pip 9.x 
> for development. Note that this does not affect end users using prepackaged 
> artifacts from central repositories.
> cc: [~robertwb]
> Script for reproduction:
> {code}
> #!/bin/bash
> set -e
> readonly MACHINE_ID=$(hexdump -n 1 -e '"%x"' /dev/random)
> readonly MACHINE="${USER}-beam-build-${MACHINE_ID}"
> readonly ZONE="us-central1-c"
> # provision building machine
> echo "Provisioning Build Machine (Ubuntu 16.04 LTS)"
> gcloud compute instances create "$MACHINE" \
>   --zone="$ZONE" \
>   --image-project="ubuntu-os-cloud" \
>   --image-family="ubuntu-1604-lts"
> # wait for ssh to be ready
> echo "Waiting for machine to finish booting"
> sleep 30
> # ssh into the machine
> # 1. install dependencies as specified by beam readme
> # 2. download beam source from github
> # 3. build with maven
> echo "Downloading and building Apache Beam (release-2.1.0)"
> gcloud compute ssh "$MACHINE" --zone="$ZONE" << EOF
> sudo apt-get --assume-yes update
> sudo apt-get --assume-yes install \
> openjdk-8-jdk \
> maven \
> python-setuptools \
> python-pip
> wget https://github.com/apache/beam/archive/release-2.1.0.tar.gz
> tar -xzf release-2.1.0.tar.gz
> cd beam-release-2.1.0
> mvn clean package
> EOF
> {code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Comment Edited] (BEAM-2587) Build fails due to python sdk

2017-08-04 Thread Ahmet Altay (JIRA)

[ 
https://issues.apache.org/jira/browse/BEAM-2587?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16114992#comment-16114992
 ] 

Ahmet Altay edited comment on BEAM-2587 at 8/4/17 9:30 PM:
---

https://builds.apache.org/view/A-D/view/Beam/job/beam_PostCommit_Python_Verify/2866/
 - failed with the same error


gen_protos.py:46: UserWarning: Installing grpcio-tools is recommended for 
development.
  warnings.warn('Installing grpcio-tools is recommended for development.')
Traceback (most recent call last):
  File "", line 1, in 
  File "/tmp/pip-wCvObr-build/setup.py", line 197, in 
'test': generate_protos_first(test),
  File "/usr/lib/python2.7/distutils/core.py", line 151, in setup
dist.run_commands()
  File "/usr/lib/python2.7/distutils/dist.py", line 953, in run_commands
self.run_command(cmd)
  File "/usr/lib/python2.7/distutils/dist.py", line 972, in run_command
cmd_obj.run()
  File 
"/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python_Verify/sdks/python/target/.tox/py27gcp/local/lib/python2.7/site-packages/setuptools/command/install.py",
 line 61, in run
return orig.install.run(self)
  File "/usr/lib/python2.7/distutils/command/install.py", line 601, in run
self.run_command('build')
  File "/usr/lib/python2.7/distutils/cmd.py", line 326, in run_command
self.distribution.run_command(command)
  File "/usr/lib/python2.7/distutils/dist.py", line 972, in run_command
cmd_obj.run()
  File "/usr/lib/python2.7/distutils/command/build.py", line 128, in run
self.run_command(cmd_name)
  File "/usr/lib/python2.7/distutils/cmd.py", line 326, in run_command
self.distribution.run_command(command)
  File "/usr/lib/python2.7/distutils/dist.py", line 972, in run_command
cmd_obj.run()
  File "/tmp/pip-wCvObr-build/setup.py", line 137, in run
gen_protos.generate_proto_files()
  File "gen_protos.py", line 65, in generate_proto_files
'Not in apache git tree; unable to find proto definitions.')
RuntimeError: Not in apache git tree; unable to find proto definitions



was (Author: altay):
https://builds.apache.org/view/A-D/view/Beam/job/beam_PostCommit_Python_Verify/2866/
 - failed with the same error

> Build fails due to python sdk
> -
>
> Key: BEAM-2587
> URL: https://issues.apache.org/jira/browse/BEAM-2587
> Project: Beam
>  Issue Type: Bug
>  Components: sdk-py
>Affects Versions: 2.1.0
>Reporter: Ahmet Altay
>
> Build fails with the following errors when {{mvn clean package}} is used on a 
> clean Ubuntu 16.04 LTS machine with pip 8.x. The issue is resolved when pip 
> is upgraded to pip 9.x
> "RuntimeError: Not in apache git tree; unable to find proto definitions."
> "DistutilsOptionError: can't combine user with prefix, exec_prefix/home, or 
> install_(plat)base​"
> We need to understand the issue and maybe add a note about requiring pip 9.x 
> for development. Note that this does not affect end users using prepackaged 
> artifacts from central repositories.
> cc: [~robertwb]
> Script for reproduction:
> {code}
> #!/bin/bash
> set -e
> readonly MACHINE_ID=$(hexdump -n 1 -e '"%x"' /dev/random)
> readonly MACHINE="${USER}-beam-build-${MACHINE_ID}"
> readonly ZONE="us-central1-c"
> # provision building machine
> echo "Provisioning Build Machine (Ubuntu 16.04 LTS)"
> gcloud compute instances create "$MACHINE" \
>   --zone="$ZONE" \
>   --image-project="ubuntu-os-cloud" \
>   --image-family="ubuntu-1604-lts"
> # wait for ssh to be ready
> echo "Waiting for machine to finish booting"
> sleep 30
> # ssh into the machine
> # 1. install dependencies as specified by beam readme
> # 2. download beam source from github
> # 3. build with maven
> echo "Downloading and building Apache Beam (release-2.1.0)"
> gcloud compute ssh "$MACHINE" --zone="$ZONE" << EOF
> sudo apt-get --assume-yes update
> sudo apt-get --assume-yes install \
> openjdk-8-jdk \
> maven \
> python-setuptools \
> python-pip
> wget https://github.com/apache/beam/archive/release-2.1.0.tar.gz
> tar -xzf release-2.1.0.tar.gz
> cd beam-release-2.1.0
> mvn clean package
> EOF
> {code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)