Thanks Michael, your assessment was correct. I needed python3.5 on my $PATH.
For completeness, I needed this <https://github.com/pyenv/pyenv/wiki/Common-build-problems#build-failed-error-the-python-zlib-extension-was-not-compiled-missing-the-zlib> to get pyenv to install python 3.5.6 on macOS: ``` CPPFLAGS="-I$(brew --prefix zlib)/include" pyenv install -v 3.5.6 ``` `./gradlew clean` worked after that. On Sat, Feb 16, 2019 at 7:28 PM Michael Luckey <[email protected]> wrote: > As far as I understand, the build got bound to 3.5 [1]. > > Could it be that you do not have python3.5 on your path? e.g. try python3.5 > --version > > If that is missing, you will not be able to run any py3 task, I guess... > > So, to get out of this state, you have to get python3.5 command working. > > At least on my machine, './gradlew clean' is working iff python3.5 is on > my path. > > michel > > [1] > https://github.com/apache/beam/blob/master/buildSrc/src/main/groovy/org/apache/beam/gradle/BeamModulePlugin.groovy#L1591 > > > On Sat, Feb 16, 2019 at 9:54 PM Ryan Williams <[email protected]> > wrote: > >> I'm seeing the same thing as Thomas above: `./gradlew clean` fails on two >> py3 setupVirtualEnv tasks, >> :beam-sdks-python-test-suites-direct-py3:setupVirtualenv and >> :beam-sdks-python-test-suites-dataflow-py3:setupVirtualenv. Here's full >> output >> <https://gist.github.com/ryan-williams/402e20131d23905163de3a4e2b178f39>. >> >> Any tips how to get out of this state, or what is causing it? >> >> On Fri, Feb 8, 2019 at 7:17 PM Kenneth Knowles <[email protected]> wrote: >> >>> Maybe add that case to https://issues.apache.org/jira/browse/BEAM-6459. >>> >>> Kenn >>> >>> On Fri, Feb 8, 2019 at 9:09 AM Thomas Weise <[email protected]> wrote: >>> >>>> Probably related, a top level ./gradlew clean fails with the following: >>>> >>>> > Task :beam-sdks-python-precommit-direct-py3:setupVirtualenv FAILED >>>> The path python3.5 (from --python=python3.5) does not exist >>>> >>>> Can we just limit clean to do cleanup?! >>>> >>>> >>>> On Mon, Jan 21, 2019 at 7:58 AM Robert Bradshaw <[email protected]> >>>> wrote: >>>> >>>>> Just some background, grpcio-tools is what's used to do the proto >>>>> generation. Unfortunately it's expensive to compile and doesn't >>>>> provide very many wheels, so we want to install it once, not every >>>>> time. (It's also used in more than just tests; one needs it every time >>>>> the .proto files change.) >>>>> >>>>> That being said, we could probably do a much cheaper clean. >>>>> >>>>> On Fri, Jan 18, 2019 at 8:56 PM Udi Meiri <[email protected]> wrote: >>>>> > >>>>> > grpcio-tools could probably be moved under the "test" tag in >>>>> setup.py. Not sure why it has to be specified in gradle configs. >>>>> > >>>>> > On Fri, Jan 18, 2019 at 11:43 AM Kenneth Knowles <[email protected]> >>>>> wrote: >>>>> >> >>>>> >> Can you `setupVirtualEnv` just enough to run `setup.py clean` >>>>> without installing gcpio-tools, etc? >>>>> >> >>>>> >> Kenn >>>>> >> >>>>> >> On Fri, Jan 18, 2019 at 11:20 AM Udi Meiri <[email protected]> >>>>> wrote: >>>>> >>> >>>>> >>> setup.py has requirements like setuptools, which are installed in >>>>> the virtual environment. >>>>> >>> So even running the clean command requires the virtualenv to be >>>>> set up. >>>>> >>> >>>>> >>> A possible fix could be to skip :beam-sdks-python:cleanPython if >>>>> setupVirtualenv has not been run. (perhaps by checking for the existence >>>>> of >>>>> its output directory) >>>>> >>> >>>>> >>> On Wed, Jan 16, 2019 at 7:03 PM Kenneth Knowles <[email protected]> >>>>> wrote: >>>>> >>>> >>>>> >>>> Filed https://issues.apache.org/jira/browse/BEAM-6459 to record >>>>> the conclusion. Doesn't require Beam knowledge so I labeled "starter". >>>>> >>>> >>>>> >>>> Kenn >>>>> >>>> >>>>> >>>> On Wed, Jan 16, 2019 at 12:14 AM Michael Luckey < >>>>> [email protected]> wrote: >>>>> >>>>> >>>>> >>>>> This seems to be on purpose [1] >>>>> >>>>> >>>>> >>>>> AFAIU setup is done to be able to call into setup.py clean. We >>>>> probably should work around that. >>>>> >>>>> >>>>> >>>>> [1] >>>>> https://github.com/apache/beam/blob/master/buildSrc/src/main/groovy/org/apache/beam/gradle/BeamModulePlugin.groovy#L1600-L1610 >>>>> >>>>> >>>>> >>>>> On Wed, Jan 16, 2019 at 7:01 AM Manu Zhang < >>>>> [email protected]> wrote: >>>>> >>>>>> >>>>> >>>>>> I have the same question. Sometimes even `./gradlew clean` >>>>> fails due to failure of `setupVirtualEnv` tasks. >>>>> >>>>>> >>>>> >>>>>> Manu Zhang >>>>> >>>>>> On Jan 16, 2019, 12:22 PM +0800, Kenneth Knowles < >>>>> [email protected]>, wrote: >>>>> >>>>>> >>>>> >>>>>> A global `./gradlew clean` runs various `setupVirtualEnv` tasks >>>>> that invoke things such as `setup.py bdist_wheel for grpcio-tools`. >>>>> Overall >>>>> it took 4 minutes. Is this intended? >>>>> >>>>>> >>>>> >>>>>> Kenn >>>>> >>>>
