Probably related, a top level ./gradlew clean fails with the following: > Task :beam-sdks-python-precommit-direct-py3:setupVirtualenv FAILED The path python3.5 (from --python=python3.5) does not exist
Can we just limit clean to do cleanup?! On Mon, Jan 21, 2019 at 7:58 AM Robert Bradshaw <rober...@google.com> wrote: > Just some background, grpcio-tools is what's used to do the proto > generation. Unfortunately it's expensive to compile and doesn't > provide very many wheels, so we want to install it once, not every > time. (It's also used in more than just tests; one needs it every time > the .proto files change.) > > That being said, we could probably do a much cheaper clean. > > On Fri, Jan 18, 2019 at 8:56 PM Udi Meiri <eh...@google.com> wrote: > > > > grpcio-tools could probably be moved under the "test" tag in setup.py. > Not sure why it has to be specified in gradle configs. > > > > On Fri, Jan 18, 2019 at 11:43 AM Kenneth Knowles <k...@google.com> wrote: > >> > >> Can you `setupVirtualEnv` just enough to run `setup.py clean` without > installing gcpio-tools, etc? > >> > >> Kenn > >> > >> On Fri, Jan 18, 2019 at 11:20 AM Udi Meiri <eh...@google.com> wrote: > >>> > >>> setup.py has requirements like setuptools, which are installed in the > virtual environment. > >>> So even running the clean command requires the virtualenv to be set up. > >>> > >>> A possible fix could be to skip :beam-sdks-python:cleanPython if > setupVirtualenv has not been run. (perhaps by checking for the existence of > its output directory) > >>> > >>> On Wed, Jan 16, 2019 at 7:03 PM Kenneth Knowles <k...@google.com> > wrote: > >>>> > >>>> Filed https://issues.apache.org/jira/browse/BEAM-6459 to record the > conclusion. Doesn't require Beam knowledge so I labeled "starter". > >>>> > >>>> Kenn > >>>> > >>>> On Wed, Jan 16, 2019 at 12:14 AM Michael Luckey <adude3...@gmail.com> > wrote: > >>>>> > >>>>> This seems to be on purpose [1] > >>>>> > >>>>> AFAIU setup is done to be able to call into setup.py clean. We > probably should work around that. > >>>>> > >>>>> [1] > https://github.com/apache/beam/blob/master/buildSrc/src/main/groovy/org/apache/beam/gradle/BeamModulePlugin.groovy#L1600-L1610 > >>>>> > >>>>> On Wed, Jan 16, 2019 at 7:01 AM Manu Zhang <owenzhang1...@gmail.com> > wrote: > >>>>>> > >>>>>> I have the same question. Sometimes even `./gradlew clean` fails > due to failure of `setupVirtualEnv` tasks. > >>>>>> > >>>>>> Manu Zhang > >>>>>> On Jan 16, 2019, 12:22 PM +0800, Kenneth Knowles <k...@apache.org>, > wrote: > >>>>>> > >>>>>> A global `./gradlew clean` runs various `setupVirtualEnv` tasks > that invoke things such as `setup.py bdist_wheel for grpcio-tools`. Overall > it took 4 minutes. Is this intended? > >>>>>> > >>>>>> Kenn >
