Just some background, grpcio-tools is what's used to do the proto generation. Unfortunately it's expensive to compile and doesn't provide very many wheels, so we want to install it once, not every time. (It's also used in more than just tests; one needs it every time the .proto files change.)
That being said, we could probably do a much cheaper clean. On Fri, Jan 18, 2019 at 8:56 PM Udi Meiri <[email protected]> wrote: > > grpcio-tools could probably be moved under the "test" tag in setup.py. Not > sure why it has to be specified in gradle configs. > > On Fri, Jan 18, 2019 at 11:43 AM Kenneth Knowles <[email protected]> wrote: >> >> Can you `setupVirtualEnv` just enough to run `setup.py clean` without >> installing gcpio-tools, etc? >> >> Kenn >> >> On Fri, Jan 18, 2019 at 11:20 AM Udi Meiri <[email protected]> wrote: >>> >>> setup.py has requirements like setuptools, which are installed in the >>> virtual environment. >>> So even running the clean command requires the virtualenv to be set up. >>> >>> A possible fix could be to skip :beam-sdks-python:cleanPython if >>> setupVirtualenv has not been run. (perhaps by checking for the existence of >>> its output directory) >>> >>> On Wed, Jan 16, 2019 at 7:03 PM Kenneth Knowles <[email protected]> wrote: >>>> >>>> Filed https://issues.apache.org/jira/browse/BEAM-6459 to record the >>>> conclusion. Doesn't require Beam knowledge so I labeled "starter". >>>> >>>> Kenn >>>> >>>> On Wed, Jan 16, 2019 at 12:14 AM Michael Luckey <[email protected]> >>>> wrote: >>>>> >>>>> This seems to be on purpose [1] >>>>> >>>>> AFAIU setup is done to be able to call into setup.py clean. We probably >>>>> should work around that. >>>>> >>>>> [1] >>>>> https://github.com/apache/beam/blob/master/buildSrc/src/main/groovy/org/apache/beam/gradle/BeamModulePlugin.groovy#L1600-L1610 >>>>> >>>>> On Wed, Jan 16, 2019 at 7:01 AM Manu Zhang <[email protected]> >>>>> wrote: >>>>>> >>>>>> I have the same question. Sometimes even `./gradlew clean` fails due to >>>>>> failure of `setupVirtualEnv` tasks. >>>>>> >>>>>> Manu Zhang >>>>>> On Jan 16, 2019, 12:22 PM +0800, Kenneth Knowles <[email protected]>, >>>>>> wrote: >>>>>> >>>>>> A global `./gradlew clean` runs various `setupVirtualEnv` tasks that >>>>>> invoke things such as `setup.py bdist_wheel for grpcio-tools`. Overall >>>>>> it took 4 minutes. Is this intended? >>>>>> >>>>>> Kenn
