What quick command to catch common issues before pushing a python PR?
I made a thread about this a while back for java, but I don't think the same commands like sptoless work for python. auto fixing lint issues running and quick checks which would fail the PR (without running the whole precommit?) Something like findbugs to detect common issues (i.e. py3 compliance) FWIW, this is what I have been using for java. It will catch pretty much everything except presubmit test failures. ./gradlew spotlessApply && ./gradlew checkstyleMain && ./gradlew checkstyleTest && ./gradlew javadoc && ./gradlew findbugsMain && ./gradlew compileTestJava && ./gradlew compileJava
Re: What quick command to catch common issues before pushing a python PR?
Hi, just curious to know, why you prefer those && instead of a plain ./gradlew spotlessApply checkstyleMain checkstyleTest javadoc findbugsMain compileJava compileTestJava michel On Mon, Feb 25, 2019 at 7:13 PM Alex Amato wrote: > I made a thread about this a while back for java, but I don't think the > same commands like sptoless work for python. > > auto fixing lint issues > running and quick checks which would fail the PR (without running the > whole precommit?) > Something like findbugs to detect common issues (i.e. py3 compliance) > > FWIW, this is what I have been using for java. It will catch pretty much > everything except presubmit test failures. > > ./gradlew spotlessApply && ./gradlew checkstyleMain && ./gradlew > checkstyleTest && ./gradlew javadoc && ./gradlew findbugsMain && ./gradlew > compileTestJava && ./gradlew compileJava >
Re: What quick command to catch common issues before pushing a python PR?
FWIW gradle is a depgraph-based build system. You can gain a few seconds by putting all but spotlessApply in one command. ./gradlew spotlessApply && ./gradlew checkstyleMain checkstyleTest javadoc findbugsMain compileTestJava compileJava It might be clever to define a meta-task. Gradle "base plugin" has the notable check (build and run tests), assemble (make artifacts), and build (assemble + check, badly named!) I think something like "everything except running tests and building artifacts" might be helpful. Kenn On Mon, Feb 25, 2019 at 10:13 AM Alex Amato wrote: > I made a thread about this a while back for java, but I don't think the > same commands like sptoless work for python. > > auto fixing lint issues > running and quick checks which would fail the PR (without running the > whole precommit?) > Something like findbugs to detect common issues (i.e. py3 compliance) > > FWIW, this is what I have been using for java. It will catch pretty much > everything except presubmit test failures. > > ./gradlew spotlessApply && ./gradlew checkstyleMain && ./gradlew > checkstyleTest && ./gradlew javadoc && ./gradlew findbugsMain && ./gradlew > compileTestJava && ./gradlew compileJava >
Re: What quick command to catch common issues before pushing a python PR?
Talking about Python: I only know of "./gradlew lint", which include style and some py3 compliance checking. There is no auto-fix like spotlessApply AFAIK. As a side-note, I really dislike our python line continuation indent rule, since pycharm can't be configured to adhere to it and I find myself manually adjusting whitespace all the time. On Mon, Feb 25, 2019 at 10:22 AM Kenneth Knowles wrote: > FWIW gradle is a depgraph-based build system. You can gain a few seconds > by putting all but spotlessApply in one command. > > ./gradlew spotlessApply && ./gradlew checkstyleMain checkstyleTest javadoc > findbugsMain compileTestJava compileJava > > It might be clever to define a meta-task. Gradle "base plugin" has the > notable check (build and run tests), assemble (make artifacts), and build > (assemble + check, badly named!) > > I think something like "everything except running tests and building > artifacts" might be helpful. > > Kenn > > On Mon, Feb 25, 2019 at 10:13 AM Alex Amato wrote: > >> I made a thread about this a while back for java, but I don't think the >> same commands like sptoless work for python. >> >> auto fixing lint issues >> running and quick checks which would fail the PR (without running the >> whole precommit?) >> Something like findbugs to detect common issues (i.e. py3 compliance) >> >> FWIW, this is what I have been using for java. It will catch pretty much >> everything except presubmit test failures. >> >> ./gradlew spotlessApply && ./gradlew checkstyleMain && ./gradlew >> checkstyleTest && ./gradlew javadoc && ./gradlew findbugsMain && ./gradlew >> compileTestJava && ./gradlew compileJava >> > smime.p7s Description: S/MIME Cryptographic Signature
Re: What quick command to catch common issues before pushing a python PR?
@Michael, no particular reason. I think Ken's suggestion makes more sense. On Mon, Feb 25, 2019 at 10:36 AM Udi Meiri wrote: > Talking about Python: > I only know of "./gradlew lint", which include style and some py3 > compliance checking. > There is no auto-fix like spotlessApply AFAIK. > > As a side-note, I really dislike our python line continuation indent rule, > since pycharm can't be configured to adhere to it and I find myself > manually adjusting whitespace all the time. > > > On Mon, Feb 25, 2019 at 10:22 AM Kenneth Knowles wrote: > >> FWIW gradle is a depgraph-based build system. You can gain a few seconds >> by putting all but spotlessApply in one command. >> >> ./gradlew spotlessApply && ./gradlew checkstyleMain checkstyleTest >> javadoc findbugsMain compileTestJava compileJava >> >> It might be clever to define a meta-task. Gradle "base plugin" has the >> notable check (build and run tests), assemble (make artifacts), and build >> (assemble + check, badly named!) >> >> I think something like "everything except running tests and building >> artifacts" might be helpful. >> >> Kenn >> >> On Mon, Feb 25, 2019 at 10:13 AM Alex Amato wrote: >> >>> I made a thread about this a while back for java, but I don't think the >>> same commands like sptoless work for python. >>> >>> auto fixing lint issues >>> running and quick checks which would fail the PR (without running the >>> whole precommit?) >>> Something like findbugs to detect common issues (i.e. py3 compliance) >>> >>> FWIW, this is what I have been using for java. It will catch pretty much >>> everything except presubmit test failures. >>> >>> ./gradlew spotlessApply && ./gradlew checkstyleMain && ./gradlew >>> checkstyleTest && ./gradlew javadoc && ./gradlew findbugsMain && ./gradlew >>> compileTestJava && ./gradlew compileJava >>> >>
Re: What quick command to catch common issues before pushing a python PR?
Out of curiosity as a light gradle user, I did a side by side comparison, and the readings confirm what Ken and Michael suggests. In the same repository, do gradle clean then followed by either of the two commands. Measure their runtime respectively. The latter one takes *1/3* running time. time ./gradlew spotlessApply && ./gradlew checkstyleMain && ./gradlew checkstyleTest && ./gradlew javadoc && ./gradlew findbugsMain && ./gradlew compileTestJava && ./gradlew compileJava real 9m29.330s user 0m11.330s sys 0m1.239s time ./gradlew spotlessApply checkstyleMain checkstyleTest javadoc findbugsMain compileJava compileTestJava real3m35.573s user0m2.701s sys 0m0.327s On Mon, Feb 25, 2019 at 10:47 AM Alex Amato wrote: > @Michael, no particular reason. I think Ken's suggestion makes more sense. > > On Mon, Feb 25, 2019 at 10:36 AM Udi Meiri wrote: > >> Talking about Python: >> I only know of "./gradlew lint", which include style and some py3 >> compliance checking. >> There is no auto-fix like spotlessApply AFAIK. >> >> As a side-note, I really dislike our python line continuation indent >> rule, since pycharm can't be configured to adhere to it and I find myself >> manually adjusting whitespace all the time. >> >> >> On Mon, Feb 25, 2019 at 10:22 AM Kenneth Knowles wrote: >> >>> FWIW gradle is a depgraph-based build system. You can gain a few seconds >>> by putting all but spotlessApply in one command. >>> >>> ./gradlew spotlessApply && ./gradlew checkstyleMain checkstyleTest >>> javadoc findbugsMain compileTestJava compileJava >>> >>> It might be clever to define a meta-task. Gradle "base plugin" has the >>> notable check (build and run tests), assemble (make artifacts), and build >>> (assemble + check, badly named!) >>> >>> I think something like "everything except running tests and building >>> artifacts" might be helpful. >>> >>> Kenn >>> >>> On Mon, Feb 25, 2019 at 10:13 AM Alex Amato wrote: >>> I made a thread about this a while back for java, but I don't think the same commands like sptoless work for python. auto fixing lint issues running and quick checks which would fail the PR (without running the whole precommit?) Something like findbugs to detect common issues (i.e. py3 compliance) FWIW, this is what I have been using for java. It will catch pretty much everything except presubmit test failures. ./gradlew spotlessApply && ./gradlew checkstyleMain && ./gradlew checkstyleTest && ./gradlew javadoc && ./gradlew findbugsMain && ./gradlew compileTestJava && ./gradlew compileJava >>> -- Ruoyun Huang
Re: What quick command to catch common issues before pushing a python PR?
Ah, that is likely caused by us having ill-defined tasks that cannot be cached. Or is it that the configuration time is so significant? Kenn On Mon, Feb 25, 2019 at 11:05 AM Ruoyun Huang wrote: > Out of curiosity as a light gradle user, I did a side by side comparison, > and the readings confirm what Ken and Michael suggests. > > In the same repository, do gradle clean then followed by either of the two > commands. Measure their runtime respectively. The latter one takes *1/3* > running time. > > time ./gradlew spotlessApply && ./gradlew checkstyleMain && ./gradlew > checkstyleTest && ./gradlew javadoc && ./gradlew findbugsMain && ./gradlew > compileTestJava && ./gradlew compileJava > real 9m29.330s user 0m11.330s sys 0m1.239s > > time ./gradlew spotlessApply checkstyleMain checkstyleTest javadoc > findbugsMain compileJava compileTestJava > real3m35.573s > user0m2.701s > sys 0m0.327s > > > > > > > > On Mon, Feb 25, 2019 at 10:47 AM Alex Amato wrote: > >> @Michael, no particular reason. I think Ken's suggestion makes more sense. >> >> On Mon, Feb 25, 2019 at 10:36 AM Udi Meiri wrote: >> >>> Talking about Python: >>> I only know of "./gradlew lint", which include style and some py3 >>> compliance checking. >>> There is no auto-fix like spotlessApply AFAIK. >>> >>> As a side-note, I really dislike our python line continuation indent >>> rule, since pycharm can't be configured to adhere to it and I find myself >>> manually adjusting whitespace all the time. >>> >>> >>> On Mon, Feb 25, 2019 at 10:22 AM Kenneth Knowles >>> wrote: >>> FWIW gradle is a depgraph-based build system. You can gain a few seconds by putting all but spotlessApply in one command. ./gradlew spotlessApply && ./gradlew checkstyleMain checkstyleTest javadoc findbugsMain compileTestJava compileJava It might be clever to define a meta-task. Gradle "base plugin" has the notable check (build and run tests), assemble (make artifacts), and build (assemble + check, badly named!) I think something like "everything except running tests and building artifacts" might be helpful. Kenn On Mon, Feb 25, 2019 at 10:13 AM Alex Amato wrote: > I made a thread about this a while back for java, but I don't think > the same commands like sptoless work for python. > > auto fixing lint issues > running and quick checks which would fail the PR (without running the > whole precommit?) > Something like findbugs to detect common issues (i.e. py3 compliance) > > FWIW, this is what I have been using for java. It will catch pretty > much everything except presubmit test failures. > > ./gradlew spotlessApply && ./gradlew checkstyleMain && ./gradlew > checkstyleTest && ./gradlew javadoc && ./gradlew findbugsMain && ./gradlew > compileTestJava && ./gradlew compileJava > > > -- > > Ruoyun Huang > >
Re: What quick command to catch common issues before pushing a python PR?
nvm. Don't take my previous non-scientific comparison (only ran it once) too seriously. :-) I tried to repeat each for multiple times and now the difference diminishes. likely there was a transient error in caching. On Mon, Feb 25, 2019 at 3:38 PM Kenneth Knowles wrote: > Ah, that is likely caused by us having ill-defined tasks that cannot be > cached. Or is it that the configuration time is so significant? > > Kenn > > On Mon, Feb 25, 2019 at 11:05 AM Ruoyun Huang wrote: > >> Out of curiosity as a light gradle user, I did a side by side comparison, >> and the readings confirm what Ken and Michael suggests. >> >> In the same repository, do gradle clean then followed by either of the >> two commands. Measure their runtime respectively. The latter one takes >> *1/3* running time. >> >> time ./gradlew spotlessApply && ./gradlew checkstyleMain && ./gradlew >> checkstyleTest && ./gradlew javadoc && ./gradlew findbugsMain && ./gradlew >> compileTestJava && ./gradlew compileJava >> real 9m29.330s user 0m11.330s sys 0m1.239s >> >> time ./gradlew spotlessApply checkstyleMain checkstyleTest javadoc >> findbugsMain compileJava compileTestJava >> real3m35.573s >> user0m2.701s >> sys 0m0.327s >> >> >> >> >> >> >> >> On Mon, Feb 25, 2019 at 10:47 AM Alex Amato wrote: >> >>> @Michael, no particular reason. I think Ken's suggestion makes more >>> sense. >>> >>> On Mon, Feb 25, 2019 at 10:36 AM Udi Meiri wrote: >>> Talking about Python: I only know of "./gradlew lint", which include style and some py3 compliance checking. There is no auto-fix like spotlessApply AFAIK. As a side-note, I really dislike our python line continuation indent rule, since pycharm can't be configured to adhere to it and I find myself manually adjusting whitespace all the time. On Mon, Feb 25, 2019 at 10:22 AM Kenneth Knowles wrote: > FWIW gradle is a depgraph-based build system. You can gain a few > seconds by putting all but spotlessApply in one command. > > ./gradlew spotlessApply && ./gradlew checkstyleMain checkstyleTest > javadoc findbugsMain compileTestJava compileJava > > It might be clever to define a meta-task. Gradle "base plugin" has the > notable check (build and run tests), assemble (make artifacts), and build > (assemble + check, badly named!) > > I think something like "everything except running tests and building > artifacts" might be helpful. > > Kenn > > On Mon, Feb 25, 2019 at 10:13 AM Alex Amato > wrote: > >> I made a thread about this a while back for java, but I don't think >> the same commands like sptoless work for python. >> >> auto fixing lint issues >> running and quick checks which would fail the PR (without running the >> whole precommit?) >> Something like findbugs to detect common issues (i.e. py3 compliance) >> >> FWIW, this is what I have been using for java. It will catch pretty >> much everything except presubmit test failures. >> >> ./gradlew spotlessApply && ./gradlew checkstyleMain && ./gradlew >> checkstyleTest && ./gradlew javadoc && ./gradlew findbugsMain && >> ./gradlew >> compileTestJava && ./gradlew compileJava >> > >> >> -- >> >> Ruoyun Huang >> >> -- Ruoyun Huang
Re: What quick command to catch common issues before pushing a python PR?
I use a selection of tox tasks. Here are the tox tasks that I use the most: - py27-gcp - py35-gcp - py27-cython - py35-cython - py35-lint - py27-lint Most recently, I'll run `tox -e py3-gcp,py3-lint`, which run fairly quickly. You can choose which subset works for you. My insight is: Lints are pretty fast, so it's fine to add a couple different lints. Unittest runs are pretty slow, so I usually go for the one with most coverage for my change (x-gcp, or x-cython). Best -P. On Mon, Feb 25, 2019 at 4:33 PM Ruoyun Huang wrote: > nvm. Don't take my previous non-scientific comparison (only ran it once) > too seriously. :-) > > I tried to repeat each for multiple times and now the difference > diminishes. likely there was a transient error in caching. > > On Mon, Feb 25, 2019 at 3:38 PM Kenneth Knowles wrote: > >> Ah, that is likely caused by us having ill-defined tasks that cannot be >> cached. Or is it that the configuration time is so significant? >> >> Kenn >> >> On Mon, Feb 25, 2019 at 11:05 AM Ruoyun Huang wrote: >> >>> Out of curiosity as a light gradle user, I did a side by side >>> comparison, and the readings confirm what Ken and Michael suggests. >>> >>> In the same repository, do gradle clean then followed by either of the >>> two commands. Measure their runtime respectively. The latter one takes >>> *1/3* running time. >>> >>> time ./gradlew spotlessApply && ./gradlew checkstyleMain && ./gradlew >>> checkstyleTest && ./gradlew javadoc && ./gradlew findbugsMain && ./gradlew >>> compileTestJava && ./gradlew compileJava >>> real 9m29.330s user 0m11.330s sys 0m1.239s >>> >>> time ./gradlew spotlessApply checkstyleMain checkstyleTest javadoc >>> findbugsMain compileJava compileTestJava >>> real3m35.573s >>> user0m2.701s >>> sys 0m0.327s >>> >>> >>> >>> >>> >>> >>> >>> On Mon, Feb 25, 2019 at 10:47 AM Alex Amato wrote: >>> @Michael, no particular reason. I think Ken's suggestion makes more sense. On Mon, Feb 25, 2019 at 10:36 AM Udi Meiri wrote: > Talking about Python: > I only know of "./gradlew lint", which include style and some py3 > compliance checking. > There is no auto-fix like spotlessApply AFAIK. > > As a side-note, I really dislike our python line continuation indent > rule, since pycharm can't be configured to adhere to it and I find myself > manually adjusting whitespace all the time. > > > On Mon, Feb 25, 2019 at 10:22 AM Kenneth Knowles > wrote: > >> FWIW gradle is a depgraph-based build system. You can gain a few >> seconds by putting all but spotlessApply in one command. >> >> ./gradlew spotlessApply && ./gradlew checkstyleMain checkstyleTest >> javadoc findbugsMain compileTestJava compileJava >> >> It might be clever to define a meta-task. Gradle "base plugin" has >> the notable check (build and run tests), assemble (make artifacts), and >> build (assemble + check, badly named!) >> >> I think something like "everything except running tests and building >> artifacts" might be helpful. >> >> Kenn >> >> On Mon, Feb 25, 2019 at 10:13 AM Alex Amato >> wrote: >> >>> I made a thread about this a while back for java, but I don't think >>> the same commands like sptoless work for python. >>> >>> auto fixing lint issues >>> running and quick checks which would fail the PR (without running >>> the whole precommit?) >>> Something like findbugs to detect common issues (i.e. py3 compliance) >>> >>> FWIW, this is what I have been using for java. It will catch pretty >>> much everything except presubmit test failures. >>> >>> ./gradlew spotlessApply && ./gradlew checkstyleMain && ./gradlew >>> checkstyleTest && ./gradlew javadoc && ./gradlew findbugsMain && >>> ./gradlew >>> compileTestJava && ./gradlew compileJava >>> >> >>> >>> -- >>> >>> Ruoyun Huang >>> >>> > > -- > > Ruoyun Huang > >
Re: What quick command to catch common issues before pushing a python PR?
Correction - the command is now: tox -e py35-gcp,py35-lint And it ran on my machine in 5min 40s. Not blazing fast, but at least significantly faster than waiting for Jenkins : ) Best -P. On Tue, Mar 19, 2019 at 5:22 PM Pablo Estrada wrote: > I use a selection of tox tasks. Here are the tox tasks that I use the most: > - py27-gcp > - py35-gcp > - py27-cython > - py35-cython > - py35-lint > - py27-lint > > Most recently, I'll run `tox -e py3-gcp,py3-lint`, which run fairly > quickly. You can choose which subset works for you. > My insight is: Lints are pretty fast, so it's fine to add a couple > different lints. Unittest runs are pretty slow, so I usually go for the one > with most coverage for my change (x-gcp, or x-cython). > Best > -P. > > On Mon, Feb 25, 2019 at 4:33 PM Ruoyun Huang wrote: > >> nvm. Don't take my previous non-scientific comparison (only ran it once) >> too seriously. :-) >> >> I tried to repeat each for multiple times and now the difference >> diminishes. likely there was a transient error in caching. >> >> On Mon, Feb 25, 2019 at 3:38 PM Kenneth Knowles wrote: >> >>> Ah, that is likely caused by us having ill-defined tasks that cannot be >>> cached. Or is it that the configuration time is so significant? >>> >>> Kenn >>> >>> On Mon, Feb 25, 2019 at 11:05 AM Ruoyun Huang wrote: >>> Out of curiosity as a light gradle user, I did a side by side comparison, and the readings confirm what Ken and Michael suggests. In the same repository, do gradle clean then followed by either of the two commands. Measure their runtime respectively. The latter one takes *1/3* running time. time ./gradlew spotlessApply && ./gradlew checkstyleMain && ./gradlew checkstyleTest && ./gradlew javadoc && ./gradlew findbugsMain && ./gradlew compileTestJava && ./gradlew compileJava real 9m29.330s user 0m11.330s sys 0m1.239s time ./gradlew spotlessApply checkstyleMain checkstyleTest javadoc findbugsMain compileJava compileTestJava real3m35.573s user0m2.701s sys 0m0.327s On Mon, Feb 25, 2019 at 10:47 AM Alex Amato wrote: > @Michael, no particular reason. I think Ken's suggestion makes more > sense. > > On Mon, Feb 25, 2019 at 10:36 AM Udi Meiri wrote: > >> Talking about Python: >> I only know of "./gradlew lint", which include style and some py3 >> compliance checking. >> There is no auto-fix like spotlessApply AFAIK. >> >> As a side-note, I really dislike our python line continuation indent >> rule, since pycharm can't be configured to adhere to it and I find myself >> manually adjusting whitespace all the time. >> >> >> On Mon, Feb 25, 2019 at 10:22 AM Kenneth Knowles >> wrote: >> >>> FWIW gradle is a depgraph-based build system. You can gain a few >>> seconds by putting all but spotlessApply in one command. >>> >>> ./gradlew spotlessApply && ./gradlew checkstyleMain checkstyleTest >>> javadoc findbugsMain compileTestJava compileJava >>> >>> It might be clever to define a meta-task. Gradle "base plugin" has >>> the notable check (build and run tests), assemble (make artifacts), and >>> build (assemble + check, badly named!) >>> >>> I think something like "everything except running tests and building >>> artifacts" might be helpful. >>> >>> Kenn >>> >>> On Mon, Feb 25, 2019 at 10:13 AM Alex Amato >>> wrote: >>> I made a thread about this a while back for java, but I don't think the same commands like sptoless work for python. auto fixing lint issues running and quick checks which would fail the PR (without running the whole precommit?) Something like findbugs to detect common issues (i.e. py3 compliance) FWIW, this is what I have been using for java. It will catch pretty much everything except presubmit test failures. ./gradlew spotlessApply && ./gradlew checkstyleMain && ./gradlew checkstyleTest && ./gradlew javadoc && ./gradlew findbugsMain && ./gradlew compileTestJava && ./gradlew compileJava >>> -- Ruoyun Huang >> >> -- >> >> Ruoyun Huang >> >>
Re: What quick command to catch common issues before pushing a python PR?
I use tox as well. Actually, I use detox and retox (parallel versions of tox, easily installable with pip) which can speed things up quite a bit. On Wed, Mar 20, 2019 at 1:33 AM Pablo Estrada wrote: > > Correction - the command is now: tox -e py35-gcp,py35-lint > > And it ran on my machine in 5min 40s. Not blazing fast, but at least > significantly faster than waiting for Jenkins : ) > Best > -P. > > On Tue, Mar 19, 2019 at 5:22 PM Pablo Estrada wrote: >> >> I use a selection of tox tasks. Here are the tox tasks that I use the most: >> - py27-gcp >> - py35-gcp >> - py27-cython >> - py35-cython >> - py35-lint >> - py27-lint >> >> Most recently, I'll run `tox -e py3-gcp,py3-lint`, which run fairly quickly. >> You can choose which subset works for you. >> My insight is: Lints are pretty fast, so it's fine to add a couple different >> lints. Unittest runs are pretty slow, so I usually go for the one with most >> coverage for my change (x-gcp, or x-cython). >> Best >> -P. >> >> On Mon, Feb 25, 2019 at 4:33 PM Ruoyun Huang wrote: >>> >>> nvm. Don't take my previous non-scientific comparison (only ran it once) >>> too seriously. :-) >>> >>> I tried to repeat each for multiple times and now the difference >>> diminishes. likely there was a transient error in caching. >>> >>> On Mon, Feb 25, 2019 at 3:38 PM Kenneth Knowles wrote: Ah, that is likely caused by us having ill-defined tasks that cannot be cached. Or is it that the configuration time is so significant? Kenn On Mon, Feb 25, 2019 at 11:05 AM Ruoyun Huang wrote: > > Out of curiosity as a light gradle user, I did a side by side comparison, > and the readings confirm what Ken and Michael suggests. > > In the same repository, do gradle clean then followed by either of the > two commands. Measure their runtime respectively. The latter one takes > 1/3 running time. > > time ./gradlew spotlessApply && ./gradlew checkstyleMain && ./gradlew > checkstyleTest && ./gradlew javadoc && ./gradlew findbugsMain && > ./gradlew compileTestJava && ./gradlew compileJava > real 9m29.330s user 0m11.330s sys 0m1.239s > > time ./gradlew spotlessApply checkstyleMain checkstyleTest javadoc > findbugsMain compileJava compileTestJava > real3m35.573s > user0m2.701s > sys 0m0.327s > > > > > > > > On Mon, Feb 25, 2019 at 10:47 AM Alex Amato wrote: >> >> @Michael, no particular reason. I think Ken's suggestion makes more >> sense. >> >> On Mon, Feb 25, 2019 at 10:36 AM Udi Meiri wrote: >>> >>> Talking about Python: >>> I only know of "./gradlew lint", which include style and some py3 >>> compliance checking. >>> There is no auto-fix like spotlessApply AFAIK. >>> >>> As a side-note, I really dislike our python line continuation indent >>> rule, since pycharm can't be configured to adhere to it and I find >>> myself manually adjusting whitespace all the time. >>> >>> >>> On Mon, Feb 25, 2019 at 10:22 AM Kenneth Knowles >>> wrote: FWIW gradle is a depgraph-based build system. You can gain a few seconds by putting all but spotlessApply in one command. ./gradlew spotlessApply && ./gradlew checkstyleMain checkstyleTest javadoc findbugsMain compileTestJava compileJava It might be clever to define a meta-task. Gradle "base plugin" has the notable check (build and run tests), assemble (make artifacts), and build (assemble + check, badly named!) I think something like "everything except running tests and building artifacts" might be helpful. Kenn On Mon, Feb 25, 2019 at 10:13 AM Alex Amato wrote: > > I made a thread about this a while back for java, but I don't think > the same commands like sptoless work for python. > > auto fixing lint issues > running and quick checks which would fail the PR (without running the > whole precommit?) > Something like findbugs to detect common issues (i.e. py3 compliance) > > FWIW, this is what I have been using for java. It will catch pretty > much everything except presubmit test failures. > > ./gradlew spotlessApply && ./gradlew checkstyleMain && ./gradlew > checkstyleTest && ./gradlew javadoc && ./gradlew findbugsMain && > ./gradlew compileTestJava && ./gradlew compileJava > > > > -- > > Ruoyun Huang > >>> >>> >>> -- >>> >>> Ruoyun Huang >>>
Re: What quick command to catch common issues before pushing a python PR?
Fancy : ) On Wed, Mar 20, 2019 at 1:25 AM Robert Bradshaw wrote: > I use tox as well. Actually, I use detox and retox (parallel versions > of tox, easily installable with pip) which can speed things up quite a > bit. > > On Wed, Mar 20, 2019 at 1:33 AM Pablo Estrada wrote: > > > > Correction - the command is now: tox -e py35-gcp,py35-lint > > > > And it ran on my machine in 5min 40s. Not blazing fast, but at least > significantly faster than waiting for Jenkins : ) > > Best > > -P. > > > > On Tue, Mar 19, 2019 at 5:22 PM Pablo Estrada > wrote: > >> > >> I use a selection of tox tasks. Here are the tox tasks that I use the > most: > >> - py27-gcp > >> - py35-gcp > >> - py27-cython > >> - py35-cython > >> - py35-lint > >> - py27-lint > >> > >> Most recently, I'll run `tox -e py3-gcp,py3-lint`, which run fairly > quickly. You can choose which subset works for you. > >> My insight is: Lints are pretty fast, so it's fine to add a couple > different lints. Unittest runs are pretty slow, so I usually go for the one > with most coverage for my change (x-gcp, or x-cython). > >> Best > >> -P. > >> > >> On Mon, Feb 25, 2019 at 4:33 PM Ruoyun Huang wrote: > >>> > >>> nvm. Don't take my previous non-scientific comparison (only ran it > once) too seriously. :-) > >>> > >>> I tried to repeat each for multiple times and now the difference > diminishes. likely there was a transient error in caching. > >>> > >>> On Mon, Feb 25, 2019 at 3:38 PM Kenneth Knowles > wrote: > > Ah, that is likely caused by us having ill-defined tasks that cannot > be cached. Or is it that the configuration time is so significant? > > Kenn > > On Mon, Feb 25, 2019 at 11:05 AM Ruoyun Huang > wrote: > > > > Out of curiosity as a light gradle user, I did a side by side > comparison, and the readings confirm what Ken and Michael suggests. > > > > In the same repository, do gradle clean then followed by either of > the two commands. Measure their runtime respectively. The latter one takes > 1/3 running time. > > > > time ./gradlew spotlessApply && ./gradlew checkstyleMain && > ./gradlew checkstyleTest && ./gradlew javadoc && ./gradlew findbugsMain && > ./gradlew compileTestJava && ./gradlew compileJava > > real 9m29.330s user 0m11.330s sys 0m1.239s > > > > time ./gradlew spotlessApply checkstyleMain checkstyleTest javadoc > findbugsMain compileJava compileTestJava > > real3m35.573s > > user0m2.701s > > sys 0m0.327s > > > > > > > > > > > > > > > > On Mon, Feb 25, 2019 at 10:47 AM Alex Amato > wrote: > >> > >> @Michael, no particular reason. I think Ken's suggestion makes more > sense. > >> > >> On Mon, Feb 25, 2019 at 10:36 AM Udi Meiri > wrote: > >>> > >>> Talking about Python: > >>> I only know of "./gradlew lint", which include style and some py3 > compliance checking. > >>> There is no auto-fix like spotlessApply AFAIK. > >>> > >>> As a side-note, I really dislike our python line continuation > indent rule, since pycharm can't be configured to adhere to it and I find > myself manually adjusting whitespace all the time. > >>> > >>> > >>> On Mon, Feb 25, 2019 at 10:22 AM Kenneth Knowles > wrote: > > FWIW gradle is a depgraph-based build system. You can gain a few > seconds by putting all but spotlessApply in one command. > > ./gradlew spotlessApply && ./gradlew checkstyleMain > checkstyleTest javadoc findbugsMain compileTestJava compileJava > > It might be clever to define a meta-task. Gradle "base plugin" > has the notable check (build and run tests), assemble (make artifacts), and > build (assemble + check, badly named!) > > I think something like "everything except running tests and > building artifacts" might be helpful. > > Kenn > > On Mon, Feb 25, 2019 at 10:13 AM Alex Amato > wrote: > > > > I made a thread about this a while back for java, but I don't > think the same commands like sptoless work for python. > > > > auto fixing lint issues > > running and quick checks which would fail the PR (without > running the whole precommit?) > > Something like findbugs to detect common issues (i.e. py3 > compliance) > > > > FWIW, this is what I have been using for java. It will catch > pretty much everything except presubmit test failures. > > > > ./gradlew spotlessApply && ./gradlew checkstyleMain && ./gradlew > checkstyleTest && ./gradlew javadoc && ./gradlew findbugsMain && ./gradlew > compileTestJava && ./gradlew compileJava > > > > > > > > -- > > > > Ruoyun Huang > > > >>> > >>> > >>> -- > >>> > >>> Ruoyun Huang > >>> >