We did turn on travis a few years ago, but ended up turning it off because it was failing (I believe because of insufficient resources) which was confusing for developers. I wouldn't be opposed to turning it on if it provides more/faster signal, but its not obvious to me that it would. In particular, do we know that given the rate PRs are created if we will hit rate limits?
Really my main feedback is, if the java linter is important we should probably have it as part of the canonical build process. I worry about having more than one set of CI infrastructure to maintain. On Mon, May 23, 2016 at 9:43 AM, Dongjoon Hyun <dongj...@apache.org> wrote: > Thank you, Steve and Hyukjin. > > And, don't worry, Ted. > > Travis launches new VMs for every PR. > > Apache Spark repository uses the following setting. > > VM: Google Compute Engine > OS: Ubuntu 14.04.3 LTS Server Edition 64bit > CPU: ~2 CORE > RAM: 7.5GB > > FYI, you can find more information about this here. > > https://docs.travis-ci.com/user/ci-environment/#Virtualization-environments > > Dongjoon. > > > > On Mon, May 23, 2016 at 6:32 AM, Ted Yu <yuzhih...@gmail.com> wrote: > >> Do you know if more than one PR would be verified on the same machine ? >> >> I wonder whether the 'mvn install' from two simultaneous PR builds may >> have conflict. >> >> On Sun, May 22, 2016 at 9:21 PM, Dongjoon Hyun <dongj...@apache.org> >> wrote: >> >>> Thank you for feedback. Sure, correctly, that's the reason why the >>> current SparkPullRequestBuilder do not run `lint-java`. :-) >>> >>> In addition, that's the same reason why contributors are reluctant to >>> run `lint-java` and causes breaking on JDK7 builds. >>> >>> Such a tedious and time-consuming job should be done by CI without human >>> interventions. >>> >>> By the way, why do you think we need to wait for that? We should not >>> wait for any CIs, we should continue our own work. >>> >>> My proposal isn't for making you wait to watch the result. There are two >>> use cases I want for us to focus here. >>> >>> Case 1: When you make a PR to Spark PR queue. >>> >>> Travis CI will finish before SparkPullRequestBuilder. >>> We will run the followings in parallel mode. >>> 1. Current SparkPullRequestBuilder: JDK8 + sbt build + (no >>> Java Linter) >>> 2. Travis: JDK7 + mvn build + Java Linter >>> 3. Travis: JDK8 + mvn build + Java Linter >>> As we know, 1 is the longest time-consuming one which have lots of >>> works (except maven building or lint- java). You don't need to wait more >>> in many cases. Yes, in many cases, not all the cases. >>> >>> >>> Case 2: When you prepare a PR on your branch. >>> >>> If you are at the final commit (maybe already-squashed), just go to >>> case 1. >>> >>> However, usually, we makes lots of commits locally while making >>> preparing our PR. >>> And, finally we squashed them into one and send a PR to Spark. >>> I mean you can use Travis CI during preparing your PRs. >>> Again, don't wait for Travis CI. Just push it sometime or at every >>> commit, and continue your work. >>> >>> At the final stage when you finish your coding, squash your commits >>> into one, >>> and amend your commit title or messages, see the Travis CI. >>> Or, you can monitor Travis CI result on status menu bar. >>> If it shows green icon, you have nothing to do. >>> >>> https://docs.travis-ci.com/user/apps/ >>> >>> To sum up, I think we don't need to wait for any CIs. It's like an >>> email. `Send and back to work.` >>> >>> Dongjoon. >>> >>> >>> On Sun, May 22, 2016 at 8:32 PM, Ted Yu <yuzhih...@gmail.com> wrote: >>> >>>> Without Zinc, 'mvn -DskipTests clean install' takes ~30 minutes. >>>> >>>> Maybe not everyone is willing to wait that long. >>>> >>>> On Sun, May 22, 2016 at 1:30 PM, Dongjoon Hyun <dongj...@apache.org> >>>> wrote: >>>> >>>>> Oh, Sure. My bad! >>>>> >>>>> - For Oracle JDK7, mvn -DskipTests install and run `dev/lint-java`. >>>>> - For Oracle JDK8, mvn -DskipTests install and run `dev/lint-java`. >>>>> >>>>> Thank you, Ted. >>>>> >>>>> Dongjoon. >>>>> >>>>> On Sun, May 22, 2016 at 1:29 PM, Ted Yu <yuzhih...@gmail.com> wrote: >>>>> >>>>>> The following line was repeated twice: >>>>>> >>>>>> - For Oracle JDK7, mvn -DskipTests install and run `dev/lint-java`. >>>>>> >>>>>> Did you intend to cover JDK 8 ? >>>>>> >>>>>> Cheers >>>>>> >>>>>> On Sun, May 22, 2016 at 1:25 PM, Dongjoon Hyun <dongj...@apache.org> >>>>>> wrote: >>>>>> >>>>>>> Hi, All. >>>>>>> >>>>>>> I want to propose the followings. >>>>>>> >>>>>>> - Turn on Travis CI for Apache Spark PR queue. >>>>>>> - Recommend this for contributors, too >>>>>>> >>>>>>> Currently, Spark provides Travis CI configuration file to help >>>>>>> contributors check Scala/Java style conformance and JDK7/8 compilation >>>>>>> easily during their preparing pull requests. Please note that it's only >>>>>>> about static analysis. >>>>>>> >>>>>>> - For Oracle JDK7, mvn -DskipTests install and run `dev/lint-java`. >>>>>>> - For Oracle JDK7, mvn -DskipTests install and run `dev/lint-java`. >>>>>>> Scalastyle is included in the step 'mvn install', too. >>>>>>> >>>>>>> Yep, if you turn on your Travis CI configuration, you can already >>>>>>> see the results on your branches before making PR. I wrote this email to >>>>>>> prevent more failures proactively and community-widely. >>>>>>> >>>>>>> For stability, I have been monitoring that for two weeks. It detects >>>>>>> the failures or recovery on JDK7 builds or Java linter on Spark master >>>>>>> branch correctly. The only exceptional case I observed rarely is >>>>>>> `timeout` >>>>>>> failure due to hangs of maven. But, as we know, it's happen in our >>>>>>> Jenkins >>>>>>> SparkPullRequestBuilder, too. I think we can ignore that. >>>>>>> >>>>>>> I'm sure that this will save much more community's efforts on the >>>>>>> static errors by preventing them at the very early stage. But, there >>>>>>> might >>>>>>> be another reason not to do this. I'm wondering about your thoughts. >>>>>>> >>>>>>> I can make a Apache INFRA Jira issue for this if there is some >>>>>>> consensus. >>>>>>> >>>>>>> Warmly, >>>>>>> Dongjoon. >>>>>>> >>>>>> >>>>>> >>>>> >>>> >>> >> >
