sigh. that should have been "is now a matrix build". On Sat, Jun 13, 2015 at 7:31 PM, Sean Busbey <[email protected]> wrote:
> The HBase-1.2 build[1] is not a matrix build that runs JDK7 and JDK8 in > parallel. > > I saved the old job for now and left it disabled[2]. > > > [1]: https://builds.apache.org/view/H-L/view/HBase/job/HBase-1.2/ > [2]: > https://builds.apache.org/view/H-L/view/HBase/job/HBase-1.2%20non-matrix/ > > On Fri, Apr 10, 2015 at 3:42 PM, Sean Busbey <[email protected]> wrote: > >> Sorry for the noise. I also updated the checkstyle step on HBase-TRUNK >> >> from >> mvn checkstyle:checkstyle >> to >> mvn -DskipTests package checkstyle:checkstyle >> >> to deal with the same issue. Looks all clear now. >> >> On Fri, Apr 10, 2015 at 3:03 PM, Sean Busbey <[email protected]> wrote: >> >>> Updated the following builds: >>> >>> * HBase-TRUNK >>> >>> moved from >>> >>> mvn clean compile findbugs:findbugs >>> >>> to >>> >>> mvn clean -DskipTests package findbugs:findbugs >>> >>> To work around the known issue where we can't do a bootstrap compile >>> without previous install or remote SNAPSHOT artifacts. (recently triggered >>> by the addition of the procedure module on master) >>> >>> On Mon, Mar 30, 2015 at 8:49 PM, 张铎 <[email protected]> wrote: >>> >>>> Make findbugs and checkstyle plugins always run. >>>> The default behavior only publishes result for stable builds, but >>>> master is >>>> not always stable and sometimes we introduce new warnings in unstable >>>> builds(see builds 6310-6314). >>>> And findbugs and checkstyle will not fail unless we abort the building >>>> process I think, so it is safe to turn it on every time. >>>> >>>> Thanks. >>>> >>>> 2015-03-22 22:44 GMT+08:00 Ted Yu <[email protected]>: >>>> >>>> > +1 on letting HBase-TRUNK jenkins show coverage report. >>>> > >>>> > Cheers >>>> > >>>> > On Sun, Mar 22, 2015 at 5:51 AM, 张铎 <[email protected]> wrote: >>>> > >>>> > > Going to change the config of HBase-TRUNK jenkins to show findbugs, >>>> > > checkstyle and jacoco coverage report. >>>> > > The config has been tested on >>>> > > https://builds.apache.org/job/HBase-TRUNK-jacoco for nearly 30 >>>> times. >>>> > Not >>>> > > much different from HBase-TRUNK unless it runs ~30% slower(the >>>> overhead >>>> > of >>>> > > collecting information for code coverage). >>>> > > Thanks. >>>> > > >>>> > > 2015-03-12 5:08 GMT+08:00 Andrew Purtell <[email protected]>: >>>> > > >>>> > > > +1, thanks a lot for improving our build hygiene. >>>> > > > >>>> > > > On Wed, Mar 11, 2015 at 2:01 PM, Enis Söztutar < >>>> [email protected]> >>>> > > wrote: >>>> > > > >>>> > > > > Thanks Sean. This is great. >>>> > > > > >>>> > > > > Enis >>>> > > > > >>>> > > > > On Wed, Mar 11, 2015 at 1:54 PM, Sean Busbey < >>>> [email protected]> >>>> > > > wrote: >>>> > > > > >>>> > > > > > FYI I just finished chasing down the breakage for mvn site on >>>> all >>>> > > patch >>>> > > > > > builds. >>>> > > > > > >>>> > > > > > HBASE-13191 consolidates the few places in the test-patch code >>>> > where >>>> > > we >>>> > > > > > hard coded MAVEN_OPTS. >>>> > > > > > >>>> > > > > > If you look at the PreCommit job now, we use the "set >>>> environment >>>> > > > > > variables" option to set MAVEN_OPTS and then everything else >>>> > respects >>>> > > > > that >>>> > > > > > setting. >>>> > > > > > >>>> > > > > > I set the initial value to be a combination of the memory >>>> > limitations >>>> > > > > we've >>>> > > > > > been actually running with (the ~6G was getting ignored) and >>>> the >>>> > > > permgen >>>> > > > > > needed for site. >>>> > > > > > >>>> > > > > > MAVEN_OPTS=-Xmx3100M -XX:-UsePerfData -XX:MaxPermSize=256m >>>> > > > > > >>>> > > > > > On Mon, Feb 23, 2015 at 11:46 PM, Stack <[email protected]> >>>> wrote: >>>> > > > > > >>>> > > > > > > I just made TRUNK and branch-1 builds use same jvm as >>>> patch-build >>>> > > > > > > (hadoopqa) -- i.e. jdku51 -- and I set the MAVEN_OPTS to be >>>> the >>>> > > same >>>> > > > as >>>> > > > > > > those of trunk build too, setting MAVEN_OPTS="-Xmx6100m"... >>>> it >>>> > had >>>> > > > been >>>> > > > > > > 3000. >>>> > > > > > > >>>> > > > > > > Yours, >>>> > > > > > > St.Ack >>>> > > > > > > >>>> > > > > > > On Wed, Dec 31, 2014 at 4:22 PM, Stack <[email protected]> >>>> wrote: >>>> > > > > > > >>>> > > > > > > > I upped hadoopqa retention to keep last 100 builds and or >>>> last >>>> > 7 >>>> > > > > days, >>>> > > > > > > > whichever comes first. >>>> > > > > > > > St.Ack >>>> > > > > > > > >>>> > > > > > > > On Tue, Nov 4, 2014 at 9:38 AM, Stack <[email protected]> >>>> > wrote: >>>> > > > > > > > >>>> > > > > > > >> Branch-1 and master have stabilized and now run mostly >>>> blue >>>> > > (give >>>> > > > or >>>> > > > > > > take >>>> > > > > > > >> the odd failure) [1][2]. Having a mostly blue branch-1 >>>> has >>>> > > helped >>>> > > > us >>>> > > > > > > >> identify at least one destabilizing commit in the last >>>> few >>>> > days, >>>> > > > > maybe >>>> > > > > > > two; >>>> > > > > > > >> this is as it should be (smile). >>>> > > > > > > >> >>>> > > > > > > >> Lets keep our builds blue. If you commit a patch, make >>>> sure >>>> > > > > subsequent >>>> > > > > > > >> builds stay blue. You can subscribe to >>>> > [email protected] >>>> > > to >>>> > > > > get >>>> > > > > > > >> notice of failures if not already subscribed. >>>> > > > > > > >> >>>> > > > > > > >> Thanks, >>>> > > > > > > >> St.Ack >>>> > > > > > > >> >>>> > > > > > > >> 1. >>>> > https://builds.apache.org/view/H-L/view/HBase/job/HBase-1.0/ >>>> > > > > > > >> 2. >>>> > > https://builds.apache.org/view/H-L/view/HBase/job/HBase-TRUNK/ >>>> > > > > > > >> >>>> > > > > > > >> >>>> > > > > > > >> On Mon, Oct 13, 2014 at 4:41 PM, Stack <[email protected] >>>> > >>>> > > wrote: >>>> > > > > > > >> >>>> > > > > > > >>> A few notes on testing. >>>> > > > > > > >>> >>>> > > > > > > >>> Too long to read, infra is more capable now and after >>>> some >>>> > > work, >>>> > > > we >>>> > > > > > are >>>> > > > > > > >>> seeing branch-1 and trunk mostly running blue. Lets try >>>> and >>>> > > keep >>>> > > > it >>>> > > > > > > this >>>> > > > > > > >>> way going forward. >>>> > > > > > > >>> >>>> > > > > > > >>> Apache Infra has new, more capable hardware. >>>> > > > > > > >>> >>>> > > > > > > >>> A recent spurt of test fixing combined with more capable >>>> > > hardware >>>> > > > > > seems >>>> > > > > > > >>> to have gotten us to a new place; tests are mostly >>>> passing >>>> > now >>>> > > on >>>> > > > > > > branch-1 >>>> > > > > > > >>> and master. Lets try and keep it this way and start to >>>> trust >>>> > > our >>>> > > > > > test >>>> > > > > > > runs >>>> > > > > > > >>> again. Just a few flakies remain. Lets try and nail >>>> them. >>>> > > > > > > >>> >>>> > > > > > > >>> Our tests now run in parallel with other test suites >>>> where >>>> > > > previous >>>> > > > > > we >>>> > > > > > > >>> ran alone. You can see this sometimes when our zombie >>>> > detector >>>> > > > > > reports >>>> > > > > > > >>> tests from another project altogether as lingerers (To >>>> be >>>> > > fixed). >>>> > > > > > > Some of >>>> > > > > > > >>> our tests are failing because a concurrent hbase run is >>>> > undoing >>>> > > > > > > classes and >>>> > > > > > > >>> data from under it. Also, lets fix. >>>> > > > > > > >>> >>>> > > > > > > >>> Our tests are brittle. It takes 75minutes for them to >>>> > complete. >>>> > > > > Many >>>> > > > > > > >>> are heavy-duty integration tests starting up multiple >>>> > clusters >>>> > > > and >>>> > > > > > > >>> mapreduce all in the one JVM. It is a miracle they pass >>>> at >>>> > all. >>>> > > > > > > Usually >>>> > > > > > > >>> integration tests have been cast as unit tests because >>>> there >>>> > > was >>>> > > > no >>>> > > > > > > where >>>> > > > > > > >>> else for them to get an airing. We have the hbase-it >>>> suite >>>> > now >>>> > > > > which >>>> > > > > > > would >>>> > > > > > > >>> be a more apt place but until these are run on a regular >>>> > basis >>>> > > in >>>> > > > > > > public >>>> > > > > > > >>> for all to see, the fat integration tests disguised as >>>> unit >>>> > > tests >>>> > > > > > will >>>> > > > > > > >>> remain. A review of our current unit tests weeding the >>>> old >>>> > > cruft >>>> > > > > and >>>> > > > > > > the >>>> > > > > > > >>> no longer relevant or duplicates would be a nice >>>> undertaking >>>> > if >>>> > > > > > > someone is >>>> > > > > > > >>> looking to contribute. >>>> > > > > > > >>> >>>> > > > > > > >>> Alex Newman has been working on making our tests work >>>> up on >>>> > > > travis >>>> > > > > > and >>>> > > > > > > >>> circle-ci. That'll be sweet when it goes end-to-end. >>>> He >>>> > also >>>> > > > > added >>>> > > > > > in >>>> > > > > > > >>> some "type" categorizations -- client, filter, >>>> mapreduce -- >>>> > > > > alongside >>>> > > > > > > our >>>> > > > > > > >>> old "sizing" categorizations of small/medium/large. His >>>> > > thinking >>>> > > > > is >>>> > > > > > > that >>>> > > > > > > >>> we can run these categorizations in parallel so we >>>> could run >>>> > > the >>>> > > > > > total >>>> > > > > > > >>> suite in about the time of the longest test, say >>>> > 20-30minutes? >>>> > > > We >>>> > > > > > > could >>>> > > > > > > >>> even change Apache to run them this way. >>>> > > > > > > >>> >>>> > > > > > > >>> FYI, >>>> > > > > > > >>> St.Ack >>>> > > > > > > >>> >>>> > > > > > > >>> >>>> > > > > > > >>> >>>> > > > > > > >>> >>>> > > > > > > >>> >>>> > > > > > > >>> >>>> > > > > > > >>> >>>> > > > > > > >> >>>> > > > > > > > >>>> > > > > > > >>>> > > > > > >>>> > > > > > >>>> > > > > > >>>> > > > > > -- >>>> > > > > > Sean >>>> > > > > > >>>> > > > > >>>> > > > >>>> > > > >>>> > > > >>>> > > > -- >>>> > > > Best regards, >>>> > > > >>>> > > > - Andy >>>> > > > >>>> > > > Problems worthy of attack prove their worth by hitting back. - >>>> Piet >>>> > Hein >>>> > > > (via Tom White) >>>> > > > >>>> > > >>>> > >>>> >>> >>> >>> >>> -- >>> Sean >>> >> >> >> >> -- >> Sean >> > > > > -- > Sean > -- Sean
