Please see this note on hadoop-common about an infra issue that's causing a temporary delay:
*http://s.apache.org/6XU <http://s.apache.org/6XU>* On Fri, Jun 19, 2015 at 7:31 PM, Enis Söztutar <[email protected]> wrote: > It seems that precommit-admin job which triggers the precommit-hbase job is > not running as frequently as it used to be. Only once every ~6 hours or so. > > https://builds.apache.org/job/PreCommit-Admin/ > > The job is starved for the slaves. > > Enis > > On Fri, Jun 19, 2015 at 5:23 PM, Enis Söztutar <[email protected]> wrote: > > > hadoopqa is acting weird for the last couple of days. It does not report > > back the results. One example is: > > https://builds.apache.org/job/PreCommit-HBASE-Build/14475/console. > > > > I've excluded ubuntu3 from the list because it does not have protoc-2.5. > > > > > > On Mon, Jun 15, 2015 at 9:55 PM, Sean Busbey <[email protected]> > wrote: > > > >> Added new HBase-1.3 build or branch-1 and changed the existing 1.2 > builds > >> to use branch-1.2. (ref HBASE-13912) > >> > >> On Sun, Jun 14, 2015 at 1:11 AM, Sean Busbey <[email protected]> > wrote: > >> > >> > I've add a new build test to run the ITs under sunny day conditions > >> using > >> > minicluster for both java 7 and java 8 on the 1.2 line. > >> > > >> > https://builds.apache.org/job/HBase-1.2-IT/ > >> > > >> > I haven't turned on notifications yet, because BulkIngest and > TestIngest > >> > are read. details on HBASE-13750 > >> > > >> > On Sat, Jun 13, 2015 at 8:28 PM, Sean Busbey <[email protected]> > >> wrote: > >> > > >> >> sigh. that should have been "is now a matrix build". > >> >> > >> >> On Sat, Jun 13, 2015 at 7:31 PM, Sean Busbey <[email protected]> > >> wrote: > >> >> > >> >>> The HBase-1.2 build[1] is not a matrix build that runs JDK7 and JDK8 > >> in > >> >>> parallel. > >> >>> > >> >>> I saved the old job for now and left it disabled[2]. > >> >>> > >> >>> > >> >>> [1]: https://builds.apache.org/view/H-L/view/HBase/job/HBase-1.2/ > >> >>> [2]: > >> >>> > >> > https://builds.apache.org/view/H-L/view/HBase/job/HBase-1.2%20non-matrix/ > >> >>> > >> >>> On Fri, Apr 10, 2015 at 3:42 PM, Sean Busbey <[email protected]> > >> >>> wrote: > >> >>> > >> >>>> Sorry for the noise. I also updated the checkstyle step on > >> HBase-TRUNK > >> >>>> > >> >>>> from > >> >>>> mvn checkstyle:checkstyle > >> >>>> to > >> >>>> mvn -DskipTests package checkstyle:checkstyle > >> >>>> > >> >>>> to deal with the same issue. Looks all clear now. > >> >>>> > >> >>>> On Fri, Apr 10, 2015 at 3:03 PM, Sean Busbey <[email protected]> > >> >>>> wrote: > >> >>>> > >> >>>>> Updated the following builds: > >> >>>>> > >> >>>>> * HBase-TRUNK > >> >>>>> > >> >>>>> moved from > >> >>>>> > >> >>>>> mvn clean compile findbugs:findbugs > >> >>>>> > >> >>>>> to > >> >>>>> > >> >>>>> mvn clean -DskipTests package findbugs:findbugs > >> >>>>> > >> >>>>> To work around the known issue where we can't do a bootstrap > compile > >> >>>>> without previous install or remote SNAPSHOT artifacts. (recently > >> triggered > >> >>>>> by the addition of the procedure module on master) > >> >>>>> > >> >>>>> On Mon, Mar 30, 2015 at 8:49 PM, 张铎 <[email protected]> > wrote: > >> >>>>> > >> >>>>>> Make findbugs and checkstyle plugins always run. > >> >>>>>> The default behavior only publishes result for stable builds, but > >> >>>>>> master is > >> >>>>>> not always stable and sometimes we introduce new warnings in > >> unstable > >> >>>>>> builds(see builds 6310-6314). > >> >>>>>> And findbugs and checkstyle will not fail unless we abort the > >> building > >> >>>>>> process I think, so it is safe to turn it on every time. > >> >>>>>> > >> >>>>>> Thanks. > >> >>>>>> > >> >>>>>> 2015-03-22 22:44 GMT+08:00 Ted Yu <[email protected]>: > >> >>>>>> > >> >>>>>> > +1 on letting HBase-TRUNK jenkins show coverage report. > >> >>>>>> > > >> >>>>>> > Cheers > >> >>>>>> > > >> >>>>>> > On Sun, Mar 22, 2015 at 5:51 AM, 张铎 <[email protected]> > >> wrote: > >> >>>>>> > > >> >>>>>> > > Going to change the config of HBase-TRUNK jenkins to show > >> >>>>>> findbugs, > >> >>>>>> > > checkstyle and jacoco coverage report. > >> >>>>>> > > The config has been tested on > >> >>>>>> > > https://builds.apache.org/job/HBase-TRUNK-jacoco for nearly > 30 > >> >>>>>> times. > >> >>>>>> > Not > >> >>>>>> > > much different from HBase-TRUNK unless it runs ~30% > slower(the > >> >>>>>> overhead > >> >>>>>> > of > >> >>>>>> > > collecting information for code coverage). > >> >>>>>> > > Thanks. > >> >>>>>> > > > >> >>>>>> > > 2015-03-12 5:08 GMT+08:00 Andrew Purtell < > [email protected] > >> >: > >> >>>>>> > > > >> >>>>>> > > > +1, thanks a lot for improving our build hygiene. > >> >>>>>> > > > > >> >>>>>> > > > On Wed, Mar 11, 2015 at 2:01 PM, Enis Söztutar < > >> >>>>>> [email protected]> > >> >>>>>> > > wrote: > >> >>>>>> > > > > >> >>>>>> > > > > Thanks Sean. This is great. > >> >>>>>> > > > > > >> >>>>>> > > > > Enis > >> >>>>>> > > > > > >> >>>>>> > > > > On Wed, Mar 11, 2015 at 1:54 PM, Sean Busbey < > >> >>>>>> [email protected]> > >> >>>>>> > > > wrote: > >> >>>>>> > > > > > >> >>>>>> > > > > > FYI I just finished chasing down the breakage for mvn > >> site > >> >>>>>> on all > >> >>>>>> > > patch > >> >>>>>> > > > > > builds. > >> >>>>>> > > > > > > >> >>>>>> > > > > > HBASE-13191 consolidates the few places in the > test-patch > >> >>>>>> code > >> >>>>>> > where > >> >>>>>> > > we > >> >>>>>> > > > > > hard coded MAVEN_OPTS. > >> >>>>>> > > > > > > >> >>>>>> > > > > > If you look at the PreCommit job now, we use the "set > >> >>>>>> environment > >> >>>>>> > > > > > variables" option to set MAVEN_OPTS and then everything > >> else > >> >>>>>> > respects > >> >>>>>> > > > > that > >> >>>>>> > > > > > setting. > >> >>>>>> > > > > > > >> >>>>>> > > > > > I set the initial value to be a combination of the > memory > >> >>>>>> > limitations > >> >>>>>> > > > > we've > >> >>>>>> > > > > > been actually running with (the ~6G was getting > ignored) > >> >>>>>> and the > >> >>>>>> > > > permgen > >> >>>>>> > > > > > needed for site. > >> >>>>>> > > > > > > >> >>>>>> > > > > > MAVEN_OPTS=-Xmx3100M -XX:-UsePerfData > >> -XX:MaxPermSize=256m > >> >>>>>> > > > > > > >> >>>>>> > > > > > On Mon, Feb 23, 2015 at 11:46 PM, Stack < > >> [email protected]> > >> >>>>>> wrote: > >> >>>>>> > > > > > > >> >>>>>> > > > > > > I just made TRUNK and branch-1 builds use same jvm as > >> >>>>>> patch-build > >> >>>>>> > > > > > > (hadoopqa) -- i.e. jdku51 -- and I set the MAVEN_OPTS > >> to > >> >>>>>> be the > >> >>>>>> > > same > >> >>>>>> > > > as > >> >>>>>> > > > > > > those of trunk build too, setting > >> >>>>>> MAVEN_OPTS="-Xmx6100m"... it > >> >>>>>> > had > >> >>>>>> > > > been > >> >>>>>> > > > > > > 3000. > >> >>>>>> > > > > > > > >> >>>>>> > > > > > > Yours, > >> >>>>>> > > > > > > St.Ack > >> >>>>>> > > > > > > > >> >>>>>> > > > > > > On Wed, Dec 31, 2014 at 4:22 PM, Stack < > >> [email protected]> > >> >>>>>> wrote: > >> >>>>>> > > > > > > > >> >>>>>> > > > > > > > I upped hadoopqa retention to keep last 100 builds > >> and > >> >>>>>> or last > >> >>>>>> > 7 > >> >>>>>> > > > > days, > >> >>>>>> > > > > > > > whichever comes first. > >> >>>>>> > > > > > > > St.Ack > >> >>>>>> > > > > > > > > >> >>>>>> > > > > > > > On Tue, Nov 4, 2014 at 9:38 AM, Stack < > >> [email protected] > >> >>>>>> > > >> >>>>>> > wrote: > >> >>>>>> > > > > > > > > >> >>>>>> > > > > > > >> Branch-1 and master have stabilized and now run > >> mostly > >> >>>>>> blue > >> >>>>>> > > (give > >> >>>>>> > > > or > >> >>>>>> > > > > > > take > >> >>>>>> > > > > > > >> the odd failure) [1][2]. Having a mostly blue > >> branch-1 > >> >>>>>> has > >> >>>>>> > > helped > >> >>>>>> > > > us > >> >>>>>> > > > > > > >> identify at least one destabilizing commit in the > >> last > >> >>>>>> few > >> >>>>>> > days, > >> >>>>>> > > > > maybe > >> >>>>>> > > > > > > two; > >> >>>>>> > > > > > > >> this is as it should be (smile). > >> >>>>>> > > > > > > >> > >> >>>>>> > > > > > > >> Lets keep our builds blue. If you commit a patch, > >> make > >> >>>>>> sure > >> >>>>>> > > > > subsequent > >> >>>>>> > > > > > > >> builds stay blue. You can subscribe to > >> >>>>>> > [email protected] > >> >>>>>> > > to > >> >>>>>> > > > > get > >> >>>>>> > > > > > > >> notice of failures if not already subscribed. > >> >>>>>> > > > > > > >> > >> >>>>>> > > > > > > >> Thanks, > >> >>>>>> > > > > > > >> St.Ack > >> >>>>>> > > > > > > >> > >> >>>>>> > > > > > > >> 1. > >> >>>>>> > https://builds.apache.org/view/H-L/view/HBase/job/HBase-1.0/ > >> >>>>>> > > > > > > >> 2. > >> >>>>>> > > > https://builds.apache.org/view/H-L/view/HBase/job/HBase-TRUNK/ > >> >>>>>> > > > > > > >> > >> >>>>>> > > > > > > >> > >> >>>>>> > > > > > > >> On Mon, Oct 13, 2014 at 4:41 PM, Stack < > >> >>>>>> [email protected]> > >> >>>>>> > > wrote: > >> >>>>>> > > > > > > >> > >> >>>>>> > > > > > > >>> A few notes on testing. > >> >>>>>> > > > > > > >>> > >> >>>>>> > > > > > > >>> Too long to read, infra is more capable now and > >> after > >> >>>>>> some > >> >>>>>> > > work, > >> >>>>>> > > > we > >> >>>>>> > > > > > are > >> >>>>>> > > > > > > >>> seeing branch-1 and trunk mostly running blue. > Lets > >> >>>>>> try and > >> >>>>>> > > keep > >> >>>>>> > > > it > >> >>>>>> > > > > > > this > >> >>>>>> > > > > > > >>> way going forward. > >> >>>>>> > > > > > > >>> > >> >>>>>> > > > > > > >>> Apache Infra has new, more capable hardware. > >> >>>>>> > > > > > > >>> > >> >>>>>> > > > > > > >>> A recent spurt of test fixing combined with more > >> >>>>>> capable > >> >>>>>> > > hardware > >> >>>>>> > > > > > seems > >> >>>>>> > > > > > > >>> to have gotten us to a new place; tests are > mostly > >> >>>>>> passing > >> >>>>>> > now > >> >>>>>> > > on > >> >>>>>> > > > > > > branch-1 > >> >>>>>> > > > > > > >>> and master. Lets try and keep it this way and > >> start > >> >>>>>> to trust > >> >>>>>> > > our > >> >>>>>> > > > > > test > >> >>>>>> > > > > > > runs > >> >>>>>> > > > > > > >>> again. Just a few flakies remain. Lets try and > >> nail > >> >>>>>> them. > >> >>>>>> > > > > > > >>> > >> >>>>>> > > > > > > >>> Our tests now run in parallel with other test > >> suites > >> >>>>>> where > >> >>>>>> > > > previous > >> >>>>>> > > > > > we > >> >>>>>> > > > > > > >>> ran alone. You can see this sometimes when our > >> zombie > >> >>>>>> > detector > >> >>>>>> > > > > > reports > >> >>>>>> > > > > > > >>> tests from another project altogether as > lingerers > >> >>>>>> (To be > >> >>>>>> > > fixed). > >> >>>>>> > > > > > > Some of > >> >>>>>> > > > > > > >>> our tests are failing because a concurrent hbase > >> run > >> >>>>>> is > >> >>>>>> > undoing > >> >>>>>> > > > > > > classes and > >> >>>>>> > > > > > > >>> data from under it. Also, lets fix. > >> >>>>>> > > > > > > >>> > >> >>>>>> > > > > > > >>> Our tests are brittle. It takes 75minutes for > them > >> to > >> >>>>>> > complete. > >> >>>>>> > > > > Many > >> >>>>>> > > > > > > >>> are heavy-duty integration tests starting up > >> multiple > >> >>>>>> > clusters > >> >>>>>> > > > and > >> >>>>>> > > > > > > >>> mapreduce all in the one JVM. It is a miracle > they > >> >>>>>> pass at > >> >>>>>> > all. > >> >>>>>> > > > > > > Usually > >> >>>>>> > > > > > > >>> integration tests have been cast as unit tests > >> >>>>>> because there > >> >>>>>> > > was > >> >>>>>> > > > no > >> >>>>>> > > > > > > where > >> >>>>>> > > > > > > >>> else for them to get an airing. We have the > >> hbase-it > >> >>>>>> suite > >> >>>>>> > now > >> >>>>>> > > > > which > >> >>>>>> > > > > > > would > >> >>>>>> > > > > > > >>> be a more apt place but until these are run on a > >> >>>>>> regular > >> >>>>>> > basis > >> >>>>>> > > in > >> >>>>>> > > > > > > public > >> >>>>>> > > > > > > >>> for all to see, the fat integration tests > disguised > >> >>>>>> as unit > >> >>>>>> > > tests > >> >>>>>> > > > > > will > >> >>>>>> > > > > > > >>> remain. A review of our current unit tests > weeding > >> >>>>>> the old > >> >>>>>> > > cruft > >> >>>>>> > > > > and > >> >>>>>> > > > > > > the > >> >>>>>> > > > > > > >>> no longer relevant or duplicates would be a nice > >> >>>>>> undertaking > >> >>>>>> > if > >> >>>>>> > > > > > > someone is > >> >>>>>> > > > > > > >>> looking to contribute. > >> >>>>>> > > > > > > >>> > >> >>>>>> > > > > > > >>> Alex Newman has been working on making our tests > >> work > >> >>>>>> up on > >> >>>>>> > > > travis > >> >>>>>> > > > > > and > >> >>>>>> > > > > > > >>> circle-ci. That'll be sweet when it goes > >> >>>>>> end-to-end. He > >> >>>>>> > also > >> >>>>>> > > > > added > >> >>>>>> > > > > > in > >> >>>>>> > > > > > > >>> some "type" categorizations -- client, filter, > >> >>>>>> mapreduce -- > >> >>>>>> > > > > alongside > >> >>>>>> > > > > > > our > >> >>>>>> > > > > > > >>> old "sizing" categorizations of > small/medium/large. > >> >>>>>> His > >> >>>>>> > > thinking > >> >>>>>> > > > > is > >> >>>>>> > > > > > > that > >> >>>>>> > > > > > > >>> we can run these categorizations in parallel so > we > >> >>>>>> could run > >> >>>>>> > > the > >> >>>>>> > > > > > total > >> >>>>>> > > > > > > >>> suite in about the time of the longest test, say > >> >>>>>> > 20-30minutes? > >> >>>>>> > > > We > >> >>>>>> > > > > > > could > >> >>>>>> > > > > > > >>> even change Apache to run them this way. > >> >>>>>> > > > > > > >>> > >> >>>>>> > > > > > > >>> FYI, > >> >>>>>> > > > > > > >>> St.Ack > >> >>>>>> > > > > > > >>> > >> >>>>>> > > > > > > >>> > >> >>>>>> > > > > > > >>> > >> >>>>>> > > > > > > >>> > >> >>>>>> > > > > > > >>> > >> >>>>>> > > > > > > >>> > >> >>>>>> > > > > > > >>> > >> >>>>>> > > > > > > >> > >> >>>>>> > > > > > > > > >> >>>>>> > > > > > > > >> >>>>>> > > > > > > >> >>>>>> > > > > > > >> >>>>>> > > > > > > >> >>>>>> > > > > > -- > >> >>>>>> > > > > > Sean > >> >>>>>> > > > > > > >> >>>>>> > > > > > >> >>>>>> > > > > >> >>>>>> > > > > >> >>>>>> > > > > >> >>>>>> > > > -- > >> >>>>>> > > > Best regards, > >> >>>>>> > > > > >> >>>>>> > > > - Andy > >> >>>>>> > > > > >> >>>>>> > > > Problems worthy of attack prove their worth by hitting > back. > >> - > >> >>>>>> Piet > >> >>>>>> > Hein > >> >>>>>> > > > (via Tom White) > >> >>>>>> > > > > >> >>>>>> > > > >> >>>>>> > > >> >>>>>> > >> >>>>> > >> >>>>> > >> >>>>> > >> >>>>> -- > >> >>>>> Sean > >> >>>>> > >> >>>> > >> >>>> > >> >>>> > >> >>>> -- > >> >>>> Sean > >> >>>> > >> >>> > >> >>> > >> >>> > >> >>> -- > >> >>> Sean > >> >>> > >> >> > >> >> > >> >> > >> >> -- > >> >> Sean > >> >> > >> > > >> > > >> > > >> > -- > >> > Sean > >> > > >> > >> > >> > >> -- > >> Sean > >> > > > > > -- Sean
