Maybe it got reenabled. I've disabled it now. Thanks Sean. St.Ack On Tue, Nov 3, 2015 at 9:50 AM, Sean Busbey <[email protected]> wrote:
> I thought HBase_TRUNK was already disabled. if I missed it, disabling > should be sufficient. There are a few older non-matrix (0.98 I think?) > that we can delete once we're confident in the matrix version > > On Tue, Nov 3, 2015 at 8:44 AM, Stack <[email protected]> wrote: > > Stuff seems to be basically working. I just committed a fix for this > > message so it should be gone now: > > > > grep: unknown devices method > > grep: write error: Broken pipe > > > > > > Hopefully we've seen last of the errant surefire kills. > > > > We have an HBASE_TRUNK build and we have an HBASE_TRUNK_MATRIX. We should > > get rid of the former now? > > > > St.Ack > > > > On Mon, Nov 2, 2015 at 11:03 PM, Stack <[email protected]> wrote: > > > >> I just did an edit of all of our jenkins configuration. I changed the > >> post-build zombie section. The surefire-killer was hanging out here it > >> seems (For detail, see HBASE-14589 Looking for the surefire-killer; > builds > >> being killed...). I may have messed up builds. Will fix in morning if so > >> (currently builds seem a bit backed up so won't wait around). > >> > >> St.Ack > >> > >> On Fri, Oct 30, 2015 at 9:18 AM, Sean Busbey <[email protected]> > wrote: > >> > >>> Hi Folks, > >>> > >>> Brief summary of cleanup/changes in our builds. ref thread '[DISCUSS] > >>> reducing our load on builds.a.o' on reasoning or if you have concerns. > >>> > >>> Our project job view now has a brief summary of what's going on: > >>> > >>> https://builds.apache.org/view/H-L/view/HBase/ > >>> > >>> * Where we had matrix builds, I confirmed they're doing the same thing > >>> (in at least one test-configuration) as the non-matrix version > >>> * I disabled jobs that were duplicating work done in matrix jobs, or > >>> that only worked on a tag > >>> * Made sure all non-disabled 0.98+ jobs are set to notify jira + > >>> builds mailing list > >>> * I updated pre-1.1 jobs to look for changes 1/day > >>> * I updated 1.1 jobs to look for changes every 8 hours > >>> * I updated 1.2, 1.3, and trunk to look for changes every 4 hours > >>> > >>> There's still some non-pressing clean up work to do, so I'll file a > >>> jira later today just to make sure it's all recorded somewhere. > >>> > >>> On Mon, Jun 15, 2015 at 11:55 PM, Sean Busbey <[email protected]> > >>> wrote: > >>> > Added new HBase-1.3 build or branch-1 and changed the existing 1.2 > >>> builds to > >>> > use branch-1.2. (ref HBASE-13912) > >>> > > >>> > On Sun, Jun 14, 2015 at 1:11 AM, Sean Busbey <[email protected]> > >>> wrote: > >>> >> > >>> >> I've add a new build test to run the ITs under sunny day conditions > >>> using > >>> >> minicluster for both java 7 and java 8 on the 1.2 line. > >>> >> > >>> >> https://builds.apache.org/job/HBase-1.2-IT/ > >>> >> > >>> >> I haven't turned on notifications yet, because BulkIngest and > >>> TestIngest > >>> >> are read. details on HBASE-13750 > >>> >> > >>> >> On Sat, Jun 13, 2015 at 8:28 PM, Sean Busbey <[email protected]> > >>> wrote: > >>> >>> > >>> >>> sigh. that should have been "is now a matrix build". > >>> >>> > >>> >>> On Sat, Jun 13, 2015 at 7:31 PM, Sean Busbey <[email protected]> > >>> wrote: > >>> >>>> > >>> >>>> The HBase-1.2 build[1] is not a matrix build that runs JDK7 and > JDK8 > >>> in > >>> >>>> parallel. > >>> >>>> > >>> >>>> I saved the old job for now and left it disabled[2]. > >>> >>>> > >>> >>>> > >>> >>>> [1]: https://builds.apache.org/view/H-L/view/HBase/job/HBase-1.2/ > >>> >>>> [2]: > >>> >>>> > >>> > https://builds.apache.org/view/H-L/view/HBase/job/HBase-1.2%20non-matrix/ > >>> >>>> > >>> >>>> On Fri, Apr 10, 2015 at 3:42 PM, Sean Busbey <[email protected] > > > >>> >>>> wrote: > >>> >>>>> > >>> >>>>> Sorry for the noise. I also updated the checkstyle step on > >>> HBase-TRUNK > >>> >>>>> > >>> >>>>> from > >>> >>>>> mvn checkstyle:checkstyle > >>> >>>>> to > >>> >>>>> mvn -DskipTests package checkstyle:checkstyle > >>> >>>>> > >>> >>>>> to deal with the same issue. Looks all clear now. > >>> >>>>> > >>> >>>>> On Fri, Apr 10, 2015 at 3:03 PM, Sean Busbey < > [email protected]> > >>> >>>>> wrote: > >>> >>>>>> > >>> >>>>>> Updated the following builds: > >>> >>>>>> > >>> >>>>>> * HBase-TRUNK > >>> >>>>>> > >>> >>>>>> moved from > >>> >>>>>> > >>> >>>>>> mvn clean compile findbugs:findbugs > >>> >>>>>> > >>> >>>>>> to > >>> >>>>>> > >>> >>>>>> mvn clean -DskipTests package findbugs:findbugs > >>> >>>>>> > >>> >>>>>> To work around the known issue where we can't do a bootstrap > >>> compile > >>> >>>>>> without previous install or remote SNAPSHOT artifacts. (recently > >>> triggered > >>> >>>>>> by the addition of the procedure module on master) > >>> >>>>>> > >>> >>>>>> On Mon, Mar 30, 2015 at 8:49 PM, 张铎 <[email protected]> > wrote: > >>> >>>>>>> > >>> >>>>>>> Make findbugs and checkstyle plugins always run. > >>> >>>>>>> The default behavior only publishes result for stable builds, > but > >>> >>>>>>> master is > >>> >>>>>>> not always stable and sometimes we introduce new warnings in > >>> unstable > >>> >>>>>>> builds(see builds 6310-6314). > >>> >>>>>>> And findbugs and checkstyle will not fail unless we abort the > >>> >>>>>>> building > >>> >>>>>>> process I think, so it is safe to turn it on every time. > >>> >>>>>>> > >>> >>>>>>> Thanks. > >>> >>>>>>> > >>> >>>>>>> 2015-03-22 22:44 GMT+08:00 Ted Yu <[email protected]>: > >>> >>>>>>> > >>> >>>>>>> > +1 on letting HBase-TRUNK jenkins show coverage report. > >>> >>>>>>> > > >>> >>>>>>> > Cheers > >>> >>>>>>> > > >>> >>>>>>> > On Sun, Mar 22, 2015 at 5:51 AM, 张铎 <[email protected]> > >>> wrote: > >>> >>>>>>> > > >>> >>>>>>> > > Going to change the config of HBase-TRUNK jenkins to show > >>> >>>>>>> > > findbugs, > >>> >>>>>>> > > checkstyle and jacoco coverage report. > >>> >>>>>>> > > The config has been tested on > >>> >>>>>>> > > https://builds.apache.org/job/HBase-TRUNK-jacoco for > nearly > >>> 30 > >>> >>>>>>> > > times. > >>> >>>>>>> > Not > >>> >>>>>>> > > much different from HBase-TRUNK unless it runs ~30% > slower(the > >>> >>>>>>> > > overhead > >>> >>>>>>> > of > >>> >>>>>>> > > collecting information for code coverage). > >>> >>>>>>> > > Thanks. > >>> >>>>>>> > > > >>> >>>>>>> > > 2015-03-12 5:08 GMT+08:00 Andrew Purtell < > [email protected] > >>> >: > >>> >>>>>>> > > > >>> >>>>>>> > > > +1, thanks a lot for improving our build hygiene. > >>> >>>>>>> > > > > >>> >>>>>>> > > > On Wed, Mar 11, 2015 at 2:01 PM, Enis Söztutar > >>> >>>>>>> > > > <[email protected]> > >>> >>>>>>> > > wrote: > >>> >>>>>>> > > > > >>> >>>>>>> > > > > Thanks Sean. This is great. > >>> >>>>>>> > > > > > >>> >>>>>>> > > > > Enis > >>> >>>>>>> > > > > > >>> >>>>>>> > > > > On Wed, Mar 11, 2015 at 1:54 PM, Sean Busbey > >>> >>>>>>> > > > > <[email protected]> > >>> >>>>>>> > > > wrote: > >>> >>>>>>> > > > > > >>> >>>>>>> > > > > > FYI I just finished chasing down the breakage for mvn > >>> site > >>> >>>>>>> > > > > > on all > >>> >>>>>>> > > patch > >>> >>>>>>> > > > > > builds. > >>> >>>>>>> > > > > > > >>> >>>>>>> > > > > > HBASE-13191 consolidates the few places in the > >>> test-patch > >>> >>>>>>> > > > > > code > >>> >>>>>>> > where > >>> >>>>>>> > > we > >>> >>>>>>> > > > > > hard coded MAVEN_OPTS. > >>> >>>>>>> > > > > > > >>> >>>>>>> > > > > > If you look at the PreCommit job now, we use the "set > >>> >>>>>>> > > > > > environment > >>> >>>>>>> > > > > > variables" option to set MAVEN_OPTS and then > everything > >>> >>>>>>> > > > > > else > >>> >>>>>>> > respects > >>> >>>>>>> > > > > that > >>> >>>>>>> > > > > > setting. > >>> >>>>>>> > > > > > > >>> >>>>>>> > > > > > I set the initial value to be a combination of the > >>> memory > >>> >>>>>>> > limitations > >>> >>>>>>> > > > > we've > >>> >>>>>>> > > > > > been actually running with (the ~6G was getting > ignored) > >>> >>>>>>> > > > > > and the > >>> >>>>>>> > > > permgen > >>> >>>>>>> > > > > > needed for site. > >>> >>>>>>> > > > > > > >>> >>>>>>> > > > > > MAVEN_OPTS=-Xmx3100M -XX:-UsePerfData > >>> -XX:MaxPermSize=256m > >>> >>>>>>> > > > > > > >>> >>>>>>> > > > > > On Mon, Feb 23, 2015 at 11:46 PM, Stack < > >>> [email protected]> > >>> >>>>>>> > > > > > wrote: > >>> >>>>>>> > > > > > > >>> >>>>>>> > > > > > > I just made TRUNK and branch-1 builds use same jvm > as > >>> >>>>>>> > > > > > > patch-build > >>> >>>>>>> > > > > > > (hadoopqa) -- i.e. jdku51 -- and I set the > MAVEN_OPTS > >>> to > >>> >>>>>>> > > > > > > be the > >>> >>>>>>> > > same > >>> >>>>>>> > > > as > >>> >>>>>>> > > > > > > those of trunk build too, setting > >>> >>>>>>> > > > > > > MAVEN_OPTS="-Xmx6100m"... it > >>> >>>>>>> > had > >>> >>>>>>> > > > been > >>> >>>>>>> > > > > > > 3000. > >>> >>>>>>> > > > > > > > >>> >>>>>>> > > > > > > Yours, > >>> >>>>>>> > > > > > > St.Ack > >>> >>>>>>> > > > > > > > >>> >>>>>>> > > > > > > On Wed, Dec 31, 2014 at 4:22 PM, Stack < > >>> [email protected]> > >>> >>>>>>> > > > > > > wrote: > >>> >>>>>>> > > > > > > > >>> >>>>>>> > > > > > > > I upped hadoopqa retention to keep last 100 > builds > >>> and > >>> >>>>>>> > > > > > > > or last > >>> >>>>>>> > 7 > >>> >>>>>>> > > > > days, > >>> >>>>>>> > > > > > > > whichever comes first. > >>> >>>>>>> > > > > > > > St.Ack > >>> >>>>>>> > > > > > > > > >>> >>>>>>> > > > > > > > On Tue, Nov 4, 2014 at 9:38 AM, Stack > >>> >>>>>>> > > > > > > > <[email protected]> > >>> >>>>>>> > wrote: > >>> >>>>>>> > > > > > > > > >>> >>>>>>> > > > > > > >> Branch-1 and master have stabilized and now run > >>> mostly > >>> >>>>>>> > > > > > > >> blue > >>> >>>>>>> > > (give > >>> >>>>>>> > > > or > >>> >>>>>>> > > > > > > take > >>> >>>>>>> > > > > > > >> the odd failure) [1][2]. Having a mostly blue > >>> branch-1 > >>> >>>>>>> > > > > > > >> has > >>> >>>>>>> > > helped > >>> >>>>>>> > > > us > >>> >>>>>>> > > > > > > >> identify at least one destabilizing commit in > the > >>> last > >>> >>>>>>> > > > > > > >> few > >>> >>>>>>> > days, > >>> >>>>>>> > > > > maybe > >>> >>>>>>> > > > > > > two; > >>> >>>>>>> > > > > > > >> this is as it should be (smile). > >>> >>>>>>> > > > > > > >> > >>> >>>>>>> > > > > > > >> Lets keep our builds blue. If you commit a > patch, > >>> make > >>> >>>>>>> > > > > > > >> sure > >>> >>>>>>> > > > > subsequent > >>> >>>>>>> > > > > > > >> builds stay blue. You can subscribe to > >>> >>>>>>> > [email protected] > >>> >>>>>>> > > to > >>> >>>>>>> > > > > get > >>> >>>>>>> > > > > > > >> notice of failures if not already subscribed. > >>> >>>>>>> > > > > > > >> > >>> >>>>>>> > > > > > > >> Thanks, > >>> >>>>>>> > > > > > > >> St.Ack > >>> >>>>>>> > > > > > > >> > >>> >>>>>>> > > > > > > >> 1. > >>> >>>>>>> > https://builds.apache.org/view/H-L/view/HBase/job/HBase-1.0/ > >>> >>>>>>> > > > > > > >> 2. > >>> >>>>>>> > > > >>> https://builds.apache.org/view/H-L/view/HBase/job/HBase-TRUNK/ > >>> >>>>>>> > > > > > > >> > >>> >>>>>>> > > > > > > >> > >>> >>>>>>> > > > > > > >> On Mon, Oct 13, 2014 at 4:41 PM, Stack > >>> >>>>>>> > > > > > > >> <[email protected]> > >>> >>>>>>> > > wrote: > >>> >>>>>>> > > > > > > >> > >>> >>>>>>> > > > > > > >>> A few notes on testing. > >>> >>>>>>> > > > > > > >>> > >>> >>>>>>> > > > > > > >>> Too long to read, infra is more capable now and > >>> after > >>> >>>>>>> > > > > > > >>> some > >>> >>>>>>> > > work, > >>> >>>>>>> > > > we > >>> >>>>>>> > > > > > are > >>> >>>>>>> > > > > > > >>> seeing branch-1 and trunk mostly running blue. > >>> Lets > >>> >>>>>>> > > > > > > >>> try and > >>> >>>>>>> > > keep > >>> >>>>>>> > > > it > >>> >>>>>>> > > > > > > this > >>> >>>>>>> > > > > > > >>> way going forward. > >>> >>>>>>> > > > > > > >>> > >>> >>>>>>> > > > > > > >>> Apache Infra has new, more capable hardware. > >>> >>>>>>> > > > > > > >>> > >>> >>>>>>> > > > > > > >>> A recent spurt of test fixing combined with > more > >>> >>>>>>> > > > > > > >>> capable > >>> >>>>>>> > > hardware > >>> >>>>>>> > > > > > seems > >>> >>>>>>> > > > > > > >>> to have gotten us to a new place; tests are > mostly > >>> >>>>>>> > > > > > > >>> passing > >>> >>>>>>> > now > >>> >>>>>>> > > on > >>> >>>>>>> > > > > > > branch-1 > >>> >>>>>>> > > > > > > >>> and master. Lets try and keep it this way and > >>> start > >>> >>>>>>> > > > > > > >>> to trust > >>> >>>>>>> > > our > >>> >>>>>>> > > > > > test > >>> >>>>>>> > > > > > > runs > >>> >>>>>>> > > > > > > >>> again. Just a few flakies remain. Lets try > and > >>> nail > >>> >>>>>>> > > > > > > >>> them. > >>> >>>>>>> > > > > > > >>> > >>> >>>>>>> > > > > > > >>> Our tests now run in parallel with other test > >>> suites > >>> >>>>>>> > > > > > > >>> where > >>> >>>>>>> > > > previous > >>> >>>>>>> > > > > > we > >>> >>>>>>> > > > > > > >>> ran alone. You can see this sometimes when our > >>> zombie > >>> >>>>>>> > detector > >>> >>>>>>> > > > > > reports > >>> >>>>>>> > > > > > > >>> tests from another project altogether as > lingerers > >>> >>>>>>> > > > > > > >>> (To be > >>> >>>>>>> > > fixed). > >>> >>>>>>> > > > > > > Some of > >>> >>>>>>> > > > > > > >>> our tests are failing because a concurrent > hbase > >>> run > >>> >>>>>>> > > > > > > >>> is > >>> >>>>>>> > undoing > >>> >>>>>>> > > > > > > classes and > >>> >>>>>>> > > > > > > >>> data from under it. Also, lets fix. > >>> >>>>>>> > > > > > > >>> > >>> >>>>>>> > > > > > > >>> Our tests are brittle. It takes 75minutes for > >>> them to > >>> >>>>>>> > complete. > >>> >>>>>>> > > > > Many > >>> >>>>>>> > > > > > > >>> are heavy-duty integration tests starting up > >>> multiple > >>> >>>>>>> > clusters > >>> >>>>>>> > > > and > >>> >>>>>>> > > > > > > >>> mapreduce all in the one JVM. It is a miracle > they > >>> >>>>>>> > > > > > > >>> pass at > >>> >>>>>>> > all. > >>> >>>>>>> > > > > > > Usually > >>> >>>>>>> > > > > > > >>> integration tests have been cast as unit tests > >>> >>>>>>> > > > > > > >>> because there > >>> >>>>>>> > > was > >>> >>>>>>> > > > no > >>> >>>>>>> > > > > > > where > >>> >>>>>>> > > > > > > >>> else for them to get an airing. We have the > >>> hbase-it > >>> >>>>>>> > > > > > > >>> suite > >>> >>>>>>> > now > >>> >>>>>>> > > > > which > >>> >>>>>>> > > > > > > would > >>> >>>>>>> > > > > > > >>> be a more apt place but until these are run on > a > >>> >>>>>>> > > > > > > >>> regular > >>> >>>>>>> > basis > >>> >>>>>>> > > in > >>> >>>>>>> > > > > > > public > >>> >>>>>>> > > > > > > >>> for all to see, the fat integration tests > >>> disguised > >>> >>>>>>> > > > > > > >>> as unit > >>> >>>>>>> > > tests > >>> >>>>>>> > > > > > will > >>> >>>>>>> > > > > > > >>> remain. A review of our current unit tests > >>> weeding > >>> >>>>>>> > > > > > > >>> the old > >>> >>>>>>> > > cruft > >>> >>>>>>> > > > > and > >>> >>>>>>> > > > > > > the > >>> >>>>>>> > > > > > > >>> no longer relevant or duplicates would be a > nice > >>> >>>>>>> > > > > > > >>> undertaking > >>> >>>>>>> > if > >>> >>>>>>> > > > > > > someone is > >>> >>>>>>> > > > > > > >>> looking to contribute. > >>> >>>>>>> > > > > > > >>> > >>> >>>>>>> > > > > > > >>> Alex Newman has been working on making our > tests > >>> work > >>> >>>>>>> > > > > > > >>> up on > >>> >>>>>>> > > > travis > >>> >>>>>>> > > > > > and > >>> >>>>>>> > > > > > > >>> circle-ci. That'll be sweet when it goes > >>> end-to-end. > >>> >>>>>>> > > > > > > >>> He > >>> >>>>>>> > also > >>> >>>>>>> > > > > added > >>> >>>>>>> > > > > > in > >>> >>>>>>> > > > > > > >>> some "type" categorizations -- client, filter, > >>> >>>>>>> > > > > > > >>> mapreduce -- > >>> >>>>>>> > > > > alongside > >>> >>>>>>> > > > > > > our > >>> >>>>>>> > > > > > > >>> old "sizing" categorizations of > >>> small/medium/large. > >>> >>>>>>> > > > > > > >>> His > >>> >>>>>>> > > thinking > >>> >>>>>>> > > > > is > >>> >>>>>>> > > > > > > that > >>> >>>>>>> > > > > > > >>> we can run these categorizations in parallel > so we > >>> >>>>>>> > > > > > > >>> could run > >>> >>>>>>> > > the > >>> >>>>>>> > > > > > total > >>> >>>>>>> > > > > > > >>> suite in about the time of the longest test, > say > >>> >>>>>>> > 20-30minutes? > >>> >>>>>>> > > > We > >>> >>>>>>> > > > > > > could > >>> >>>>>>> > > > > > > >>> even change Apache to run them this way. > >>> >>>>>>> > > > > > > >>> > >>> >>>>>>> > > > > > > >>> FYI, > >>> >>>>>>> > > > > > > >>> St.Ack > >>> >>>>>>> > > > > > > >>> > >>> >>>>>>> > > > > > > >>> > >>> >>>>>>> > > > > > > >>> > >>> >>>>>>> > > > > > > >>> > >>> >>>>>>> > > > > > > >>> > >>> >>>>>>> > > > > > > >>> > >>> >>>>>>> > > > > > > >>> > >>> >>>>>>> > > > > > > >> > >>> >>>>>>> > > > > > > > > >>> >>>>>>> > > > > > > > >>> >>>>>>> > > > > > > >>> >>>>>>> > > > > > > >>> >>>>>>> > > > > > > >>> >>>>>>> > > > > > -- > >>> >>>>>>> > > > > > Sean > >>> >>>>>>> > > > > > > >>> >>>>>>> > > > > > >>> >>>>>>> > > > > >>> >>>>>>> > > > > >>> >>>>>>> > > > > >>> >>>>>>> > > > -- > >>> >>>>>>> > > > Best regards, > >>> >>>>>>> > > > > >>> >>>>>>> > > > - Andy > >>> >>>>>>> > > > > >>> >>>>>>> > > > Problems worthy of attack prove their worth by hitting > >>> back. - > >>> >>>>>>> > > > Piet > >>> >>>>>>> > Hein > >>> >>>>>>> > > > (via Tom White) > >>> >>>>>>> > > > > >>> >>>>>>> > > > >>> >>>>>>> > > >>> >>>>>> > >>> >>>>>> > >>> >>>>>> > >>> >>>>>> > >>> >>>>>> -- > >>> >>>>>> Sean > >>> >>>>> > >>> >>>>> > >>> >>>>> > >>> >>>>> > >>> >>>>> -- > >>> >>>>> Sean > >>> >>>> > >>> >>>> > >>> >>>> > >>> >>>> > >>> >>>> -- > >>> >>>> Sean > >>> >>> > >>> >>> > >>> >>> > >>> >>> > >>> >>> -- > >>> >>> Sean > >>> >> > >>> >> > >>> >> > >>> >> > >>> >> -- > >>> >> Sean > >>> > > >>> > > >>> > > >>> > > >>> > -- > >>> > Sean > >>> > >>> > >>> > >>> -- > >>> Sean > >>> > >> > >> > > > > -- > Sean >
