Oh, btw, flaky dashboard job needs to be changed now to use those builds instead. I'll try to give it sometime.
On Wed, Nov 29, 2017 at 1:19 PM, Apekshit Sharma <[email protected]> wrote: > Yeah, i liked that breakup a lot! One look, and you know which part needs > fixing. > fyi: It might take few seconds before the table we are talking about > shows up. > > -- Appy > > On Wed, Nov 29, 2017 at 8:06 AM, Stack <[email protected]> wrote: > >> Example of the new nice reporting: vhttps:// >> builds.apache.org/view/H-L/view/HBase/job/HBase%20Nightly/job/branch-1.2/ >> S >> >> On Wed, Nov 29, 2017 at 8:06 AM, Stack <[email protected]> wrote: >> >> > Note that I have disabled the HBase-1.2-JDK7, HBase-1.2-JDK8, >> > HBase-1.3-JDK7, and HBase-1.3-JDK8 jobs. They have been broken for a >> good >> > while now. In their place, refer to an ongoing Sean "Nightly" project, >> an >> > effort he has been at for a while. It does more checking with pretty >> > reports that will help figuring general stability over time. See under >> > https://builds.apache.org/view/H-L/view/HBase/job/HBase%20Nightly/ >> > See the nightly builds for 1.2 and 1.3. They have some teething issues >> > still but are almost there. See the 1.2 build from last night. In recent >> > days, the 1.2 branch went from trash-can fire to stable. See how all >> tests >> > passed in the last build but then we failed generating the src bundle on >> > the end (this is what I mean by 'teething' issue). Will work on fixing >> this >> > last step and moving over 1.4, etc., in the next few days. >> > >> > FYI, >> > St.Ack >> > >> > >> > On Tue, Nov 7, 2017 at 7:45 AM, Stack <[email protected]> wrote: >> > >> >> On Tue, Nov 7, 2017 at 6:10 AM, Sean Busbey <[email protected]> wrote: >> >> >> >>> > Should I be able to see the machine dir when I look at nightlies >> >>> output? >> >>> > (Was trying to see what else is running). >> >>> >> >>> Ah. we don't have the same machine sampling on nightly as we do in >> >>> precommit. I am 80% on a patch for HBASE-19189 (run test ad-hoc >> >>> repeatedly) that includes pulling that information gathering into a >> >>> place where we could also use it in nightly. >> >>> >> >>> >> >> Sweet. >> >> >> >> >> >> >> >>> Did we ever figure out how many cores we expect our tests to need? It >> >>> looks like the Hadoop nodes have 8 cores. (with 2 executors that means >> >>> 4 is our fair share) >> >>> >> >>> >> >> At the end of the thread inquiry I suggested that we don't use enough >> >> cores, that we could up our fork counts and tests would complete in >> less >> >> time. I wanted to experiment some w/ high fork counts -- 16 or so -- >> to see >> >> if concurrent running brought on more failure. >> >> >> >> St.Ack >> >> >> >> >> >> >> >> >> >>> On Tue, Nov 7, 2017 at 8:05 AM, Sean Busbey <[email protected]> >> wrote: >> >>> > surefire results get zipped up (we were filling the jenkins hosts >> with >> >>> > old test logs previously) and stored in a file called >> "test_logs.zip" >> >>> > for each jvm run. So if that happend in the jdk7 run for branch-1.2, >> >>> > it'd be in artifacts -> output-jdk7 -> test_logs.zip. >> >>> > >> >>> > I don't know if the archival process grabs things from surefire that >> >>> > aren't the surefire XML files, but we can update it to do so if it >> >>> > doesn't. >> >>> > >> >>> > On Mon, Nov 6, 2017 at 11:39 PM, Stack <[email protected]> wrote: >> >>> >> I see this in the 1.2 nightly just when it gives up the ghost.... >> >>> >> >> >>> >> [WARNING] Corrupted STDOUT by directly writing to native stream in >> >>> >> forked JVM 2. See FAQ web page and the dump file >> >>> >> /testptch/hbase/hbase-server/target/surefire-reports/2017-11 >> >>> -06T20-11-30_219-jvmRun2.dumpstream >> >>> >> >> >>> >> .. but the pointed to dumpstream doesn't seem to be around post >> build. >> >>> >> I am looking in wrong place? >> >>> >> >> >>> >> >> >>> >> Thanks, >> >>> >> >> >>> >> S >> >>> >> >> >>> >> >> >>> >> On Mon, Nov 6, 2017 at 8:20 PM, Stack <[email protected]> wrote: >> >>> >> >> >>> >>> On Mon, Nov 6, 2017 at 8:35 AM, Sean Busbey < >> [email protected]> >> >>> wrote: >> >>> >>> >> >>> >>>> Given that all of the old post-commit tests have been posting >> that >> >>> >>>> they're failing to JIRAs for what looks like a month, is there >> any >> >>> >>>> reason not to switch to the new tests that also say they're >> failing? >> >>> >>>> >> >>> >>>> >> >>> >>> No reason. >> >>> >>> >> >>> >>> >> >>> >>> >> >>> >>>> The reason HBASE-18467 has been sitting on hold this whole time >> has >> >>> >>>> been because the new nightly branch tests keep complaining about >> >>> >>>> failures. >> >>> >>>> >> >>> >>>> >> >>> >>> Looking just now, it looks like killed-off test runs. >> >>> >>> >> >>> >>> +1 on move to nightlies. >> >>> >>> >> >>> >>> Can I help? >> >>> >>> >> >>> >>> Should I be able to see the machine dir when I look at nightlies >> >>> output? >> >>> >>> (Was trying to see what else is running). >> >>> >>> >> >>> >>> Thanks Sean, >> >>> >>> St.Ack >> >>> >>> >> >>> >>> >> >>> >>> >> >>> >>> >> >>> >>> >> >>> >>> >> >>> >>>> On Mon, Nov 6, 2017 at 10:21 AM, Sean Busbey < >> [email protected] >> >>> > >> >>> >>>> wrote: >> >>> >>>> > It looks like old tests branch-1.2 and branch-1.3 are failing >> with >> >>> >>>> > some maven enforcer problem that we thought we had fixed a few >> >>> times >> >>> >>>> > before. It's probably fixable by changing the version of maven >> >>> they >> >>> >>>> > use, but I'd much rather any test effort go into the last mile >> of >> >>> >>>> > getting our new nightly tests working. >> >>> >>>> > >> >>> >>>> > I'll start picking this up as soon as I close out HBASE-18784. >> >>> >>>> > >> >>> >>>> > Please consider branch-1.2 release blocked. :( >> >>> >>>> > >> >>> >>>> > On Mon, Nov 6, 2017 at 10:19 AM, Stack <[email protected]> >> wrote: >> >>> >>>> >> Our builds seem pretty sick up on builds.apache.org even >> after >> >>> the >> >>> >>>> miracle >> >>> >>>> >> work by Allen W containing errant hadoop processes. Looking at >> >>> 1.2 and >> >>> >>>> 1.3, >> >>> >>>> >> we don't even get off the ground. Anyone been taking a look? >> >>> >>>> >> >> >>> >>>> >> When I try to run the branch-1.2 and branch-1.3 unit tests >> >>> locally, >> >>> >>>> about >> >>> >>>> >> ten tests or so timeout. Have others tried branch-1 test runs >> >>> recently? >> >>> >>>> >> >> >>> >>>> >> Thanks, >> >>> >>>> >> S >> >>> >>>> >> >> >>> >>>> >> >> >>> >>>> >> On Mon, Aug 21, 2017 at 1:54 PM, Stack <[email protected]> >> wrote: >> >>> >>>> >> >> >>> >>>> >>> Loads of tests timing out in test runs -- then they all pass. >> >>> Anyone >> >>> >>>> have >> >>> >>>> >>> an input? I'm trying to take a look as background task... >> >>> >>>> >>> >> >>> >>>> >>> S >> >>> >>>> >>> >> >>> >>>> >>> On Tue, Jul 11, 2017 at 7:05 PM, Stack <[email protected]> >> >>> wrote: >> >>> >>>> >>> >> >>> >>>> >>>> Thanks Appy. >> >>> >>>> >>>> >> >>> >>>> >>>> Any one looking at the 'ERROR ExecutionException Java heap >> >>> space...' >> >>> >>>> >>>> errors on patch builds or failed forking? Seems common >> enough. >> >>> Here >> >>> >>>> are >> >>> >>>> >>>> complaints that remote JVM went away: >> >>> >>>> >>>> >> >>> >>>> >>>> https://builds.apache.org/view >> /H-L/view/HBase/job/PreCommit- >> >>> >>>> >>>> HBASE-Build/7617/artifact/patc >> hprocess/patch-unit-hbase-serv >> >>> er.txt >> >>> >>>> >>>> https://builds.apache.org/view >> /H-L/view/HBase/job/PreCommit- >> >>> >>>> >>>> HBASE-Build/7616/artifact/patc >> hprocess/patch-unit-hbase-serv >> >>> er.txt >> >>> >>>> >>>> >> >>> >>>> >>>> Then this succeeds.... >> >>> >>>> >>>> >> >>> >>>> >>>> https://builds.apache.org/view >> /H-L/view/HBase/job/PreCommit- >> >>> >>>> >>>> HBASE-Build/7614/artifact/patc >> hprocess/patch-unit-hbase-serv >> >>> er.txt >> >>> >>>> >>>> >> >>> >>>> >>>> And we are good for a while. >> >>> >>>> >>>> >> >>> >>>> >>>> Then heap issues: >> >>> >>>> >>>> >> >>> >>>> >>>> https://builds.apache.org/view >> /H-L/view/HBase/job/PreCommit- >> >>> >>>> >>>> HBASE-Build/7607/artifact/patc >> hprocess/patch-unit-hbase-serv >> >>> er.txt >> >>> >>>> >>>> >> >>> >>>> >>>> Are the zombies back? >> >>> >>>> >>>> >> >>> >>>> >>>> St.Ack >> >>> >>>> >>>> >> >>> >>>> >>>> On Tue, Jul 11, 2017 at 12:33 AM, Apekshit Sharma < >> >>> [email protected] >> >>> >>>> > >> >>> >>>> >>>> wrote: >> >>> >>>> >>>> >> >>> >>>> >>>>> Fixed 'trends' in flaky dashboard. Since i changed the test >> >>> names >> >>> >>>> in last >> >>> >>>> >>>>> fix, the dots in the name were messing up with CSS >> selectors. >> >>> :) >> >>> >>>> >>>>> >> >>> >>>> >>>>> >> >>> >>>> >>>>> On Mon, Jul 10, 2017 at 11:34 AM, Apekshit Sharma < >> >>> >>>> [email protected]> >> >>> >>>> >>>>> wrote: >> >>> >>>> >>>>> >> >>> >>>> >>>>> > Quick update on flaky dashboard: >> >>> >>>> >>>>> > Flaky dashboard wasn't working earlier because our trunk >> >>> build was >> >>> >>>> >>>>> broken. >> >>> >>>> >>>>> > After trunk was fixed, the format of log lines in >> >>> consoleText was >> >>> >>>> not >> >>> >>>> >>>>> the >> >>> >>>> >>>>> > same, so findHangingTests.py was not able to parse it >> >>> correctly >> >>> >>>> for >> >>> >>>> >>>>> > broken/hanging/timeout tests. That's been fixed now >> >>> HBASE-18341 >> >>> >>>> >>>>> > <https://issues.apache.org/jira/browse/HBASE-18341>. >> >>> >>>> >>>>> > Drob brought up in other thread that 'treads' isn't >> >>> working. It's >> >>> >>>> >>>>> probably >> >>> >>>> >>>>> > because i changed tests names (which are used as keys in >> >>> python >> >>> >>>> dicts) >> >>> >>>> >>>>> from >> >>> >>>> >>>>> > just class name to package name+classname (without common >> >>> >>>> >>>>> > org.apache.hadoop.hbase prefix). I had to do it because >> we >> >>> have >> >>> >>>> some >> >>> >>>> >>>>> tests >> >>> >>>> >>>>> > with same class name but in different packages. >> >>> >>>> >>>>> > >> >>> >>>> >>>>> > I'll take a look at it sometime this week (unless someone >> >>> wants to >> >>> >>>> >>>>> take it >> >>> >>>> >>>>> > up and work on this beautiful piece of infra ;) ) >> >>> >>>> >>>>> > >> >>> >>>> >>>>> > >> >>> >>>> >>>>> > On Thu, Jul 6, 2017 at 11:25 PM, Stack <[email protected] >> > >> >>> wrote: >> >>> >>>> >>>>> > >> >>> >>>> >>>>> >> On Thu, Jul 6, 2017 at 3:45 PM, Sean Busbey < >> >>> [email protected]> >> >>> >>>> >>>>> wrote: >> >>> >>>> >>>>> >> >> >>> >>>> >>>>> >> > that sounds like our project structure is broken. >> Please >> >>> make >> >>> >>>> sure >> >>> >>>> >>>>> >> there's >> >>> >>>> >>>>> >> > a jira that tracks it and I'll take a look later. >> >>> >>>> >>>>> >> > >> >>> >>>> >>>>> >> > >> >>> >>>> >>>>> >> >> >>> >>>> >>>>> >> Filed HBASE-18331 for now. >> >>> >>>> >>>>> >> >> >>> >>>> >>>>> >> I can take a look too later. >> >>> >>>> >>>>> >> >> >>> >>>> >>>>> >> St.Ack >> >>> >>>> >>>>> >> >> >>> >>>> >>>>> >> >> >>> >>>> >>>>> >> >> >>> >>>> >>>>> >> > On Thu, Jul 6, 2017 at 6:15 PM, Stack < >> [email protected]> >> >>> >>>> wrote: >> >>> >>>> >>>>> >> > >> >>> >>>> >>>>> >> > > I tried publishing hbase-3.0.0-SNAPSHOT... so >> >>> >>>> hbase-checkstyle >> >>> >>>> >>>>> was up >> >>> >>>> >>>>> >> in >> >>> >>>> >>>>> >> > > repo (presuming it relied on an aged-out snapshot). >> >>> Seems to >> >>> >>>> have >> >>> >>>> >>>>> >> 'fixed' >> >>> >>>> >>>>> >> > > it for now.... >> >>> >>>> >>>>> >> > > >> >>> >>>> >>>>> >> > > St.Ack >> >>> >>>> >>>>> >> > > >> >>> >>>> >>>>> >> > > On Thu, Jul 6, 2017 at 12:50 PM, Stack < >> >>> [email protected]> >> >>> >>>> wrote: >> >>> >>>> >>>>> >> > > >> >>> >>>> >>>>> >> > > > The 3.0.0-SNAPSHOT looks suspicious ... the hbase >> >>> >>>> version.... >> >>> >>>> >>>>> >> > > > St.Ack >> >>> >>>> >>>>> >> > > > >> >>> >>>> >>>>> >> > > > On Thu, Jul 6, 2017 at 12:49 PM, Stack < >> >>> [email protected]> >> >>> >>>> >>>>> wrote: >> >>> >>>> >>>>> >> > > > >> >>> >>>> >>>>> >> > > >> On Thu, Jul 6, 2017 at 12:48 PM, Stack < >> >>> [email protected]> >> >>> >>>> >>>>> wrote: >> >>> >>>> >>>>> >> > > >> >> >>> >>>> >>>>> >> > > >>> Checkstyle is currently broke on our builds... >> >>> looking. >> >>> >>>> >>>>> >> > > >>> St.Ack >> >>> >>>> >>>>> >> > > >>> >> >>> >>>> >>>>> >> > > >>> >> >>> >>>> >>>>> >> > > >> Works if I run it locally (of course) >> >>> >>>> >>>>> >> > > >> St.Ack >> >>> >>>> >>>>> >> > > >> >> >>> >>>> >>>>> >> > > >> >> >>> >>>> >>>>> >> > > >> >> >>> >>>> >>>>> >> > > >> >> >>> >>>> >>>>> >> > > >>> >> >>> >>>> >>>>> >> > > >>> >> >>> >>>> >>>>> >> > > >>> [ERROR] Failed to execute goal >> >>> org.apache.maven.plugins: >> >>> >>>> >>>>> >> > > maven-checkstyle-plugin:2.17:checkstyle >> (default-cli) >> >>> on >> >>> >>>> project >> >>> >>>> >>>>> >> hbase: >> >>> >>>> >>>>> >> > > Execution default-cli of goal >> org.apache.maven.plugins: >> >>> >>>> >>>>> >> > > maven-checkstyle-plugin:2.17:checkstyle failed: >> Plugin >> >>> >>>> >>>>> >> > > org.apache.maven.plugins:maven >> -checkstyle-plugin:2.17 >> >>> or >> >>> >>>> one of >> >>> >>>> >>>>> its >> >>> >>>> >>>>> >> > > dependencies could not be resolved: Could not find >> >>> artifact >> >>> >>>> >>>>> >> > > org.apache.hbase:hbase-checkst >> yle:jar:3.0.0-SNAPSHOT >> >>> in >> >>> >>>> Nexus ( >> >>> >>>> >>>>> >> > > http://repository.apache.org/snapshots) -> [Help >> >>> 1][ERROR] >> >>> >>>> >>>>> [ERROR] To >> >>> >>>> >>>>> >> > see >> >>> >>>> >>>>> >> > > the full stack trace of the errors, re-run Maven >> with >> >>> the -e >> >>> >>>> >>>>> >> > switch.[ERROR] >> >>> >>>> >>>>> >> > > Re-run Maven using the -X switch to enable full >> debug >> >>> >>>> >>>>> logging.[ERROR] >> >>> >>>> >>>>> >> > > [ERROR] For more information about the errors and >> >>> possible >> >>> >>>> >>>>> solutions, >> >>> >>>> >>>>> >> > > please read the following articles:[ERROR] [Help 1] >> >>> >>>> >>>>> >> > > http://cwiki.apache.org/confluence/display/MAVEN/ >> >>> >>>> >>>>> >> > > PluginResolutionExceptionBuild step 'Invoke >> top-level >> >>> Maven >> >>> >>>> >>>>> targets' >> >>> >>>> >>>>> >> > > marked build as failure >> >>> >>>> >>>>> >> > > >>> Performing Post build task... >> >>> >>>> >>>>> >> > > >>> Match found for :.* : True >> >>> >>>> >>>>> >> > > >>> Logical operation result is TRUE >> >>> >>>> >>>>> >> > > >>> Running script : # Run zombie detector script >> >>> >>>> >>>>> >> > > >>> ./dev-support/zombie-detector.sh --jenkins >> >>> ${BUILD_ID} >> >>> >>>> >>>>> >> > > >>> [a3159d73] $ /bin/bash -xe >> >>> /tmp/hudson1697041977582083402 >> >>> >>>> .sh >> >>> >>>> >>>>> >> > > >>> + ./dev-support/zombie-detector.sh --jenkins >> 3320 >> >>> >>>> >>>>> >> > > >>> Thu Jul 6 01:37:09 UTC 2017 We're ok: there is >> no >> >>> >>>> zombie test >> >>> >>>> >>>>> >> > > >>> >> >>> >>>> >>>>> >> > > >>> >> >>> >>>> >>>>> >> > > >>> >> >>> >>>> >>>>> >> > > >>> >> >>> >>>> >>>>> >> > > >>> On Fri, Jun 30, 2017 at 2:43 PM, Sean Busbey < >> >>> >>>> >>>>> [email protected]> >> >>> >>>> >>>>> >> > > wrote: >> >>> >>>> >>>>> >> > > >>> >> >>> >>>> >>>>> >> > > >>>> jacoco was added ages ago. I'd guess that >> >>> something >> >>> >>>> changed >> >>> >>>> >>>>> on >> >>> >>>> >>>>> >> the >> >>> >>>> >>>>> >> > > >>>> machines >> >>> >>>> >>>>> >> > > >>>> we use to cause it to stop working. >> >>> >>>> >>>>> >> > > >>>> >> >>> >>>> >>>>> >> > > >>>> On Thu, Jun 29, 2017 at 12:02 PM, Stack < >> >>> >>>> [email protected]> >> >>> >>>> >>>>> >> wrote: >> >>> >>>> >>>>> >> > > >>>> >> >>> >>>> >>>>> >> > > >>>> > On Wed, Jun 28, 2017 at 8:43 AM, Josh Elser < >> >>> >>>> >>>>> [email protected] >> >>> >>>> >>>>> >> > >> >>> >>>> >>>>> >> > > >>>> wrote: >> >>> >>>> >>>>> >> > > >>>> > >> >>> >>>> >>>>> >> > > >>>> > > >> >>> >>>> >>>>> >> > > >>>> > > >> >>> >>>> >>>>> >> > > >>>> > > On 6/27/17 7:20 PM, Stack wrote: >> >>> >>>> >>>>> >> > > >>>> > > >> >>> >>>> >>>>> >> > > >>>> > >> * test-patch's whitespace plugin can >> >>> configured to >> >>> >>>> >>>>> ignore >> >>> >>>> >>>>> >> some >> >>> >>>> >>>>> >> > > >>>> files >> >>> >>>> >>>>> >> > > >>>> > (but >> >>> >>>> >>>>> >> > > >>>> > >>> I >> >>> >>>> >>>>> >> > > >>>> > >>> can't think of any we'd care to so >> >>> whitelist) >> >>> >>>> >>>>> >> > > >>>> > >>> >> >>> >>>> >>>>> >> > > >>>> > >>> Generated files. >> >>> >>>> >>>>> >> > > >>>> > >> >> >>> >>>> >>>>> >> > > >>>> > > >> >>> >>>> >>>>> >> > > >>>> > > Oh my goodness, yes, please. This has been >> >>> such a >> >>> >>>> pain >> >>> >>>> >>>>> in the >> >>> >>>> >>>>> >> > rear >> >>> >>>> >>>>> >> > > >>>> for me >> >>> >>>> >>>>> >> > > >>>> > > as I've been rebasing space quota patches. >> >>> >>>> Sometimes, the >> >>> >>>> >>>>> >> spaces >> >>> >>>> >>>>> >> > > in >> >>> >>>> >>>>> >> > > >>>> > > pb-gen'ed code are removed by folks before >> >>> commit, >> >>> >>>> other >> >>> >>>> >>>>> >> times >> >>> >>>> >>>>> >> > > they >> >>> >>>> >>>>> >> > > >>>> > aren't. >> >>> >>>> >>>>> >> > > >>>> > > >> >>> >>>> >>>>> >> > > >>>> > >> >>> >>>> >>>>> >> > > >>>> > Agree sir. Its a distraction at least. >> >>> >>>> >>>>> >> > > >>>> > >> >>> >>>> >>>>> >> > > >>>> > I see Jacoco report here now: >> >>> >>>> >>>>> >> > > >>>> > https://builds.apache.org/job/ >> >>> >>>> HBase-Trunk_matrix/jdk=JDK% >> >>> >>>> >>>>> >> > > >>>> > 201.8%20(latest),label=Hadoop/3277/ >> >>> >>>> >>>>> >> > > >>>> > >> >>> >>>> >>>>> >> > > >>>> > Maybe it has been there always and I just >> >>> haven't >> >>> >>>> noticed. >> >>> >>>> >>>>> >> > > >>>> > >> >>> >>>> >>>>> >> > > >>>> > Its all 0%. We need to turn on stuff? >> >>> >>>> >>>>> >> > > >>>> > >> >>> >>>> >>>>> >> > > >>>> > St.Ack >> >>> >>>> >>>>> >> > > >>>> > >> >>> >>>> >>>>> >> > > >>>> >> >>> >>>> >>>>> >> > > >>> >> >>> >>>> >>>>> >> > > >>> >> >>> >>>> >>>>> >> > > >> >> >>> >>>> >>>>> >> > > > >> >>> >>>> >>>>> >> > > >> >>> >>>> >>>>> >> > >> >>> >>>> >>>>> >> >> >>> >>>> >>>>> > >> >>> >>>> >>>>> > >> >>> >>>> >>>>> > >> >>> >>>> >>>>> > -- >> >>> >>>> >>>>> > >> >>> >>>> >>>>> > -- Appy >> >>> >>>> >>>>> > >> >>> >>>> >>>>> >> >>> >>>> >>>>> >> >>> >>>> >>>>> >> >>> >>>> >>>>> -- >> >>> >>>> >>>>> >> >>> >>>> >>>>> -- Appy >> >>> >>>> >>>>> >> >>> >>>> >>>> >> >>> >>>> >>>> >> >>> >>>> >>> >> >>> >>>> > >> >>> >>>> > >> >>> >>>> > >> >>> >>>> > -- >> >>> >>>> > Sean >> >>> >>>> >> >>> >>>> >> >>> >>>> >> >>> >>>> -- >> >>> >>>> Sean >> >>> >>>> >> >>> >>> >> >>> >>> >> >>> >> >> >> >> >> > >> > > > > -- > > -- Appy > -- -- Appy
