Fun times.
1) Turns out our 1.3-IT jobs have been running against branch-1.2.
Don't know how long, but as long as we have history.
2) I deleted the failing-since-august 1.2-IT job.
3) I renamed the passing 1.3-IT job that runs against branch-1.2 to be
the 1.2-IT job
4) I copied the now renamed 1.2-IT job and made a 1.3-IT job that runs
against branch-1.3
I kicked off jobs after all this shuffling. We'll see how it goes.
On Tue, Feb 21, 2017 at 5:49 PM, Sean Busbey <bus...@apache.org> wrote:
> FYI, I updated the 1.2-IT and 1.3-IT jobs today to use Appy's
> suggested "custom child workspace" of "${SHORT_COMBINATION}", since
> spaces in paths had caused them to fail for a v long time.
>
> On Fri, Oct 14, 2016 at 4:44 PM, Andrew Purtell <apurt...@apache.org> wrote:
>> Thanks Ted, that would be a nice contribution, thank you.
>>
>>
>> On Fri, Oct 14, 2016 at 12:07 PM, Apekshit Sharma <a...@cloudera.com> wrote:
>>
>>> @Ted, here's the old jira, HBASE-14167. Use that.
>>>
>>> On Fri, Oct 14, 2016 at 12:02 PM, Ted Yu <yuzhih...@gmail.com> wrote:
>>>
>>> > I just ran the tests in hbase-spark module using 'mvn verify'.
>>> >
>>> > All passed.
>>> >
>>> > I am testing a patch locally where hbase-spark tests are run in test
>>> phase.
>>> >
>>> > If the tests pass, I will log a JIRA.
>>> >
>>> > Thanks
>>> >
>>> > > On Oct 14, 2016, at 11:41 AM, Andrew Purtell <apurt...@apache.org>
>>> > wrote:
>>> > >
>>> > > The hbase-spark integration tests run (and fail) for me locally
>>> whenever
>>> > I
>>> > > build master with 'mvn clean install -DskipITs' .
>>> > >
>>> > > HBaseConnectionCacheSuite:
>>> > > - all test cases *** FAILED ***
>>> > > 2 did not equal 1 (HBaseConnectionCacheSuite.scala:92)
>>> > >
>>> > > Saw it but had to ignore/triage to get something else done.
>>> > >
>>> > > We have a weird situation where integration tests run when they
>>> shouldn't
>>> > > locally yet no tests run at all for patch process?
>>> > >
>>> > > I would like to see Spark behave like the other modules. I remember
>>> > filing
>>> > > a JIRA asking that hbase-spark honor -DskipITs. It still doesn't.
>>> > > Meanwhile, it does its own thing with '-DskipSparkTests', which is not
>>> > > appropriate given that none of the other modules have their own
>>> distinct
>>> > > control parameters. There also doesn't seem to be a distinction between
>>> > > unit tests and integration tests. The 'test' target does nothing.
>>> > > Everything happens during the 'integration-test' phase. Is this a Spark
>>> > > limitation?
>>> > >
>>> > >
>>> > >> On Fri, Oct 14, 2016 at 11:27 AM, Sean Busbey <bus...@cloudera.com>
>>> > wrote:
>>> > >>
>>> > >> Do the HBase Spark tests only run during the maven verify command?
>>> > >> We'll need to update our personality to say that that command should
>>> > >> be used for unit tests when in the hbase spark module. ugh.
>>> > >>
>>> > >> On Thu, Oct 13, 2016 at 7:42 PM, Apekshit Sharma <a...@cloudera.com>
>>> > >> wrote:
>>> > >>> Our patch process isn't running hbase-spark tests. See this for
>>> > example:
>>> > >>>
>>> > >>> https://builds.apache.org/job/PreCommit-HBASE-Build/3842/
>>> > >>> https://builds.apache.org/job/PreCommit-HBASE-Build/3842/
>>> > >> artifact/patchprocess/patch-unit-hbase-spark.txt/*view*/
>>> > >>>
>>> > >>> Found it when trying to debug cause of trunk failures. Part of the
>>> > cause
>>> > >> is
>>> > >>> hbase-spark's HBaseConnectionCacheSuite test failure (
>>> > >>> https://builds.apache.org/view/All/job/HBase-Trunk_
>>> > >> matrix/jdk=JDK%201.8%20(latest),label=yahoo-not-h2/1776/consoleFull
>>> > >> <https://builds.apache.org/view/All/job/HBase-Trunk_
>>> > matrix/jdk=JDK%201.8%20%28latest%29,label=yahoo-not-h2/1776/consoleFull>
>>> > >> )
>>> > >>> which was added in HBASE-16638. However, to be fair, QA was green and
>>> > >>> reported passing hbase-spark tests for that jira.
>>> > >>>
>>> > >>>> On Mon, Sep 19, 2016 at 12:57 PM, Stack <st...@duboce.net> wrote:
>>> > >>>>
>>> > >>>> childCustomWorkspace seems to be just the ticket. Nice find Appy.
>>> > >>>> St.Ack
>>> > >>>>
>>> > >>>> On Mon, Sep 19, 2016 at 10:03 AM, Sean Busbey <bus...@cloudera.com>
>>> > >> wrote:
>>> > >>>>
>>> > >>>>> Option 2c looks to be working really well. Thanks for tackling this
>>> > >> Appy!
>>> > >>>>>
>>> > >>>>> We still have some failures on the master build, but it looks like
>>> > >>>>> actual problems (or perhaps a flakey). There are several passing
>>> > >>>>> builds.
>>> > >>>>>
>>> > >>>>> This should be pretty easy to replicate on the other jobs. I don't
>>> > see
>>> > >>>>> a downside. Anyone else have concerns?
>>> > >>>>>
>>> > >>>>>
>>> > >>>>> On Fri, Sep 16, 2016 at 6:15 PM, Apekshit Sharma <
>>> a...@cloudera.com>
>>> > >>>>> wrote:
>>> > >>>>>> So this all started with spaces-in-path issue, right? I think it
>>> > >> has
>>> > >>>>>> gobbled up a lot of time of a lot of people.
>>> > >>>>>> Let's discuss our options and try to fix it for good. Here are
>>> what
>>> > >> i
>>> > >>>> can
>>> > >>>>>> think of, and my opinion about them.
>>> > >>>>>>
>>> > >>>>>> 1. Not use matrix build
>>> > >>>>>> Temporary fix. Not preferred since not applicable to
>>> other
>>> > >>>>>> branches' builds.
>>> > >>>>>>
>>> > >>>>>> 2. Use matrix build
>>> > >>>>>>
>>> > >>>>>> a. Use tool environment trick
>>> > >>>>>> I applied this few days ago. Seemed to work until we
>>> > >>>>> discovered
>>> > >>>>>> scalatest issue. While the solution looks legitimate, we can't
>>> trust
>>> > >>>> that
>>> > >>>>>> all tools will use JAVA_HOME instead of directly using java
>>> command.
>>> > >>>>>>
>>> > >>>>>> b. Use JDK axix
>>> > >>>>>> Doesn't work right now. I don't have much idea of what's
>>> > >> the
>>> > >>>>> cost
>>> > >>>>>> for fixing it.
>>> > >>>>>>
>>> > >>>>>> c. Use JDK axis with custom child workspace
>>> > >>>>>>
>>> > >>>>>> https://github.com/jenkinsci/matrix-project-plugin/blob/
>>> > >>>>> master/src/main/resources/hudson/matrix/MatrixProject/
>>> > >>>>> help-childCustomWorkspace.html
>>> > >>>>>> Just found this one, and it might solve things for good.
>>> I
>>> > >>>> have
>>> > >>>>>> updated the job to use this. Let's see how it works.
>>> > >>>>>>
>>> > >>>>>> What do others think?
>>> > >>>>>>
>>> > >>>>>>> On Fri, Sep 16, 2016 at 3:31 PM, Stack <st...@duboce.net> wrote:
>>> > >>>>>>>
>>> > >>>>>>> The profile (or define) skipSparkTests looks like it will skip
>>> > >> spark
>>> > >>>>> tests.
>>> > >>>>>>> Setting skipIntegrationTests to true will skip it.
>>> > >>>>>>> S
>>> > >>>>>>>
>>> > >>>>>>> On Fri, Sep 16, 2016 at 1:40 PM, Dima Spivak <
>>> > >> dimaspi...@apache.org>
>>> > >>>>>>> wrote:
>>> > >>>>>>>
>>> > >>>>>>>> Doesn't seem we need a matrix project for master anymore since
>>> > >> we're
>>> > >>>>> just
>>> > >>>>>>>> doing JDK 8 now, right? Also, it looks like the hbase-spark
>>> > >>>>>>>> integration-test phase is what's tripping up the build. Why not
>>> > >> just
>>> > >>>>>>>> temporarily disable that to unblock testing?
>>> > >>>>>>>>
>>> > >>>>>>>> On Friday, September 16, 2016, Apekshit Sharma <
>>> > >> a...@cloudera.com>
>>> > >>>>>>> wrote:
>>> > >>>>>>>>
>>> > >>>>>>>>> So the issue is, we set JAVA_HOME to jdk8 based on matrix param
>>> > >>>> and
>>> > >>>>>>> using
>>> > >>>>>>>>> tool environment. Since mvn uses the env variable, it compiles
>>> > >>>> with
>>> > >>>>> jdk
>>> > >>>>>>>> 8.
>>> > >>>>>>>>> But i suspect that scalatest isn't using the env variable,
>>> > >> instead
>>> > >>>>> it
>>> > >>>>>>>> might
>>> > >>>>>>>>> be directly using 'java' cmd, which can be jdk 7 or 8, and can
>>> > >>>> vary
>>> > >>>>> by
>>> > >>>>>>>>> machine.
>>> > >>>>>>>>> Build succeed if 'java' points to jdk 8, otherwise fails.
>>> > >>>>>>>>> Note that we didn't have this issue earlier since we were using
>>> > >>>>> jenkins
>>> > >>>>>>>>> 'JDK' axis which would set the 'java' to the appropriate
>>> > >> version.
>>> > >>>>> But
>>> > >>>>>>>> that
>>> > >>>>>>>>> methods had spaces-in-path issue, so i had to change it.
>>> > >>>>>>>>>
>>> > >>>>>>>>>
>>> > >>>>>>>>> On Fri, Sep 16, 2016 at 3:46 AM, aman poonia <
>>> > >>>>> aman.poonia...@gmail.com
>>> > >>>>>>>>> <javascript:;>>
>>> > >>>>>>>>> wrote:
>>> > >>>>>>>>>
>>> > >>>>>>>>>> I am not sure if this will help. But it looks like it is
>>> > >> because
>>> > >>>>> of
>>> > >>>>>>>>> version
>>> > >>>>>>>>>> mismatch, that is, it is expecting JDK1.7 and we are
>>> > >> compiling
>>> > >>>>> with
>>> > >>>>>>>>> jdk1.8.
>>> > >>>>>>>>>> That means there is some library which has to be compiled
>>> > >> with
>>> > >>>>> jdk8
>>> > >>>>>>> or
>>> > >>>>>>>>>> needs to be updated to a jdk8 compatible version.
>>> > >>>>>>>>>>
>>> > >>>>>>>>>>
>>> > >>>>>>>>>> --
>>> > >>>>>>>>>> *With Regards:-*
>>> > >>>>>>>>>> *Aman Poonia*
>>> > >>>>>>>>>>
>>> > >>>>>>>>>> On Fri, Sep 16, 2016 at 2:40 AM, Apekshit Sharma <
>>> > >>>>> a...@cloudera.com
>>> > >>>>>>>>> <javascript:;>>
>>> > >>>>>>>>>> wrote:
>>> > >>>>>>>>>>
>>> > >>>>>>>>>>> And....everything is back to red.
>>> > >>>>>>>>>>> Because something is plaguing our builds again. :(
>>> > >>>>>>>>>>>
>>> > >>>>>>>>>>> If anyone knows what's problem in this case, please reply
>>> > >> on
>>> > >>>>> this
>>> > >>>>>>>>> thread,
>>> > >>>>>>>>>>> otherwise i'll try to fix it later sometime today.
>>> > >>>>>>>>>>>
>>> > >>>>>>>>>>> [INFO] *--- scalatest-maven-plugin:1.0:test
>>> > >>>> (integration-test)
>>> > >>>>> @
>>> > >>>>>>>>>>> hbase-spark ---
>>> > >>>>>>>>>>> * [36mDiscovery starting. [0m
>>> > >>>>>>>>>>> [31m*** RUN ABORTED *** [0m
>>> > >>>>>>>>>>> [31m java.lang.UnsupportedClassVersionError:
>>> > >>>>>>>>>>> org/apache/hadoop/hbase/spark/example/hbasecontext/
>>> > >>>>>>>>>>> JavaHBaseDistributedScan
>>> > >>>>>>>>>>> : Unsupported major.minor version 52.0 [0m
>>> > >>>>>>>>>>> [31m at java.lang.ClassLoader.defineClass1(Native
>>> > >> Method)
>>> > >>>> [0m
>>> > >>>>>>>>>>> [31m at java.lang.ClassLoader.
>>> > >> defineClass(ClassLoader.java:
>>> > >>>>> 803)
>>> > >>>>>>>> [0m
>>> > >>>>>>>>>>> [31m at java.security.SecureClassLoader.defineClass(
>>> > >>>>>>>>>>> SecureClassLoader.java:142)
>>> > >>>>>>>>>>> [0m
>>> > >>>>>>>>>>> [31m at java.net.URLClassLoader.
>>> > >> defineClass(URLClassLoader.
>>> > >>>>>>>> java:449)
>>> > >>>>>>>>>> [0m
>>> > >>>>>>>>>>> [31m at java.net.URLClassLoader.
>>> > >> access$100(URLClassLoader.
>>> > >>>>>>> java:71)
>>> > >>>>>>>>> [0m
>>> > >>>>>>>>>>> [31m at java.net.URLClassLoader$1.run(
>>> > >>>>> URLClassLoader.java:361)
>>> > >>>>>>> [0m
>>> > >>>>>>>>>>> [31m at java.net.URLClassLoader$1.run(
>>> > >>>>> URLClassLoader.java:355)
>>> > >>>>>>> [0m
>>> > >>>>>>>>>>> [31m at java.security.AccessController.doPrivileged(
>>> > >> Native
>>> > >>>>>>> Method)
>>> > >>>>>>>>> [0m
>>> > >>>>>>>>>>> [31m at java.net.URLClassLoader.
>>> > >>>>> findClass(URLClassLoader.java:
>>> > >>>>>>> 354)
>>> > >>>>>>>>> [0m
>>> > >>>>>>>>>>> [31m at java.lang.ClassLoader.
>>> > >> loadClass(ClassLoader.java:
>>> > >>>> 425)
>>> > >>>>>>> [0m
>>> > >>>>>>>>>>>
>>> > >>>>>>>>>>>
>>> > >>>>>>>>>>>
>>> > >>>>>>>>>>> On Mon, Sep 12, 2016 at 5:01 PM, Mikhail Antonov <
>>> > >>>>>>>> olorinb...@gmail.com
>>> > >>>>>>>>> <javascript:;>>
>>> > >>>>>>>>>>> wrote:
>>> > >>>>>>>>>>>
>>> > >>>>>>>>>>>> Great work indeed!
>>> > >>>>>>>>>>>>
>>> > >>>>>>>>>>>> Agreed, occasional failed runs may not be that bad, but
>>> > >>>> fairly
>>> > >>>>>>>>> regular
>>> > >>>>>>>>>>>> failed runs ruin the idea of CI. Especially for released
>>> > >> or
>>> > >>>>>>>> otherwise
>>> > >>>>>>>>>>>> supposedly stable branches.
>>> > >>>>>>>>>>>>
>>> > >>>>>>>>>>>> -Mikhail
>>> > >>>>>>>>>>>>
>>> > >>>>>>>>>>>> On Mon, Sep 12, 2016 at 4:53 PM, Sean Busbey <
>>> > >>>>>>> bus...@cloudera.com
>>> > >>>>>>>>> <javascript:;>>
>>> > >>>>>>>>>>> wrote:
>>> > >>>>>>>>>>>>
>>> > >>>>>>>>>>>>> awesome work Appy!
>>> > >>>>>>>>>>>>>
>>> > >>>>>>>>>>>>> That's certainly good news to hear.
>>> > >>>>>>>>>>>>>
>>> > >>>>>>>>>>>>> On Mon, Sep 12, 2016 at 2:14 PM, Apekshit Sharma <
>>> > >>>>>>>>> a...@cloudera.com <javascript:;>>
>>> > >>>>>>>>>>>>> wrote:
>>> > >>>>>>>>>>>>>> On a separate note:
>>> > >>>>>>>>>>>>>> Trunk had 8 green runs in last 3 days! (
>>> > >>>>>>>>>>>>>> https://builds.apache.org/job/HBase-Trunk_matrix/)
>>> > >>>>>>>>>>>>>> This was due to fixing just the mass failures on
>>> > >> trunk
>>> > >>>>> and no
>>> > >>>>>>>>>> change
>>> > >>>>>>>>>>> in
>>> > >>>>>>>>>>>>>> flaky infra. Which made me to conclude two things:
>>> > >>>>>>>>>>>>>> 1. Flaky infra works.
>>> > >>>>>>>>>>>>>> 2. It relies heavily on the post-commit build's
>>> > >>>> stability
>>> > >>>>>>>> (which
>>> > >>>>>>>>>>> every
>>> > >>>>>>>>>>>>>> project should anyways strive for). If the build
>>> > >> fails
>>> > >>>>>>>>>>> catastrophically
>>> > >>>>>>>>>>>>>> once in a while, we can just exclude that one run
>>> > >> using
>>> > >>>> a
>>> > >>>>>>> flag
>>> > >>>>>>>>> and
>>> > >>>>>>>>>>>>>> everything will work, but if it happens frequently,
>>> > >> then
>>> > >>>>> it
>>> > >>>>>>>> won't
>>> > >>>>>>>>>>> work
>>> > >>>>>>>>>>>>>> right.
>>> > >>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>> I have re-enabled Flaky tests job (
>>> > >>>>>>>>>>>>>> https://builds.apache.org/
>>> > >>>> view/H-L/view/HBase/job/HBASE-
>>> > >>>>>>>>>> Flaky-Tests/
>>> > >>>>>>>>>>> )
>>> > >>>>>>>>>>>>> which
>>> > >>>>>>>>>>>>>> was disabled for almost a month due to trunk being on
>>> > >>>>> fire.
>>> > >>>>>>>>>>>>>> I will keep an eye on how things are going.
>>> > >>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>> On Mon, Sep 12, 2016 at 2:02 PM, Apekshit Sharma <
>>> > >>>>>>>>>> a...@cloudera.com <javascript:;>>
>>> > >>>>>>>>>>>>> wrote:
>>> > >>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>> @Sean, Mikhail: I found the alternate solution.
>>> > >> Using
>>> > >>>>> user
>>> > >>>>>>>>> defined
>>> > >>>>>>>>>>>> axis,
>>> > >>>>>>>>>>>>>>> tool environment and env variable injection.
>>> > >>>>>>>>>>>>>>> See latest diff to https://builds.apache.org/job/
>>> > >>>>>>>>>>> HBase-Trunk_matrix/
>>> > >>>>>>>>>>>>> job
>>> > >>>>>>>>>>>>>>> for reference.
>>> > >>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>> On Tue, Aug 30, 2016 at 7:39 PM, Mikhail Antonov <
>>> > >>>>>>>>>>>> olorinb...@gmail.com <javascript:;>>
>>> > >>>>>>>>>>>>>>> wrote:
>>> > >>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>> FYI, I did the same for branch-1.3 builds. I've
>>> > >>>>> disabled
>>> > >>>>>>>>>> hbase-1.3
>>> > >>>>>>>>>>>> and
>>> > >>>>>>>>>>>>>>>> hbase-1.3-IT jobs and instead created
>>> > >>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>> https://builds.apache.org/job/HBase-1.3-JDK8 and
>>> > >>>>>>>>>>>>>>>> https://builds.apache.org/job/HBase-1.3-JDK7
>>> > >>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>> This should work for now until we figure out how to
>>> > >>>> move
>>> > >>>>>>>>> forward.
>>> > >>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>> Thanks,
>>> > >>>>>>>>>>>>>>>> Mikhail
>>> > >>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>> On Wed, Aug 17, 2016 at 1:41 PM, Sean Busbey <
>>> > >>>>>>>>>> bus...@cloudera.com <javascript:;>>
>>> > >>>>>>>>>>>>> wrote:
>>> > >>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>> /me smacks forehead
>>> > >>>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>> these replacement jobs, of course, also have
>>> > >> special
>>> > >>>>>>>>> characters
>>> > >>>>>>>>>>> in
>>> > >>>>>>>>>>>>>>>>> their names which then show up in the working
>>> > >> path.
>>> > >>>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>> renaming them to skip spaces and parens.
>>> > >>>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>> On Wed, Aug 17, 2016 at 1:34 PM, Sean Busbey <
>>> > >>>>>>>>>>>> sean.bus...@gmail.com <javascript:;>>
>>> > >>>>>>>>>>>>>>>>> wrote:
>>> > >>>>>>>>>>>>>>>>>> FYI, it looks like essentially our entire CI
>>> > >> suite
>>> > >>>>> is
>>> > >>>>>>>> red,
>>> > >>>>>>>>>>>> probably
>>> > >>>>>>>>>>>>>>>> due
>>> > >>>>>>>>>>>>>>>>> to
>>> > >>>>>>>>>>>>>>>>>> parts of our codebase not tolerating spaces or
>>> > >>>> other
>>> > >>>>>>>>> special
>>> > >>>>>>>>>>>>>>>> characters
>>> > >>>>>>>>>>>>>>>>> in
>>> > >>>>>>>>>>>>>>>>>> the working directory.
>>> > >>>>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>>> I've made a stop-gap non-multi-configuration
>>> > >> set
>>> > >>>> of
>>> > >>>>>>> jobs
>>> > >>>>>>>>> for
>>> > >>>>>>>>>>>>> running
>>> > >>>>>>>>>>>>>>>> unit
>>> > >>>>>>>>>>>>>>>>>> tests for the 1.2 branch against JDK 7 and JDK
>>> > >> 8:
>>> > >>>>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>>> https://builds.apache.org/
>>> > >>>>>>> view/H-L/view/HBase/job/HBase%
>>> > >>>>>>>>>>>>>>>>> 201.2%20(JDK%201.7)/
>>> > >>>>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>>> https://builds.apache.org/
>>> > >>>>>>> view/H-L/view/HBase/job/HBase%
>>> > >>>>>>>>>>>>>>>>> 201.2%20(JDK%201.8)/
>>> > >>>>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>>> Due to the lack of response from infra@ I
>>> > >> suspect
>>> > >>>>> our
>>> > >>>>>>>> only
>>> > >>>>>>>>>>>> options
>>> > >>>>>>>>>>>>>>>> for
>>> > >>>>>>>>>>>>>>>>>> continuing on ASF infra is to fix whatever
>>> > >> part of
>>> > >>>>> our
>>> > >>>>>>>>> build
>>> > >>>>>>>>>>>>> doesn't
>>> > >>>>>>>>>>>>>>>>>> tolerate the new paths, or stop using
>>> > >>>>>>> multiconfiguration
>>> > >>>>>>>>>>>>> deployments.
>>> > >>>>>>>>>>>>>>>> I
>>> > >>>>>>>>>>>>>>>>> am
>>> > >>>>>>>>>>>>>>>>>> obviously less than thrilled at the idea of
>>> > >> having
>>> > >>>>>>>> several
>>> > >>>>>>>>>>>>> multiples
>>> > >>>>>>>>>>>>>>>> of
>>> > >>>>>>>>>>>>>>>>>> current jobs.
>>> > >>>>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>>> On Wed, Aug 10, 2016 at 6:28 PM, Sean Busbey <
>>> > >>>>>>>>>>>> bus...@cloudera.com <javascript:;>>
>>> > >>>>>>>>>>>>>>>>> wrote:
>>> > >>>>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>>>> Ugh.
>>> > >>>>>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>>>> I sent a reply to Gav on builds@ about maybe
>>> > >>>>> getting
>>> > >>>>>>>>> names
>>> > >>>>>>>>>>> that
>>> > >>>>>>>>>>>>>>>> don't
>>> > >>>>>>>>>>>>>>>>>>> have spaces in them:
>>> > >>>>>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>>>> https://lists.apache.org/thread.html/
>>> > >>>>>>>>>>>>> 8ac03dc62f9d6862d4f3d5eb37119c
>>> > >>>>>>>>>>>>>>>>>>> 9c73b4059aaa3ebba52fc63bb6@%3C
>>> > >> builds.apache.org
>>> > >>>> %3E
>>> > >>>>>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>>>> In the mean time, is this an issue we need
>>> > >> file
>>> > >>>>> with
>>> > >>>>>>>>> Hadoop
>>> > >>>>>>>>>> or
>>> > >>>>>>>>>>>>>>>>>>> something we need to fix in our own code?
>>> > >>>>>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>>>> On Wed, Aug 10, 2016 at 6:04 PM, Matteo
>>> > >> Bertozzi
>>> > >>>>>>>>>>>>>>>>>>> <theo.berto...@gmail.com <javascript:;>>
>>> > >> wrote:
>>> > >>>>>>>>>>>>>>>>>>>> There are a bunch of builds that have most
>>> > >> of
>>> > >>>> the
>>> > >>>>>>> test
>>> > >>>>>>>>>>>> failing.
>>> > >>>>>>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>>>>> Example:
>>> > >>>>>>>>>>>>>>>>>>>> https://builds.apache.org/job/
>>> > >>>>>>>>>> HBase-Trunk_matrix/1392/jdk=
>>> > >>>>>>>>>>>>>>>>>>> JDK%201.7%20(latest),label=
>>> > >>>>>>>> yahoo-not-h2/testReport/junit/
>>> > >>>>>>>>>>>>>>>>>>> org.apache.hadoop.hbase/
>>> > >> TestLocalHBaseCluster/
>>> > >>>>>>>>>>>>> testLocalHBaseCluster/
>>> > >>>>>>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>>>>> from the stack trace looks like the problem
>>> > >> is
>>> > >>>>> with
>>> > >>>>>>>> the
>>> > >>>>>>>>>> jdk
>>> > >>>>>>>>>>>> name
>>> > >>>>>>>>>>>>>>>> that
>>> > >>>>>>>>>>>>>>>>> has
>>> > >>>>>>>>>>>>>>>>>>>> spaces:
>>> > >>>>>>>>>>>>>>>>>>>> the hadoop FsVolumeImpl calls
>>> > >>>> setNameFormat(... +
>>> > >>>>>>>>>>>>>>>> fileName.toString()
>>> > >>>>>>>>>>>>>>>>> +
>>> > >>>>>>>>>>>>>>>>>>> ...)
>>> > >>>>>>>>>>>>>>>>>>>> and this seems to not be escaped
>>> > >>>>>>>>>>>>>>>>>>>> so we end up with JDK%25201.7%2520(latest)
>>> > >> in
>>> > >>>> the
>>> > >>>>>>>> string
>>> > >>>>>>>>>>>> format
>>> > >>>>>>>>>>>>>>>> and we
>>> > >>>>>>>>>>>>>>>>>>> get
>>> > >>>>>>>>>>>>>>>>>>>> a IllegalFormatPrecisionException: 7
>>> > >>>>>>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>>>>> 2016-08-10 22:07:46,108 WARN [DataNode:
>>> > >>>>>>>>>>>>>>>>>>>> [[[DISK]file:/home/jenkins/
>>> > >>>>>>>>> jenkins-slave/workspace/HBase-
>>> > >>>>>>>>>>>>>>>>>>> Trunk_matrix/jdk/JDK%25201.7%
>>> > >>>>>>>>> 2520(latest)/label/yahoo-not-
>>> > >>>>>>>>>>>>>>>>>>> h2/hbase-server/target/test-
>>> > >>>>>>>> data/e7099624-ecfa-4674-87de-
>>> > >>>>>>>>>>>>>>>>>>> a8733d13b582/dfscluster_
>>> > >> 10fdcfc3-cd1b-45be-9b5a-
>>> > >>>>>>>>>>>>>>>>>>> 9c88f385e6f1/dfs/data/data1/,
>>> > >>>>>>>>>>>>>>>>>>>> [DISK]file:/home/jenkins/
>>> > >>>>>>>> jenkins-slave/workspace/HBase-
>>> > >>>>>>>>>>>>>>>>>>> Trunk_matrix/jdk/JDK%25201.7%
>>> > >>>>>>>>> 2520(latest)/label/yahoo-not-
>>> > >>>>>>>>>>>>>>>>>>> h2/hbase-server/target/test-
>>> > >>>>>>>> data/e7099624-ecfa-4674-87de-
>>> > >>>>>>>>>>>>>>>>>>> a8733d13b582/dfscluster_
>>> > >> 10fdcfc3-cd1b-45be-9b5a-
>>> > >>>>>>>>>>>>>>>>>>> 9c88f385e6f1/dfs/data/data2/]]
>>> > >>>>>>>>>>>>>>>>>>>> heartbeating to localhost/127.0.0.1:34629]
>>> > >>>>>>>>>>>>>>>>>>>> datanode.BPServiceActor(831): Unexpected
>>> > >>>>> exception
>>> > >>>>>>> in
>>> > >>>>>>>>>> block
>>> > >>>>>>>>>>>> pool
>>> > >>>>>>>>>>>>>>>> Block
>>> > >>>>>>>>>>>>>>>>>>>> pool <registering> (Datanode Uuid
>>> > >> unassigned)
>>> > >>>>>>> service
>>> > >>>>>>>> to
>>> > >>>>>>>>>>>>>>>>>>>> localhost/127.0.0.1:34629
>>> > >>>>>>>>>>>>>>>>>>>> java.util.IllegalFormatPrecisionException:
>>> > >> 7
>>> > >>>>>>>>>>>>>>>>>>>> at java.util.Formatter$
>>> > >>>>>>>>> FormatSpecifier.checkText(
>>> > >>>>>>>>>>>>>>>>>>> Formatter.java:2984)
>>> > >>>>>>>>>>>>>>>>>>>> at java.util.Formatter$
>>> > >>>>>>>> FormatSpecifier.<init>(
>>> > >>>>>>>>>>>>>>>>>>> Formatter.java:2688)
>>> > >>>>>>>>>>>>>>>>>>>> at java.util.Formatter.parse(
>>> > >>>>>>>>> Formatter.java:2528)
>>> > >>>>>>>>>>>>>>>>>>>> at java.util.Formatter.format(
>>> > >>>>>>>>>> Formatter.java:2469)
>>> > >>>>>>>>>>>>>>>>>>>> at java.util.Formatter.format(
>>> > >>>>>>>>>> Formatter.java:2423)
>>> > >>>>>>>>>>>>>>>>>>>> at java.lang.String.format(
>>> > >>>>>>> String.java:2792)
>>> > >>>>>>>>>>>>>>>>>>>> at com.google.common.util.
>>> > >> concurrent.
>>> > >>>>>>>>>>>>> ThreadFactoryBuilder.
>>> > >>>>>>>>>>>>>>>>>>> setNameFormat(ThreadFactoryBuilder.java:68)
>>> > >>>>>>>>>>>>>>>>>>>> at org.apache.hadoop.hdfs.server.
>>> > >>>>>>>>>>>>> datanode.fsdataset.impl.
>>> > >>>>>>>>>>>>>>>>>>> FsVolumeImpl.initializeCacheExecutor(
>>> > >>>>>>>>> FsVolumeImpl.java:140)
>>> > >>>>>>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>>>>> Matteo
>>> > >>>>>>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>>>>> On Tue, Aug 9, 2016 at 9:55 AM, Stack <
>>> > >>>>>>>> st...@duboce.net
>>> > >>>>>>>>> <javascript:;>>
>>> > >>>>>>>>>>>> wrote:
>>> > >>>>>>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>>>>>> Good on you Sean.
>>> > >>>>>>>>>>>>>>>>>>>>> S
>>> > >>>>>>>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>>>>>> On Mon, Aug 8, 2016 at 9:43 PM, Sean
>>> > >> Busbey <
>>> > >>>>>>>>>>>> bus...@apache.org <javascript:;>
>>> > >>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>> wrote:
>>> > >>>>>>>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>>>>>>> I updated all of our jobs to use the
>>> > >> updated
>>> > >>>>> JDK
>>> > >>>>>>>>>> versions
>>> > >>>>>>>>>>>>> from
>>> > >>>>>>>>>>>>>>>>> infra.
>>> > >>>>>>>>>>>>>>>>>>>>>> These have spaces in the names, and those
>>> > >>>>> names
>>> > >>>>>>> end
>>> > >>>>>>>>> up
>>> > >>>>>>>>>> in
>>> > >>>>>>>>>>>> our
>>> > >>>>>>>>>>>>>>>>>>>>>> workspace path, so try to keep an eye
>>> > >> out.
>>> > >>>>>>>>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>>>>>>> On Mon, Aug 8, 2016 at 10:42 AM, Sean
>>> > >>>> Busbey <
>>> > >>>>>>>>>>>>>>>> bus...@cloudera.com <javascript:;>>
>>> > >>>>>>>>>>>>>>>>>>>>> wrote:
>>> > >>>>>>>>>>>>>>>>>>>>>>> running in docker is the default now.
>>> > >>>>> relying
>>> > >>>>>>> on
>>> > >>>>>>>>> the
>>> > >>>>>>>>>>>>> default
>>> > >>>>>>>>>>>>>>>>> docker
>>> > >>>>>>>>>>>>>>>>>>>>>>> image that comes with Yetus means that
>>> > >> our
>>> > >>>>>>> protoc
>>> > >>>>>>>>>>> checks
>>> > >>>>>>>>>>>>> are
>>> > >>>>>>>>>>>>>>>>>>>>>>> failing[1].
>>> > >>>>>>>>>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>>>>>>>> [1]: https://issues.apache.org/
>>> > >>>>>>>>>> jira/browse/HBASE-16373
>>> > >>>>>>>>>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>>>>>>>> On Sat, Aug 6, 2016 at 5:03 PM, Sean
>>> > >>>> Busbey
>>> > >>>>> <
>>> > >>>>>>>>>>>>>>>> bus...@apache.org <javascript:;>>
>>> > >>>>>>>>>>>>>>>>>>> wrote:
>>> > >>>>>>>>>>>>>>>>>>>>>>>> Hi folks!
>>> > >>>>>>>>>>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>>>>>>>>> this morning I merged the patch that
>>> > >>>>> updates
>>> > >>>>>>> us
>>> > >>>>>>>> to
>>> > >>>>>>>>>>> Yetus
>>> > >>>>>>>>>>>>>>>>> 0.3.0[1]
>>> > >>>>>>>>>>>>>>>>>>> and
>>> > >>>>>>>>>>>>>>>>>>>>>> updated the precommit job appropriately.
>>> > >> I
>>> > >>>>> also
>>> > >>>>>>>>> changed
>>> > >>>>>>>>>>> it
>>> > >>>>>>>>>>>> to
>>> > >>>>>>>>>>>>>>>> use
>>> > >>>>>>>>>>>>>>>>> one
>>> > >>>>>>>>>>>>>>>>>>> of
>>> > >>>>>>>>>>>>>>>>>>>>>> the Java versions post the puppet
>>> > >> changes to
>>> > >>>>> asf
>>> > >>>>>>>>> build.
>>> > >>>>>>>>>>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>>>>>>>>> The last three builds look normal
>>> > >> (#2975
>>> > >>>> -
>>> > >>>>>>>> #2977).
>>> > >>>>>>>>>> I'm
>>> > >>>>>>>>>>>>> gonna
>>> > >>>>>>>>>>>>>>>> try
>>> > >>>>>>>>>>>>>>>>>>>>>> running things in docker next. I'll email
>>> > >>>>> again
>>> > >>>>>>>> when
>>> > >>>>>>>>> I
>>> > >>>>>>>>>>> make
>>> > >>>>>>>>>>>>> it
>>> > >>>>>>>>>>>>>>>> the
>>> > >>>>>>>>>>>>>>>>>>>>> default.
>>> > >>>>>>>>>>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>>>>>>>>> [1]: https://issues.apache.org/
>>> > >>>>>>>>>>> jira/browse/HBASE-15882
>>> > >>>>>>>>>>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>>>>>>>>> On 2016-06-16 10:43 (-0500), Sean
>>> > >> Busbey
>>> > >>>> <
>>> > >>>>>>>>>>>>> bus...@apache.org <javascript:;>>
>>> > >>>>>>>>>>>>>>>>>>> wrote:
>>> > >>>>>>>>>>>>>>>>>>>>>>>>> FYI, today our precommit jobs started
>>> > >>>>> failing
>>> > >>>>>>>>>> because
>>> > >>>>>>>>>>>> our
>>> > >>>>>>>>>>>>>>>>> chosen
>>> > >>>>>>>>>>>>>>>>>>> jdk
>>> > >>>>>>>>>>>>>>>>>>>>>>>>> (1.7.0.79) disappeared (mentioned on
>>> > >>>>>>>>> HBASE-16032).
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>>>>>>>>>> Initially we were doing something
>>> > >> wrong,
>>> > >>>>>>> namely
>>> > >>>>>>>>>>>> directly
>>> > >>>>>>>>>>>>>>>>>>> referencing
>>> > >>>>>>>>>>>>>>>>>>>>>>>>> the jenkins build tools area without
>>> > >>>>> telling
>>> > >>>>>>>>>> jenkins
>>> > >>>>>>>>>>> to
>>> > >>>>>>>>>>>>> give
>>> > >>>>>>>>>>>>>>>>> us an
>>> > >>>>>>>>>>>>>>>>>>>>> env
>>> > >>>>>>>>>>>>>>>>>>>>>>>>> variable that stated where the jdk is
>>> > >>>>>>> located.
>>> > >>>>>>>>>>> However,
>>> > >>>>>>>>>>>>>>>> after
>>> > >>>>>>>>>>>>>>>>>>>>>>>>> attempting to switch to the
>>> > >> appropriate
>>> > >>>>>>> tooling
>>> > >>>>>>>>>>>> variable
>>> > >>>>>>>>>>>>> for
>>> > >>>>>>>>>>>>>>>>> jdk
>>> > >>>>>>>>>>>>>>>>>>>>>>>>> 1.7.0.79, I found that it didn't
>>> > >> point
>>> > >>>> to
>>> > >>>>> a
>>> > >>>>>>>> place
>>> > >>>>>>>>>>> that
>>> > >>>>>>>>>>>>>>>> worked.
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>>>>>>>>>> I've now updated the job to rely on
>>> > >> the
>>> > >>>>>>> latest
>>> > >>>>>>>>> 1.7
>>> > >>>>>>>>>>> jdk,
>>> > >>>>>>>>>>>>>>>> which
>>> > >>>>>>>>>>>>>>>>> is
>>> > >>>>>>>>>>>>>>>>>>>>>>>>> currently 1.7.0.80. I don't know how
>>> > >>>> often
>>> > >>>>>>>>> "latest"
>>> > >>>>>>>>>>>>> updates.
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>>>>>>>>>> Personally, I think this is a sign
>>> > >> that
>>> > >>>> we
>>> > >>>>>>> need
>>> > >>>>>>>>> to
>>> > >>>>>>>>>>>>>>>> prioritize
>>> > >>>>>>>>>>>>>>>>>>>>>>>>> HBASE-15882 so that we can switch
>>> > >> back
>>> > >>>> to
>>> > >>>>>>> using
>>> > >>>>>>>>>>>> Docker. I
>>> > >>>>>>>>>>>>>>>> won't
>>> > >>>>>>>>>>>>>>>>>>> have
>>> > >>>>>>>>>>>>>>>>>>>>>>>>> time this week, so if anyone else
>>> > >> does
>>> > >>>>> please
>>> > >>>>>>>>> pick
>>> > >>>>>>>>>> up
>>> > >>>>>>>>>>>> the
>>> > >>>>>>>>>>>>>>>>> ticket.
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>>>>>>>>>> On Thu, Mar 17, 2016 at 5:19 PM,
>>> > >> Stack <
>>> > >>>>>>>>>>>> st...@duboce.net <javascript:;>
>>> > >>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>> wrote:
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>> Thanks Sean.
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>> St.Ack
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>> On Wed, Mar 16, 2016 at 12:04 PM,
>>> > >> Sean
>>> > >>>>>>>> Busbey <
>>> > >>>>>>>>>>>>>>>>>>> bus...@cloudera.com <javascript:;>
>>> > >>>>>>>>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>>>>>>> wrote:
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>> FYI, I updated the precommit job
>>> > >>>> today
>>> > >>>>> to
>>> > >>>>>>>>>> specify
>>> > >>>>>>>>>>>> that
>>> > >>>>>>>>>>>>>>>> only
>>> > >>>>>>>>>>>>>>>>>>>>> compile
>>> > >>>>>>>>>>>>>>>>>>>>>> time
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>> checks should be done against jdks
>>> > >>>>> other
>>> > >>>>>>>> than
>>> > >>>>>>>>>> the
>>> > >>>>>>>>>>>>> primary
>>> > >>>>>>>>>>>>>>>>> jdk7
>>> > >>>>>>>>>>>>>>>>>>>>>> instance.
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Mar 7, 2016 at 8:43 PM,
>>> > >> Sean
>>> > >>>>>>> Busbey
>>> > >>>>>>>> <
>>> > >>>>>>>>>>>>>>>>>>> bus...@cloudera.com <javascript:;>>
>>> > >>>>>>>>>>>>>>>>>>>>>> wrote:
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> I tested things out, and while
>>> > >>>>>>>> YETUS-297[1]
>>> > >>>>>>>>> is
>>> > >>>>>>>>>>>>> present
>>> > >>>>>>>>>>>>>>>> the
>>> > >>>>>>>>>>>>>>>>>>>>>> default runs
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> all plugins that can do multiple
>>> > >>>> jdks
>>> > >>>>>>>>> against
>>> > >>>>>>>>>>>> those
>>> > >>>>>>>>>>>>>>>>> available
>>> > >>>>>>>>>>>>>>>>>>>>>> (jdk7 and
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> jdk8 in our case).
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> We can configure things to only
>>> > >> do
>>> > >>>> a
>>> > >>>>>>>> single
>>> > >>>>>>>>>> run
>>> > >>>>>>>>>>> of
>>> > >>>>>>>>>>>>> unit
>>> > >>>>>>>>>>>>>>>>>>> tests.
>>> > >>>>>>>>>>>>>>>>>>>>>> They'll be
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> against jdk7, since that is our
>>> > >>>>> default
>>> > >>>>>>>> jdk.
>>> > >>>>>>>>>>> That
>>> > >>>>>>>>>>>>> fine
>>> > >>>>>>>>>>>>>>>> by
>>> > >>>>>>>>>>>>>>>>>>>>>> everyone? It'll
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> save ~1.5 hours on any build
>>> > >> that
>>> > >>>>> hits
>>> > >>>>>>>>>>>> hbase-server.
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Mar 7, 2016 at 1:22 PM,
>>> > >>>>> Stack <
>>> > >>>>>>>>>>>>>>>> st...@duboce.net <javascript:;>>
>>> > >>>>>>>>>>>>>>>>>>> wrote:
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hurray!
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> It looks like YETUS-96 is in
>>> > >> there
>>> > >>>>> and
>>> > >>>>>>> we
>>> > >>>>>>>>> are
>>> > >>>>>>>>>>>> only
>>> > >>>>>>>>>>>>>>>>> running
>>> > >>>>>>>>>>>>>>>>>>> on
>>> > >>>>>>>>>>>>>>>>>>>>>> jdk build
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> now, the default (but testing
>>> > >>>>> compile
>>> > >>>>>>>>> against
>>> > >>>>>>>>>>>>>>>> both)....
>>> > >>>>>>>>>>>>>>>>> Will
>>> > >>>>>>>>>>>>>>>>>>>>>> keep an
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>> eye.
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> St.Ack
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Mar 7, 2016 at 10:27
>>> > >> AM,
>>> > >>>>> Sean
>>> > >>>>>>>>> Busbey
>>> > >>>>>>>>>> <
>>> > >>>>>>>>>>>>>>>>>>>>>> bus...@cloudera.com <javascript:;>>
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> FYI, I've just updated our
>>> > >>>>> precommit
>>> > >>>>>>>> jobs
>>> > >>>>>>>>>> to
>>> > >>>>>>>>>>>> use
>>> > >>>>>>>>>>>>> the
>>> > >>>>>>>>>>>>>>>>> 0.2.0
>>> > >>>>>>>>>>>>>>>>>>>>>> release of
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yetus
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that came out today.
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> After keeping an eye out for
>>> > >>>>>>>> strangeness
>>> > >>>>>>>>>>> today
>>> > >>>>>>>>>>>>> I'll
>>> > >>>>>>>>>>>>>>>>> turn
>>> > >>>>>>>>>>>>>>>>>>>>>> docker mode
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> back
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> on by default tonight.
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Wed, Jan 13, 2016 at 10:14
>>> > >>>> AM,
>>> > >>>>>>> Sean
>>> > >>>>>>>>>>> Busbey <
>>> > >>>>>>>>>>>>>>>>>>>>>> bus...@apache.org <javascript:;>>
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> FYI, I added a new
>>> > >> parameter
>>> > >>>> to
>>> > >>>>> the
>>> > >>>>>>>>>>> precommit
>>> > >>>>>>>>>>>>> job:
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * USE_YETUS_PRERELEASE -
>>> > >>>> causes
>>> > >>>>> us
>>> > >>>>>>> to
>>> > >>>>>>>>> use
>>> > >>>>>>>>>>> the
>>> > >>>>>>>>>>>>>>>> HEAD of
>>> > >>>>>>>>>>>>>>>>>>> the
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>> apache/yetus
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> repo rather than our chosen
>>> > >>>>> release
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It defaults to inactive,
>>> > >> but
>>> > >>>>> can be
>>> > >>>>>>>>> used
>>> > >>>>>>>>>> in
>>> > >>>>>>>>>>>>>>>>>>>>>> manually-triggered runs
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>> to
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> test a solution to a
>>> > >> problem
>>> > >>>> in
>>> > >>>>> the
>>> > >>>>>>>>> yetus
>>> > >>>>>>>>>>>>>>>> library. At
>>> > >>>>>>>>>>>>>>>>>>> the
>>> > >>>>>>>>>>>>>>>>>>>>>> moment,
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>> I'm
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> using it to test a
>>> > >> solution to
>>> > >>>>>>>> default
>>> > >>>>>>>>>>> module
>>> > >>>>>>>>>>>>>>>>> ordering
>>> > >>>>>>>>>>>>>>>>>>> as
>>> > >>>>>>>>>>>>>>>>>>>>>> seen in
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> HBASE-15075.
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Fri, Jan 8, 2016 at 7:58
>>> > >>>> AM,
>>> > >>>>>>> Sean
>>> > >>>>>>>>>>> Busbey <
>>> > >>>>>>>>>>>>>>>>>>>>>> bus...@cloudera.com <javascript:;>>
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> FYI, I just pushed
>>> > >>>> HBASE-13525
>>> > >>>>>>>>> (switch
>>> > >>>>>>>>>> to
>>> > >>>>>>>>>>>>> Apache
>>> > >>>>>>>>>>>>>>>>> Yetus
>>> > >>>>>>>>>>>>>>>>>>>>> for
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>> precommit
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> tests)
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and updated our jenkins
>>> > >>>>> precommit
>>> > >>>>>>>>> build
>>> > >>>>>>>>>>> to
>>> > >>>>>>>>>>>>> use
>>> > >>>>>>>>>>>>>>>> it.
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Jenkins job has some
>>> > >>>>> explanation:
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>> https://builds.apache.org/
>>> > >>>>>>>>>>>>> view/PreCommit%20Builds/job/
>>> > >>>>>>>>>>>>>>>>>>>>>> PreCommit-HBASE-Build/
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Release note from
>>> > >>>> HBASE-13525
>>> > >>>>>>> does
>>> > >>>>>>>> as
>>> > >>>>>>>>>>> well.
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The old job will stick
>>> > >>>> around
>>> > >>>>>>> here
>>> > >>>>>>>>> for
>>> > >>>>>>>>>> a
>>> > >>>>>>>>>>>>> couple
>>> > >>>>>>>>>>>>>>>> of
>>> > >>>>>>>>>>>>>>>>>>> weeks,
>>> > >>>>>>>>>>>>>>>>>>>>>> in case
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>> we
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> need
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to refer back to it:
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>> https://builds.apache.org/
>>> > >>>>>>>>>>>>> view/PreCommit%20Builds/job/
>>> > >>>>>>>>>>>>>>>>>>>>>> PreCommit-HBASE-Build-deprecated/
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If something looks awry,
>>> > >>>>> please
>>> > >>>>>>>> drop
>>> > >>>>>>>>> a
>>> > >>>>>>>>>>> note
>>> > >>>>>>>>>>>>> on
>>> > >>>>>>>>>>>>>>>>>>>>> HBASE-13525
>>> > >>>>>>>>>>>>>>>>>>>>>> while
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>> it
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> remains
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> open (and make a new
>>> > >> issue
>>> > >>>>>>> after).
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Wed, Dec 2, 2015 at
>>> > >> 3:22
>>> > >>>>> PM,
>>> > >>>>>>>>> Stack <
>>> > >>>>>>>>>>>>>>>>>>> st...@duboce.net <javascript:;>>
>>> > >>>>>>>>>>>>>>>>>>>>>> wrote:
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> As part of my continuing
>>> > >>>>>>> advocacy
>>> > >>>>>>>> of
>>> > >>>>>>>>>>>>>>>>>>> builds.apache.org
>>> > >>>>>>>>>>>>>>>>>>>>>> and that
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> their
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> results are now worthy
>>> > >> of
>>> > >>>> our
>>> > >>>>>>>> trust
>>> > >>>>>>>>>> and
>>> > >>>>>>>>>>>>>>>> nurture,
>>> > >>>>>>>>>>>>>>>>> here
>>> > >>>>>>>>>>>>>>>>>>>>> are
>>> > >>>>>>>>>>>>>>>>>>>>>> some
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> highlights
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> from the last few days
>>> > >> of
>>> > >>>>>>> builds:
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> + hadoopqa is now
>>> > >> finding
>>> > >>>>>>> zombies
>>> > >>>>>>>>>> before
>>> > >>>>>>>>>>>> the
>>> > >>>>>>>>>>>>>>>>> patch is
>>> > >>>>>>>>>>>>>>>>>>>>>> committed.
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> HBASE-14888 showed "-1
>>> > >> core
>>> > >>>>>>> tests.
>>> > >>>>>>>>> The
>>> > >>>>>>>>>>>> patch
>>> > >>>>>>>>>>>>>>>>> failed
>>> > >>>>>>>>>>>>>>>>>>>>> these
>>> > >>>>>>>>>>>>>>>>>>>>>> unit
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> tests:"
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> but
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> didn't have any failed
>>> > >>>> tests
>>> > >>>>>>>> listed
>>> > >>>>>>>>>> (I'm
>>> > >>>>>>>>>>>>>>>> trying to
>>> > >>>>>>>>>>>>>>>>>>> see
>>> > >>>>>>>>>>>>>>>>>>>>> if
>>> > >>>>>>>>>>>>>>>>>>>>>> I can
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>> do
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> anything
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> about this...). Running
>>> > >> our
>>> > >>>>>>> little
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>> ./dev-tools/findHangingTests.py
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> against
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the consoleText, it
>>> > >> showed
>>> > >>>> a
>>> > >>>>>>>> hanging
>>> > >>>>>>>>>>> test.
>>> > >>>>>>>>>>>>>>>> Running
>>> > >>>>>>>>>>>>>>>>>>>>>> locally, I see
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> same
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hang. This is before the
>>> > >>>>> patch
>>> > >>>>>>>>> landed.
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> + Our branch runs are
>>> > >> now
>>> > >>>>> near
>>> > >>>>>>>>> totally
>>> > >>>>>>>>>>>>> zombie
>>> > >>>>>>>>>>>>>>>> and
>>> > >>>>>>>>>>>>>>>>>>> flakey
>>> > >>>>>>>>>>>>>>>>>>>>>> free --
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> still
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> some
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> work to do -- but a
>>> > >> recent
>>> > >>>>> patch
>>> > >>>>>>>>> that
>>> > >>>>>>>>>>>> seemed
>>> > >>>>>>>>>>>>>>>>> harmless
>>> > >>>>>>>>>>>>>>>>>>>>> was
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>> causing a
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> reliable flake fail in
>>> > >> the
>>> > >>>>>>>> backport
>>> > >>>>>>>>> to
>>> > >>>>>>>>>>>>>>>> branch-1*
>>> > >>>>>>>>>>>>>>>>>>>>>> confirmed by
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>> local
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> runs.
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The flakeyness was
>>> > >> plain to
>>> > >>>>> see
>>> > >>>>>>> up
>>> > >>>>>>>>> in
>>> > >>>>>>>>>>>>>>>>>>> builds.apache.org
>>> > >>>>>>>>>>>>>>>>>>>>> .
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> + In the last few days
>>> > >> I've
>>> > >>>>>>>>> committed
>>> > >>>>>>>>>> a
>>> > >>>>>>>>>>>>> patch
>>> > >>>>>>>>>>>>>>>> that
>>> > >>>>>>>>>>>>>>>>>>>>>> included
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>> javadoc
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> warnings even though
>>> > >>>> hadoopqa
>>> > >>>>>>> said
>>> > >>>>>>>>> the
>>> > >>>>>>>>>>>> patch
>>> > >>>>>>>>>>>>>>>>>>> introduced
>>> > >>>>>>>>>>>>>>>>>>>>>> javadoc
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> issues
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (I
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> missed it). This messed
>>> > >> up
>>> > >>>>> life
>>> > >>>>>>>> for
>>> > >>>>>>>>>>> folks
>>> > >>>>>>>>>>>>>>>>>>> subsequently
>>> > >>>>>>>>>>>>>>>>>>>>> as
>>> > >>>>>>>>>>>>>>>>>>>>>> their
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> patches
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> now
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> reported javadoc
>>> > >> issues....
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> In short, I suggest that
>>> > >>>>>>>>>>>> builds.apache.org
>>> > >>>>>>>>>>>>> is
>>> > >>>>>>>>>>>>>>>>> worth
>>> > >>>>>>>>>>>>>>>>>>>>>> keeping an
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>> eye
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> on,
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> make
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> sure you get a clean
>>> > >> build
>>> > >>>>> out
>>> > >>>>>>> of
>>> > >>>>>>>>>>> hadoopqa
>>> > >>>>>>>>>>>>>>>> before
>>> > >>>>>>>>>>>>>>>>>>>>>> committing
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> anything,
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> lets all work together
>>> > >> to
>>> > >>>> try
>>> > >>>>>>> and
>>> > >>>>>>>>> keep
>>> > >>>>>>>>>>> our
>>> > >>>>>>>>>>>>>>>> builds
>>> > >>>>>>>>>>>>>>>>>>> blue:
>>> > >>>>>>>>>>>>>>>>>>>>>> it'll
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>> save
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> us
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> all
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> work in the long run.
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> St.Ack
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Nov 4, 2014 at
>>> > >> 9:38
>>> > >>>>> AM,
>>> > >>>>>>>>> Stack
>>> > >>>>>>>>>> <
>>> > >>>>>>>>>>>>>>>>>>> st...@duboce.net <javascript:;>
>>> > >>>>>>>>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>>>>>>> wrote:
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Branch-1 and master
>>> > >> have
>>> > >>>>>>>>> stabilized
>>> > >>>>>>>>>>> and
>>> > >>>>>>>>>>>>> now
>>> > >>>>>>>>>>>>>>>> run
>>> > >>>>>>>>>>>>>>>>>>> mostly
>>> > >>>>>>>>>>>>>>>>>>>>>> blue
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> (give or
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> take
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the odd failure)
>>> > >> [1][2].
>>> > >>>>>>> Having
>>> > >>>>>>>> a
>>> > >>>>>>>>>>> mostly
>>> > >>>>>>>>>>>>> blue
>>> > >>>>>>>>>>>>>>>>>>> branch-1
>>> > >>>>>>>>>>>>>>>>>>>>>> has
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> helped us
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> identify at least one
>>> > >>>>>>>>> destabilizing
>>> > >>>>>>>>>>>>> commit in
>>> > >>>>>>>>>>>>>>>>> the
>>> > >>>>>>>>>>>>>>>>>>> last
>>> > >>>>>>>>>>>>>>>>>>>>>> few
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>> days,
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> maybe
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> two;
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> this is as it should
>>> > >> be
>>> > >>>>>>> (smile).
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Lets keep our builds
>>> > >>>> blue.
>>> > >>>>> If
>>> > >>>>>>>> you
>>> > >>>>>>>>>>>> commit a
>>> > >>>>>>>>>>>>>>>>> patch,
>>> > >>>>>>>>>>>>>>>>>>> make
>>> > >>>>>>>>>>>>>>>>>>>>>> sure
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> subsequent
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> builds stay blue. You
>>> > >> can
>>> > >>>>>>>>> subscribe
>>> > >>>>>>>>>> to
>>> > >>>>>>>>>>>>>>>>>>>>>> bui...@hbase.apache.org <javascript:;>
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> get
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> notice of failures if
>>> > >> not
>>> > >>>>>>>> already
>>> > >>>>>>>>>>>>> subscribed.
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> St.Ack
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1.
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>> https://builds.apache.org/
>>> > >>>>>>>>>>>>> view/H-L/view/HBase/job/HBase-
>>> > >>>>>>>>>>>>>>>>> 1.0/
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2.
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> https://builds.apache.org/view
>>> > >>>>>>>>>>>>>>>> /H-L/view/HBase/job/HBase-
>>> > >>>>>>>>>>>>>>>>>>> TRUNK/
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Oct 13, 2014
>>> > >> at
>>> > >>>>> 4:41
>>> > >>>>>>> PM,
>>> > >>>>>>>>>>> Stack <
>>> > >>>>>>>>>>>>>>>>>>>>>> st...@duboce.net <javascript:;>>
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> A few notes on
>>> > >> testing.
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Too long to read,
>>> > >> infra
>>> > >>>> is
>>> > >>>>>>> more
>>> > >>>>>>>>>>> capable
>>> > >>>>>>>>>>>>> now
>>> > >>>>>>>>>>>>>>>> and
>>> > >>>>>>>>>>>>>>>>>>> after
>>> > >>>>>>>>>>>>>>>>>>>>>> some
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> work, we
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> are
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> seeing branch-1 and
>>> > >>>> trunk
>>> > >>>>>>>> mostly
>>> > >>>>>>>>>>>> running
>>> > >>>>>>>>>>>>>>>> blue.
>>> > >>>>>>>>>>>>>>>>>>> Lets
>>> > >>>>>>>>>>>>>>>>>>>>>> try and
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> keep it
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> this
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> way going forward.
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Apache Infra has new,
>>> > >>>> more
>>> > >>>>>>>>> capable
>>> > >>>>>>>>>>>>> hardware.
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> A recent spurt of
>>> > >> test
>>> > >>>>> fixing
>>> > >>>>>>>>>>> combined
>>> > >>>>>>>>>>>>> with
>>> > >>>>>>>>>>>>>>>>> more
>>> > >>>>>>>>>>>>>>>>>>>>>> capable
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> hardware
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> seems
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to have gotten us to
>>> > >> a
>>> > >>>> new
>>> > >>>>>>>> place;
>>> > >>>>>>>>>>> tests
>>> > >>>>>>>>>>>>> are
>>> > >>>>>>>>>>>>>>>>> mostly
>>> > >>>>>>>>>>>>>>>>>>>>>> passing now
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> on
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> branch-1
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and master. Lets try
>>> > >>>> and
>>> > >>>>>>> keep
>>> > >>>>>>>> it
>>> > >>>>>>>>>>> this
>>> > >>>>>>>>>>>>> way
>>> > >>>>>>>>>>>>>>>> and
>>> > >>>>>>>>>>>>>>>>>>> start
>>> > >>>>>>>>>>>>>>>>>>>>>> to trust
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> our
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> test
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> runs
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> again. Just a few
>>> > >>>> flakies
>>> > >>>>>>>>> remain.
>>> > >>>>>>>>>>>> Lets
>>> > >>>>>>>>>>>>> try
>>> > >>>>>>>>>>>>>>>>> and
>>> > >>>>>>>>>>>>>>>>>>> nail
>>> > >>>>>>>>>>>>>>>>>>>>>> them.
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Our tests now run in
>>> > >>>>> parallel
>>> > >>>>>>>>> with
>>> > >>>>>>>>>>>> other
>>> > >>>>>>>>>>>>>>>> test
>>> > >>>>>>>>>>>>>>>>>>> suites
>>> > >>>>>>>>>>>>>>>>>>>>>> where
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> previous
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> we
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ran alone. You can
>>> > >> see
>>> > >>>>> this
>>> > >>>>>>>>>> sometimes
>>> > >>>>>>>>>>>>> when
>>> > >>>>>>>>>>>>>>>> our
>>> > >>>>>>>>>>>>>>>>>>> zombie
>>> > >>>>>>>>>>>>>>>>>>>>>> detector
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> reports
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> tests from another
>>> > >>>> project
>>> > >>>>>>>>>> altogether
>>> > >>>>>>>>>>>> as
>>> > >>>>>>>>>>>>>>>>> lingerers
>>> > >>>>>>>>>>>>>>>>>>>>> (To
>>> > >>>>>>>>>>>>>>>>>>>>>> be
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> fixed).
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Some
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> our tests are failing
>>> > >>>>>>> because a
>>> > >>>>>>>>>>>>> concurrent
>>> > >>>>>>>>>>>>>>>>> hbase
>>> > >>>>>>>>>>>>>>>>>>> run
>>> > >>>>>>>>>>>>>>>>>>>>> is
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>> undoing
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> classes
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> data from under it.
>>> > >>>> Also,
>>> > >>>>>>> lets
>>> > >>>>>>>>> fix.
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Our tests are
>>> > >> brittle.
>>> > >>>> It
>>> > >>>>>>> takes
>>> > >>>>>>>>>>>> 75minutes
>>> > >>>>>>>>>>>>>>>> for
>>> > >>>>>>>>>>>>>>>>>>> them to
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>> complete.
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Many
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> are
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> heavy-duty
>>> > >> integration
>>> > >>>>> tests
>>> > >>>>>>>>>> starting
>>> > >>>>>>>>>>>> up
>>> > >>>>>>>>>>>>>>>>> multiple
>>> > >>>>>>>>>>>>>>>>>>>>>> clusters and
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mapreduce
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> all in the one JVM.
>>> > >> It
>>> > >>>> is
>>> > >>>>> a
>>> > >>>>>>>>> miracle
>>> > >>>>>>>>>>>> they
>>> > >>>>>>>>>>>>>>>> pass
>>> > >>>>>>>>>>>>>>>>> at
>>> > >>>>>>>>>>>>>>>>>>> all.
>>> > >>>>>>>>>>>>>>>>>>>>>> Usually
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> integration
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> tests have been cast
>>> > >> as
>>> > >>>>> unit
>>> > >>>>>>>>> tests
>>> > >>>>>>>>>>>>> because
>>> > >>>>>>>>>>>>>>>>> there
>>> > >>>>>>>>>>>>>>>>>>> was
>>> > >>>>>>>>>>>>>>>>>>>>>> no where
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> else
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> them
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to get an airing. We
>>> > >>>> have
>>> > >>>>>>> the
>>> > >>>>>>>>>>> hbase-it
>>> > >>>>>>>>>>>>>>>> suite
>>> > >>>>>>>>>>>>>>>>> now
>>> > >>>>>>>>>>>>>>>>>>>>>> which would
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> be a
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> more
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> apt
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> place but until these
>>> > >>>> are
>>> > >>>>> run
>>> > >>>>>>>> on
>>> > >>>>>>>>> a
>>> > >>>>>>>>>>>>> regular
>>> > >>>>>>>>>>>>>>>>> basis
>>> > >>>>>>>>>>>>>>>>>>> in
>>> > >>>>>>>>>>>>>>>>>>>>>> public for
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> all
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> see,
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the fat integration
>>> > >>>> tests
>>> > >>>>>>>>> disguised
>>> > >>>>>>>>>>> as
>>> > >>>>>>>>>>>>> unit
>>> > >>>>>>>>>>>>>>>>> tests
>>> > >>>>>>>>>>>>>>>>>>>>> will
>>> > >>>>>>>>>>>>>>>>>>>>>> remain.
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> A
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> review of
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> our current unit
>>> > >> tests
>>> > >>>>>>> weeding
>>> > >>>>>>>>> the
>>> > >>>>>>>>>>> old
>>> > >>>>>>>>>>>>> cruft
>>> > >>>>>>>>>>>>>>>>> and
>>> > >>>>>>>>>>>>>>>>>>> the
>>> > >>>>>>>>>>>>>>>>>>>>>> no longer
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> relevant
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> or
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> duplicates would be a
>>> > >>>> nice
>>> > >>>>>>>>>>> undertaking
>>> > >>>>>>>>>>>> if
>>> > >>>>>>>>>>>>>>>>> someone
>>> > >>>>>>>>>>>>>>>>>>> is
>>> > >>>>>>>>>>>>>>>>>>>>>> looking
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>> to
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> contribute.
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Alex Newman has been
>>> > >>>>> working
>>> > >>>>>>> on
>>> > >>>>>>>>>>> making
>>> > >>>>>>>>>>>>> our
>>> > >>>>>>>>>>>>>>>>> tests
>>> > >>>>>>>>>>>>>>>>>>> work
>>> > >>>>>>>>>>>>>>>>>>>>>> up on
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> travis
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> circle-ci. That'll
>>> > >> be
>>> > >>>>> sweet
>>> > >>>>>>>> when
>>> > >>>>>>>>>> it
>>> > >>>>>>>>>>>> goes
>>> > >>>>>>>>>>>>>>>>>>> end-to-end.
>>> > >>>>>>>>>>>>>>>>>>>>>> He also
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> added
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> some "type"
>>> > >>>>> categorizations
>>> > >>>>>>> --
>>> > >>>>>>>>>>> client,
>>> > >>>>>>>>>>>>>>>> filter,
>>> > >>>>>>>>>>>>>>>>>>>>>> mapreduce --
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> alongside
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> our
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> old "sizing"
>>> > >>>>> categorizations
>>> > >>>>>>> of
>>> > >>>>>>>>>>>>>>>>>>> small/medium/large.
>>> > >>>>>>>>>>>>>>>>>>>>>> His
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> thinking
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> we can run these
>>> > >>>>>>>> categorizations
>>> > >>>>>>>>> in
>>> > >>>>>>>>>>>>> parallel
>>> > >>>>>>>>>>>>>>>>> so we
>>> > >>>>>>>>>>>>>>>>>>>>>> could run
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> total
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> suite in about the
>>> > >> time
>>> > >>>> of
>>> > >>>>>>> the
>>> > >>>>>>>>>>> longest
>>> > >>>>>>>>>>>>> test,
>>> > >>>>>>>>>>>>>>>>> say
>>> > >>>>>>>>>>>>>>>>>>>>>> 20-30minutes?
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> We
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> could
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> even change Apache to
>>> > >>>> run
>>> > >>>>>>> them
>>> > >>>>>>>>> this
>>> > >>>>>>>>>>>> way.
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> FYI,
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> St.Ack
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Sean
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> busbey
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> busbey
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>> busbey
>>> > >>>>>>>>>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>>>>>>>> --
>>> > >>>>>>>>>>>>>>>>>>>>>>> busbey
>>> > >>>>>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>>>> --
>>> > >>>>>>>>>>>>>>>>>>> busbey
>>> > >>>>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>>> --
>>> > >>>>>>>>>>>>>>>>>> Sean
>>> > >>>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>> --
>>> > >>>>>>>>>>>>>>>>> busbey
>>> > >>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>> --
>>> > >>>>>>>>>>>>>>>> Thanks,
>>> > >>>>>>>>>>>>>>>> Michael Antonov
>>> > >>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>> --
>>> > >>>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>> -- Appy
>>> > >>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>> --
>>> > >>>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>> -- Appy
>>> > >>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>
>>> > >>>>>>>>>>>>>
>>> > >>>>>>>>>>>>> --
>>> > >>>>>>>>>>>>> busbey
>>> > >>>>>>>>>>>>
>>> > >>>>>>>>>>>>
>>> > >>>>>>>>>>>>
>>> > >>>>>>>>>>>> --
>>> > >>>>>>>>>>>> Thanks,
>>> > >>>>>>>>>>>> Michael Antonov
>>> > >>>>>>>>>>>
>>> > >>>>>>>>>>>
>>> > >>>>>>>>>>>
>>> > >>>>>>>>>>> --
>>> > >>>>>>>>>>>
>>> > >>>>>>>>>>> -- Appy
>>> > >>>>>>>>>
>>> > >>>>>>>>>
>>> > >>>>>>>>>
>>> > >>>>>>>>> --
>>> > >>>>>>>>>
>>> > >>>>>>>>> -- Appy
>>> > >>>>>>>>
>>> > >>>>>>>>
>>> > >>>>>>>> --
>>> > >>>>>>>> -Dima
>>> > >>>>>>
>>> > >>>>>>
>>> > >>>>>>
>>> > >>>>>> --
>>> > >>>>>>
>>> > >>>>>> -- Appy
>>> > >>>>>
>>> > >>>>>
>>> > >>>>>
>>> > >>>>> --
>>> > >>>>> busbey
>>> > >>>
>>> > >>>
>>> > >>>
>>> > >>> --
>>> > >>>
>>> > >>> -- Appy
>>> > >>
>>> > >>
>>> > >>
>>> > >> --
>>> > >> busbey
>>> > >
>>> > >
>>> > >
>>> > > --
>>> > > Best regards,
>>> > >
>>> > > - Andy
>>> > >
>>> > > Problems worthy of attack prove their worth by hitting back. - Piet
>>> Hein
>>> > > (via Tom White)
>>> >
>>> >
>>>
>>>
>>> --
>>>
>>> -- Appy
>>>
>>
>>
>>
>> --
>> Best regards,
>>
>> - Andy
>>
>> Problems worthy of attack prove their worth by hitting back. - Piet Hein
>> (via Tom White)
