So applying the bit mask workaround from HBASE-5711 fixes the HFileTargetIT test, but I have this deja vu-like feeling that we ran into something like this before, but the Internet at this hotel isn't quite solid enough for me to figure out when it was. Will keep poking around.
J On Tue, Oct 14, 2014 at 9:17 PM, Josh Wills <[email protected]> wrote: > I can replicate the original error- looks like dfs.data.dir permissions > aren't set properly in test mode for some reason. Will investigate and > figure out a patch. > > > On Tuesday, October 14, 2014, Micah Whitacre <[email protected]> wrote: > >> One thing I noticed is that is when testing we don't force IPv4. >> Zookeeper >> does not handle IPv6 well and can actually significantly slow down tests >> trying to resolve the localhost. This doesn't affect everyone because >> your >> /etc/hosts could help it resolve faster but does make machine setup >> important. We could try adding an argline like this to surefire/failsafe >> to enforce IPv4. >> >> <argLine>-Xmx512m -XX:MaxPermSize=128m >> -Djava.net.preferIPv4Stack=true</argLine> >> >> On Tue, Oct 14, 2014 at 9:28 AM, Danushka Menikkumbura < >> [email protected]> wrote: >> >> > Thanks J! >> > >> > On Tue, Oct 14, 2014 at 9:57 AM, Josh Wills <[email protected]> >> wrote: >> > >> > > I'm still rocking 1.7. Will give 1.8 a whirl this evening. >> > > >> > > J >> > > >> > > On Tue, Oct 14, 2014 at 6:53 AM, Danushka Menikkumbura < >> > > [email protected]> wrote: >> > > >> > > > I am on Ubuntu 14.04.1 LTS and Java 1.8.0_20. >> > > > >> > > > BTW maybe it is a result of the commit >> > > > 3f98411364cec32a0a8c6681dfaabd43caa4dd60?. >> > > > >> > > > Thanks, >> > > > Danushka >> > > > >> > > > >> > > > >> > > > On Tue, Oct 14, 2014 at 9:44 AM, Josh Wills <[email protected]> >> > wrote: >> > > > >> > > > > No, master is compatible w/Hadoop 1; the error you're seeing is >> > caused >> > > by >> > > > > the HBase testing code being flaky. I've experienced the flaky >> > > > > HFileTargetIT test on my machine before, but never on a regular >> > basis. >> > > > Can >> > > > > you give me the basics of your setup-- OS, java version, etc.? >> > > > > >> > > > > On Tue, Oct 14, 2014 at 6:38 AM, Danushka Menikkumbura < >> > > > > [email protected]> wrote: >> > > > > >> > > > > > No I don't. >> > > > > > >> > > > > > That means the master branch is not compatible with Hadoop 1? >> > > > > > >> > > > > > Thanks, >> > > > > > Danushka >> > > > > > >> > > > > > On Tue, Oct 14, 2014 at 9:26 AM, Josh Wills < >> [email protected]> >> > > > wrote: >> > > > > > >> > > > > > > Hrm, okay. Do you get it if you use the -Dcrunch.platform=2 >> > option? >> > > > > > > >> > > > > > > On Tue, Oct 14, 2014 at 6:23 AM, Danushka Menikkumbura < >> > > > > > > [email protected]> wrote: >> > > > > > > >> > > > > > > > Yes. I get it every time I try to build HBase module. >> > > > > > > > >> > > > > > > > FYI : org.apache.crunch.io.hbase.HFileTargetIT is the test >> in >> > > > error. >> > > > > > > > >> > > > > > > > Thanks, >> > > > > > > > Danushka >> > > > > > > > >> > > > > > > > On Tue, Oct 14, 2014 at 9:15 AM, Josh Wills < >> > [email protected] >> > > > >> > > > > > wrote: >> > > > > > > > >> > > > > > > > > That can happen intermittently if the local HBase cluster >> > gets >> > > > hung >> > > > > > > up-- >> > > > > > > > do >> > > > > > > > > you get it regularly (i.e., every time you run?) >> > > > > > > > > >> > > > > > > > > J >> > > > > > > > > >> > > > > > > > > On Tue, Oct 14, 2014 at 6:08 AM, Danushka Menikkumbura < >> > > > > > > > > [email protected]> wrote: >> > > > > > > > > >> > > > > > > > > > Hi, >> > > > > > > > > > >> > > > > > > > > > I am getting the following test failure while building >> > > Crunch. >> > > > > Have >> > > > > > > you >> > > > > > > > > got >> > > > > > > > > > an idea as to what may be the issue here?. >> > > > > > > > > > >> > > > > > > > > > 34787 [Thread-2057] INFO >> > > > > > > > > > >> > > > > > > > >> > > > >> org.apache.crunch.hadoop.mapreduce.lib.jobcontrol.CrunchControlledJob >> > > > > > - >> > > > > > > > > > Job status available at: http://localhost:8080/ >> > > > > > > > > > 44548 [M:0;danushka:38318.oldLogCleaner] ERROR >> > > > > > > > > > org.apache.hadoop.hbase.client.HConnectionManager - >> > > Connection >> > > > > not >> > > > > > > > found >> > > > > > > > > > in the list, can't delete it (connection >> > > > > > > > > > >> > > > key=HConnectionKey{properties={hbase.zookeeper.quorum=localhost, >> > > > > > > > > > hbase.rpc.timeout=60000, >> > > > > hbase.zookeeper.property.clientPort=57963, >> > > > > > > > > > zookeeper.znode.parent=/hbase, >> > > hbase.client.retries.number=350, >> > > > > > > > > > hbase.client.pause=100}, username='danushka'}). May be >> the >> > > key >> > > > > was >> > > > > > > > > > modified? >> > > > > > > > > > java.lang.Exception >> > > > > > > > > > at >> > > > > > > > > > >> > > > > > > > > > >> > > > > > > > > >> > > > > > > > >> > > > > > > >> > > > > > >> > > > > >> > > > >> > > >> > >> org.apache.hadoop.hbase.client.HConnectionManager.deleteConnection(HConnectionManager.java:493) >> > > > > > > > > > at >> > > > > > > > > > >> > > > > > > > > > >> > > > > > > > > >> > > > > > > > >> > > > > > > >> > > > > > >> > > > > >> > > > >> > > >> > >> org.apache.hadoop.hbase.client.HConnectionManager.deleteConnection(HConnectionManager.java:429) >> > > > > > > > > > at >> > > > > > > > > > >> > > > > > > > > > >> > > > > > > > > >> > > > > > > > >> > > > > > > >> > > > > > >> > > > > >> > > > >> > > >> > >> org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner.stop(ReplicationLogCleaner.java:141) >> > > > > > > > > > at >> > > > > > > > > > >> > > > > > > > > > >> > > > > > > > > >> > > > > > > > >> > > > > > > >> > > > > > >> > > > > >> > > > >> > > >> > >> org.apache.hadoop.hbase.master.cleaner.CleanerChore.cleanup(CleanerChore.java:276) >> > > > > > > > > > at org.apache.hadoop.hbase.Chore.run(Chore.java:94) >> > > > > > > > > > at java.lang.Thread.run(Thread.java:745) >> > > > > > > > > > Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time >> > > elapsed: >> > > > > > > 21.625 >> > > > > > > > > sec >> > > > > > > > > > >> > > > > > > > > > Thanks, >> > > > > > > > > > Danushka >> > > > > > > > > > >> > > > > > > > > >> > > > > > > > > >> > > > > > > > > >> > > > > > > > > -- >> > > > > > > > > Director of Data Science >> > > > > > > > > Cloudera <http://www.cloudera.com> >> > > > > > > > > Twitter: @josh_wills <http://twitter.com/josh_wills> >> > > > > > > > > >> > > > > > > > >> > > > > > > >> > > > > > > >> > > > > > > >> > > > > > > -- >> > > > > > > Director of Data Science >> > > > > > > Cloudera <http://www.cloudera.com> >> > > > > > > Twitter: @josh_wills <http://twitter.com/josh_wills> >> > > > > > > >> > > > > > >> > > > > >> > > > > >> > > > > >> > > > > -- >> > > > > Director of Data Science >> > > > > Cloudera <http://www.cloudera.com> >> > > > > Twitter: @josh_wills <http://twitter.com/josh_wills> >> > > > > >> > > > >> > > >> > > >> > > >> > > -- >> > > Director of Data Science >> > > Cloudera <http://www.cloudera.com> >> > > Twitter: @josh_wills <http://twitter.com/josh_wills> >> > > >> > >> > > > -- > Director of Data Science > Cloudera <http://www.cloudera.com> > Twitter: @josh_wills <http://twitter.com/josh_wills> > > -- Director of Data Science Cloudera <http://www.cloudera.com> Twitter: @josh_wills <http://twitter.com/josh_wills>
