The HDFS tests fail unless your umask is (from memory) 0022. Otherwise it can't delete files it creates or something.
The exec source thing appears to be a race condition. I keep running the build until i get success. Yes, I agree, frustrating on Bamboo. Mike said, "I've done a small proof of concept on some flume sink customizations in 1.3.1. I want to incorporate these customizations into our build server (Bamboo). I started by creating a local git repo of 1.4 and I'm able to run a mvn clean install just fine. However when I push the raw un-altered code to our build server the build fails and not consistently. Half the time TestExecSource fails in flume-ng-core: build 01-Aug-2013 22:56:11 testMonitoredCounterGroup(org. apache.flume.source.TestExecSource) Time elapsed: 1264 sec <<< FAILURE! build 01-Aug-2013 22:56:11 org.junit.ComparisonFailure: Expected Value: 5 expected:<[5]> but was:<[0]> build 01-Aug-2013 22:56:11 at org.junit.Assert.assertEquals( Assert.java:125) build 01-Aug-2013 22:56:11 at org.apache.flume.source. TestExecSource.testMonitoredCounterGroup(TestExecSource.java:215) When that passes my build fails in flume-hdfs-sink build 02-Aug-2013 00:03:30 simpleHDFSGZipCompressedTest( org.apache.flume.sink.hdfs.TestHDFSEventSinkOnMiniCluster) Time elapsed: 2676 sec <<< ERROR! build 02-Aug-2013 00:03:30 java.lang.NullPointerException build 02-Aug-2013 00:03:30 at org.apache.hadoop.hdfs. MiniDFSCluster.startDataNodes(MiniDFSCluster.java:422) build 02-Aug-2013 00:03:30 at org.apache.hadoop.hdfs. MiniDFSCluster.<init>(MiniDFSCluster.java:280) build 02-Aug-2013 00:03:30 at org.apache.hadoop.hdfs. MiniDFSCluster.<init>(MiniDFSCluster.java:124) build 02-Aug-2013 00:03:30 at org.apache.flume.sink.hdfs. TestHDFSEventSinkOnMiniCluster.simpleHDFSGZipCompressedTest( TestHDFSEventSinkOnMiniCluster.java:164) ... build 02-Aug-2013 00:03:30 underReplicationTest(org. apache.flume.sink.hdfs.TestHDFSEventSinkOnMiniCluster) Time elapsed: 31 sec <<< ERROR! build 02-Aug-2013 00:03:30 java.io.IOException: Cannot lock storage target/test/dfs/dfs/name1. The directory is already locked. build 02-Aug-2013 00:03:30 at org.apache.hadoop.hdfs.server. common.Storage$StorageDirectory.lock(Storage.java:602) build 02-Aug-2013 00:03:30 at org.apache.hadoop.hdfs.server. namenode.FSImage.format(FSImage.java:1219) build 02-Aug-2013 00:03:30 at org.apache.hadoop.hdfs.server. namenode.FSImage.format(FSImage.java:1237) build 02-Aug-2013 00:03:30 at org.apache.hadoop.hdfs.server. namenode.NameNode.format(NameNode.java:1164) build 02-Aug-2013 00:03:30 at org.apache.hadoop.hdfs.server. namenode.NameNode.format(NameNode.java:184) build 02-Aug-2013 00:03:30 at org.apache.hadoop.hdfs. MiniDFSCluster.<init>(MiniDFSCluster.java:267) build 02-Aug-2013 00:03:30 at org.apache.hadoop.hdfs. MiniDFSCluster.<init>(MiniDFSCluster.java:124) I'm at a loss to explain the inconsistency of flume-ng-core and when it does succeed how to solve the failure of HDFS-sink. Any advice would be appreciated. Thanks, Mike"
