[ https://issues.apache.org/jira/browse/HBASE-5659?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13420314#comment-13420314 ]
Lars Hofhansl commented on HBASE-5659: -------------------------------------- In the latest failed runs I see this: {quote} org.apache.hadoop.hbase.DroppedSnapshotException: region: testtable,,1342893920897.74c4711d8d09b8d84618388ce125cedd. at org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:1474) at org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:1353) at org.apache.hadoop.hbase.regionserver.HRegion.flushcache(HRegion.java:1294) at org.apache.hadoop.hbase.regionserver.TestAtomicOperation$1.run(TestAtomicOperation.java:278) Caused by: java.io.FileNotFoundException: /home/hudson/hudson-slave/workspace/HBase-0.94/trunk/target/test-data/bca87d6a-50fb-4cdc-9e8c-72912928f4ed/TestAtomicOperationtestRowMutationMultiThreads/testtable/74c4711d8d09b8d84618388ce125cedd/.tmp/62b35712575041f28c04f6f953830b80 (Too many open files) at java.io.FileOutputStream.open(Native Method) at java.io.FileOutputStream.<init>(FileOutputStream.java:194) at org.apache.hadoop.fs.RawLocalFileSystem$LocalFSFileOutputStream.<init>(RawLocalFileSystem.java:188) at org.apache.hadoop.fs.RawLocalFileSystem$LocalFSFileOutputStream.<init>(RawLocalFileSystem.java:184) at org.apache.hadoop.fs.RawLocalFileSystem.create(RawLocalFileSystem.java:255) at org.apache.hadoop.fs.RawLocalFileSystem.create(RawLocalFileSystem.java:236) at org.apache.hadoop.fs.ChecksumFileSystem$ChecksumFSOutputSummer.<init>(ChecksumFileSystem.java:335) at org.apache.hadoop.fs.ChecksumFileSystem.create(ChecksumFileSystem.java:381) at org.apache.hadoop.fs.ChecksumFileSystem.create(ChecksumFileSystem.java:364) at org.apache.hadoop.hbase.util.FSUtils.create(FSUtils.java:151) at org.apache.hadoop.hbase.util.FSUtils.create(FSUtils.java:126) at org.apache.hadoop.hbase.io.hfile.AbstractHFileWriter.createOutputStream(AbstractHFileWriter.java:271) at org.apache.hadoop.hbase.io.hfile.HFile$WriterFactory.create(HFile.java:393) at org.apache.hadoop.hbase.regionserver.StoreFile$Writer.<init>(StoreFile.java:956) at org.apache.hadoop.hbase.regionserver.StoreFile$Writer.<init>(StoreFile.java:901) at org.apache.hadoop.hbase.regionserver.StoreFile$WriterBuilder.build(StoreFile.java:815) at org.apache.hadoop.hbase.regionserver.Store.createWriterInTmp(Store.java:836) at org.apache.hadoop.hbase.regionserver.Store.createWriterInTmp(Store.java:816) at org.apache.hadoop.hbase.regionserver.Store.internalFlushCache(Store.java:717) at org.apache.hadoop.hbase.regionserver.Store.flushCache(Store.java:674) at org.apache.hadoop.hbase.regionserver.Store.access$400(Store.java:108) at org.apache.hadoop.hbase.regionserver.Store$StoreFlusherImpl.flushCache(Store.java:2247) at org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:1449) ... 3 more {quote} > TestAtomicOperation.testMultiRowMutationMultiThreads is still failing > occasionally > ---------------------------------------------------------------------------------- > > Key: HBASE-5659 > URL: https://issues.apache.org/jira/browse/HBASE-5659 > Project: HBase > Issue Type: Sub-task > Reporter: Lars Hofhansl > Assignee: Lars Hofhansl > Priority: Minor > Fix For: 0.96.0, 0.94.2 > > Attachments: 5659.txt > > > See run here: > https://builds.apache.org/job/PreCommit-HBASE-Build/1318//testReport/org.apache.hadoop.hbase.regionserver/TestAtomicOperation/testMultiRowMutationMultiThreads/ > {quote} > 2012-03-27 04:36:12,627 DEBUG [Thread-118] regionserver.StoreScanner(499): > Storescanner.peek() is changed where before = > rowB/colfamily11:qual1/7202/Put/vlen=6/ts=7922,and after = > rowB/colfamily11:qual1/7199/DeleteColumn/vlen=0/ts=0 > 2012-03-27 04:36:12,629 INFO [Thread-121] regionserver.HRegion(1558): > Finished memstore flush of ~2.9k/2952, currentsize=1.6k/1640 for region > testtable,,1332822963417.7cd30e219714cfc5e91f69def66e7f81. in 14ms, > sequenceid=7927, compaction requested=true > 2012-03-27 04:36:12,629 DEBUG [Thread-126] > regionserver.TestAtomicOperation$2(362): flushing > 2012-03-27 04:36:12,630 DEBUG [Thread-126] regionserver.HRegion(1426): > Started memstore flush for > testtable,,1332822963417.7cd30e219714cfc5e91f69def66e7f81., current region > memstore size 1.9k > 2012-03-27 04:36:12,630 DEBUG [Thread-126] regionserver.HRegion(1474): > Finished snapshotting > testtable,,1332822963417.7cd30e219714cfc5e91f69def66e7f81., commencing wait > for mvcc, flushsize=1968 > 2012-03-27 04:36:12,630 DEBUG [Thread-126] regionserver.HRegion(1484): > Finished snapshotting, commencing flushing stores > 2012-03-27 04:36:12,630 DEBUG [Thread-126] util.FSUtils(153): Creating > file=/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/trunk/target/test-data/b9091c3c-961e-4035-850a-83ad14d517cc/TestAtomicOperationtestMultiRowMutationMultiThreads/testtable/7cd30e219714cfc5e91f69def66e7f81/.tmp/61954619003e469baf1a34be5ff2ec57 > with permission=rwxrwxrwx > 2012-03-27 04:36:12,631 DEBUG [Thread-126] hfile.HFileWriterV2(143): > Initialized with CacheConfig:enabled [cacheDataOnRead=true] > [cacheDataOnWrite=false] [cacheIndexesOnWrite=false] > [cacheBloomsOnWrite=false] [cacheEvictOnClose=false] [cacheCompressed=false] > 2012-03-27 04:36:12,631 INFO [Thread-126] > regionserver.StoreFile$Writer(997): Delete Family Bloom filter type for > /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/trunk/target/test-data/b9091c3c-961e-4035-850a-83ad14d517cc/TestAtomicOperationtestMultiRowMutationMultiThreads/testtable/7cd30e219714cfc5e91f69def66e7f81/.tmp/61954619003e469baf1a34be5ff2ec57: > CompoundBloomFilterWriter > 2012-03-27 04:36:12,632 INFO [Thread-126] > regionserver.StoreFile$Writer(1220): NO General Bloom and NO DeleteFamily was > added to HFile > (/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/trunk/target/test-data/b9091c3c-961e-4035-850a-83ad14d517cc/TestAtomicOperationtestMultiRowMutationMultiThreads/testtable/7cd30e219714cfc5e91f69def66e7f81/.tmp/61954619003e469baf1a34be5ff2ec57) > > 2012-03-27 04:36:12,632 INFO [Thread-126] regionserver.Store(770): Flushed , > sequenceid=7934, memsize=1.9k, into tmp file > /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/trunk/target/test-data/b9091c3c-961e-4035-850a-83ad14d517cc/TestAtomicOperationtestMultiRowMutationMultiThreads/testtable/7cd30e219714cfc5e91f69def66e7f81/.tmp/61954619003e469baf1a34be5ff2ec57 > 2012-03-27 04:36:12,632 DEBUG [Thread-126] regionserver.Store(795): Renaming > flushed file at > /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/trunk/target/test-data/b9091c3c-961e-4035-850a-83ad14d517cc/TestAtomicOperationtestMultiRowMutationMultiThreads/testtable/7cd30e219714cfc5e91f69def66e7f81/.tmp/61954619003e469baf1a34be5ff2ec57 > to > /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/trunk/target/test-data/b9091c3c-961e-4035-850a-83ad14d517cc/TestAtomicOperationtestMultiRowMutationMultiThreads/testtable/7cd30e219714cfc5e91f69def66e7f81/colfamily11/61954619003e469baf1a34be5ff2ec57 > 2012-03-27 04:36:12,634 INFO [Thread-126] regionserver.Store(818): Added > /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/trunk/target/test-data/b9091c3c-961e-4035-850a-83ad14d517cc/TestAtomicOperationtestMultiRowMutationMultiThreads/testtable/7cd30e219714cfc5e91f69def66e7f81/colfamily11/61954619003e469baf1a34be5ff2ec57, > entries=12, sequenceid=7934, filesize=1.3k > 2012-03-27 04:36:12,642 DEBUG [Thread-118] > regionserver.TestAtomicOperation$2(392): [] > Exception in thread "Thread-118" junit.framework.AssertionFailedError at > junit.framework.Assert.fail(Assert.java:48) > at junit.framework.Assert.fail(Assert.java:56) > at > org.apache.hadoop.hbase.regionserver.TestAtomicOperation$2.run(TestAtomicOperation.java:394) > 2012-03-27 04:36:12,643 INFO [Thread-126] regionserver.HRegion(1558): > Finished memstore flush of ~1.9k/1968, currentsize=1.3k/1312 for region > testtable,,1332822963417.7cd30e219714cfc5e91f69def66e7f81. in 14ms, > sequenceid=7934, compaction requested=true > {quote} > I think this issue is different from parent. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira