[
https://issues.apache.org/jira/browse/HDDS-6114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17460994#comment-17460994
]
Attila Doroszlai edited comment on HDDS-6114 at 12/16/21, 7:29 PM:
-------------------------------------------------------------------
These failures appeared after HDDS-5208.
Previous fix attempt: HDDS-5632. Seems like it was not enough:
{code:title=https://github.com/elek/ozone-build-results/blob/master/2021/10/14/10887/it-ozone/hadoop-ozone/integration-test/org.apache.hadoop.ozone.om.TestOzoneManagerBootstrap.txt}
Tests run: 3, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 133.918 s <<<
FAILURE! - in org.apache.hadoop.ozone.om.TestOzoneManagerBootstrap
testBootstrapTwoNewOMs Time elapsed: 16.205 s <<< ERROR!
java.io.IOException: Failed init RocksDB, db path :
/home/runner/work/ozone/ozone/hadoop-ozone/integration-test/target/test-dir/MiniOzoneClusterImpl-e4ff273a-68e1-4f64-ab71-586128804aef/omNode-1/om.db,
exception :org.rocksdb.RocksDBException lock hold by current process, acquire
time 1634239155 acquiring thread 140708968617728:
/home/runner/work/ozone/ozone/hadoop-ozone/integration-test/target/test-dir/MiniOzoneClusterImpl-e4ff273a-68e1-4f64-ab71-586128804aef/omNode-1/om.db/LOCK:
No locks available; status : IOError; message : lock hold by current process,
acquire time 1634239155 acquiring thread 140708968617728:
/home/runner/work/ozone/ozone/hadoop-ozone/integration-test/target/test-dir/MiniOzoneClusterImpl-e4ff273a-68e1-4f64-ab71-586128804aef/omNode-1/om.db/LOCK:
No locks available
at
org.apache.hadoop.hdds.utils.HddsServerUtil.toIOException(HddsServerUtil.java:564)
at org.apache.hadoop.hdds.utils.db.RDBStore.<init>(RDBStore.java:164)
at
org.apache.hadoop.hdds.utils.db.DBStoreBuilder.build(DBStoreBuilder.java:191)
at
org.apache.hadoop.ozone.om.OmMetadataManagerImpl.loadDB(OmMetadataManagerImpl.java:400)
at
org.apache.hadoop.ozone.om.OmMetadataManagerImpl.loadDB(OmMetadataManagerImpl.java:390)
at
org.apache.hadoop.ozone.om.OmMetadataManagerImpl.start(OmMetadataManagerImpl.java:382)
at
org.apache.hadoop.ozone.om.OmMetadataManagerImpl.<init>(OmMetadataManagerImpl.java:247)
at
org.apache.hadoop.ozone.om.OzoneManager.instantiateServices(OzoneManager.java:612)
at org.apache.hadoop.ozone.om.OzoneManager.<init>(OzoneManager.java:521)
at
org.apache.hadoop.ozone.om.OzoneManager.createOm(OzoneManager.java:578)
at
org.apache.hadoop.ozone.MiniOzoneHAClusterImpl$Builder.createOMService(MiniOzoneHAClusterImpl.java:476)
at
org.apache.hadoop.ozone.MiniOzoneHAClusterImpl$Builder.build(MiniOzoneHAClusterImpl.java:387)
at
org.apache.hadoop.ozone.om.TestOzoneManagerBootstrap.setupCluster(TestOzoneManagerBootstrap.java:91)
at
org.apache.hadoop.ozone.om.TestOzoneManagerBootstrap.setupCluster(TestOzoneManagerBootstrap.java:80)
at
org.apache.hadoop.ozone.om.TestOzoneManagerBootstrap.testBootstrapTwoNewOMs(TestOzoneManagerBootstrap.java:179)
{code}
was (Author: adoroszlai):
These failures appeared after HDDS-5208. Related fix: HDDS-5632.
> Intermittent error due to Failed to init RocksDB
> ------------------------------------------------
>
> Key: HDDS-6114
> URL: https://issues.apache.org/jira/browse/HDDS-6114
> Project: Apache Ozone
> Issue Type: Bug
> Components: test
> Reporter: Attila Doroszlai
> Priority: Major
>
> {code:title=https://github.com/elek/ozone-build-results/blob/master/2021/11/23/11642/it-ozone/hadoop-ozone/integration-test/org.apache.hadoop.ozone.scm.TestStorageContainerManagerHA.txt}
> Tests run: 3, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 176.373 s <<<
> FAILURE! - in org.apache.hadoop.ozone.scm.TestStorageContainerManagerHA
> testBootStrapSCM Time elapsed: 23.788 s <<< ERROR!
> java.io.IOException: Failed init RocksDB, db path :
> /home/runner/work/ozone/ozone/hadoop-ozone/integration-test/target/test-dir/MiniOzoneClusterImpl-7ab73b5c-8ac7-4e67-b2fd-a2c0bde55e77/omNode-2/om.db,
> exception :org.rocksdb.RocksDBException lock hold by current process,
> acquire time 1637654016 acquiring thread 139638819440384:
> /home/runner/work/ozone/ozone/hadoop-ozone/integration-test/target/test-dir/MiniOzoneClusterImpl-7ab73b5c-8ac7-4e67-b2fd-a2c0bde55e77/omNode-2/om.db/LOCK:
> No locks available; status : IOError; message : lock hold by current
> process, acquire time 1637654016 acquiring thread 139638819440384:
> /home/runner/work/ozone/ozone/hadoop-ozone/integration-test/target/test-dir/MiniOzoneClusterImpl-7ab73b5c-8ac7-4e67-b2fd-a2c0bde55e77/omNode-2/om.db/LOCK:
> No locks available
> at
> org.apache.hadoop.hdds.utils.HddsServerUtil.toIOException(HddsServerUtil.java:564)
> at org.apache.hadoop.hdds.utils.db.RDBStore.<init>(RDBStore.java:164)
> at
> org.apache.hadoop.hdds.utils.db.DBStoreBuilder.build(DBStoreBuilder.java:191)
> at
> org.apache.hadoop.ozone.om.OmMetadataManagerImpl.loadDB(OmMetadataManagerImpl.java:397)
> at
> org.apache.hadoop.ozone.om.OmMetadataManagerImpl.loadDB(OmMetadataManagerImpl.java:387)
> at
> org.apache.hadoop.ozone.om.OmMetadataManagerImpl.start(OmMetadataManagerImpl.java:379)
> at
> org.apache.hadoop.ozone.om.OmMetadataManagerImpl.<init>(OmMetadataManagerImpl.java:246)
> at
> org.apache.hadoop.ozone.om.OzoneManager.instantiateServices(OzoneManager.java:640)
> at org.apache.hadoop.ozone.om.OzoneManager.<init>(OzoneManager.java:528)
> at
> org.apache.hadoop.ozone.om.OzoneManager.createOm(OzoneManager.java:606)
> at
> org.apache.hadoop.ozone.MiniOzoneHAClusterImpl$Builder.createOMService(MiniOzoneHAClusterImpl.java:494)
> at
> org.apache.hadoop.ozone.MiniOzoneHAClusterImpl$Builder.build(MiniOzoneHAClusterImpl.java:405)
> at
> org.apache.hadoop.ozone.scm.TestStorageContainerManagerHA.init(TestStorageContainerManagerHA.java:103)
> {code}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
