[
https://issues.apache.org/jira/browse/HBASE-14247?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Liu Shaohui updated HBASE-14247:
--------------------------------
Description:
Currently all old WALs of regionservers are achieved into the single directory
of oldWALs. In big clusters, because of long TTL of WAL or disabled
replications, the number of files under oldWALs may reach the
max-directory-items limit of HDFS, which will make the hbase cluster crashed.
{quote}
Caused by:
org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.hdfs.protocol.FSLimitException$MaxDirectoryItemsExceededException):
The directory item limit of /hbase/lgprc-xiaomi/.oldlogs is exceeded:
limit=1048576 items=1048576
{quote}
A simple solution is to separate the old WALs into different directories
according to the server name of the WAL.
Suggestions are welcomed~ Thanks
was:
Currently all old WALs of regionservers are achieved into the single directory
of oldWALs. In big clusters, because of long TTL of WAL or disabled
replications, the number of files under oldWALs may reach the
max-directory-items limit of HDFS, which will make the hbase cluster crashed.
```
Caused by:
org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.hdfs.protocol.FSLimitException$MaxDirectoryItemsExceededException):
The directory item limit of /hbase/lgprc-xiaomi/.oldlogs is exceeded:
limit=1048576 items=1048576
```
A simple solution is to separate the old WALs into different directories
according to the server name of the WAL.
Suggestions are welcomed~ Thanks
> Separate the old WALs into different regionserver directories
> -------------------------------------------------------------
>
> Key: HBASE-14247
> URL: https://issues.apache.org/jira/browse/HBASE-14247
> Project: HBase
> Issue Type: Improvement
> Reporter: Liu Shaohui
> Assignee: Liu Shaohui
> Priority: Minor
>
> Currently all old WALs of regionservers are achieved into the single
> directory of oldWALs. In big clusters, because of long TTL of WAL or disabled
> replications, the number of files under oldWALs may reach the
> max-directory-items limit of HDFS, which will make the hbase cluster crashed.
> {quote}
> Caused by:
> org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.hdfs.protocol.FSLimitException$MaxDirectoryItemsExceededException):
> The directory item limit of /hbase/lgprc-xiaomi/.oldlogs is exceeded:
> limit=1048576 items=1048576
> {quote}
> A simple solution is to separate the old WALs into different directories
> according to the server name of the WAL.
> Suggestions are welcomed~ Thanks
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
