Hi Ling,
We have identified this as an issue and are working on a
solution for 0.20.
See https://issues.apache.org/jira/browse/HBASE-1410
In the meantime, there may be a way to recover your current
situation, for 0.19. The trick may be the following dual
strategy:
1) Increase the available heap for the region server
temporarily to the maximum available for the amount of
RAM in the server;
2) Temporarily skip a substantial number of index entries
to lessen the heap load required to hold the storefile
indexes in memory, by adding something like the
following in conf/hbase-site.xml:
<property>
<name>io.map.index.skip</name>
<value>32</value>
</property>
There is a trade off here between reducing the number
of keys read into the in memory index and yet leaving
enough keys in place so compaction/split can find a
midkey. If 16 does not work, you can consider trying
64, 128, 256, 512, 1024.
Please consider trying this approach. If it does not work
to clear the issue, then come back and we can think about a
next step.
> From: Ling Qian
> Subject: Re: OOME when restarting hbase
> To: [email protected], [email protected]
> Date: Tuesday, May 12, 2009, 2:12 AM
>
> Is there any method to save the 4450 hstorefiles? ( about
> 53GB on DFS)
- Andy
