Export is a MapReduce job, and HBase will only configure a maximum of
one Mapper per Region in the table being scanned.
If you have multiple regions for your tsdb table, then it's possible
that you need to tweak the concurrency on the YARN side such that you
have multiple Mappers running in parallel?
Sounds like looking at the YARN Application log and UI is your next best
bet.
On 8/18/21 4:52 AM, Nguyen, Tai Van (EXT - VN) wrote:
Hi HBase Team
Image can see here :
* Export with single regionserver: https://imgur.com/86wSUMV
<https://imgur.com/86wSUMV>
* Export with two regionservers: https://imgur.com/a/XMovlZx
<https://imgur.com/a/XMovlZx>
Log show about time was:
root@solaltiplano-track4-master:~/hbase-exporting/latest# cat
hbase_export_compress_default.log | grep export
Starting hbase export at Fri Jun 11 12:22:46 UTC 2021
tsdb table exported in 6279 seconds
tsdb-meta table exported in 6 seconds
tsdb-tree table exported in 7 seconds
tsdb-uid table exported in 90 seconds
Ending hbase export at Fri Jun 11 14:09:08 UTC 2021
*
Thanks,
Tai
------------------------------------------------------------------------
*From:* Mathews, Jacob 1. (Nokia - IN/Bangalore) <[email protected]>
*Sent:* Monday, August 16, 2021 6:47 PM
*To:* Nguyen, Tai Van (EXT - VN) <[email protected]>
*Subject:* FW: Hbase export is very slow - help needed
*From:*Mathews, Jacob 1. (Nokia - IN/Bangalore)
*Sent:* Friday, August 6, 2021 12:38 PM
*To:* [email protected]
*Subject:* Hbase export is very slow - help needed
Hi HBase team,
We are trying to use Hbase export mentioned here:
http://hbase.apache.org/book.html#export
<http://hbase.apache.org/book.html#export>
But it is happening sequentially row by row as seen from the logs.
we tried many options of the Hbase export, but all were taking long time.
Backup folder contents size:
bash-4.2$ du -kh
16K ./tsdb-tree
16K ./tsdb-meta
60M ./tsdb-uid
5.9G ./tsdb
6.0G .
took around 104 minutes for 6gb compressed data.
Is there a way we can parallelise this and improve the export time.
Below are the charts from Hbase .
Export with single regionserver:
Export with two regionservers:
Scaling the HBase Region server also did not help, the export still
happens sequentially.
Thanks
Jacob Mathews
