Ok, thanks for confirming that the Impala table is backed by Parquet. It's not surprising to hear that you can get higher throughput while loading a Parquet table compared to Phoenix. Even when using Parquet, the file structure used by Impala is quite lightweight compared to HFiles. For example, each column in non-key row in Phoenix contains the full row key and the name of the column qualifier, and all of these need to be sorted before writing them as HFiles.
The trade-off is of course that you can query single rows extremely quickly in HBase, whereas that won't be as performant in Impala. By using the suggestions I gave, you will likely be able to improve performance of the loading, but I think that creating parquet files for Impala will always give substantially higher throughput. - Gabriel On Tue, Dec 23, 2014 at 1:40 PM, Bulvik, Noam <[email protected]> wrote: > Thanks for the answer we will look in to it and update > The impala is impala parquet table > > -----Original Message----- > From: Gabriel Reid [mailto:[email protected]] > Sent: Tuesday, December 23, 2014 2:27 PM > To: [email protected] > Subject: Re: CSV bulk loading using map reduce > > Hi Noam, > > I think that the things that most typically can affect MR loading performance > are: > * number of regions (as this affects the number of reducers used to create > the HFiles) > * amount of memory used for sort buffers > * use of compression on map output > > With your 32-region salted table, it sounds like things should be > well-balanced, so that shouldn't be an issue. > > I would expect the biggest impact you can possibly get is by tuning the sort > buffers. You can spot if there are potential issues here by looking at the > "Spilled records" counter in the list of job counters in the job tracker. If > you're getting multiple times the number of spilled records vs the number of > output records, then it'll probably be advantageous to up the sort buffer > settings. There's an old but good overview of these concepts starting from > slide 12 in this > presentation: http://www.slideshare.net/cloudera/mr-perf > > About the compression on map output, the mapreduce.map.output.compress should > almost certainly be set to true, with (probably) snappy used for compression. > Having this set to true compresses the output of the map task before it's > sent across the network to the reducers. Since HBase KeyValues are very > bulky, having this setting turned on can have quite an impact on throughput. > > As far as the performance comparison with Impala, I assume you're referring > to impala that is backed by text files or Parquet, correct? > > - Gabriel > > > On Tue, Dec 23, 2014 at 9:43 AM, Bulvik, Noam <[email protected]> wrote: >> We are testing the CSV bulk loader tool (using MR) and trying to >> understand what are the parameter that can be used to enhanced its >> performance. >> >> we have 8 nodes with 4 disks each (1 core per disk) we created 32 >> regions and pre-split our table using salt, the table has 55 columns >> >> >> >> So far we managed to load ~40,000 rows per second (from csv file of ~ >> 8G), when we compare this loading to impala loading on the same >> cluster, same table structure we see rates of about 300,000 rows per seconds. >> >> >> >> Any idea which parameters of Hbase /Phoenix can we use to enhance the >> performance >> >> >> >> Regards, >> >> Noam >> >> >> >> Information in this e-mail and its attachments is confidential and >> privileged under the TEOCO confidentiality terms that can be reviewed here. > Information in this e-mail and its attachments is confidential and privileged > under the TEOCO confidentiality terms that can be reviewed > here<http://www.teoco.com/email-disclaimer>.
