Hi Josh Sharing steps and my findings for better understanding:
I have tested on below table creation policy (considering that I am 100% aware of pre-splitting but can't use as per our rowkey design) I have to opt some different policy which can evenly distribute the data to all Regions #1 hbase org.apache.hadoop.hbase.util.RegionSplitter test_table HexStringSplit -c 10 -f f1 alter 'test_table', { NAME => 'si', DATA_BLOCK_ENCODING => 'FAST_DIFF' } alter 'test_table', {NAME => 'si', COMPRESSION => 'SNAPPY' #2 create 'TEST_TABLE_KeyPrefixRegionSplitPolicy', {NAME => 'si'}, CONFIG => {'KeyPrefixRegionSplitPolicy.prefix_length'=> '5'} alter 'TEST_TABLE_KeyPrefixRegionSplitPolicy', { NAME => 'si', DATA_BLOCK_ENCODING => 'FAST_DIFF' } alter 'TEST_TABLE_KeyPrefixRegionSplitPolicy', {NAME => 'si', COMPRESSION => 'SNAPPY' #3 Currently I am consdring it and want to distribute data only based on rowkey create 'TEST_TABLE','si',{ NAME => 'si', COMPRESSION => 'SNAPPY' } alter 'TEST_TABLE', { NAME => 'si', DATA_BLOCK_ENCODING => 'FAST_DIFF' } alter 'TEST_TABLE', {NAME => 'si', COMPRESSION => 'SNAPPY' } Thanks Manjeet Singh On Fri, Aug 31, 2018 at 6:49 PM, Josh Elser <els...@apache.org> wrote: > I'd like to remind you again that we're all volunteers and we're helping > you because we choose to do so. Antagonizing those who are helping you is a > great way to stop receiving any free help. > > If you do not create more than one Region, HBase will not distribute your > data on more than one RegionServer. Full stop. > > > On 8/30/18 2:16 PM, Manjeet Singh wrote: > >> Hi Elser >> >> I have clearly total about rowkey does I am talking about data? see below >> what I have told about rowkey >> >> SALT_ID_DayStartTimestamp_DayEndTimeStamp_IDTimeStamp >> >> Problem is this you are not understanding the question and just telling >> what you know, even on slack you are saying same thing. >> Question is simple if I put salt (which can be any arbit char or genrated >> hash any thing) at the begning of the rowkey why my data not getting >> distributed >> Please note this is not pre splitted table. >> >> Thanks >> Manjeet Singh >> >> On Thu, Aug 30, 2018 at 9:11 PM Josh Elser <els...@apache.org> wrote: >> >> As I've been trying to explain in Slack: >>> >>> 1. Are you including the salt in the data that you are writing, such >>> that you are spreading the data across all Regions per their boundaries? >>> Or, as I think you are, just creating split points with this arbitrary >>> "salt" and not including it when you write data? >>> >>> If, as I am assuming, you are not, all of your data will go into the >>> first or last region. If you are still not getting my point, I'd suggest >>> that you share the exact splitpoints and one rowkey that you are writing >>> to HBase. That will make it quite clear if my guess is correct or not. >>> >>> 2. The number of Regions controls the number of RegionServers that will >>> be involved with reads/writes against that table. This is a calculation >>> that you need to figure out based on your cluster configuration and the >>> magnitude of your workload. >>> >>> On 8/30/18 1:11 AM, Manjeet Singh wrote: >>> >>>> Hi All, >>>> >>>> >>>> >>>> I have two Question >>>> >>>> *Question 1 : * >>>> >>>> I want to understand how rowkey distribution happen if I create my table >>>> with out applying any policy but opting prefix salting. >>>> >>>> Example I have rowkey like >>>> >>>> SALT_ID_DayStartTimestamp_DayEndTimeStamp_IDTimeStamp >>>> >>>> So it will look like as below >>>> >>>> *_99_1516838400_1516924800_1516865160 >>>> >>>> Question is : now I can not see that my data is getting distributed only >>>> because of salt. >>>> >>>> So does I have only choice of pre splitting? Or do I have any other >>>> >>> option? >>> >>>> >>>> I have seen two more approaches >>>> >>>> i.e. >>>> >>>> hbase org.apache.hadoop.hbase.util.RegionSplitter test_table >>>> >>> HexStringSplit >>> >>>> -c 10 -f f1 >>>> >>>> I guess its scope is limited as number of region created at the time >>>> >>> table >>> >>>> creation and it will fix? Not sure. >>>> >>>> and >>>> >>>> *UniformSplit >>>> < >>>> >>> https://hbase.apache.org/0.94/apidocs/org/apache/hadoop/hbas >>> e/util/RegionSplitter.UniformSplit.html >>> >>>> * >>>> >>>> >>>> >>>> *Second 2: Does number of split point anywhere related to the number of >>>> >>> RS >>> >>>> in cluster, If yes what is the calculation? * >>>> >>>> >>> >> >> -- luv all