Original Message-
From: Amit Sela [mailto:am...@infolinks.com]
Sent: Tuesday, December 17, 2013 7:47 PM
To: user@hbase.apache.org
Subject: Re: Bulk load moving HFiles to the wrong region
Indeed there are more than 2 split points, there are 4 split points for 5 new
regions added each day.
t
me how did you run the split for each new day? It
> seems there may have more than 2 split points.
>
> Thanks.
> Jieshan
>
> -Original Message-
> From: Amit Sela [mailto:am...@infolinks.com]
> Sent: Tuesday, December 17, 2013 6:10 PM
> To: user@hbase.apache.org
>
t;> regions) after pre-split and before the bulkload, but that
>> >>>> didn't
>> help.
>> >>>>
>> >>>>
>> >>>>
>> >>>> On Mon, Dec 16, 2013 at 3:07 PM, Bijieshan > >wrote:
>> >>>
) to MyHFileOutputFormat.configureIncrementalLoad() so that
>> getRegionStartKeys returns only the corresponding keys.
>> >I did this in order to avoid having 2000 reducers when my target is 15
>> regions...
>>
>> We always do like this:). Only configure the necessary
returns only the corresponding keys.
> >I did this in order to avoid having 2000 reducers when my target is 15
> regions...
>
> We always do like this:). Only configure the necessary regions.
>
> Sorry for the lately reply.
>
> Jieshan
> -----Original Message-
&
ry regions.
Sorry for the lately reply.
Jieshan
-Original Message-
From: Amit Sela [mailto:am...@infolinks.com]
Sent: Tuesday, December 17, 2013 12:19 AM
To: user@hbase.apache.org
Subject: Re: Bulk load moving HFiles to the wrong region
I've managed to isolate the problem.
I implemented
old region indicated the old region directory was
>>>>> still there. So you started bulk load just after region split? (Old region
>>>>> directory will be deleted soon by CatalogJanitor after region-split once
>>>>> compaction finished)
>>>>>
>&g
>>>> moved to the previous old region indicated the old region directory was
>>>> still there. So you started bulk load just after region split? (Old region
>>>> directory will be deleted soon by CatalogJanitor after region-split once
>>>> compaction finished
ory was
>>> still there. So you started bulk load just after region split? (Old region
>>> directory will be deleted soon by CatalogJanitor after region-split once
>>> compaction finished)
>>>
>>> I suggest to check the regionserver logs.
>>>
>&g
--Original Message-----
>> From: Amit Sela [mailto:am...@infolinks.com]
>> Sent: Monday, December 16, 2013 2:29 PM
>> To: user@hbase.apache.org
>> Subject: RE: Bulk load moving HFiles to the wrong region
>>
>> Every split executed is a new day. The row key design is y
16, 2013 2:29 PM
> To: user@hbase.apache.org
> Subject: RE: Bulk load moving HFiles to the wrong region
>
> Every split executed is a new day. The row key design is MMdd_URL. And
> the split points are MMdd_x, MMdd_y etc. In a way that the entire
> load is (almost) ev
.
-Original Message-
From: Amit Sela [mailto:am...@infolinks.com]
Sent: Monday, December 16, 2013 2:29 PM
To: user@hbase.apache.org
Subject: RE: Bulk load moving HFiles to the wrong region
Every split executed is a new day. The row key design is MMdd_URL. And the
split points are
Every split executed is a new day. The row key design is MMdd_URL. And
the split points are MMdd_x, MMdd_y etc. In a way that the entire
load is (almost) evenly spread.
The problem I described causes the bulk load to load all files to to the
last region of the previous day.
Thanks.
On D
Hi Amit:
Can you provide the split-keys of the new regions and your row-key design?
Thank you.
Jieshan.
-Original Message-
From: Amit Sela [mailto:am...@infolinks.com]
Sent: Monday, December 16, 2013 7:09 AM
To: user@hbase.apache.org
Subject: Bulk load moving HFiles to the wrong region
H
14 matches
Mail list logo
