It seems LoadIncrementalHFiles is still running. Can you run "jstack" on 1
RegionServer process also?
Which version are you using?
Jieshan.
-Original Message-
From: Tao Xiao [mailto:xiaotao.cs@gmail.com]
Sent: Wednesday, December 18, 2013 1:49 PM
To: user@hbase.apache.org
Subject:
I did jstack one such process and can see the following output in the
terminal, and I guess this info told us that the processes started by the
command "LoadIncrementalHFiles" never exit. Why didn't they exit after
finished running ?
... ...
... ...
"LoadIncrementalHFiles-0.LruBlockCache.Eviction
Hey Ted Yu,
I had digging the name node log and so far I've found nothing special. No
Exception, FATAL or ERROR message nor anything other peculiarities.
Only I see a lot of messages like this:
2013-12-12 13:53:22,541 INFO org.apache.hadoop.hdfs.StateChange: Removing lease
on
/hbase/Sessions_
Hey Azuryy Yu,
yep, checked the GC log, nothing there.
I think, there is no special JVM configuration:
export HBASE_OPTS="-XX:+UseConcMarkSweepGC"
export SERVER_GC_OPTS="-verbose:gc -XX:+PrintGCDetails -XX:+PrintGCDateStamps
-XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=1 -XX:GCLogFileSize=5
Hey,
sorry for the answer delay, I had a flight to San Francisco and fighting with
the jetleg. I am here on vacation, maybe I can visit some interesting talks
about HBase/Hadoop :).
Am 14.12.2013 um 13:14 schrieb lars hofhansl :
> Did you observe anything interesting with such a large Java hea
Tao:
Can you jstack one such process next time you see them hanging ?
Thanks
On Tue, Dec 17, 2013 at 6:31 PM, Tao Xiao wrote:
> BTW, I noticed another problem. I bulk load data into HBase every five
> minutes, but I found that whenever the following command was executed
> hbase org.apache.
BTW, I noticed another problem. I bulk load data into HBase every five
minutes, but I found that whenever the following command was executed
hbase org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles
HFiles-Dir MyTable
there is a new process called "LoadIncrementalHFiles"
I can see many p
Ted,
thanks. these two patches marked "deleteConnection(Configuration conf,
boolean stopProxy)" as deprecated, and left
"deleteConnection(Configuration conf)" as the API for 94 . However,
deleteConnection(Configuration conf) is marked as deprecated on 96.0 now.
Is there a way to search which pa
See:
HBASE-7626 Backport client connection cleanup from HBASE-7460
Cheers
On Tue, Dec 17, 2013 at 1:17 PM, Demai Ni wrote:
> hi, folks,
>
> we are currently using both calls. They are being deprecated. Wondering
> what APIs should be used to replacement them? many thanks
>
> Demai
>
hi, folks,
we are currently using both calls. They are being deprecated. Wondering
what APIs should be used to replacement them? many thanks
Demai
I was afraid of this answer and suspected it ;). I knew that the answer
would depend on the actual setting, but I hoped, that there is is a
little hint.
Thanks a lot for your time and the answer. I will try it out with test
data (and a simple table design) and will share my experiments when they
a
Have you looked at this filter ?
src/main/java/org/apache/hadoop/hbase/filter/PrefixFilter.java
Cheers
On Tue, Dec 17, 2013 at 7:14 AM, fateme Abiri wrote:
> hi my freiend...
> tanx for your feedback...
>
> i m sorry, i take a mistake when i wrote in my email, i write correctly in
> my IDE,
> b
hi my freiend...
tanx for your feedback...
i m sorry, i take a mistake when i wrote in my email, i write correctly in my
IDE,
but the rows which was returned only match with one of the prefix
filters!!!
On Tuesday, December 17, 2013 5:57 PM, Ted Yu wrote:
An 'E' is missing from MUST_PAS
An 'E' is missing from MUST_PASS_ON
The for loop has 5 iterations instead of 4.
Cheers
On Dec 17, 2013, at 6:14 AM, fateme Abiri wrote:
> hi friends
> I want to use a filter in hbase to return rows with different prefix...
>
> for eg. my rows structure are +URLStrings
>
> so i want to re
hi friends
I want to use a filter in hbase to return rows with different prefix...
for eg. my rows structure are +URLStrings
so i want to return rows with 4 prefix :
<12234>
<4534>
<134>
<4234>
how can I do that?
i use
FilterLis RowFilterlist; RowFilterlist= new
FilterLis(FilterList.Op
In my opinion, it really depends on your queries.
The first one achieves data locality. There is no additional data transmit
between different nodes. But this strategy sacrifices parallelism and the
node which stores A will be a hot node if too many applications try to
access A.
The second appro
>Where does the RegionServer save the partitions file written to
>DistributedCache ?
There's no need for RegionServer to save the partitions file.
It seems you added the new regions directly into META, and didn't change the
endkey of the last previous region?
Jieshan.
-Original Message
Yes, it should be cleaned up. But not included in current code in my
understanding.
Jieshan.
-Original Message-
From: Ted Yu [mailto:yuzhih...@gmail.com]
Sent: Tuesday, December 17, 2013 10:55 AM
To: user@hbase.apache.org
Subject: Re: Why so many unexpected files like partitions_ are
Indeed there are more than 2 split points, there are 4 split points for 5
new regions added each day.
the new data bulk loaded each day belongs to he new regions.
It seems like the partitions read are from the previous insertion, and if
that is the case, the comparator will surely indicate that the
>> The previous last region is not supposed to delete I'm just
>> adding new regions (always following lexicographically) so that
>> the last region before the pre-split is not the last anymore.
You mean you added the new regions into META? Sorry if I misunderstood you
here. But
Using a custom InputFormat with dedicated getsplit() allow you to use a
single scan object when initiating job. It is cloned later by each
mapper setting startrow and stoprow according the list returned by
getsplit().
Getsplit would return a list of couple (startrow, stoprow) calculated
based on r
Thanks guys, we have fixed it by reinstall the python depended Thrift lib.
Cheers
Ramon
Region server logs in region servers that were supposed to get the loaded
data show that they get request to open the (correct) region, and they open
it.
But only in the region server where the data is actually loaded in to have
the move in the log, for all file..
The log actually shows it copies t
Thanks for your reply, Damien.
So this solution still use one scan object, and sent it to initTableMapperJob?
Does modified getsplit() function set the salt Bucket number to the number of
mapper?If I set 256 salt buckets, and the mapper number will be 256, right?
Another question is can this buck
As per the line no it comes as
* byte*[][] famAndQf = KeyValue.*parseColumn*(*getBytes*(m.column));
column inside Mutation comes as null... Can you check client code
-Anoop-
On Tue, Dec 17, 2013 at 2:59 PM, ramkrishna vasudevan <
ramkrishna.s.vasude...@gmail.com> wrote:
> Due to some reason th
Like I mentioned before, running with all reducers works fine. Running with
the extension of HFileOutputFormat fails, sometimes, on some tables.
.META. encoded qualifier points to different directories for the different
regions files are supposedly loaded into. The directories actually do
exist, an
Due to some reason the row that is created inside the BatchMutation is
null. Can you check your Thrift client code where BatchMutation is created?
On Tue, Dec 17, 2013 at 2:45 PM, Ramon Wang wrote:
> Hi Folks
>
> We upgraded our cluster to CDH4.5.0 recently, HBase version is
> 0.94.6-cdh4.5.0
Hello,
yes you need 256 scans range or a full (almost) scan with combination of
filters for each 256 ranges
(https://hbase.apache.org/apidocs/org/apache/hadoop/hbase/filter/FilterList.Operator.html#MUST_PASS_ONE)
For mapreduce, the getsplit() method should be modified from
TableInputFormatBase to
Hi Folks
We upgraded our cluster to CDH4.5.0 recently, HBase version is
0.94.6-cdh4.5.0 now. Our client program(written in Python) cannot save data
by using Thrift, there are errors happen when we are trying to save data
with many columns(more than 7 or 8), and here is the error log:
2013-12-17 0
Hello,
@Alex Baranau
Thanks for your salt solution. In my understanding, the salt solution is divide
the data into several partial(if 2 letters,00~FF, then 255 parts will be
devided). My question is when I want to scan data, do I need scan 256 times for
the following situation:rowkey: salt pref
30 matches
Mail list logo