Oops, sorry, misread your question.
Lars George wrote:
From the download page, see
http://hadoop.apache.org/hbase/releases.html
for reference.
llpind wrote:
Andrew Purtell-2 wrote:
Is the patch for HADOOP- 4681 applied? See
https://issues.apache.org/jira/browse/HADOOP-4681
- Andy
org.apache.hadoop.hdfs.server.datanode.DataXceiver.writeBlock(DataXceiver.java:357)
>at
>
> org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:103)
> at java.lang.Thread.run(Thread.java:619)
> 2009-06-02 12:32:20,289 INFO
> or
Or you can apply the patch and recompile? Or I can make a patched Hadoop 0.19.1
tarball for you?
Let me know.
- Andy
From: llpind
To: hbase-user@hadoop.apache.org
Sent: Wednesday, June 3, 2009 9:27:01 AM
Subject: Re: HBase looses regions.
Andrew
Hadoop 0.19.2 has not yet been released, though a release is likely
forthcoming based on mailing list posts. This patch applies cleanly
to 0.19.1 if you want to apply it yourself -- I think this is what has
been suggested.
Michael
llpind wrote:
>
> Andrew Purtell-2 wrote:
>>
>> Is the patch for
From the download page, see
http://hadoop.apache.org/hbase/releases.html
for reference.
llpind wrote:
Andrew Purtell-2 wrote:
Is the patch for HADOOP- 4681 applied? See
https://issues.apache.org/jira/browse/HADOOP-4681
- Andy
I'm on Hadoop 0.19.1. It appears it has been ap
message in context:
http://www.nabble.com/HBase-looses-regions.-tp23657983p23854753.html
Sent from the HBase User mailing list archive at Nabble.com.
Is the patch for HADOOP- 4681 applied? See
https://issues.apache.org/jira/browse/HADOOP-4681
- Andy
From: llpind
To: hbase-user@hadoop.apache.org
Sent: Tuesday, June 2, 2009 1:00:24 PM
Subject: Re: HBase looses regions.
Here are the changes I've
one now, that seems useful -- but if you have co-deployed mapreduce
> with your region servers, you could use JobClient.getClusterStatus() to
> programmatically determine the size of the cluster. See
> http://hadoop.apache.org/core/docs/current/api/org/apache/hadoop/mapred/JobClient.html#getCl
red/JobClient.html#getClusterStatus()
Best regards,
- Andy
From: llpind
To: hbase-user@hadoop.apache.org
Sent: Wednesday, May 27, 2009 10:49:45 AM
Subject: Re: HBase looses regions.
Andrew Purtell-2 wrote:
>
> Also the program that is pounding the
help. Right now I'm not seeing any splits happening, so
I don't know how much adding more boxes will help. It seems to not be
balanced. All writes go to a single slave, when that box dies, it moves to
the next.
--
View this message in context:
http://www.nabble.com/HBase-looses-regio
http://www.nabble.com/HBase-looses-regions.-tp23657983p23746026.html
Sent from the HBase User mailing list archive at Nabble.com.
Go to the hosts where you think you run DataNodes and check what is set
for "HADOOP_LOG_DIR" in
$HADOOP_HOME/conf/hadoop-env.sh
(It is "$HADOOP_HOME/logs" by default.)
If there is no log like
"$HADOOP_LOG_DIR/hadoop-user-datanode-hostname.log"
in that dir I'd assume you are not running D
Subject: Re: HBase looses regions.
Finally failed between 7M-8M records. below is the last tail output. The
other two region server don't have much activity in the logs, but i can post
those if necessary.
===
2009-05-26 10:28:06,550
Where is the datanode log? The logs I have above are from the datanode
(hbase/logs). I don't see any logs from hadoop/logs directory.
--
View this message in context:
http://www.nabble.com/HBase-looses-regions.-tp23657983p23733204.html
Sent from the HBase User mailing list archi
5-26 10:28:14,270 INFO org.apache.hadoop.hbase.Leases:
> regionserver/0.0.0.0:60020.leaseChecker closing leases
> 2009-05-26 10:28:14,271 INFO org.apache.hadoop.hbase.Leases:
> regionserver/0.0.0.0:60020.leaseChecker closed leases
> 2009-05-26 10:28:14,273 INFO
> org.apache.hadoop.hbase.regionserver.HRegionServer: Starting shutdown
> thread.
> 2009-05-26 10:28:14,273 INFO
> org.apache.hadoop.hbase.regionserver.HRegionServer: Shutdown thread complete
>
>
> ===
> --
> View this message in context:
> http://www.nabble.com/HBase-looses-regions.-tp23657983p23727987.html
> Sent from the HBase User mailing list archive at Nabble.com.
>
>
he.hadoop.hbase.regionserver.HRegion:
> Closing tableA,,1243363849039: compactions & flushes disabled
> 2009-05-26 12:09:26,929 DEBUG org.apache.hadoop.hbase.regionserver.HRegion:
> Updates disabled for region, no outstanding scanners on
> tableA,,1243363849039
> 2009-05-26 12:09:26,929 DEBUG org.apache.hadoop.hbase.regionserver.HRegion:
> No more row locks outstanding on region tableA,,1243363849039
> 2009-05-26 12:09:26,929 DEBUG org.apache.hadoop.hbase.regionserver.HStore:
> closed 407623107/entity
> 2009-05-26 12:09:26,929 DEBUG org.apache.hadoop.hbase.regionserver.HStore:
> closed 407623107/link
> 2009-05-26 12:09:26,929 INFO org.apache.hadoop.hbase.regionserver.HRegion:
> Closed tableA,,1243363849039
> 2009-05-26 12:09:26,929 INFO
> org.apache.hadoop.hbase.regionserver.HRegionServer: aborting server at:
> 192.168.240.175:60020
> 2009-05-26 12:09:27,033 INFO
> org.apache.hadoop.hbase.regionserver.HRegionServer:
> regionserver/0.0.0.0:60020 exiting
> 2009-05-26 12:09:27,034 INFO
> org.apache.hadoop.hbase.regionserver.HRegionServer: Starting shutdown
> thread.
> 2009-05-26 12:09:27,034 INFO
> org.apache.hadoop.hbase.regionserver.HRegionServer: Shutdown thread complete
>
>
>
>
>
> stack-3 wrote:
>>
>> That looks sick. Different log files can't close? Enable DEBUG in
>> your logs. See FAQ for how. You sure your HDFS healthy? Is it even
>> working?
>> St.Ack
>>
>
> --
> View this message in context:
> http://www.nabble.com/HBase-looses-regions.-tp23657983p23730737.html
> Sent from the HBase User mailing list archive at Nabble.com.
>
>
: servers: 2 requests=0, regions=4
--
View this message in context:
http://www.nabble.com/HBase-looses-regions.-tp23657983p23731055.html
Sent from the HBase User mailing list archive at Nabble.com.
0
==PROCESSING RECORD: == @200
==PROCESSING RECORD: == @300
==PROCESSING RECORD: == @400
==PROCESSING RECORD: == @500
--
View this message in context:
http://www.nabble.com/HBase-looses-regions.-tp23657983p23730864.html
Sent from the HBase User mailing
Enable DEBUG in
> your logs. See FAQ for how. You sure your HDFS healthy? Is it even
> working?
> St.Ack
>
--
View this message in context:
http://www.nabble.com/HBase-looses-regions.-tp23657983p23730737.html
Sent from the HBase User mailing list archive at Nabble.com.
out on dfsclient to 0.
>>
>> For more detail on how to do above configurations, their exact names
>> and whether client or server-side config, see the FAQ and
>> troubleshooting.
>>
>> Let us know the particular issues you are running into. We'd like to help
&g
53 INFO org.apache.hadoop.hbase.regionserver.HRegion:
> starting compaction on region tableA,ROW_KEY,1243357190459
> 2009-05-26 10:01:40,903 INFO org.apache.hadoop.hbase.regionserver.HLog:
> Closed
> hdfs://ats181:54310/hbase/log_192.168.240.175_1243356336827_60020/hlog.dat.1243357263846,
> entries=11. New log writer:
> /hbase/log_192.168.240.175_1243356336827_60020/hlog.dat.1243357300890
> 2009-05-26 10:01:40,904 INFO org.apache.hadoop.hbase.regionserver.HLog:
> removing old log file
> /hbase/log_192.168.240.175_1243356336827_60020/hlog.dat.1243357129598 whose
> highest sequence/edit id is 1275010
> 2009-05-26 10:01:40,912 INFO org.apache.hadoop.hbase.regionserver.HLog:
> removing old log file
> /hbase/log_192.168.240.175_1243356336827_60020/hlog.dat.1243357169473 whose
> highest sequence/edit id is 1380010
> 2009-05-26 10:01:59,165 INFO org.apache.hadoop.hdfs.DFSClient: Could not
> complete file
> /hbase/tableA/compaction.dir/336579621/link/mapfiles/6229032204546731894/data
> retrying...
> 2009-05-26 10:01:59,569 INFO org.apache.hadoop.hdfs.DFSClient: Could not
> complete file
> /hbase/tableA/compaction.dir/336579621/link/mapfiles/6229032204546731894/data
> retrying...
> 2009-05-26 10:01:59,973 INFO org.apache.hadoop.hdfs.DFSClient: Could not
> complete file
> /hbase/tableA/compaction.dir/336579621/link/mapfiles/6229032204546731894/data
> retrying...
> 2009-05-26 10:02:00,377 INFO org.apache.hadoop.hdfs.DFSClient: Could not
> complete file
> /hbase/tableA/compaction.dir/336579621/link/mapfiles/6229032204546731894/data
> retrying...
> 2009-05-26 10:02:00,781 INFO org.apache.hadoop.hdfs.DFSClient: Could not
> complete file
> /hbase/tableA/compaction.dir/336579621/link/mapfiles/6229032204546731894/data
> retrying...
> 2009-05-26 10:02:01,185 INFO org.apache.hadoop.hdfs.DFSClient: Could not
> complete file
> /hbase/tableA/compaction.dir/336579621/link/mapfiles/6229032204546731894/data
> retrying...
> 2009-05-26 10:02:01,589 INFO org.apache.hadoop.hdfs.DFSClient: Could not
> complete file
> /hbase/tableA/compaction.dir/336579621/link/mapfiles/6229032204546731894/data
> retrying...
> 2009-05-26 10:02:01,993 INFO org.apache.hadoop.hdfs.DFSClient: Could not
> complete file
> /hbase/tableA/compaction.dir/336579621/link/mapfiles/6229032204546731894/data
> retrying...
>
> --
> View this message in context:
> http://www.nabble.com/HBase-looses-regions.-tp23657983p23727376.html
> Sent from the HBase User mailing list archive at Nabble.com.
>
>
53 INFO org.apache.hadoop.hbase.regionserver.HRegion:
> starting compaction on region tableA,ROW_KEY,1243357190459
> 2009-05-26 10:01:40,903 INFO org.apache.hadoop.hbase.regionserver.HLog:
> Closed
> hdfs://ats181:54310/hbase/log_192.168.240.175_1243356336827_60020/hlog.dat.1243357263846,
> entries=11. New log writer:
> /hbase/log_192.168.240.175_1243356336827_60020/hlog.dat.1243357300890
> 2009-05-26 10:01:40,904 INFO org.apache.hadoop.hbase.regionserver.HLog:
> removing old log file
> /hbase/log_192.168.240.175_1243356336827_60020/hlog.dat.1243357129598 whose
> highest sequence/edit id is 1275010
> 2009-05-26 10:01:40,912 INFO org.apache.hadoop.hbase.regionserver.HLog:
> removing old log file
> /hbase/log_192.168.240.175_1243356336827_60020/hlog.dat.1243357169473 whose
> highest sequence/edit id is 1380010
> 2009-05-26 10:01:59,165 INFO org.apache.hadoop.hdfs.DFSClient: Could not
> complete file
> /hbase/tableA/compaction.dir/336579621/link/mapfiles/6229032204546731894/data
> retrying...
> 2009-05-26 10:01:59,569 INFO org.apache.hadoop.hdfs.DFSClient: Could not
> complete file
> /hbase/tableA/compaction.dir/336579621/link/mapfiles/6229032204546731894/data
> retrying...
> 2009-05-26 10:01:59,973 INFO org.apache.hadoop.hdfs.DFSClient: Could not
> complete file
> /hbase/tableA/compaction.dir/336579621/link/mapfiles/6229032204546731894/data
> retrying...
> 2009-05-26 10:02:00,377 INFO org.apache.hadoop.hdfs.DFSClient: Could not
> complete file
> /hbase/tableA/compaction.dir/336579621/link/mapfiles/6229032204546731894/data
> retrying...
> 2009-05-26 10:02:00,781 INFO org.apache.hadoop.hdfs.DFSClient: Could not
> complete file
> /hbase/tableA/compaction.dir/336579621/link/mapfiles/6229032204546731894/data
> retrying...
> 2009-05-26 10:02:01,185 INFO org.apache.hadoop.hdfs.DFSClient: Could not
> complete file
> /hbase/tableA/compaction.dir/336579621/link/mapfiles/6229032204546731894/data
> retrying...
> 2009-05-26 10:02:01,589 INFO org.apache.hadoop.hdfs.DFSClient: Could not
> complete file
> /hbase/tableA/compaction.dir/336579621/link/mapfiles/6229032204546731894/data
> retrying...
> 2009-05-26 10:02:01,993 INFO org.apache.hadoop.hdfs.DFSClient: Could not
> complete file
> /hbase/tableA/compaction.dir/336579621/link/mapfiles/6229032204546731894/data
> retrying...
>
> --
> View this message in context:
> http://www.nabble.com/HBase-looses-regions.-tp23657983p23727376.html
> Sent from the HBase User mailing list archive at Nabble.com.
>
>
4,273 INFO
org.apache.hadoop.hbase.regionserver.HRegionServer: Shutdown thread complete
===
--
View this message in context:
http://www.nabble.com/HBase-looses-regions.-tp23657983p23727987.html
Sent from the HBase User mailing list archive at Nabble.com.
using TRUNK (If you have answered this question already,
>>>>> please excuse my not remembering)?
>>>>>
>>>>> St.Ack
>>>>>
>>>>> On Sat, May 23, 2009 at 2:17 PM, llpind
>>>>> wrote:
>>>>>&g
7 PM, llpind wrote:
>>>>>
>>>>> I see similar behavior in my small cluster. (1 master, 3 datanodes)
>>>>>
>>>>> I am also planning on trying this RC version. I've tried various
>>>>> configurations, and I continue to lo
://www.nabble.com/HBase-looses-regions.-tp23657983p23727376.html
Sent from the HBase User mailing list archive at Nabble.com.
;>
>>>> I see similar behavior in my small cluster. (1 master, 3 datanodes)
>>>>
>>>> I am also planning on trying this RC version. I've tried various
>>>> configurations, and I continue to lose Regions with intensive writes.
>
On Mon, May 25, 2009 at 12:58 PM, Kirill Shabunov wrote:
> What should I do to vote for the release? I just subscribed for hbase-dev
> list.
>
Can you find the message in our archives? Its title is "[ANN]
hbase-0.19.3 release candidate 1" If not, just +1 it and we'll know
what you are at.
St.A
Thanks a lot for the advice!
What should I do to vote for the release? I just subscribed for
hbase-dev list.
--Kirill
stack wrote:
On Fri, May 22, 2009 at 7:21 AM, Kirill Shabunov wrote:
Thanks a lot!
I've tried, it seems to be much more stable overall. No component have
crashed so far.
really hope something like this will work, because we are starting to
>>> consider other options now.
>>>
>>> --
>>> View this message in context:
>>> http://www.nabble.com/HBase-looses-regions.-tp23657983p23688361.html
>>> Sent from the HBase User
nsive writes. I
>> really hope something like this will work, because we are starting to
>> consider other options now.
>>
>> --
>> View this message in context:
>> http://www.nabble.com/HBase-looses-regions.-tp23657983p23688361.html
>> Sent from
message in context:
http://www.nabble.com/HBase-looses-regions.-tp23657983p23690429.html
Sent from the HBase User mailing list archive at Nabble.com.
writes. I
> really hope something like this will work, because we are starting to
> consider other options now.
>
> --
> View this message in context:
> http://www.nabble.com/HBase-looses-regions.-tp23657983p23688361.html
> Sent from the HBase User mailing list archive at Nabble.com.
>
>
I've tried various
> configurations, and I continue to lose Regions with intensive writes. I
> really hope something like this will work, because we are starting to
> consider other options now.
>
> --
> View this message in context:
> http://www.nabble.com/HBase-looses-re
other options now.
--
View this message in context:
http://www.nabble.com/HBase-looses-regions.-tp23657983p23688361.html
Sent from the HBase User mailing list archive at Nabble.com.
On Fri, May 22, 2009 at 7:21 AM, Kirill Shabunov wrote:
> Thanks a lot!
>
> I've tried, it seems to be much more stable overall. No component have
> crashed so far.
I'm glad. If the RC is working for you, please vote for it over in
hbase-dev mailing list.
>
>
> I see some new messages, like
Thanks a lot!
I've tried, it seems to be much more stable overall. No component have
crashed so far.
I see some new messages, like this:
WARN org.apache.hadoop.hbase.regionserver.HLog:
regionserver/0:0:0:0:0:0:0:0:60020.logFlusher took 1661ms optional sync
to HLog; editcount=0
I also chan
Can you try the 0.19.3 RC1 posted yesterday:
http://people.apache.org/~stack/hbase-0.19.3-candidate-1/
It has fixes that should help stop master and regionserver get into state of
disagreement (In particular, HBASE-1421 and HBASE-1344).
Thanks,
St.Ack
On Thu, May 21, 2009 at 11:15 AM, Kirill S
Hi!
I am running HBase 0.19.2, r771918 on top of Hadoop 0.19.1, r745977.
When I stress the system with lots of uploaded data it often happens a
Regionserver gets overloaded and is lost by the cluster. This is
understandable. However, right after that Hbase becomes generally
unstable and then
39 matches
Mail list logo
