Re: Hbck errors

2011-08-12 Thread Stack
On Fri, Aug 12, 2011 at 10:48 AM, Rohit Nigam wrote: > Tried scanning  the '.META' table > scan > '.META.',{STARTROW=>'NAM_CLUSTERKEYS3,|0USA| > 240b|000LAKE',LIMIT=>1} > > > > Getting this error:-- > I think you need to add two commas to the end of the ke

Hbck errors

2011-08-12 Thread Rohit Nigam
Hi We are having a issue while running the job as it hangs forever at 99% (reducer) inserting into the table , hbase hbck -details finds inconsistency in the table I am trying to insert data as :-- Chain of regions in table NAM_CLUSTERKEYS3 is broken; edges does not contain |

Re: Hbck errors in 0.90.3

2011-07-25 Thread Matthias Hofschen
Hi Stack, finally the migration worked. We copied table data from 0.20.4 hbase cloud to cdh3u1 cloud (0.90.3 hbase) by using the mozilla approach to copying the files on hdfs level. ( http://blog.mozilla.com/data/2011/02/04/migrating-hbase-in-the-trenches/) As described in the post from mozilla i

Re: Hbck errors in 0.90.3

2011-07-11 Thread Stack
On Mon, Jul 11, 2011 at 7:39 AM, Mat Hofschen wrote: > Hi Stack, > the scan of META does not contain any 'offline' or 'split' attributes. OK. So the daughters and parents are all 'online'. > After executing add_table I restart hbase. Have not used disable/enable. > add_table.rb seems to be pic

Re: Hbck errors in 0.90.3

2011-07-11 Thread Mat Hofschen
Hi Stack, the scan of META does not contain any 'offline' or 'split' attributes. Still parent reqion is present in META. I tried to merge the parent with one child but received the following error: (on 0.90.3) 11/07/06 11:59:35 FATAL util.Merge: Merge failed java.lang.NullPointerException at or

Re: Hbck errors in 0.90.3

2011-07-07 Thread Stack
On Thu, Jul 7, 2011 at 2:39 AM, Mat Hofschen wrote: > looking at the old 0.20.4 cluster the parent region is not written to any > more. (no data on filesystem) In META table I can not identify that this > parent region is offlined though. Where can I find that key? Why is the > region not being wr

Re: Hbck errors in 0.90.3

2011-07-07 Thread Mat Hofschen
Hi Stack, looking at the old 0.20.4 cluster the parent region is not written to any more. (no data on filesystem) In META table I can not identify that this parent region is offlined though. Where can I find that key? Why is the region not being written to any more if there is no offline flag set?

Re: Hbck errors in 0.90.3

2011-07-06 Thread Stack
On Wed, Jul 6, 2011 at 8:22 AM, Mat Hofschen wrote: > With hbck there are a few errors (52). Now I am wondering how to fix these. > For example hbck complains about two regions starting with the same key. > Next it complains about 2 regions overlapping. From looking at the META > table there seems

Hbck errors in 0.90.3

2011-07-06 Thread Mat Hofschen
Hello, I am wondering what the best way is to fix errors reported with hbck in 0.90.3. We did a migration from 0.20.4 to 0.90.3 by copying over hbase tables from 0.20.4 to a new cluster with 0.90.3. Then we used add_table.rb to create META table from scratch. (we stopped all writes to source clust

Re: Hbck errors

2011-04-15 Thread 茅旭峰
Hi Marc, Can you give some clues/links about how to manipulate the .META. in details? For example, how to build an encoded region name, for filling up the holes in .META., and how to assign the hole region then compact it. Finally, I also tried check_meta.rb, it does not work for me. Any suggestio

Re: Hbck errors

2011-03-22 Thread Adam Phelps
On 3/21/11 10:13 PM, Stack wrote: On Mon, Mar 21, 2011 at 7:19 PM, Adam Phelps wrote: It looks like we've come up against a problem that looks identical to the one you described. How did you go about manually inserting the two child regions? You know the daughter regions because they should

Re: Hbck errors

2011-03-21 Thread Stack
On Mon, Mar 21, 2011 at 7:19 PM, Adam Phelps wrote: > It looks like we've come up against a problem that looks identical to the > one you described.  How did you go about manually inserting the two child > regions? > You know the daughter regions because they should be listed when you look at the

Re: Hbck errors

2011-03-21 Thread Adam Phelps
On 3/6/11 10:08 AM, Marc Limotte wrote: 1. split started on region A 2. region A was offlined 3. The daughter regions were created in HDFS with the reference files 4. .META. was updated for region A 5. server crashed So, the new daughter entries were never added to .META

Re: Hbck errors

2011-03-06 Thread Marc Limotte
We have resolved the issue. Details follow as "lessons learned". Yes, Stack, the info:splitA/B columns followed, as part of the offline'd parents .META. row. But the regions that splitA/B point to *did not exist*in .META. Also, after the original email, we checked in HDFS and found the parent r

Re: Hbck errors

2011-03-06 Thread Stack
So, yeah Marc, what are the rows that follow the ones you post below? Are they the info:splitA and info:splitB or something else? Thanks, St.Ack On Sat, Mar 5, 2011 at 4:22 PM, Marc Limotte wrote: > We had an issue a day ago with some OOME's on the region servers.  The > master shutdown ok, but m

Re: Hbck errors

2011-03-05 Thread Stack
Are the daughter regions in the meta table? Are both there? Are they both online? You could try enabling the parent region On Mar 5, 2011, at 16:22, Marc Limotte wrote: > We had an issue a day ago with some OOME's on the region servers. The > master shutdown ok, but most of the RegionSer

Hbck errors

2011-03-05 Thread Marc Limotte
We had an issue a day ago with some OOME's on the region servers. The master shutdown ok, but most of the RegionServers didn't and so eventually had to kill -9 them. Brought it all back up and ran a major compaction to change the hbase block size. This seemed to work, but now we have an inconsis