date:20110623

[jira] [Commented] (HBASE-3872) Hole in split transaction rollback; edits to .META. need to be rolled back even if it seems like they didn't make it

2011-06-23 Thread Aaron Kimball (JIRA)

[
https://issues.apache.org/jira/browse/HBASE-3872?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13053667#comment-13053667
]

Aaron Kimball commented on HBASE-3872:
--

The parent is *not* in .META. There is a hole in the list of regions, as seen
by running a scan on .META. from the hbase shell and/or by looking at table.jsp
on the master server's website.

Also, running {{hbase hbck}} identified one of the missing regions (chain of
regions in table ... is broken; edges does not contain rowkey). It did not
notice the second missing region. Is that because the process that checks the
region chain gives up after the first error? Or could that be unrelated?

Hole in split transaction rollback; edits to .META. need to be rolled back
even if it seems like they didn't make it

Key: HBASE-3872
URL: https://issues.apache.org/jira/browse/HBASE-3872
Project: HBase
Issue Type: Bug
Components: regionserver
Affects Versions: 0.90.3
Reporter: stack
Assignee: stack
Priority: Blocker
Fix For: 0.90.4

Attachments: 3872.txt

Saw this interesting one on a cluster of ours. The cluster was configured
with too few handlers so lots of the phenomeneon where actions were queued
but then by the time they got into the server and tried respond to the
client, the client had disconnected because of the timeout of 60 seconds.
Well, the meta edits for a split were queued at the regionserver carrying
.META. and by the time it went to write back, the client had gone (the first
insert of parent offline with daughter regions added as info:splitA and
info:splitB). The client presumed the edits failed and 'successfully' rolled
back the transaction (failing to undo .META. edits thinking they didn't go
through).
A few minutes later the .META. scanner on master runs. It sees 'no
references' in daughters -- the daughters had been cleaned up as part of the
split transaction rollback -- so it thinks its safe to delete the parent.
Two things:
+ Tighten up check in master... need to check daughter region at least exists
and possibly the daughter region has an entry in .META.
+ Dependent on the edit that fails, schedule rollback edits though it will
seem like they didn't go through.
This is pretty critical one.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-3996) Support multiple tables and scanners as input to the mapper in map/reduce jobs

2011-06-23 Thread Eran Kutner (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-3996?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Eran Kutner updated HBASE-3996:
---

Attachment: (was: MultiTableInputFormat.patch)

 Support multiple tables and scanners as input to the mapper in map/reduce jobs
 --

 Key: HBASE-3996
 URL: https://issues.apache.org/jira/browse/HBASE-3996
 Project: HBase
  Issue Type: Improvement
  Components: mapreduce
Reporter: Eran Kutner
 Fix For: 0.90.4

 Attachments: MultiTableInputFormat.patch, 
 TestMultiTableInputFormat.java.patch


 It seems that in many cases feeding data from multiple tables or multiple 
 scanners on a single table can save a lot of time when running map/reduce 
 jobs.
 I propose a new MultiTableInputFormat class that would allow doing this.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-3996) Support multiple tables and scanners as input to the mapper in map/reduce jobs

2011-06-23 Thread Eran Kutner (JIRA)

[
https://issues.apache.org/jira/browse/HBASE-3996?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13053681#comment-13053681
]

Eran Kutner commented on HBASE-3996:

Thanks stack.

I hope I finally got Eclipse to properly manage the tabs and line lengths (I'm
not really a Java developer so this is all new to me).

{quote}In TableSplit you create an HTable instance. Do you need to? And when
you create it, though I believe it will be less of a problem going forward, can
you use the constructor that takes a Configuration and table name? Is there a
close in Split interface? If so, you might want to call close of your HTable in
there. (Where is it used? Each split needs its own HTable?) Use the constructor
that takes a Configuration here too...{quote}

There are actually two issues here, I added the configuration and closed the
table in getSplits(), that's the easy one.
HTable per split is needed because it is used for reading the data from the
split by the cluster nodes when the job is running. However, in order to
support passing the configuration, I move the Htable creation out of TableSplit
and into MutiTableInputFormatBase. I also modified TableRecordReaderImpl to
close the table after reading all the records in the split. I believe this is
OK, and the tests are passing fine, but it wasn't like that in the existing,
single table, implementation so I hope I'm not missing (and messing) anything.

{quote}You don't need the e.printStackTrace in below{quote}
Right, removed and fixed the spelling in the warning.

{quote}By any chance is the code here in MultiTableInputFormatBase where we are
checking start and end rows copied from elsewhere?{quote}
It's copied from TableInputFormatBase, as I said my code is closely based on
the single table code.

{quote}You remove the hashCode in TableSplit. Should it have one?{quote}
I actually don't know if it needs one or not (it does seem to work fine without
it) but I didn't remove it intentionally. I wrote my original code based on the
0.90.3 branch and when I copied to trunk I missed this change. It's back
now.{quote}

{quote}therwise patch looks great. Test too.{quote}
Thanks!

Hope that's it.

Support multiple tables and scanners as input to the mapper in map/reduce jobs
--

Key: HBASE-3996
URL: https://issues.apache.org/jira/browse/HBASE-3996
Project: HBase
Issue Type: Improvement
Components: mapreduce
Reporter: Eran Kutner
Fix For: 0.90.4

Attachments: MultiTableInputFormat.patch,
TestMultiTableInputFormat.java.patch

It seems that in many cases feeding data from multiple tables or multiple
scanners on a single table can save a lot of time when running map/reduce
jobs.
I propose a new MultiTableInputFormat class that would allow doing this.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-3996) Support multiple tables and scanners as input to the mapper in map/reduce jobs

2011-06-23 Thread Eran Kutner (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-3996?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Eran Kutner updated HBASE-3996:
---

Attachment: (was: MultiTableInputFormat.patch)

 Support multiple tables and scanners as input to the mapper in map/reduce jobs
 --

 Key: HBASE-3996
 URL: https://issues.apache.org/jira/browse/HBASE-3996
 Project: HBase
  Issue Type: Improvement
  Components: mapreduce
Reporter: Eran Kutner
 Fix For: 0.90.4

 Attachments: MultiTableInputFormat.patch, 
 TestMultiTableInputFormat.java.patch


 It seems that in many cases feeding data from multiple tables or multiple 
 scanners on a single table can save a lot of time when running map/reduce 
 jobs.
 I propose a new MultiTableInputFormat class that would allow doing this.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-3996) Support multiple tables and scanners as input to the mapper in map/reduce jobs

2011-06-23 Thread Eran Kutner (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-3996?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Eran Kutner updated HBASE-3996:
---

Attachment: MultiTableInputFormat.patch

 Support multiple tables and scanners as input to the mapper in map/reduce jobs
 --

 Key: HBASE-3996
 URL: https://issues.apache.org/jira/browse/HBASE-3996
 Project: HBase
  Issue Type: Improvement
  Components: mapreduce
Reporter: Eran Kutner
 Fix For: 0.90.4

 Attachments: MultiTableInputFormat.patch, 
 TestMultiTableInputFormat.java.patch


 It seems that in many cases feeding data from multiple tables or multiple 
 scanners on a single table can save a lot of time when running map/reduce 
 jobs.
 I propose a new MultiTableInputFormat class that would allow doing this.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-4013) Make ZooKeeperListener Abstract

2011-06-23 Thread Akash Ashok (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-4013?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Akash Ashok updated HBASE-4013:
---

Status: Patch Available  (was: Open)

 Make ZooKeeperListener Abstract
 ---

 Key: HBASE-4013
 URL: https://issues.apache.org/jira/browse/HBASE-4013
 Project: HBase
  Issue Type: Task
  Components: zookeeper
Reporter: Akash Ashok
Priority: Minor
  Labels: zookeeper

 org.apache.hadoop.hbase.zookeeper.ZooKeeperListener seems to have all 
 unimplemented methods. This should be made an abstract class.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-4013) Make ZooKeeperListener Abstract

2011-06-23 Thread Akash Ashok (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-4013?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Akash Ashok updated HBASE-4013:
---

Status: Open  (was: Patch Available)

 Make ZooKeeperListener Abstract
 ---

 Key: HBASE-4013
 URL: https://issues.apache.org/jira/browse/HBASE-4013
 Project: HBase
  Issue Type: Task
  Components: zookeeper
Reporter: Akash Ashok
Priority: Minor
  Labels: zookeeper

 org.apache.hadoop.hbase.zookeeper.ZooKeeperListener seems to have all 
 unimplemented methods. This should be made an abstract class.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-4013) Make ZooKeeperListener Abstract

2011-06-23 Thread Akash Ashok (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-4013?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Akash Ashok updated HBASE-4013:
---

Attachment: hbase-4013.patch

ZooKeeperListener made abstract 

 Make ZooKeeperListener Abstract
 ---

 Key: HBASE-4013
 URL: https://issues.apache.org/jira/browse/HBASE-4013
 Project: HBase
  Issue Type: Task
  Components: zookeeper
Reporter: Akash Ashok
Priority: Minor
  Labels: zookeeper
 Attachments: hbase-4013.patch


 org.apache.hadoop.hbase.zookeeper.ZooKeeperListener seems to have all 
 unimplemented methods. This should be made an abstract class.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-4013) Make ZooKeeperListener Abstract

2011-06-23 Thread Akash Ashok (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-4013?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Akash Ashok updated HBASE-4013:
---

Status: Patch Available  (was: Open)

 Make ZooKeeperListener Abstract
 ---

 Key: HBASE-4013
 URL: https://issues.apache.org/jira/browse/HBASE-4013
 Project: HBase
  Issue Type: Task
  Components: zookeeper
Reporter: Akash Ashok
Priority: Minor
  Labels: zookeeper
 Attachments: hbase-4013.patch


 org.apache.hadoop.hbase.zookeeper.ZooKeeperListener seems to have all 
 unimplemented methods. This should be made an abstract class.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

72 matches

Mail list logo