[jira] [Created] (HBASE-7369) HConnectionManager should remove aborted connections
Bryan Baugher created HBASE-7369: Summary: HConnectionManager should remove aborted connections Key: HBASE-7369 URL: https://issues.apache.org/jira/browse/HBASE-7369 Project: HBase Issue Type: Improvement Components: Client Affects Versions: 0.94.3 Reporter: Bryan Baugher Priority: Minor When an HConnection is abort()'ed (i.e. if numerous services are lost) the connection becomes unusable. HConnectionManager cache of HConnections currently does not have any logic around removing aborted connections automatically. Currently it is up to the consumer to do so using HConnectionManager.deleteStaleConnection(HConnection). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7369) HConnectionManager should remove aborted connections
[ https://issues.apache.org/jira/browse/HBASE-7369?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Baugher updated HBASE-7369: - Attachment: patch.diff > HConnectionManager should remove aborted connections > > > Key: HBASE-7369 > URL: https://issues.apache.org/jira/browse/HBASE-7369 > Project: HBase > Issue Type: Improvement > Components: Client >Affects Versions: 0.94.3 >Reporter: Bryan Baugher >Priority: Minor > Attachments: patch.diff > > > When an HConnection is abort()'ed (i.e. if numerous services are lost) the > connection becomes unusable. HConnectionManager cache of HConnections > currently does not have any logic around removing aborted connections > automatically. Currently it is up to the consumer to do so using > HConnectionManager.deleteStaleConnection(HConnection). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7369) HConnectionManager should remove aborted connections
[ https://issues.apache.org/jira/browse/HBASE-7369?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Baugher updated HBASE-7369: - Status: Patch Available (was: Open) Patch provided > HConnectionManager should remove aborted connections > > > Key: HBASE-7369 > URL: https://issues.apache.org/jira/browse/HBASE-7369 > Project: HBase > Issue Type: Improvement > Components: Client >Affects Versions: 0.94.3 >Reporter: Bryan Baugher >Priority: Minor > Attachments: patch.diff > > > When an HConnection is abort()'ed (i.e. if numerous services are lost) the > connection becomes unusable. HConnectionManager cache of HConnections > currently does not have any logic around removing aborted connections > automatically. Currently it is up to the consumer to do so using > HConnectionManager.deleteStaleConnection(HConnection). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7369) HConnectionManager should remove aborted connections
[ https://issues.apache.org/jira/browse/HBASE-7369?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Baugher updated HBASE-7369: - Attachment: patch2.diff Addressed comment by Ted Yu and also fixed the test in which aborting the HConnection caused other tests/infrastructure to fail. > HConnectionManager should remove aborted connections > > > Key: HBASE-7369 > URL: https://issues.apache.org/jira/browse/HBASE-7369 > Project: HBase > Issue Type: Improvement > Components: Client >Affects Versions: 0.94.3 >Reporter: Bryan Baugher >Priority: Minor > Attachments: patch2.diff, patch.diff > > > When an HConnection is abort()'ed (i.e. if numerous services are lost) the > connection becomes unusable. HConnectionManager cache of HConnections > currently does not have any logic around removing aborted connections > automatically. Currently it is up to the consumer to do so using > HConnectionManager.deleteStaleConnection(HConnection). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7369) HConnectionManager should remove aborted connections
[ https://issues.apache.org/jira/browse/HBASE-7369?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13542423#comment-13542423 ] Bryan Baugher commented on HBASE-7369: -- Anything I can do to help this along? > HConnectionManager should remove aborted connections > > > Key: HBASE-7369 > URL: https://issues.apache.org/jira/browse/HBASE-7369 > Project: HBase > Issue Type: Improvement > Components: Client >Affects Versions: 0.94.3 >Reporter: Bryan Baugher >Priority: Minor > Attachments: patch2.diff, patch.diff > > > When an HConnection is abort()'ed (i.e. if numerous services are lost) the > connection becomes unusable. HConnectionManager cache of HConnections > currently does not have any logic around removing aborted connections > automatically. Currently it is up to the consumer to do so using > HConnectionManager.deleteStaleConnection(HConnection). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7369) HConnectionManager should remove aborted connections
[ https://issues.apache.org/jira/browse/HBASE-7369?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Baugher updated HBASE-7369: - Attachment: patch3.diff Re-attaching patch2.diff as patch3.diff > HConnectionManager should remove aborted connections > > > Key: HBASE-7369 > URL: https://issues.apache.org/jira/browse/HBASE-7369 > Project: HBase > Issue Type: Improvement > Components: Client >Affects Versions: 0.94.3 >Reporter: Bryan Baugher >Priority: Minor > Attachments: patch2.diff, patch3.diff, patch.diff > > > When an HConnection is abort()'ed (i.e. if numerous services are lost) the > connection becomes unusable. HConnectionManager cache of HConnections > currently does not have any logic around removing aborted connections > automatically. Currently it is up to the consumer to do so using > HConnectionManager.deleteStaleConnection(HConnection). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7369) HConnectionManager should remove aborted connections
[ https://issues.apache.org/jira/browse/HBASE-7369?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13543001#comment-13543001 ] Bryan Baugher commented on HBASE-7369: -- [~zjushch] I like the idea of moving the logic into close() and will do that shortly. The reason I created this issue and patch is because I was fighting an issue where our HConnection would become aborted and no matter how many times we closed and retrieved new HTables the connection was still aborted (because of the caching mechanism). With this patch that should no longer be true. My thought is that HTables should be short lived or it should be understandable that if an HTable throws an exception one should close it and retrieve a new one, which would fix the issue. > HConnectionManager should remove aborted connections > > > Key: HBASE-7369 > URL: https://issues.apache.org/jira/browse/HBASE-7369 > Project: HBase > Issue Type: Improvement > Components: Client >Affects Versions: 0.94.3 >Reporter: Bryan Baugher >Priority: Minor > Attachments: patch2.diff, patch3.diff, patch.diff > > > When an HConnection is abort()'ed (i.e. if numerous services are lost) the > connection becomes unusable. HConnectionManager cache of HConnections > currently does not have any logic around removing aborted connections > automatically. Currently it is up to the consumer to do so using > HConnectionManager.deleteStaleConnection(HConnection). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7369) HConnectionManager should remove aborted connections
[ https://issues.apache.org/jira/browse/HBASE-7369?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13543003#comment-13543003 ] Bryan Baugher commented on HBASE-7369: -- I had forgot what the code looked like, abort actually never calls close() but sets closed=true. > HConnectionManager should remove aborted connections > > > Key: HBASE-7369 > URL: https://issues.apache.org/jira/browse/HBASE-7369 > Project: HBase > Issue Type: Improvement > Components: Client >Affects Versions: 0.94.3 >Reporter: Bryan Baugher >Priority: Minor > Attachments: patch2.diff, patch3.diff, patch.diff > > > When an HConnection is abort()'ed (i.e. if numerous services are lost) the > connection becomes unusable. HConnectionManager cache of HConnections > currently does not have any logic around removing aborted connections > automatically. Currently it is up to the consumer to do so using > HConnectionManager.deleteStaleConnection(HConnection). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7369) HConnectionManager should remove aborted connections
[ https://issues.apache.org/jira/browse/HBASE-7369?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Baugher updated HBASE-7369: - Attachment: HBASE-7369_HCM-remove-aborted-cnxs.txt Sorry I mis-read what some of you had wrote. I believe this should address all concerns. > HConnectionManager should remove aborted connections > > > Key: HBASE-7369 > URL: https://issues.apache.org/jira/browse/HBASE-7369 > Project: HBase > Issue Type: Improvement > Components: Client >Affects Versions: 0.94.3 >Reporter: Bryan Baugher >Priority: Minor > Attachments: HBASE-7369_HCM-remove-aborted-cnxs.txt, patch2.diff, > patch3.diff, patch.diff > > > When an HConnection is abort()'ed (i.e. if numerous services are lost) the > connection becomes unusable. HConnectionManager cache of HConnections > currently does not have any logic around removing aborted connections > automatically. Currently it is up to the consumer to do so using > HConnectionManager.deleteStaleConnection(HConnection). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7369) HConnectionManager should remove aborted connections
[ https://issues.apache.org/jira/browse/HBASE-7369?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Baugher updated HBASE-7369: - Attachment: HBASE-7369_HCM-remove-aborted-cnxs-2.txt [~yuzhih...@gmail.com] Addressed comments with new patch. When we call close(), this.closed = true is not taken care of (at least not when managed = true). close() calls another function close(boolean) that will set this.closed=true when called but also seems to have some duplicate logic similar to HCM.deleteConnection(..). > HConnectionManager should remove aborted connections > > > Key: HBASE-7369 > URL: https://issues.apache.org/jira/browse/HBASE-7369 > Project: HBase > Issue Type: Improvement > Components: Client >Affects Versions: 0.94.3 >Reporter: Bryan Baugher >Priority: Minor > Attachments: HBASE-7369_HCM-remove-aborted-cnxs-2.txt, > HBASE-7369_HCM-remove-aborted-cnxs.txt, patch2.diff, patch3.diff, patch.diff > > > When an HConnection is abort()'ed (i.e. if numerous services are lost) the > connection becomes unusable. HConnectionManager cache of HConnections > currently does not have any logic around removing aborted connections > automatically. Currently it is up to the consumer to do so using > HConnectionManager.deleteStaleConnection(HConnection). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7369) HConnectionManager should remove aborted connections
[ https://issues.apache.org/jira/browse/HBASE-7369?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Baugher updated HBASE-7369: - Attachment: HBASE-7369_HCM-remove-aborted-cnxs-3.txt Addressed all comments with latest patch > HConnectionManager should remove aborted connections > > > Key: HBASE-7369 > URL: https://issues.apache.org/jira/browse/HBASE-7369 > Project: HBase > Issue Type: Improvement > Components: Client >Affects Versions: 0.94.3 >Reporter: Bryan Baugher >Priority: Minor > Attachments: HBASE-7369_HCM-remove-aborted-cnxs-2.txt, > HBASE-7369_HCM-remove-aborted-cnxs-3.txt, > HBASE-7369_HCM-remove-aborted-cnxs.txt, patch2.diff, patch3.diff, patch.diff > > > When an HConnection is abort()'ed (i.e. if numerous services are lost) the > connection becomes unusable. HConnectionManager cache of HConnections > currently does not have any logic around removing aborted connections > automatically. Currently it is up to the consumer to do so using > HConnectionManager.deleteStaleConnection(HConnection). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7369) HConnectionManager should remove aborted connections
[ https://issues.apache.org/jira/browse/HBASE-7369?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Baugher updated HBASE-7369: - Attachment: HBASE-7369_HCM-remove-aborted-cnxs-4.txt Resolved conflict > HConnectionManager should remove aborted connections > > > Key: HBASE-7369 > URL: https://issues.apache.org/jira/browse/HBASE-7369 > Project: HBase > Issue Type: Improvement > Components: Client >Affects Versions: 0.94.3 >Reporter: Bryan Baugher >Priority: Minor > Attachments: HBASE-7369_HCM-remove-aborted-cnxs-2.txt, > HBASE-7369_HCM-remove-aborted-cnxs-3.txt, > HBASE-7369_HCM-remove-aborted-cnxs-4.txt, > HBASE-7369_HCM-remove-aborted-cnxs.txt, patch2.diff, patch3.diff, patch.diff > > > When an HConnection is abort()'ed (i.e. if numerous services are lost) the > connection becomes unusable. HConnectionManager cache of HConnections > currently does not have any logic around removing aborted connections > automatically. Currently it is up to the consumer to do so using > HConnectionManager.deleteStaleConnection(HConnection). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-3996) Support multiple tables and scanners as input to the mapper in map/reduce jobs
[ https://issues.apache.org/jira/browse/HBASE-3996?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Baugher updated HBASE-3996: - Attachment: 3996-v8.txt I would like to offer to finish this issue. If you would rather close this issue or start a new one that is fine just let me know. Here is what I have done from the previous version, * Removed random formatting changes * Removed table.close() in TableRecordReaderImpl * Replaced MultiTableInputCollection with List * Brought up to date with trunk Remaining questions, * Since most of enum Version code is copied, we may want to factor the base enum to its own class. Would org.apache.hadoop.hbase.util be a good namespace for the enum class ? * The changes to TableSplit would not allow a new version of it to be deserialized by an old server. Is that OK for a M/R job? * It has been mentioned to scope this to scans (of a single table) rather then multiple tables. I can't seem to get the tests to run for me (getting OOM errors) but I would imagine most everything still works. > Support multiple tables and scanners as input to the mapper in map/reduce jobs > -- > > Key: HBASE-3996 > URL: https://issues.apache.org/jira/browse/HBASE-3996 > Project: HBase > Issue Type: Improvement > Components: mapreduce >Reporter: Eran Kutner >Assignee: Lars Hofhansl > Fix For: 0.96.0, 0.94.5 > > Attachments: 3996-v2.txt, 3996-v3.txt, 3996-v4.txt, 3996-v5.txt, > 3996-v6.txt, 3996-v7.txt, 3996-v8.txt, HBase-3996.patch > > > It seems that in many cases feeding data from multiple tables or multiple > scanners on a single table can save a lot of time when running map/reduce > jobs. > I propose a new MultiTableInputFormat class that would allow doing this. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-3996) Support multiple tables and scanners as input to the mapper in map/reduce jobs
[ https://issues.apache.org/jira/browse/HBASE-3996?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Baugher updated HBASE-3996: - Attachment: 3996-v9.txt Well thats not a great way to start off. Lets try this again. > Support multiple tables and scanners as input to the mapper in map/reduce jobs > -- > > Key: HBASE-3996 > URL: https://issues.apache.org/jira/browse/HBASE-3996 > Project: HBase > Issue Type: Improvement > Components: mapreduce >Reporter: Eran Kutner >Assignee: Lars Hofhansl > Fix For: 0.96.0, 0.94.5 > > Attachments: 3996-v2.txt, 3996-v3.txt, 3996-v4.txt, 3996-v5.txt, > 3996-v6.txt, 3996-v7.txt, 3996-v8.txt, 3996-v9.txt, HBase-3996.patch > > > It seems that in many cases feeding data from multiple tables or multiple > scanners on a single table can save a lot of time when running map/reduce > jobs. > I propose a new MultiTableInputFormat class that would allow doing this. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-3996) Support multiple tables and scanners as input to the mapper in map/reduce jobs
[ https://issues.apache.org/jira/browse/HBASE-3996?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Baugher updated HBASE-3996: - Attachment: 3996-v10.txt Well that would be because I forgot to include my changes to Scan. Done. > Support multiple tables and scanners as input to the mapper in map/reduce jobs > -- > > Key: HBASE-3996 > URL: https://issues.apache.org/jira/browse/HBASE-3996 > Project: HBase > Issue Type: Improvement > Components: mapreduce >Reporter: Eran Kutner >Assignee: Lars Hofhansl > Fix For: 0.96.0, 0.94.5 > > Attachments: 3996-v10.txt, 3996-v2.txt, 3996-v3.txt, 3996-v4.txt, > 3996-v5.txt, 3996-v6.txt, 3996-v7.txt, 3996-v8.txt, 3996-v9.txt, > HBase-3996.patch > > > It seems that in many cases feeding data from multiple tables or multiple > scanners on a single table can save a lot of time when running map/reduce > jobs. > I propose a new MultiTableInputFormat class that would allow doing this. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-3996) Support multiple tables and scanners as input to the mapper in map/reduce jobs
[ https://issues.apache.org/jira/browse/HBASE-3996?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Baugher updated HBASE-3996: - Attachment: 3996-v11.txt Fixed test error, line length and added stability annotation. I was finally able to get the test to run so I may try to clean up / add more tests in the meantime. > Support multiple tables and scanners as input to the mapper in map/reduce jobs > -- > > Key: HBASE-3996 > URL: https://issues.apache.org/jira/browse/HBASE-3996 > Project: HBase > Issue Type: Improvement > Components: mapreduce >Reporter: Eran Kutner >Assignee: Lars Hofhansl > Fix For: 0.96.0, 0.94.5 > > Attachments: 3996-v10.txt, 3996-v11.txt, 3996-v2.txt, 3996-v3.txt, > 3996-v4.txt, 3996-v5.txt, 3996-v6.txt, 3996-v7.txt, 3996-v8.txt, 3996-v9.txt, > HBase-3996.patch > > > It seems that in many cases feeding data from multiple tables or multiple > scanners on a single table can save a lot of time when running map/reduce > jobs. > I propose a new MultiTableInputFormat class that would allow doing this. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-3996) Support multiple tables and scanners as input to the mapper in map/reduce jobs
[ https://issues.apache.org/jira/browse/HBASE-3996?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Baugher updated HBASE-3996: - Attachment: 3996-v12.txt Update to latest trunk which had a conflict > Support multiple tables and scanners as input to the mapper in map/reduce jobs > -- > > Key: HBASE-3996 > URL: https://issues.apache.org/jira/browse/HBASE-3996 > Project: HBase > Issue Type: Improvement > Components: mapreduce >Reporter: Eran Kutner >Assignee: Bryan Baugher >Priority: Critical > Fix For: 0.96.0, 0.94.5 > > Attachments: 3996-v10.txt, 3996-v11.txt, 3996-v12.txt, 3996-v2.txt, > 3996-v3.txt, 3996-v4.txt, 3996-v5.txt, 3996-v6.txt, 3996-v7.txt, 3996-v8.txt, > 3996-v9.txt, HBase-3996.patch > > > It seems that in many cases feeding data from multiple tables or multiple > scanners on a single table can save a lot of time when running map/reduce > jobs. > I propose a new MultiTableInputFormat class that would allow doing this. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-3996) Support multiple tables and scanners as input to the mapper in map/reduce jobs
[ https://issues.apache.org/jira/browse/HBASE-3996?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13558890#comment-13558890 ] Bryan Baugher commented on HBASE-3996: -- I believe there are two possible questions left unanswered as well as some +1's still needed, * The changes to TableSplit would not allow a new version of it to be deserialized by an old server. Is that OK for a M/R job? * It has been mentioned to scope this to scans (of a single table) rather then multiple tables. > Support multiple tables and scanners as input to the mapper in map/reduce jobs > -- > > Key: HBASE-3996 > URL: https://issues.apache.org/jira/browse/HBASE-3996 > Project: HBase > Issue Type: Improvement > Components: mapreduce >Reporter: Eran Kutner >Assignee: Bryan Baugher >Priority: Critical > Fix For: 0.96.0, 0.94.5 > > Attachments: 3996-v10.txt, 3996-v11.txt, 3996-v12.txt, 3996-v2.txt, > 3996-v3.txt, 3996-v4.txt, 3996-v5.txt, 3996-v6.txt, 3996-v7.txt, 3996-v8.txt, > 3996-v9.txt, HBase-3996.patch > > > It seems that in many cases feeding data from multiple tables or multiple > scanners on a single table can save a lot of time when running map/reduce > jobs. > I propose a new MultiTableInputFormat class that would allow doing this. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-3996) Support multiple tables and scanners as input to the mapper in map/reduce jobs
[ https://issues.apache.org/jira/browse/HBASE-3996?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13558899#comment-13558899 ] Bryan Baugher commented on HBASE-3996: -- Done, https://reviews.apache.org/r/9042/diff/ > Support multiple tables and scanners as input to the mapper in map/reduce jobs > -- > > Key: HBASE-3996 > URL: https://issues.apache.org/jira/browse/HBASE-3996 > Project: HBase > Issue Type: Improvement > Components: mapreduce >Reporter: Eran Kutner >Assignee: Bryan Baugher >Priority: Critical > Fix For: 0.96.0, 0.94.5 > > Attachments: 3996-v10.txt, 3996-v11.txt, 3996-v12.txt, 3996-v2.txt, > 3996-v3.txt, 3996-v4.txt, 3996-v5.txt, 3996-v6.txt, 3996-v7.txt, 3996-v8.txt, > 3996-v9.txt, HBase-3996.patch > > > It seems that in many cases feeding data from multiple tables or multiple > scanners on a single table can save a lot of time when running map/reduce > jobs. > I propose a new MultiTableInputFormat class that would allow doing this. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-3996) Support multiple tables and scanners as input to the mapper in map/reduce jobs
[ https://issues.apache.org/jira/browse/HBASE-3996?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Baugher updated HBASE-3996: - Attachment: 3996-v13.txt Attached latest patch address review comments > Support multiple tables and scanners as input to the mapper in map/reduce jobs > -- > > Key: HBASE-3996 > URL: https://issues.apache.org/jira/browse/HBASE-3996 > Project: HBase > Issue Type: Improvement > Components: mapreduce >Reporter: Eran Kutner >Assignee: Bryan Baugher >Priority: Critical > Fix For: 0.96.0, 0.94.5 > > Attachments: 3996-v10.txt, 3996-v11.txt, 3996-v12.txt, 3996-v13.txt, > 3996-v2.txt, 3996-v3.txt, 3996-v4.txt, 3996-v5.txt, 3996-v6.txt, 3996-v7.txt, > 3996-v8.txt, 3996-v9.txt, HBase-3996.patch > > > It seems that in many cases feeding data from multiple tables or multiple > scanners on a single table can save a lot of time when running map/reduce > jobs. > I propose a new MultiTableInputFormat class that would allow doing this. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-3996) Support multiple tables and scanners as input to the mapper in map/reduce jobs
[ https://issues.apache.org/jira/browse/HBASE-3996?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Baugher updated HBASE-3996: - Attachment: 3996-v14.txt Fixed findbug error in MultiTableInputFormatBase > Support multiple tables and scanners as input to the mapper in map/reduce jobs > -- > > Key: HBASE-3996 > URL: https://issues.apache.org/jira/browse/HBASE-3996 > Project: HBase > Issue Type: Improvement > Components: mapreduce >Reporter: Eran Kutner >Assignee: Bryan Baugher >Priority: Critical > Fix For: 0.96.0, 0.94.5 > > Attachments: 3996-v10.txt, 3996-v11.txt, 3996-v12.txt, 3996-v13.txt, > 3996-v14.txt, 3996-v2.txt, 3996-v3.txt, 3996-v4.txt, 3996-v5.txt, > 3996-v6.txt, 3996-v7.txt, 3996-v8.txt, 3996-v9.txt, HBase-3996.patch > > > It seems that in many cases feeding data from multiple tables or multiple > scanners on a single table can save a lot of time when running map/reduce > jobs. > I propose a new MultiTableInputFormat class that would allow doing this. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-3996) Support multiple tables and scanners as input to the mapper in map/reduce jobs
[ https://issues.apache.org/jira/browse/HBASE-3996?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13566521#comment-13566521 ] Bryan Baugher commented on HBASE-3996: -- No I have not. We primarily run CDH4 here (currently at 92.1). > Support multiple tables and scanners as input to the mapper in map/reduce jobs > -- > > Key: HBASE-3996 > URL: https://issues.apache.org/jira/browse/HBASE-3996 > Project: HBase > Issue Type: Improvement > Components: mapreduce >Reporter: Eran Kutner >Assignee: Bryan Baugher >Priority: Critical > Fix For: 0.96.0, 0.94.5 > > Attachments: 3996-v10.txt, 3996-v11.txt, 3996-v12.txt, 3996-v13.txt, > 3996-v14.txt, 3996-v2.txt, 3996-v3.txt, 3996-v4.txt, 3996-v5.txt, > 3996-v6.txt, 3996-v7.txt, 3996-v8.txt, 3996-v9.txt, HBase-3996.patch > > > It seems that in many cases feeding data from multiple tables or multiple > scanners on a single table can save a lot of time when running map/reduce > jobs. > I propose a new MultiTableInputFormat class that would allow doing this. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-7894) Update Filter javadoc about the use of read/write fields for multiple regions
Bryan Baugher created HBASE-7894: Summary: Update Filter javadoc about the use of read/write fields for multiple regions Key: HBASE-7894 URL: https://issues.apache.org/jira/browse/HBASE-7894 Project: HBase Issue Type: Improvement Reporter: Bryan Baugher Priority: Minor Following this[1] on the mailing list, it is not obvious how hbase will use a filter's read/write fields when there are multiple regions. Javadoc should be added explaining that for scans spanning multiple regions hbase will re-create the filter using the previous filter's writeFields instead of the original scan. [1] - http://mail-archives.apache.org/mod_mbox/hbase-user/201302.mbox/%3CCANZ-JHE-8FATa69ucHQmmZAUEkN1dKshiRKLJyb_bFf%3D6ZNm2A%40mail.gmail.com%3E -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6956) Do not return back to HTablePool closed connections
[
https://issues.apache.org/jira/browse/HBASE-6956?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13507345#comment-13507345
]
Bryan Baugher commented on HBASE-6956:
--
We have also ran into this issue specifically during major failure of the
cluster (i.e. hmaster, zookeeper, regionservers are down).
[~amuraru] did you manage to find a good way to flush a bad table from your
pool when the connection becomes closed?
> Do not return back to HTablePool closed connections
> ---
>
> Key: HBASE-6956
> URL: https://issues.apache.org/jira/browse/HBASE-6956
> Project: HBase
> Issue Type: Bug
> Components: Client
>Affects Versions: 0.90.6
>Reporter: Igor Yurinok
>
> Sometimes we see a lot of Exception about closed connections:
> {code}
>
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@553fd068
> closed
> org.apache.hadoop.hbase.client.ClosedConnectionException:
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@553fd068
> closed
> {code}
> After investigation we assumed that it occurs because closed connection
> returns back into HTablePool.
> For our opinion best solution is check whether the table is closed in method
> HTablePool.putTable and if true don't add it into the queue and release such
> HTableInterface.
> But unfortunatly right now there are no access to HTable#closed field through
> HTableInterface
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
