[jira] [Updated] (HBASE-6617) ReplicationSourceManager should be able to track multiple WAL paths
[ https://issues.apache.org/jira/browse/HBASE-6617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Elliott Clark updated HBASE-6617: - Fix Version/s: (was: 1.20) 1.2.0 > ReplicationSourceManager should be able to track multiple WAL paths > --- > > Key: HBASE-6617 > URL: https://issues.apache.org/jira/browse/HBASE-6617 > Project: HBase > Issue Type: Improvement > Components: Replication >Reporter: Ted Yu >Assignee: Yu Li > Fix For: 2.0.0, 1.2.0, 1.3.0 > > Attachments: 6617-v11.patch, HBASE-6617.branch-1.patch, > HBASE-6617.branch-1.v2.patch, HBASE-6617.branch-1.v2.patch, > HBASE-6617.branch-1.v2.patch, HBASE-6617.patch, HBASE-6617_v10.patch, > HBASE-6617_v11.patch, HBASE-6617_v12.patch, HBASE-6617_v2.patch, > HBASE-6617_v3.patch, HBASE-6617_v4.patch, HBASE-6617_v7.patch, > HBASE-6617_v9.patch > > > Currently ReplicationSourceManager uses logRolled() to receive notification > about new HLog and remembers it in latestPath. > When region server has multiple WAL support, we need to keep track of > multiple Path's in ReplicationSourceManager -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-6617) ReplicationSourceManager should be able to track multiple WAL paths
[ https://issues.apache.org/jira/browse/HBASE-6617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Elliott Clark updated HBASE-6617: - Fix Version/s: 1.20 > ReplicationSourceManager should be able to track multiple WAL paths > --- > > Key: HBASE-6617 > URL: https://issues.apache.org/jira/browse/HBASE-6617 > Project: HBase > Issue Type: Improvement > Components: Replication >Reporter: Ted Yu >Assignee: Yu Li > Fix For: 2.0.0, 1.2.0, 1.3.0 > > Attachments: 6617-v11.patch, HBASE-6617.branch-1.patch, > HBASE-6617.branch-1.v2.patch, HBASE-6617.branch-1.v2.patch, > HBASE-6617.branch-1.v2.patch, HBASE-6617.patch, HBASE-6617_v10.patch, > HBASE-6617_v11.patch, HBASE-6617_v12.patch, HBASE-6617_v2.patch, > HBASE-6617_v3.patch, HBASE-6617_v4.patch, HBASE-6617_v7.patch, > HBASE-6617_v9.patch > > > Currently ReplicationSourceManager uses logRolled() to receive notification > about new HLog and remembers it in latestPath. > When region server has multiple WAL support, we need to keep track of > multiple Path's in ReplicationSourceManager -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-6617) ReplicationSourceManager should be able to track multiple WAL paths
[ https://issues.apache.org/jira/browse/HBASE-6617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-6617: -- Resolution: Fixed Status: Resolved (was: Patch Available) Thanks Yu for the contribution. Thanks for the reviews Sean and Chunhui. > ReplicationSourceManager should be able to track multiple WAL paths > --- > > Key: HBASE-6617 > URL: https://issues.apache.org/jira/browse/HBASE-6617 > Project: HBase > Issue Type: Improvement > Components: Replication >Reporter: Ted Yu >Assignee: Yu Li > Fix For: 2.0.0, 1.3.0 > > Attachments: 6617-v11.patch, HBASE-6617.branch-1.patch, > HBASE-6617.branch-1.v2.patch, HBASE-6617.branch-1.v2.patch, > HBASE-6617.branch-1.v2.patch, HBASE-6617.patch, HBASE-6617_v10.patch, > HBASE-6617_v11.patch, HBASE-6617_v12.patch, HBASE-6617_v2.patch, > HBASE-6617_v3.patch, HBASE-6617_v4.patch, HBASE-6617_v7.patch, > HBASE-6617_v9.patch > > > Currently ReplicationSourceManager uses logRolled() to receive notification > about new HLog and remembers it in latestPath. > When region server has multiple WAL support, we need to keep track of > multiple Path's in ReplicationSourceManager -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-6617) ReplicationSourceManager should be able to track multiple WAL paths
[ https://issues.apache.org/jira/browse/HBASE-6617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yu Li updated HBASE-6617: - Release Note: ReplicationSourceManager now could track multiple wal paths. Notice that although most changes are internal and all metrics names remain the same, signature of below methods in MetricsSource are changed: 1. refreshAgeOfLastShippedOp now requires a String parameter which indicates the wal group id of the reporter 2. setAgeOfLastShippedOp also adds a String parameter for wal group id > ReplicationSourceManager should be able to track multiple WAL paths > --- > > Key: HBASE-6617 > URL: https://issues.apache.org/jira/browse/HBASE-6617 > Project: HBase > Issue Type: Improvement > Components: Replication >Reporter: Ted Yu >Assignee: Yu Li > Fix For: 2.0.0, 1.3.0 > > Attachments: 6617-v11.patch, HBASE-6617.branch-1.patch, > HBASE-6617.branch-1.v2.patch, HBASE-6617.branch-1.v2.patch, > HBASE-6617.branch-1.v2.patch, HBASE-6617.patch, HBASE-6617_v10.patch, > HBASE-6617_v11.patch, HBASE-6617_v12.patch, HBASE-6617_v2.patch, > HBASE-6617_v3.patch, HBASE-6617_v4.patch, HBASE-6617_v7.patch, > HBASE-6617_v9.patch > > > Currently ReplicationSourceManager uses logRolled() to receive notification > about new HLog and remembers it in latestPath. > When region server has multiple WAL support, we need to keep track of > multiple Path's in ReplicationSourceManager -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-6617) ReplicationSourceManager should be able to track multiple WAL paths
[ https://issues.apache.org/jira/browse/HBASE-6617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-6617: -- Hadoop Flags: Reviewed > ReplicationSourceManager should be able to track multiple WAL paths > --- > > Key: HBASE-6617 > URL: https://issues.apache.org/jira/browse/HBASE-6617 > Project: HBase > Issue Type: Improvement > Components: Replication >Reporter: Ted Yu >Assignee: Yu Li > Fix For: 2.0.0, 1.3.0 > > Attachments: 6617-v11.patch, HBASE-6617.branch-1.patch, > HBASE-6617.branch-1.v2.patch, HBASE-6617.branch-1.v2.patch, > HBASE-6617.branch-1.v2.patch, HBASE-6617.patch, HBASE-6617_v10.patch, > HBASE-6617_v11.patch, HBASE-6617_v12.patch, HBASE-6617_v2.patch, > HBASE-6617_v3.patch, HBASE-6617_v4.patch, HBASE-6617_v7.patch, > HBASE-6617_v9.patch > > > Currently ReplicationSourceManager uses logRolled() to receive notification > about new HLog and remembers it in latestPath. > When region server has multiple WAL support, we need to keep track of > multiple Path's in ReplicationSourceManager -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-6617) ReplicationSourceManager should be able to track multiple WAL paths
[ https://issues.apache.org/jira/browse/HBASE-6617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-6617: -- Attachment: HBASE-6617.branch-1.v2.patch > ReplicationSourceManager should be able to track multiple WAL paths > --- > > Key: HBASE-6617 > URL: https://issues.apache.org/jira/browse/HBASE-6617 > Project: HBase > Issue Type: Improvement > Components: Replication >Reporter: Ted Yu >Assignee: Yu Li > Fix For: 2.0.0, 1.3.0 > > Attachments: 6617-v11.patch, HBASE-6617.branch-1.patch, > HBASE-6617.branch-1.v2.patch, HBASE-6617.branch-1.v2.patch, > HBASE-6617.branch-1.v2.patch, HBASE-6617.patch, HBASE-6617_v10.patch, > HBASE-6617_v11.patch, HBASE-6617_v12.patch, HBASE-6617_v2.patch, > HBASE-6617_v3.patch, HBASE-6617_v4.patch, HBASE-6617_v7.patch, > HBASE-6617_v9.patch > > > Currently ReplicationSourceManager uses logRolled() to receive notification > about new HLog and remembers it in latestPath. > When region server has multiple WAL support, we need to keep track of > multiple Path's in ReplicationSourceManager -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-6617) ReplicationSourceManager should be able to track multiple WAL paths
[ https://issues.apache.org/jira/browse/HBASE-6617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-6617: -- Attachment: HBASE-6617.branch-1.v2.patch > ReplicationSourceManager should be able to track multiple WAL paths > --- > > Key: HBASE-6617 > URL: https://issues.apache.org/jira/browse/HBASE-6617 > Project: HBase > Issue Type: Improvement > Components: Replication >Reporter: Ted Yu >Assignee: Yu Li > Fix For: 2.0.0, 1.3.0 > > Attachments: 6617-v11.patch, HBASE-6617.branch-1.patch, > HBASE-6617.branch-1.v2.patch, HBASE-6617.branch-1.v2.patch, HBASE-6617.patch, > HBASE-6617_v10.patch, HBASE-6617_v11.patch, HBASE-6617_v12.patch, > HBASE-6617_v2.patch, HBASE-6617_v3.patch, HBASE-6617_v4.patch, > HBASE-6617_v7.patch, HBASE-6617_v9.patch > > > Currently ReplicationSourceManager uses logRolled() to receive notification > about new HLog and remembers it in latestPath. > When region server has multiple WAL support, we need to keep track of > multiple Path's in ReplicationSourceManager -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-6617) ReplicationSourceManager should be able to track multiple WAL paths
[ https://issues.apache.org/jira/browse/HBASE-6617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yu Li updated HBASE-6617: - Attachment: HBASE-6617.branch-1.v2.patch v2 patch for branch-1, fix the checkstyle issues and ask HadoopQA to rerun UT > ReplicationSourceManager should be able to track multiple WAL paths > --- > > Key: HBASE-6617 > URL: https://issues.apache.org/jira/browse/HBASE-6617 > Project: HBase > Issue Type: Improvement > Components: Replication >Reporter: Ted Yu >Assignee: Yu Li > Fix For: 2.0.0, 1.3.0 > > Attachments: 6617-v11.patch, HBASE-6617.branch-1.patch, > HBASE-6617.branch-1.v2.patch, HBASE-6617.patch, HBASE-6617_v10.patch, > HBASE-6617_v11.patch, HBASE-6617_v12.patch, HBASE-6617_v2.patch, > HBASE-6617_v3.patch, HBASE-6617_v4.patch, HBASE-6617_v7.patch, > HBASE-6617_v9.patch > > > Currently ReplicationSourceManager uses logRolled() to receive notification > about new HLog and remembers it in latestPath. > When region server has multiple WAL support, we need to keep track of > multiple Path's in ReplicationSourceManager -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-6617) ReplicationSourceManager should be able to track multiple WAL paths
[ https://issues.apache.org/jira/browse/HBASE-6617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yu Li updated HBASE-6617: - Attachment: HBASE-6617.branch-1.patch Upload the patch for branch-1 > ReplicationSourceManager should be able to track multiple WAL paths > --- > > Key: HBASE-6617 > URL: https://issues.apache.org/jira/browse/HBASE-6617 > Project: HBase > Issue Type: Improvement > Components: Replication >Reporter: Ted Yu >Assignee: Yu Li > Fix For: 2.0.0, 1.3.0 > > Attachments: 6617-v11.patch, HBASE-6617.branch-1.patch, > HBASE-6617.patch, HBASE-6617_v10.patch, HBASE-6617_v11.patch, > HBASE-6617_v12.patch, HBASE-6617_v2.patch, HBASE-6617_v3.patch, > HBASE-6617_v4.patch, HBASE-6617_v7.patch, HBASE-6617_v9.patch > > > Currently ReplicationSourceManager uses logRolled() to receive notification > about new HLog and remembers it in latestPath. > When region server has multiple WAL support, we need to keep track of > multiple Path's in ReplicationSourceManager -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-6617) ReplicationSourceManager should be able to track multiple WAL paths
[ https://issues.apache.org/jira/browse/HBASE-6617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yu Li updated HBASE-6617: - Attachment: HBASE-6617_v12.patch Fix UT failures in the new patch. The configuration used for multiwal in the newly-added UT cases in patch v11 was based on HBASE-14306, which will cause ClassNotFound issue in current code base. Also checked the UT failure of TestImportExport#testImport94Table and TestProcessBasedCluster#testHomePath, both could pass on my local run. Let's see what HadoopQA will say with the new patch. > ReplicationSourceManager should be able to track multiple WAL paths > --- > > Key: HBASE-6617 > URL: https://issues.apache.org/jira/browse/HBASE-6617 > Project: HBase > Issue Type: Improvement > Components: Replication >Reporter: Ted Yu >Assignee: Yu Li > Fix For: 2.0.0, 1.3.0 > > Attachments: 6617-v11.patch, HBASE-6617.patch, HBASE-6617_v10.patch, > HBASE-6617_v11.patch, HBASE-6617_v12.patch, HBASE-6617_v2.patch, > HBASE-6617_v3.patch, HBASE-6617_v4.patch, HBASE-6617_v7.patch, > HBASE-6617_v9.patch > > > Currently ReplicationSourceManager uses logRolled() to receive notification > about new HLog and remembers it in latestPath. > When region server has multiple WAL support, we need to keep track of > multiple Path's in ReplicationSourceManager -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-6617) ReplicationSourceManager should be able to track multiple WAL paths
[ https://issues.apache.org/jira/browse/HBASE-6617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-6617: -- Attachment: 6617-v11.patch Re-attaching patch v11. > ReplicationSourceManager should be able to track multiple WAL paths > --- > > Key: HBASE-6617 > URL: https://issues.apache.org/jira/browse/HBASE-6617 > Project: HBase > Issue Type: Improvement > Components: Replication >Reporter: Ted Yu >Assignee: Yu Li > Fix For: 2.0.0, 1.3.0 > > Attachments: 6617-v11.patch, HBASE-6617.patch, HBASE-6617_v10.patch, > HBASE-6617_v11.patch, HBASE-6617_v2.patch, HBASE-6617_v3.patch, > HBASE-6617_v4.patch, HBASE-6617_v7.patch, HBASE-6617_v9.patch > > > Currently ReplicationSourceManager uses logRolled() to receive notification > about new HLog and remembers it in latestPath. > When region server has multiple WAL support, we need to keep track of > multiple Path's in ReplicationSourceManager -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-6617) ReplicationSourceManager should be able to track multiple WAL paths
[ https://issues.apache.org/jira/browse/HBASE-6617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yu Li updated HBASE-6617: - Attachment: HBASE-6617_v11.patch Refactor the patch based on latest code base after HBASE-12988 > ReplicationSourceManager should be able to track multiple WAL paths > --- > > Key: HBASE-6617 > URL: https://issues.apache.org/jira/browse/HBASE-6617 > Project: HBase > Issue Type: Improvement > Components: Replication >Reporter: Ted Yu >Assignee: Yu Li > Fix For: 2.0.0, 1.3.0 > > Attachments: HBASE-6617.patch, HBASE-6617_v10.patch, > HBASE-6617_v11.patch, HBASE-6617_v2.patch, HBASE-6617_v3.patch, > HBASE-6617_v4.patch, HBASE-6617_v7.patch, HBASE-6617_v9.patch > > > Currently ReplicationSourceManager uses logRolled() to receive notification > about new HLog and remembers it in latestPath. > When region server has multiple WAL support, we need to keep track of > multiple Path's in ReplicationSourceManager -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-6617) ReplicationSourceManager should be able to track multiple WAL paths
[ https://issues.apache.org/jira/browse/HBASE-6617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yu Li updated HBASE-6617: - Attachment: HBASE-6617_v10.patch > ReplicationSourceManager should be able to track multiple WAL paths > --- > > Key: HBASE-6617 > URL: https://issues.apache.org/jira/browse/HBASE-6617 > Project: HBase > Issue Type: Improvement > Components: Replication >Reporter: Ted Yu >Assignee: Yu Li > Fix For: 2.0.0, 1.3.0 > > Attachments: HBASE-6617.patch, HBASE-6617_v10.patch, > HBASE-6617_v2.patch, HBASE-6617_v3.patch, HBASE-6617_v4.patch, > HBASE-6617_v7.patch, HBASE-6617_v9.patch > > > Currently ReplicationSourceManager uses logRolled() to receive notification > about new HLog and remembers it in latestPath. > When region server has multiple WAL support, we need to keep track of > multiple Path's in ReplicationSourceManager -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-6617) ReplicationSourceManager should be able to track multiple WAL paths
[ https://issues.apache.org/jira/browse/HBASE-6617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yu Li updated HBASE-6617: - Attachment: (was: HBASE-6617_v10.patch) > ReplicationSourceManager should be able to track multiple WAL paths > --- > > Key: HBASE-6617 > URL: https://issues.apache.org/jira/browse/HBASE-6617 > Project: HBase > Issue Type: Improvement > Components: Replication >Reporter: Ted Yu >Assignee: Yu Li > Fix For: 2.0.0, 1.3.0 > > Attachments: HBASE-6617.patch, HBASE-6617_v10.patch, > HBASE-6617_v2.patch, HBASE-6617_v3.patch, HBASE-6617_v4.patch, > HBASE-6617_v7.patch, HBASE-6617_v9.patch > > > Currently ReplicationSourceManager uses logRolled() to receive notification > about new HLog and remembers it in latestPath. > When region server has multiple WAL support, we need to keep track of > multiple Path's in ReplicationSourceManager -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-6617) ReplicationSourceManager should be able to track multiple WAL paths
[ https://issues.apache.org/jira/browse/HBASE-6617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yu Li updated HBASE-6617: - Status: Open (was: Patch Available) > ReplicationSourceManager should be able to track multiple WAL paths > --- > > Key: HBASE-6617 > URL: https://issues.apache.org/jira/browse/HBASE-6617 > Project: HBase > Issue Type: Improvement > Components: Replication >Reporter: Ted Yu >Assignee: Yu Li > Fix For: 2.0.0, 1.3.0 > > Attachments: HBASE-6617.patch, HBASE-6617_v10.patch, > HBASE-6617_v2.patch, HBASE-6617_v3.patch, HBASE-6617_v4.patch, > HBASE-6617_v7.patch, HBASE-6617_v9.patch > > > Currently ReplicationSourceManager uses logRolled() to receive notification > about new HLog and remembers it in latestPath. > When region server has multiple WAL support, we need to keep track of > multiple Path's in ReplicationSourceManager -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-6617) ReplicationSourceManager should be able to track multiple WAL paths
[ https://issues.apache.org/jira/browse/HBASE-6617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yu Li updated HBASE-6617: - Status: Patch Available (was: Open) Resubmit the patch again per requested > ReplicationSourceManager should be able to track multiple WAL paths > --- > > Key: HBASE-6617 > URL: https://issues.apache.org/jira/browse/HBASE-6617 > Project: HBase > Issue Type: Improvement > Components: Replication >Reporter: Ted Yu >Assignee: Yu Li > Fix For: 2.0.0, 1.3.0 > > Attachments: HBASE-6617.patch, HBASE-6617_v10.patch, > HBASE-6617_v2.patch, HBASE-6617_v3.patch, HBASE-6617_v4.patch, > HBASE-6617_v7.patch, HBASE-6617_v9.patch > > > Currently ReplicationSourceManager uses logRolled() to receive notification > about new HLog and remembers it in latestPath. > When region server has multiple WAL support, we need to keep track of > multiple Path's in ReplicationSourceManager -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-6617) ReplicationSourceManager should be able to track multiple WAL paths
[ https://issues.apache.org/jira/browse/HBASE-6617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yu Li updated HBASE-6617: - Status: Patch Available (was: Open) > ReplicationSourceManager should be able to track multiple WAL paths > --- > > Key: HBASE-6617 > URL: https://issues.apache.org/jira/browse/HBASE-6617 > Project: HBase > Issue Type: Improvement > Components: Replication >Reporter: Ted Yu >Assignee: Yu Li > Fix For: 2.0.0, 1.3.0 > > Attachments: HBASE-6617.patch, HBASE-6617_v10.patch, > HBASE-6617_v2.patch, HBASE-6617_v3.patch, HBASE-6617_v4.patch, > HBASE-6617_v7.patch, HBASE-6617_v9.patch > > > Currently ReplicationSourceManager uses logRolled() to receive notification > about new HLog and remembers it in latestPath. > When region server has multiple WAL support, we need to keep track of > multiple Path's in ReplicationSourceManager -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-6617) ReplicationSourceManager should be able to track multiple WAL paths
[ https://issues.apache.org/jira/browse/HBASE-6617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yu Li updated HBASE-6617: - Attachment: (was: HBASE-6617_v10.patch) > ReplicationSourceManager should be able to track multiple WAL paths > --- > > Key: HBASE-6617 > URL: https://issues.apache.org/jira/browse/HBASE-6617 > Project: HBase > Issue Type: Improvement > Components: Replication >Reporter: Ted Yu >Assignee: Yu Li > Fix For: 2.0.0, 1.3.0 > > Attachments: HBASE-6617.patch, HBASE-6617_v10.patch, > HBASE-6617_v2.patch, HBASE-6617_v3.patch, HBASE-6617_v4.patch, > HBASE-6617_v7.patch, HBASE-6617_v9.patch > > > Currently ReplicationSourceManager uses logRolled() to receive notification > about new HLog and remembers it in latestPath. > When region server has multiple WAL support, we need to keep track of > multiple Path's in ReplicationSourceManager -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-6617) ReplicationSourceManager should be able to track multiple WAL paths
[ https://issues.apache.org/jira/browse/HBASE-6617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yu Li updated HBASE-6617: - Attachment: HBASE-6617_v10.patch > ReplicationSourceManager should be able to track multiple WAL paths > --- > > Key: HBASE-6617 > URL: https://issues.apache.org/jira/browse/HBASE-6617 > Project: HBase > Issue Type: Improvement > Components: Replication >Reporter: Ted Yu >Assignee: Yu Li > Fix For: 2.0.0, 1.3.0 > > Attachments: HBASE-6617.patch, HBASE-6617_v10.patch, > HBASE-6617_v10.patch, HBASE-6617_v2.patch, HBASE-6617_v3.patch, > HBASE-6617_v4.patch, HBASE-6617_v7.patch, HBASE-6617_v9.patch > > > Currently ReplicationSourceManager uses logRolled() to receive notification > about new HLog and remembers it in latestPath. > When region server has multiple WAL support, we need to keep track of > multiple Path's in ReplicationSourceManager -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-6617) ReplicationSourceManager should be able to track multiple WAL paths
[ https://issues.apache.org/jira/browse/HBASE-6617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yu Li updated HBASE-6617: - Status: Open (was: Patch Available) Resubmit patch for HadoopQA to check > ReplicationSourceManager should be able to track multiple WAL paths > --- > > Key: HBASE-6617 > URL: https://issues.apache.org/jira/browse/HBASE-6617 > Project: HBase > Issue Type: Improvement > Components: Replication >Reporter: Ted Yu >Assignee: Yu Li > Fix For: 2.0.0, 1.3.0 > > Attachments: HBASE-6617.patch, HBASE-6617_v10.patch, > HBASE-6617_v2.patch, HBASE-6617_v3.patch, HBASE-6617_v4.patch, > HBASE-6617_v7.patch, HBASE-6617_v9.patch > > > Currently ReplicationSourceManager uses logRolled() to receive notification > about new HLog and remembers it in latestPath. > When region server has multiple WAL support, we need to keep track of > multiple Path's in ReplicationSourceManager -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-6617) ReplicationSourceManager should be able to track multiple WAL paths
[ https://issues.apache.org/jira/browse/HBASE-6617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yu Li updated HBASE-6617: - Attachment: HBASE-6617_v10.patch New patch in sync with rb, changes include: 1. Add 3 UT cases using multiwal provider in setup for the most problematic replication cases 2. Add a waiting for wal roll logic in TestReplicationEndpoint#setup > ReplicationSourceManager should be able to track multiple WAL paths > --- > > Key: HBASE-6617 > URL: https://issues.apache.org/jira/browse/HBASE-6617 > Project: HBase > Issue Type: Improvement > Components: Replication >Reporter: Ted Yu >Assignee: Yu Li > Fix For: 2.0.0, 1.3.0 > > Attachments: HBASE-6617.patch, HBASE-6617_v10.patch, > HBASE-6617_v2.patch, HBASE-6617_v3.patch, HBASE-6617_v4.patch, > HBASE-6617_v7.patch, HBASE-6617_v9.patch > > > Currently ReplicationSourceManager uses logRolled() to receive notification > about new HLog and remembers it in latestPath. > When region server has multiple WAL support, we need to keep track of > multiple Path's in ReplicationSourceManager -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-6617) ReplicationSourceManager should be able to track multiple WAL paths
[ https://issues.apache.org/jira/browse/HBASE-6617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yu Li updated HBASE-6617: - Attachment: HBASE-6617_v9.patch Update the patch in sync with rb, also addressing the javadoc warning. > ReplicationSourceManager should be able to track multiple WAL paths > --- > > Key: HBASE-6617 > URL: https://issues.apache.org/jira/browse/HBASE-6617 > Project: HBase > Issue Type: Improvement > Components: Replication >Reporter: Ted Yu >Assignee: Yu Li > Fix For: 2.0.0, 1.3.0 > > Attachments: HBASE-6617.patch, HBASE-6617_v2.patch, > HBASE-6617_v3.patch, HBASE-6617_v4.patch, HBASE-6617_v7.patch, > HBASE-6617_v9.patch > > > Currently ReplicationSourceManager uses logRolled() to receive notification > about new HLog and remembers it in latestPath. > When region server has multiple WAL support, we need to keep track of > multiple Path's in ReplicationSourceManager -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-6617) ReplicationSourceManager should be able to track multiple WAL paths
[ https://issues.apache.org/jira/browse/HBASE-6617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yu Li updated HBASE-6617: - Status: Patch Available (was: Open) > ReplicationSourceManager should be able to track multiple WAL paths > --- > > Key: HBASE-6617 > URL: https://issues.apache.org/jira/browse/HBASE-6617 > Project: HBase > Issue Type: Improvement > Components: Replication >Reporter: Ted Yu >Assignee: Yu Li > Fix For: 2.0.0, 1.3.0 > > Attachments: HBASE-6617.patch, HBASE-6617_v2.patch, > HBASE-6617_v3.patch, HBASE-6617_v4.patch, HBASE-6617_v7.patch > > > Currently ReplicationSourceManager uses logRolled() to receive notification > about new HLog and remembers it in latestPath. > When region server has multiple WAL support, we need to keep track of > multiple Path's in ReplicationSourceManager -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-6617) ReplicationSourceManager should be able to track multiple WAL paths
[ https://issues.apache.org/jira/browse/HBASE-6617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yu Li updated HBASE-6617: - Attachment: HBASE-6617_v7.patch Sync-up with [rb|https://reviews.apache.org/r/37717/] After an offline discussion with Chunhui, I agreed to take the new design. In the new design, we will launch multiple threads(in name of ReplicationSourceWorkerThread) to track multiple wal groups inside ReplicationSource, thus keeping the changes inside ReplicationSource to minimize the affect to the outer part. The uploaded v7 patch is based on the new design, and got all Replication UT cases passed with both default provider and multiwal provider. [~busbey], [~tedyu] and [~zjushch], Please help review the new patch, and let me know your thoughts. Thanks. > ReplicationSourceManager should be able to track multiple WAL paths > --- > > Key: HBASE-6617 > URL: https://issues.apache.org/jira/browse/HBASE-6617 > Project: HBase > Issue Type: Improvement > Components: Replication >Reporter: Ted Yu >Assignee: Yu Li > Fix For: 2.0.0, 1.3.0 > > Attachments: HBASE-6617.patch, HBASE-6617_v2.patch, > HBASE-6617_v3.patch, HBASE-6617_v4.patch, HBASE-6617_v7.patch > > > Currently ReplicationSourceManager uses logRolled() to receive notification > about new HLog and remembers it in latestPath. > When region server has multiple WAL support, we need to keep track of > multiple Path's in ReplicationSourceManager -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-6617) ReplicationSourceManager should be able to track multiple WAL paths
[ https://issues.apache.org/jira/browse/HBASE-6617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-6617: -- Status: Open (was: Patch Available) > ReplicationSourceManager should be able to track multiple WAL paths > --- > > Key: HBASE-6617 > URL: https://issues.apache.org/jira/browse/HBASE-6617 > Project: HBase > Issue Type: Improvement > Components: Replication >Reporter: Ted Yu >Assignee: Yu Li > Fix For: 2.0.0, 1.3.0 > > Attachments: HBASE-6617.patch, HBASE-6617_v2.patch, > HBASE-6617_v3.patch, HBASE-6617_v4.patch > > > Currently ReplicationSourceManager uses logRolled() to receive notification > about new HLog and remembers it in latestPath. > When region server has multiple WAL support, we need to keep track of > multiple Path's in ReplicationSourceManager -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-6617) ReplicationSourceManager should be able to track multiple WAL paths
[ https://issues.apache.org/jira/browse/HBASE-6617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yu Li updated HBASE-6617: - Attachment: HBASE-6617_v4.patch Upload v4 patch to sync-up with rb > ReplicationSourceManager should be able to track multiple WAL paths > --- > > Key: HBASE-6617 > URL: https://issues.apache.org/jira/browse/HBASE-6617 > Project: HBase > Issue Type: Improvement > Components: Replication >Reporter: Ted Yu >Assignee: Yu Li > Fix For: 2.0.0, 1.3.0 > > Attachments: HBASE-6617.patch, HBASE-6617_v2.patch, > HBASE-6617_v3.patch, HBASE-6617_v4.patch > > > Currently ReplicationSourceManager uses logRolled() to receive notification > about new HLog and remembers it in latestPath. > When region server has multiple WAL support, we need to keep track of > multiple Path's in ReplicationSourceManager -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-6617) ReplicationSourceManager should be able to track multiple WAL paths
[ https://issues.apache.org/jira/browse/HBASE-6617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yu Li updated HBASE-6617: - Attachment: HBASE-6617_v3.patch More changes according to review comments on RB and fix javadoc/line-wrap warnings reported by HadoopQA Also checked the UT failures/zombies reported by HadoopQA and should be irrelevant to the changes here. Will see how HadoopQA says with the new patch > ReplicationSourceManager should be able to track multiple WAL paths > --- > > Key: HBASE-6617 > URL: https://issues.apache.org/jira/browse/HBASE-6617 > Project: HBase > Issue Type: Improvement > Components: Replication >Reporter: Ted Yu >Assignee: Yu Li > Fix For: 2.0.0, 1.3.0 > > Attachments: HBASE-6617.patch, HBASE-6617_v2.patch, > HBASE-6617_v3.patch > > > Currently ReplicationSourceManager uses logRolled() to receive notification > about new HLog and remembers it in latestPath. > When region server has multiple WAL support, we need to keep track of > multiple Path's in ReplicationSourceManager -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-6617) ReplicationSourceManager should be able to track multiple WAL paths
[ https://issues.apache.org/jira/browse/HBASE-6617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-6617: -- Assignee: Yu Li (was: Sean Busbey) Status: Patch Available (was: Open) > ReplicationSourceManager should be able to track multiple WAL paths > --- > > Key: HBASE-6617 > URL: https://issues.apache.org/jira/browse/HBASE-6617 > Project: HBase > Issue Type: Improvement > Components: Replication >Reporter: Ted Yu >Assignee: Yu Li > Fix For: 2.0.0, 1.3.0 > > Attachments: HBASE-6617.patch, HBASE-6617_v2.patch > > > Currently ReplicationSourceManager uses logRolled() to receive notification > about new HLog and remembers it in latestPath. > When region server has multiple WAL support, we need to keep track of > multiple Path's in ReplicationSourceManager -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-6617) ReplicationSourceManager should be able to track multiple WAL paths
[ https://issues.apache.org/jira/browse/HBASE-6617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yu Li updated HBASE-6617: - Attachment: HBASE-6617_v2.patch Update the patch (sync-up with RB), solved NPE in TestBoundedRegionGroupingProvider and done some variable name refactor > ReplicationSourceManager should be able to track multiple WAL paths > --- > > Key: HBASE-6617 > URL: https://issues.apache.org/jira/browse/HBASE-6617 > Project: HBase > Issue Type: Improvement > Components: Replication >Reporter: Ted Yu >Assignee: Sean Busbey > Fix For: 2.0.0, 1.3.0 > > Attachments: HBASE-6617.patch, HBASE-6617_v2.patch > > > Currently ReplicationSourceManager uses logRolled() to receive notification > about new HLog and remembers it in latestPath. > When region server has multiple WAL support, we need to keep track of > multiple Path's in ReplicationSourceManager -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-6617) ReplicationSourceManager should be able to track multiple WAL paths
[ https://issues.apache.org/jira/browse/HBASE-6617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yu Li updated HBASE-6617: - Attachment: HBASE-6617.patch > ReplicationSourceManager should be able to track multiple WAL paths > --- > > Key: HBASE-6617 > URL: https://issues.apache.org/jira/browse/HBASE-6617 > Project: HBase > Issue Type: Improvement > Components: Replication >Reporter: Ted Yu >Assignee: Sean Busbey > Fix For: 2.0.0, 1.3.0 > > Attachments: HBASE-6617.patch > > > Currently ReplicationSourceManager uses logRolled() to receive notification > about new HLog and remembers it in latestPath. > When region server has multiple WAL support, we need to keep track of > multiple Path's in ReplicationSourceManager -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-6617) ReplicationSourceManager should be able to track multiple WAL paths
[ https://issues.apache.org/jira/browse/HBASE-6617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-6617: --- Fix Version/s: (was: 1.2.0) 1.3.0 > ReplicationSourceManager should be able to track multiple WAL paths > --- > > Key: HBASE-6617 > URL: https://issues.apache.org/jira/browse/HBASE-6617 > Project: HBase > Issue Type: Improvement > Components: Replication >Reporter: Ted Yu >Assignee: Sean Busbey > Fix For: 2.0.0, 1.3.0 > > > Currently ReplicationSourceManager uses logRolled() to receive notification > about new HLog and remembers it in latestPath. > When region server has multiple WAL support, we need to keep track of > multiple Path's in ReplicationSourceManager -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-6617) ReplicationSourceManager should be able to track multiple WAL paths
[ https://issues.apache.org/jira/browse/HBASE-6617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk updated HBASE-6617: Fix Version/s: (was: 1.1.0) 1.2.0 > ReplicationSourceManager should be able to track multiple WAL paths > --- > > Key: HBASE-6617 > URL: https://issues.apache.org/jira/browse/HBASE-6617 > Project: HBase > Issue Type: Improvement > Components: Replication >Reporter: Ted Yu >Assignee: Sean Busbey > Fix For: 2.0.0, 1.2.0 > > > Currently ReplicationSourceManager uses logRolled() to receive notification > about new HLog and remembers it in latestPath. > When region server has multiple WAL support, we need to keep track of > multiple Path's in ReplicationSourceManager -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-6617) ReplicationSourceManager should be able to track multiple WAL paths
[
https://issues.apache.org/jira/browse/HBASE-6617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Busbey updated HBASE-6617:
---
Fix Version/s: 1.1.0
2.0.0
Copied from my follow up on HBASE-5699's rb
{quote}
there's an optimization of sorts still outstanding with HBASE-6617.
Essentially, the replication code keeps around the last wal it saw so that when
a new replication target comes online there's something to give it right away.
HBASE-6617 would make it keep one-per-provider so that in the case of multiwal
you don't just have 1/n of the entries you would have had with a single wal.
As is, if someone enables multiwal prior to HBASE-6617 getting finished they'll
have fewer recent entries replayed as a new source comes online, but things
will still function as expected from that point on.
{quote}
> ReplicationSourceManager should be able to track multiple WAL paths
> ---
>
> Key: HBASE-6617
> URL: https://issues.apache.org/jira/browse/HBASE-6617
> Project: HBase
> Issue Type: Improvement
> Components: Replication
>Reporter: Ted Yu
>Assignee: Sean Busbey
> Fix For: 2.0.0, 1.1.0
>
>
> Currently ReplicationSourceManager uses logRolled() to receive notification
> about new HLog and remembers it in latestPath.
> When region server has multiple WAL support, we need to keep track of
> multiple Path's in ReplicationSourceManager
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-6617) ReplicationSourceManager should be able to track multiple WAL paths
[ https://issues.apache.org/jira/browse/HBASE-6617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-6617: --- Issue Type: Improvement (was: Sub-task) Parent: (was: HBASE-5699) > ReplicationSourceManager should be able to track multiple WAL paths > --- > > Key: HBASE-6617 > URL: https://issues.apache.org/jira/browse/HBASE-6617 > Project: HBase > Issue Type: Improvement > Components: Replication >Reporter: Ted Yu >Assignee: Sean Busbey > > Currently ReplicationSourceManager uses logRolled() to receive notification > about new HLog and remembers it in latestPath. > When region server has multiple WAL support, we need to keep track of > multiple Path's in ReplicationSourceManager -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-6617) ReplicationSourceManager should be able to track multiple WAL paths
[ https://issues.apache.org/jira/browse/HBASE-6617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-6617: --- Component/s: Replication > ReplicationSourceManager should be able to track multiple WAL paths > --- > > Key: HBASE-6617 > URL: https://issues.apache.org/jira/browse/HBASE-6617 > Project: HBase > Issue Type: Sub-task > Components: Replication >Reporter: Ted Yu > > Currently ReplicationSourceManager uses logRolled() to receive notification > about new HLog and remembers it in latestPath. > When region server has multiple WAL support, we need to keep track of > multiple Path's in ReplicationSourceManager -- This message was sent by Atlassian JIRA (v6.2#6252)
