[jira] [Commented] (HBASE-26233) The region replication framework should not be built upon the general replication framework
[ https://issues.apache.org/jira/browse/HBASE-26233?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17824044#comment-17824044 ] Bryan Beaudreault commented on HBASE-26233: --- I guess we never picked up the backport of this? > The region replication framework should not be built upon the general > replication framework > --- > > Key: HBASE-26233 > URL: https://issues.apache.org/jira/browse/HBASE-26233 > Project: HBase > Issue Type: Umbrella > Components: read replicas >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 3.0.0-alpha-3 > > > At least, at the source path, where we track the edits, we should not make > region replication rely on general replication framework. > The difficulty here for switching to a table based storage is that, the WAL > system and replication system highly depend on each other. There will be > cyclic dependency if we want to store replication peer and queue data in a > hbase table. > And after HBASE-18070, even meta wal provider will be integrated together > with replication system, which makes things more difficult. > But in general, for region replication, it is not a big deal to lose some > edits, a flush can fix everything, which means we do not so heavy tracking > system in the general replication system. > We should find a more light-weighted way to do region replication. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (HBASE-26233) The region replication framework should not be built upon the general replication framework
[
https://issues.apache.org/jira/browse/HBASE-26233?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17536279#comment-17536279
]
Huaxiang Sun commented on HBASE-26233:
--
Sorry, [~zhangduo], catching up the topic so late! I am starting to review the
feature now.
{quote}And when discussing around HBASE-18070, I recall that we talked about
only replicate the 'info' family. So have we already done this, i.e, only
replicate 'info' family but not other families? Reading the section in ref guide
[http://hbase.apache.org/book.html#async.wal.replication.meta]
I haven't seen any related topics. So my question is do we still need to
implement this feature?
{quote}
Yeah, since it only needs the info family for region locations, it really does
not need to replicate other families. I can create Jira and implement it during
the reviewing process if it has not been in the place, thanks.
> The region replication framework should not be built upon the general
> replication framework
> ---
>
> Key: HBASE-26233
> URL: https://issues.apache.org/jira/browse/HBASE-26233
> Project: HBase
> Issue Type: Umbrella
> Components: read replicas
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Fix For: 3.0.0-alpha-3
>
>
> At least, at the source path, where we track the edits, we should not make
> region replication rely on general replication framework.
> The difficulty here for switching to a table based storage is that, the WAL
> system and replication system highly depend on each other. There will be
> cyclic dependency if we want to store replication peer and queue data in a
> hbase table.
> And after HBASE-18070, even meta wal provider will be integrated together
> with replication system, which makes things more difficult.
> But in general, for region replication, it is not a big deal to lose some
> edits, a flush can fix everything, which means we do not so heavy tracking
> system in the general replication system.
> We should find a more light-weighted way to do region replication.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Commented] (HBASE-26233) The region replication framework should not be built upon the general replication framework
[
https://issues.apache.org/jira/browse/HBASE-26233?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17468969#comment-17468969
]
Duo Zhang commented on HBASE-26233:
---
{quote}
Can we start a discussion on backporting it to branch-2?
{quote}
Of course.
And on the techinal part, the main problem is we highly rely on the
AsyncClusterConnection in the implementation but we do not have it on branch-2,
if we want to backport it then we need to implement the logic in
ClusterConnection. I will not say it is impossible, but it is a bit strange...
> The region replication framework should not be built upon the general
> replication framework
> ---
>
> Key: HBASE-26233
> URL: https://issues.apache.org/jira/browse/HBASE-26233
> Project: HBase
> Issue Type: Umbrella
> Components: read replicas
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Fix For: 3.0.0-alpha-3
>
>
> At least, at the source path, where we track the edits, we should not make
> region replication rely on general replication framework.
> The difficulty here for switching to a table based storage is that, the WAL
> system and replication system highly depend on each other. There will be
> cyclic dependency if we want to store replication peer and queue data in a
> hbase table.
> And after HBASE-18070, even meta wal provider will be integrated together
> with replication system, which makes things more difficult.
> But in general, for region replication, it is not a big deal to lose some
> edits, a flush can fix everything, which means we do not so heavy tracking
> system in the general replication system.
> We should find a more light-weighted way to do region replication.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (HBASE-26233) The region replication framework should not be built upon the general replication framework
[ https://issues.apache.org/jira/browse/HBASE-26233?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17468916#comment-17468916 ] Nick Dimiduk commented on HBASE-26233: -- This is great [~zhangduo] , congratulations! Can we start a discussion on backporting it to branch-2? :D > The region replication framework should not be built upon the general > replication framework > --- > > Key: HBASE-26233 > URL: https://issues.apache.org/jira/browse/HBASE-26233 > Project: HBase > Issue Type: Umbrella > Components: read replicas >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 3.0.0-alpha-3 > > > At least, at the source path, where we track the edits, we should not make > region replication rely on general replication framework. > The difficulty here for switching to a table based storage is that, the WAL > system and replication system highly depend on each other. There will be > cyclic dependency if we want to store replication peer and queue data in a > hbase table. > And after HBASE-18070, even meta wal provider will be integrated together > with replication system, which makes things more difficult. > But in general, for region replication, it is not a big deal to lose some > edits, a flush can fix everything, which means we do not so heavy tracking > system in the general replication system. > We should find a more light-weighted way to do region replication. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (HBASE-26233) The region replication framework should not be built upon the general replication framework
[
https://issues.apache.org/jira/browse/HBASE-26233?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17467260#comment-17467260
]
Hudson commented on HBASE-26233:
Results for branch HBASE-26233
[build #15 on
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/HBASE-26233/15/]:
(/) *{color:green}+1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/HBASE-26233/15/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/HBASE-26233/15/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/HBASE-26233/15/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> The region replication framework should not be built upon the general
> replication framework
> ---
>
> Key: HBASE-26233
> URL: https://issues.apache.org/jira/browse/HBASE-26233
> Project: HBase
> Issue Type: Umbrella
> Components: read replicas
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
>
> At least, at the source path, where we track the edits, we should not make
> region replication rely on general replication framework.
> The difficulty here for switching to a table based storage is that, the WAL
> system and replication system highly depend on each other. There will be
> cyclic dependency if we want to store replication peer and queue data in a
> hbase table.
> And after HBASE-18070, even meta wal provider will be integrated together
> with replication system, which makes things more difficult.
> But in general, for region replication, it is not a big deal to lose some
> edits, a flush can fix everything, which means we do not so heavy tracking
> system in the general replication system.
> We should find a more light-weighted way to do region replication.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (HBASE-26233) The region replication framework should not be built upon the general replication framework
[
https://issues.apache.org/jira/browse/HBASE-26233?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17466037#comment-17466037
]
Hudson commented on HBASE-26233:
Results for branch HBASE-26233
[build #14 on
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/HBASE-26233/14/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/HBASE-26233/14/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/HBASE-26233/14/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(x) {color:red}-1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/HBASE-26233/14/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> The region replication framework should not be built upon the general
> replication framework
> ---
>
> Key: HBASE-26233
> URL: https://issues.apache.org/jira/browse/HBASE-26233
> Project: HBase
> Issue Type: Umbrella
> Components: read replicas
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
>
> At least, at the source path, where we track the edits, we should not make
> region replication rely on general replication framework.
> The difficulty here for switching to a table based storage is that, the WAL
> system and replication system highly depend on each other. There will be
> cyclic dependency if we want to store replication peer and queue data in a
> hbase table.
> And after HBASE-18070, even meta wal provider will be integrated together
> with replication system, which makes things more difficult.
> But in general, for region replication, it is not a big deal to lose some
> edits, a flush can fix everything, which means we do not so heavy tracking
> system in the general replication system.
> We should find a more light-weighted way to do region replication.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (HBASE-26233) The region replication framework should not be built upon the general replication framework
[
https://issues.apache.org/jira/browse/HBASE-26233?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17465079#comment-17465079
]
Hudson commented on HBASE-26233:
Results for branch HBASE-26233
[build #13 on
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/HBASE-26233/13/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/HBASE-26233/13/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/HBASE-26233/13/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/HBASE-26233/13/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> The region replication framework should not be built upon the general
> replication framework
> ---
>
> Key: HBASE-26233
> URL: https://issues.apache.org/jira/browse/HBASE-26233
> Project: HBase
> Issue Type: Umbrella
> Components: read replicas
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
>
> At least, at the source path, where we track the edits, we should not make
> region replication rely on general replication framework.
> The difficulty here for switching to a table based storage is that, the WAL
> system and replication system highly depend on each other. There will be
> cyclic dependency if we want to store replication peer and queue data in a
> hbase table.
> And after HBASE-18070, even meta wal provider will be integrated together
> with replication system, which makes things more difficult.
> But in general, for region replication, it is not a big deal to lose some
> edits, a flush can fix everything, which means we do not so heavy tracking
> system in the general replication system.
> We should find a more light-weighted way to do region replication.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (HBASE-26233) The region replication framework should not be built upon the general replication framework
[
https://issues.apache.org/jira/browse/HBASE-26233?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17464961#comment-17464961
]
Hudson commented on HBASE-26233:
Results for branch HBASE-26233
[build #12 on
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/HBASE-26233/12/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/HBASE-26233/12/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/HBASE-26233/12/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/HBASE-26233/12/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> The region replication framework should not be built upon the general
> replication framework
> ---
>
> Key: HBASE-26233
> URL: https://issues.apache.org/jira/browse/HBASE-26233
> Project: HBase
> Issue Type: Umbrella
> Components: read replicas
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
>
> At least, at the source path, where we track the edits, we should not make
> region replication rely on general replication framework.
> The difficulty here for switching to a table based storage is that, the WAL
> system and replication system highly depend on each other. There will be
> cyclic dependency if we want to store replication peer and queue data in a
> hbase table.
> And after HBASE-18070, even meta wal provider will be integrated together
> with replication system, which makes things more difficult.
> But in general, for region replication, it is not a big deal to lose some
> edits, a flush can fix everything, which means we do not so heavy tracking
> system in the general replication system.
> We should find a more light-weighted way to do region replication.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (HBASE-26233) The region replication framework should not be built upon the general replication framework
[
https://issues.apache.org/jira/browse/HBASE-26233?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17462334#comment-17462334
]
Hudson commented on HBASE-26233:
Results for branch HBASE-26233
[build #11 on
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/HBASE-26233/11/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/HBASE-26233/11/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/HBASE-26233/11/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(x) {color:red}-1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/HBASE-26233/11/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> The region replication framework should not be built upon the general
> replication framework
> ---
>
> Key: HBASE-26233
> URL: https://issues.apache.org/jira/browse/HBASE-26233
> Project: HBase
> Issue Type: Umbrella
> Components: read replicas
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
>
> At least, at the source path, where we track the edits, we should not make
> region replication rely on general replication framework.
> The difficulty here for switching to a table based storage is that, the WAL
> system and replication system highly depend on each other. There will be
> cyclic dependency if we want to store replication peer and queue data in a
> hbase table.
> And after HBASE-18070, even meta wal provider will be integrated together
> with replication system, which makes things more difficult.
> But in general, for region replication, it is not a big deal to lose some
> edits, a flush can fix everything, which means we do not so heavy tracking
> system in the general replication system.
> We should find a more light-weighted way to do region replication.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (HBASE-26233) The region replication framework should not be built upon the general replication framework
[
https://issues.apache.org/jira/browse/HBASE-26233?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17462115#comment-17462115
]
Hudson commented on HBASE-26233:
Results for branch HBASE-26233
[build #10 on
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/HBASE-26233/10/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/HBASE-26233/10/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/HBASE-26233/10/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(x) {color:red}-1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/HBASE-26233/10/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> The region replication framework should not be built upon the general
> replication framework
> ---
>
> Key: HBASE-26233
> URL: https://issues.apache.org/jira/browse/HBASE-26233
> Project: HBase
> Issue Type: Umbrella
> Components: read replicas
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
>
> At least, at the source path, where we track the edits, we should not make
> region replication rely on general replication framework.
> The difficulty here for switching to a table based storage is that, the WAL
> system and replication system highly depend on each other. There will be
> cyclic dependency if we want to store replication peer and queue data in a
> hbase table.
> And after HBASE-18070, even meta wal provider will be integrated together
> with replication system, which makes things more difficult.
> But in general, for region replication, it is not a big deal to lose some
> edits, a flush can fix everything, which means we do not so heavy tracking
> system in the general replication system.
> We should find a more light-weighted way to do region replication.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (HBASE-26233) The region replication framework should not be built upon the general replication framework
[
https://issues.apache.org/jira/browse/HBASE-26233?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17461824#comment-17461824
]
Hudson commented on HBASE-26233:
Results for branch HBASE-26233
[build #9 on
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/HBASE-26233/9/]:
(/) *{color:green}+1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/HBASE-26233/9/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/HBASE-26233/9/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/HBASE-26233/9/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> The region replication framework should not be built upon the general
> replication framework
> ---
>
> Key: HBASE-26233
> URL: https://issues.apache.org/jira/browse/HBASE-26233
> Project: HBase
> Issue Type: Umbrella
> Components: read replicas
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
>
> At least, at the source path, where we track the edits, we should not make
> region replication rely on general replication framework.
> The difficulty here for switching to a table based storage is that, the WAL
> system and replication system highly depend on each other. There will be
> cyclic dependency if we want to store replication peer and queue data in a
> hbase table.
> And after HBASE-18070, even meta wal provider will be integrated together
> with replication system, which makes things more difficult.
> But in general, for region replication, it is not a big deal to lose some
> edits, a flush can fix everything, which means we do not so heavy tracking
> system in the general replication system.
> We should find a more light-weighted way to do region replication.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (HBASE-26233) The region replication framework should not be built upon the general replication framework
[
https://issues.apache.org/jira/browse/HBASE-26233?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17459786#comment-17459786
]
Hudson commented on HBASE-26233:
Results for branch HBASE-26233
[build #8 on
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/HBASE-26233/8/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/HBASE-26233/8/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/HBASE-26233/8/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(x) {color:red}-1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/HBASE-26233/8/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> The region replication framework should not be built upon the general
> replication framework
> ---
>
> Key: HBASE-26233
> URL: https://issues.apache.org/jira/browse/HBASE-26233
> Project: HBase
> Issue Type: Umbrella
> Components: read replicas
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
>
> At least, at the source path, where we track the edits, we should not make
> region replication rely on general replication framework.
> The difficulty here for switching to a table based storage is that, the WAL
> system and replication system highly depend on each other. There will be
> cyclic dependency if we want to store replication peer and queue data in a
> hbase table.
> And after HBASE-18070, even meta wal provider will be integrated together
> with replication system, which makes things more difficult.
> But in general, for region replication, it is not a big deal to lose some
> edits, a flush can fix everything, which means we do not so heavy tracking
> system in the general replication system.
> We should find a more light-weighted way to do region replication.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (HBASE-26233) The region replication framework should not be built upon the general replication framework
[
https://issues.apache.org/jira/browse/HBASE-26233?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17458031#comment-17458031
]
Hudson commented on HBASE-26233:
Results for branch HBASE-26233
[build #7 on
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/HBASE-26233/7/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/HBASE-26233/7/General_20Nightly_20Build_20Report/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/HBASE-26233/7/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(x) {color:red}-1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/HBASE-26233/7/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> The region replication framework should not be built upon the general
> replication framework
> ---
>
> Key: HBASE-26233
> URL: https://issues.apache.org/jira/browse/HBASE-26233
> Project: HBase
> Issue Type: Umbrella
> Components: read replicas
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
>
> At least, at the source path, where we track the edits, we should not make
> region replication rely on general replication framework.
> The difficulty here for switching to a table based storage is that, the WAL
> system and replication system highly depend on each other. There will be
> cyclic dependency if we want to store replication peer and queue data in a
> hbase table.
> And after HBASE-18070, even meta wal provider will be integrated together
> with replication system, which makes things more difficult.
> But in general, for region replication, it is not a big deal to lose some
> edits, a flush can fix everything, which means we do not so heavy tracking
> system in the general replication system.
> We should find a more light-weighted way to do region replication.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (HBASE-26233) The region replication framework should not be built upon the general replication framework
[
https://issues.apache.org/jira/browse/HBASE-26233?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17454502#comment-17454502
]
Hudson commented on HBASE-26233:
Results for branch HBASE-26233
[build #6 on
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/HBASE-26233/6/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/HBASE-26233/6/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/HBASE-26233/6/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/HBASE-26233/6/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> The region replication framework should not be built upon the general
> replication framework
> ---
>
> Key: HBASE-26233
> URL: https://issues.apache.org/jira/browse/HBASE-26233
> Project: HBase
> Issue Type: Umbrella
> Components: read replicas
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
>
> At least, at the source path, where we track the edits, we should not make
> region replication rely on general replication framework.
> The difficulty here for switching to a table based storage is that, the WAL
> system and replication system highly depend on each other. There will be
> cyclic dependency if we want to store replication peer and queue data in a
> hbase table.
> And after HBASE-18070, even meta wal provider will be integrated together
> with replication system, which makes things more difficult.
> But in general, for region replication, it is not a big deal to lose some
> edits, a flush can fix everything, which means we do not so heavy tracking
> system in the general replication system.
> We should find a more light-weighted way to do region replication.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (HBASE-26233) The region replication framework should not be built upon the general replication framework
[
https://issues.apache.org/jira/browse/HBASE-26233?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17453700#comment-17453700
]
Hudson commented on HBASE-26233:
Results for branch HBASE-26233
[build #5 on
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/HBASE-26233/5/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/HBASE-26233/5/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/HBASE-26233/5/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/HBASE-26233/5/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(x) {color:red}-1 source release artifact{color}
-- See build output for details.
(x) {color:red}-1 client integration test{color}
-- Something went wrong with this stage, [check relevant console
output|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/HBASE-26233/5//console].
> The region replication framework should not be built upon the general
> replication framework
> ---
>
> Key: HBASE-26233
> URL: https://issues.apache.org/jira/browse/HBASE-26233
> Project: HBase
> Issue Type: Umbrella
> Components: read replicas
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
>
> At least, at the source path, where we track the edits, we should not make
> region replication rely on general replication framework.
> The difficulty here for switching to a table based storage is that, the WAL
> system and replication system highly depend on each other. There will be
> cyclic dependency if we want to store replication peer and queue data in a
> hbase table.
> And after HBASE-18070, even meta wal provider will be integrated together
> with replication system, which makes things more difficult.
> But in general, for region replication, it is not a big deal to lose some
> edits, a flush can fix everything, which means we do not so heavy tracking
> system in the general replication system.
> We should find a more light-weighted way to do region replication.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (HBASE-26233) The region replication framework should not be built upon the general replication framework
[
https://issues.apache.org/jira/browse/HBASE-26233?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17453589#comment-17453589
]
Hudson commented on HBASE-26233:
Results for branch HBASE-26233
[build #4 on
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/HBASE-26233/4/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/HBASE-26233/4/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/HBASE-26233/4/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(x) {color:red}-1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/HBASE-26233/4/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> The region replication framework should not be built upon the general
> replication framework
> ---
>
> Key: HBASE-26233
> URL: https://issues.apache.org/jira/browse/HBASE-26233
> Project: HBase
> Issue Type: Umbrella
> Components: read replicas
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
>
> At least, at the source path, where we track the edits, we should not make
> region replication rely on general replication framework.
> The difficulty here for switching to a table based storage is that, the WAL
> system and replication system highly depend on each other. There will be
> cyclic dependency if we want to store replication peer and queue data in a
> hbase table.
> And after HBASE-18070, even meta wal provider will be integrated together
> with replication system, which makes things more difficult.
> But in general, for region replication, it is not a big deal to lose some
> edits, a flush can fix everything, which means we do not so heavy tracking
> system in the general replication system.
> We should find a more light-weighted way to do region replication.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (HBASE-26233) The region replication framework should not be built upon the general replication framework
[
https://issues.apache.org/jira/browse/HBASE-26233?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17453339#comment-17453339
]
Hudson commented on HBASE-26233:
Results for branch HBASE-26233
[build #3 on
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/HBASE-26233/3/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/HBASE-26233/3/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/HBASE-26233/3/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(x) {color:red}-1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/HBASE-26233/3/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> The region replication framework should not be built upon the general
> replication framework
> ---
>
> Key: HBASE-26233
> URL: https://issues.apache.org/jira/browse/HBASE-26233
> Project: HBase
> Issue Type: Umbrella
> Components: read replicas
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
>
> At least, at the source path, where we track the edits, we should not make
> region replication rely on general replication framework.
> The difficulty here for switching to a table based storage is that, the WAL
> system and replication system highly depend on each other. There will be
> cyclic dependency if we want to store replication peer and queue data in a
> hbase table.
> And after HBASE-18070, even meta wal provider will be integrated together
> with replication system, which makes things more difficult.
> But in general, for region replication, it is not a big deal to lose some
> edits, a flush can fix everything, which means we do not so heavy tracking
> system in the general replication system.
> We should find a more light-weighted way to do region replication.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (HBASE-26233) The region replication framework should not be built upon the general replication framework
[
https://issues.apache.org/jira/browse/HBASE-26233?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17449950#comment-17449950
]
Hudson commented on HBASE-26233:
Results for branch HBASE-26233
[build #2 on
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/HBASE-26233/2/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/HBASE-26233/2/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/HBASE-26233/2/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(x) {color:red}-1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/HBASE-26233/2/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> The region replication framework should not be built upon the general
> replication framework
> ---
>
> Key: HBASE-26233
> URL: https://issues.apache.org/jira/browse/HBASE-26233
> Project: HBase
> Issue Type: Umbrella
> Components: read replicas
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
>
> At least, at the source path, where we track the edits, we should not make
> region replication rely on general replication framework.
> The difficulty here for switching to a table based storage is that, the WAL
> system and replication system highly depend on each other. There will be
> cyclic dependency if we want to store replication peer and queue data in a
> hbase table.
> And after HBASE-18070, even meta wal provider will be integrated together
> with replication system, which makes things more difficult.
> But in general, for region replication, it is not a big deal to lose some
> edits, a flush can fix everything, which means we do not so heavy tracking
> system in the general replication system.
> We should find a more light-weighted way to do region replication.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (HBASE-26233) The region replication framework should not be built upon the general replication framework
[
https://issues.apache.org/jira/browse/HBASE-26233?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17449851#comment-17449851
]
Hudson commented on HBASE-26233:
Results for branch HBASE-26233
[build #1 on
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/HBASE-26233/1/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/HBASE-26233/1/General_20Nightly_20Build_20Report/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/HBASE-26233/1/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(x) {color:red}-1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/HBASE-26233/1/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> The region replication framework should not be built upon the general
> replication framework
> ---
>
> Key: HBASE-26233
> URL: https://issues.apache.org/jira/browse/HBASE-26233
> Project: HBase
> Issue Type: Umbrella
> Components: read replicas
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
>
> At least, at the source path, where we track the edits, we should not make
> region replication rely on general replication framework.
> The difficulty here for switching to a table based storage is that, the WAL
> system and replication system highly depend on each other. There will be
> cyclic dependency if we want to store replication peer and queue data in a
> hbase table.
> And after HBASE-18070, even meta wal provider will be integrated together
> with replication system, which makes things more difficult.
> But in general, for region replication, it is not a big deal to lose some
> edits, a flush can fix everything, which means we do not so heavy tracking
> system in the general replication system.
> We should find a more light-weighted way to do region replication.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (HBASE-26233) The region replication framework should not be built upon the general replication framework
[ https://issues.apache.org/jira/browse/HBASE-26233?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17449378#comment-17449378 ] Duo Zhang commented on HBASE-26233: --- For enabling the feature by default on master, since I plan to also port this feature to branch-2, we'd better keep it off by default, so the backport does not need to change this flag. And then we open another issue to discuss whether to enable it by default on master. Thanks. > The region replication framework should not be built upon the general > replication framework > --- > > Key: HBASE-26233 > URL: https://issues.apache.org/jira/browse/HBASE-26233 > Project: HBase > Issue Type: Umbrella > Components: read replicas >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > > At least, at the source path, where we track the edits, we should not make > region replication rely on general replication framework. > The difficulty here for switching to a table based storage is that, the WAL > system and replication system highly depend on each other. There will be > cyclic dependency if we want to store replication peer and queue data in a > hbase table. > And after HBASE-18070, even meta wal provider will be integrated together > with replication system, which makes things more difficult. > But in general, for region replication, it is not a big deal to lose some > edits, a flush can fix everything, which means we do not so heavy tracking > system in the general replication system. > We should find a more light-weighted way to do region replication. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (HBASE-26233) The region replication framework should not be built upon the general replication framework
[ https://issues.apache.org/jira/browse/HBASE-26233?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17443332#comment-17443332 ] Duo Zhang commented on HBASE-26233: --- [~huaxiangsun] This is a redo of the region replication framework at server side, for decoupling the replication framework and wal framework, so we can continue the work for HBASE-15867. And when discussing around HBASE-18070, I recall that we talked about only replicate the 'info' family. So have we already done this, i.e, only replicate 'info' family but not other families? Reading the section in ref guide http://hbase.apache.org/book.html#async.wal.replication.meta I haven't seen any related topics. So my question is do we still need to implement this feature? Thanks. > The region replication framework should not be built upon the general > replication framework > --- > > Key: HBASE-26233 > URL: https://issues.apache.org/jira/browse/HBASE-26233 > Project: HBase > Issue Type: Umbrella > Components: read replicas >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > > At least, at the source path, where we track the edits, we should not make > region replication rely on general replication framework. > The difficulty here for switching to a table based storage is that, the WAL > system and replication system highly depend on each other. There will be > cyclic dependency if we want to store replication peer and queue data in a > hbase table. > And after HBASE-18070, even meta wal provider will be integrated together > with replication system, which makes things more difficult. > But in general, for region replication, it is not a big deal to lose some > edits, a flush can fix everything, which means we do not so heavy tracking > system in the general replication system. > We should find a more light-weighted way to do region replication. -- This message was sent by Atlassian Jira (v8.20.1#820001)
