[jira] [Commented] (HBASE-24666) Offload the replication source/sink job to independent Replication Server
[
https://issues.apache.org/jira/browse/HBASE-24666?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17151929#comment-17151929
]
Guanghao Zhang commented on HBASE-24666:
{quote}you don execute any CP hooks as part of compaction? May be you don have
such a need ?
{quote}
Yes. For internal usage, we didn't consider this problem at all..
> Offload the replication source/sink job to independent Replication Server
> -
>
> Key: HBASE-24666
> URL: https://issues.apache.org/jira/browse/HBASE-24666
> Project: HBase
> Issue Type: Umbrella
>Reporter: Guanghao Zhang
>Priority: Major
>
> The basic idea is add a role "ReplicationServer" to take the replication
> source/sink job. HMaster is responsible for scheduling the replication job to
> different ReplicationServer.
> [link Design
> doc|https://docs.google.com/document/d/16kRPVGctFSf__nC3yaVZmAm3GTxIbHefekKC_rMmTw8/edit?usp=sharing]
> Suggestions are welcomed. Thanks.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-24666) Offload the replication source/sink job to independent Replication Server
[ https://issues.apache.org/jira/browse/HBASE-24666?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17151809#comment-17151809 ] ramkrishna.s.vasudevan commented on HBASE-24666: As per your doc -the external compaction based process (currently MR based) you don execute any CP hooks as part of compaction? May be you don have such a need ? > Offload the replication source/sink job to independent Replication Server > - > > Key: HBASE-24666 > URL: https://issues.apache.org/jira/browse/HBASE-24666 > Project: HBase > Issue Type: Umbrella >Reporter: Guanghao Zhang >Priority: Major > > The basic idea is add a role "ReplicationServer" to take the replication > source/sink job. HMaster is responsible for scheduling the replication job to > different ReplicationServer. > [link Design > doc|https://docs.google.com/document/d/16kRPVGctFSf__nC3yaVZmAm3GTxIbHefekKC_rMmTw8/edit?usp=sharing] > Suggestions are welcomed. Thanks. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-24666) Offload the replication source/sink job to independent Replication Server
[
https://issues.apache.org/jira/browse/HBASE-24666?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17151793#comment-17151793
]
Guanghao Zhang commented on HBASE-24666:
{quote} I found that the CP hooks is becoming a blocker in moving things out of
RS.
{quote}
Yes. CP is a blocker if it need access RS during Replication/Compaction. Need
to check when the CP was called.
> Offload the replication source/sink job to independent Replication Server
> -
>
> Key: HBASE-24666
> URL: https://issues.apache.org/jira/browse/HBASE-24666
> Project: HBase
> Issue Type: Umbrella
>Reporter: Guanghao Zhang
>Priority: Major
>
> The basic idea is add a role "ReplicationServer" to take the replication
> source/sink job. HMaster is responsible for scheduling the replication job to
> different ReplicationServer.
> [link Design
> doc|https://docs.google.com/document/d/16kRPVGctFSf__nC3yaVZmAm3GTxIbHefekKC_rMmTw8/edit?usp=sharing]
> Suggestions are welcomed. Thanks.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-24666) Offload the replication source/sink job to independent Replication Server
[ https://issues.apache.org/jira/browse/HBASE-24666?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17151790#comment-17151790 ] Anoop Sam John commented on HBASE-24666: [~zghao] Even around Replication also we will have CP hooks. I did not check in detail.. But will it need access to the RSs during the replication? I found that the CP hooks is becoming a blocker in moving things out of RS. > Offload the replication source/sink job to independent Replication Server > - > > Key: HBASE-24666 > URL: https://issues.apache.org/jira/browse/HBASE-24666 > Project: HBase > Issue Type: Umbrella >Reporter: Guanghao Zhang >Priority: Major > > The basic idea is add a role "ReplicationServer" to take the replication > source/sink job. HMaster is responsible for scheduling the replication job to > different ReplicationServer. > [link Design > doc|https://docs.google.com/document/d/16kRPVGctFSf__nC3yaVZmAm3GTxIbHefekKC_rMmTw8/edit?usp=sharing] > Suggestions are welcomed. Thanks. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-24666) Offload the replication source/sink job to independent Replication Server
[ https://issues.apache.org/jira/browse/HBASE-24666?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17151785#comment-17151785 ] Zheng Wang commented on HBASE-24666: Maybe it is better to use a generic name(eg: ChoreServer), so we can move some other works such as compaction and log split to it in future. > Offload the replication source/sink job to independent Replication Server > - > > Key: HBASE-24666 > URL: https://issues.apache.org/jira/browse/HBASE-24666 > Project: HBase > Issue Type: Umbrella >Reporter: Guanghao Zhang >Priority: Major > > The basic idea is add a role "ReplicationServer" to take the replication > source/sink job. HMaster is responsible for scheduling the replication job to > different ReplicationServer. > [link Design > doc|https://docs.google.com/document/d/16kRPVGctFSf__nC3yaVZmAm3GTxIbHefekKC_rMmTw8/edit?usp=sharing] > Suggestions are welcomed. Thanks. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-24666) Offload the replication source/sink job to independent Replication Server
[ https://issues.apache.org/jira/browse/HBASE-24666?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17149734#comment-17149734 ] Bharath Vissapragada commented on HBASE-24666: -- I like this idea of de-coupling replication subsystem from the region server. We are brainstorming similar ideas at my employer for the very same reasons of write amplification to multiple peers and we have plans to run customer replication endpoints (with 3rd party code and external dependencies). This decoupling provides a clean way to implement all of these. I think this is also a good opportunity to clean up the nexus of Replication* classes. Added my thoughts to the design doc, thanks for putting it together. > Offload the replication source/sink job to independent Replication Server > - > > Key: HBASE-24666 > URL: https://issues.apache.org/jira/browse/HBASE-24666 > Project: HBase > Issue Type: Umbrella >Reporter: Guanghao Zhang >Priority: Major > > The basic idea is add a role "ReplicationServer" to take the replication > source/sink job. HMaster is responsible for scheduling the replication job to > different ReplicationServer. > [link Design > doc|https://docs.google.com/document/d/16kRPVGctFSf__nC3yaVZmAm3GTxIbHefekKC_rMmTw8/edit?usp=sharing] > Suggestions are welcomed. Thanks. -- This message was sent by Atlassian Jira (v8.3.4#803005)
