[jira] [Commented] (HBASE-11367) Pluggable replication endpoint
[ https://issues.apache.org/jira/browse/HBASE-11367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14222152#comment-14222152 ] Hudson commented on HBASE-11367: FAILURE: Integrated in HBase-0.98 #695 (See [https://builds.apache.org/job/HBase-0.98/695/]) HBASE-11992 Backport HBASE-11367 (Pluggable replication endpoint) to 0.98 (Ramkrishna S. Vasudevan) (apurtell: rev 25be14686f281949c0e9e16d0c3875a11e990518) * hbase-client/src/main/java/org/apache/hadoop/hbase/replication/ReplicationPeers.java * hbase-client/src/main/java/org/apache/hadoop/hbase/replication/ReplicationQueuesZKImpl.java * hbase-client/src/main/java/org/apache/hadoop/hbase/replication/ReplicationPeerConfig.java * hbase-common/src/main/java/org/apache/hadoop/hbase/HBaseInterfaceAudience.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/HLogKey.java * hbase-server/src/main/java/org/apache/hadoop/hbase/replication/ScopeWALEntryFilter.java * hbase-server/src/main/java/org/apache/hadoop/hbase/replication/ReplicationEndpoint.java * hbase-client/src/main/java/org/apache/hadoop/hbase/replication/ReplicationFactory.java * hbase-client/src/main/java/org/apache/hadoop/hbase/zookeeper/ZKClusterId.java * hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSinkManager.java * hbase-protocol/src/main/protobuf/ZooKeeper.proto * hbase-server/src/test/java/org/apache/hadoop/hbase/replication/ReplicationSourceDummy.java * hbase-server/src/test/java/org/apache/hadoop/hbase/client/replication/TestReplicationAdmin.java * hbase-server/src/test/java/org/apache/hadoop/hbase/replication/TestReplicationStateBasic.java * hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSourceInterface.java * hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/MetricsSource.java * hbase-client/src/main/java/org/apache/hadoop/hbase/replication/ReplicationQueuesClient.java * hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/ZooKeeperProtos.java * hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSourceManager.java * hbase-server/src/main/java/org/apache/hadoop/hbase/replication/ChainWALEntryFilter.java * hbase-server/src/test/java/org/apache/hadoop/hbase/replication/TestReplicationWALEntryFilters.java * hbase-server/src/test/java/org/apache/hadoop/hbase/replication/TestReplicationTrackerZKImpl.java * hbase-server/src/test/java/org/apache/hadoop/hbase/replication/regionserver/TestReplicationSinkManager.java * hbase-client/src/main/java/org/apache/hadoop/hbase/replication/ReplicationPeer.java * hbase-server/src/main/java/org/apache/hadoop/hbase/replication/WALEntryFilter.java * hbase-client/src/main/java/org/apache/hadoop/hbase/replication/ReplicationQueuesClientZKImpl.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/HLog.java * hbase-server/src/main/java/org/apache/hadoop/hbase/replication/HBaseReplicationEndpoint.java * hbase-server/src/test/java/org/apache/hadoop/hbase/replication/TestPerTableCFReplication.java * hbase-client/src/main/java/org/apache/hadoop/hbase/replication/ReplicationPeersZKImpl.java * hbase-server/src/main/java/org/apache/hadoop/hbase/replication/master/ReplicationLogCleaner.java * hbase-server/src/main/java/org/apache/hadoop/hbase/replication/SystemTableWALEntryFilter.java * hbase-server/src/main/java/org/apache/hadoop/hbase/replication/TableCfWALEntryFilter.java * hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSource.java * hbase-client/src/main/java/org/apache/hadoop/hbase/replication/ReplicationPeerZKImpl.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/WALEdit.java * hbase-client/src/main/java/org/apache/hadoop/hbase/replication/ReplicationTrackerZKImpl.java * hbase-client/src/main/java/org/apache/hadoop/hbase/replication/ReplicationStateZKBase.java * hbase-client/src/main/java/org/apache/hadoop/hbase/client/replication/ReplicationAdmin.java * hbase-server/src/main/java/org/apache/hadoop/hbase/replication/BaseReplicationEndpoint.java * hbase-server/src/test/java/org/apache/hadoop/hbase/replication/TestReplicationEndpoint.java * hbase-client/src/main/java/org/apache/hadoop/hbase/replication/ReplicationQueueInfo.java * hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/HBaseInterClusterReplicationEndpoint.java * hbase-server/src/main/java/org/apache/hadoop/hbase/mapreduce/replication/VerifyReplication.java Pluggable replication endpoint -- Key: HBASE-11367 URL: https://issues.apache.org/jira/browse/HBASE-11367 Project: HBase Issue Type: Sub-task Reporter: Enis Soztutar Assignee: Enis Soztutar Priority: Blocker Fix
[jira] [Commented] (HBASE-11367) Pluggable replication endpoint
[ https://issues.apache.org/jira/browse/HBASE-11367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14222168#comment-14222168 ] Hudson commented on HBASE-11367: FAILURE: Integrated in HBase-0.98-on-Hadoop-1.1 #662 (See [https://builds.apache.org/job/HBase-0.98-on-Hadoop-1.1/662/]) HBASE-11992 Backport HBASE-11367 (Pluggable replication endpoint) to 0.98 (Ramkrishna S. Vasudevan) (apurtell: rev 25be14686f281949c0e9e16d0c3875a11e990518) * hbase-server/src/main/java/org/apache/hadoop/hbase/replication/WALEntryFilter.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/HLogKey.java * hbase-client/src/main/java/org/apache/hadoop/hbase/replication/ReplicationQueuesClient.java * hbase-server/src/test/java/org/apache/hadoop/hbase/replication/ReplicationSourceDummy.java * hbase-server/src/test/java/org/apache/hadoop/hbase/replication/TestReplicationTrackerZKImpl.java * hbase-server/src/main/java/org/apache/hadoop/hbase/replication/HBaseReplicationEndpoint.java * hbase-server/src/test/java/org/apache/hadoop/hbase/replication/TestReplicationStateBasic.java * hbase-client/src/main/java/org/apache/hadoop/hbase/replication/ReplicationStateZKBase.java * hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSinkManager.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/WALEdit.java * hbase-client/src/main/java/org/apache/hadoop/hbase/replication/ReplicationQueueInfo.java * hbase-client/src/main/java/org/apache/hadoop/hbase/replication/ReplicationPeersZKImpl.java * hbase-client/src/main/java/org/apache/hadoop/hbase/replication/ReplicationPeer.java * hbase-protocol/src/main/protobuf/ZooKeeper.proto * hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSource.java * hbase-server/src/main/java/org/apache/hadoop/hbase/replication/BaseReplicationEndpoint.java * hbase-server/src/main/java/org/apache/hadoop/hbase/replication/ScopeWALEntryFilter.java * hbase-server/src/main/java/org/apache/hadoop/hbase/replication/ChainWALEntryFilter.java * hbase-client/src/main/java/org/apache/hadoop/hbase/replication/ReplicationQueuesZKImpl.java * hbase-server/src/test/java/org/apache/hadoop/hbase/replication/TestPerTableCFReplication.java * hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSourceManager.java * hbase-client/src/main/java/org/apache/hadoop/hbase/replication/ReplicationQueuesClientZKImpl.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/HLog.java * hbase-server/src/main/java/org/apache/hadoop/hbase/mapreduce/replication/VerifyReplication.java * hbase-client/src/main/java/org/apache/hadoop/hbase/replication/ReplicationTrackerZKImpl.java * hbase-server/src/test/java/org/apache/hadoop/hbase/client/replication/TestReplicationAdmin.java * hbase-client/src/main/java/org/apache/hadoop/hbase/client/replication/ReplicationAdmin.java * hbase-server/src/test/java/org/apache/hadoop/hbase/replication/TestReplicationWALEntryFilters.java * hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/MetricsSource.java * hbase-client/src/main/java/org/apache/hadoop/hbase/replication/ReplicationPeers.java * hbase-client/src/main/java/org/apache/hadoop/hbase/replication/ReplicationFactory.java * hbase-server/src/test/java/org/apache/hadoop/hbase/replication/TestReplicationEndpoint.java * hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/HBaseInterClusterReplicationEndpoint.java * hbase-server/src/main/java/org/apache/hadoop/hbase/replication/ReplicationEndpoint.java * hbase-client/src/main/java/org/apache/hadoop/hbase/replication/ReplicationPeerZKImpl.java * hbase-server/src/test/java/org/apache/hadoop/hbase/replication/regionserver/TestReplicationSinkManager.java * hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSourceInterface.java * hbase-common/src/main/java/org/apache/hadoop/hbase/HBaseInterfaceAudience.java * hbase-client/src/main/java/org/apache/hadoop/hbase/replication/ReplicationPeerConfig.java * hbase-server/src/main/java/org/apache/hadoop/hbase/replication/master/ReplicationLogCleaner.java * hbase-client/src/main/java/org/apache/hadoop/hbase/zookeeper/ZKClusterId.java * hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/ZooKeeperProtos.java * hbase-server/src/main/java/org/apache/hadoop/hbase/replication/TableCfWALEntryFilter.java * hbase-server/src/main/java/org/apache/hadoop/hbase/replication/SystemTableWALEntryFilter.java Pluggable replication endpoint -- Key: HBASE-11367 URL: https://issues.apache.org/jira/browse/HBASE-11367 Project: HBase Issue Type: Sub-task Reporter: Enis Soztutar Assignee: Enis Soztutar
[jira] [Commented] (HBASE-11367) Pluggable replication endpoint
[ https://issues.apache.org/jira/browse/HBASE-11367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14135069#comment-14135069 ] Enis Soztutar commented on HBASE-11367: --- bq. I need couple of days before I start on that. BTW Anoop has given comments on HBASE-11920. I will update that patch and can we commit that to master and branch-1? Sorry, I was not able to get to it yet. I have to read through the parent jira. Will do that tomorrow. Pluggable replication endpoint -- Key: HBASE-11367 URL: https://issues.apache.org/jira/browse/HBASE-11367 Project: HBase Issue Type: Sub-task Reporter: Enis Soztutar Assignee: Enis Soztutar Priority: Blocker Fix For: 0.99.0, 2.0.0 Attachments: 0001-11367.patch, hbase-11367_v1.patch, hbase-11367_v2.patch, hbase-11367_v3.patch, hbase-11367_v4.patch, hbase-11367_v4.patch, hbase-11367_v5.patch We need a pluggable endpoint for replication for more flexibility. See parent jira for more context. ReplicationSource tails the logs for each peer. This jira introduces ReplicationEndpoint which is customizable per peer. ReplicationEndpoint is run in the same RS process and instantiated per replication peer per region server. Implementations of this interface handle the actual shipping of WAL edits to the remote cluster. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-11367) Pluggable replication endpoint
[ https://issues.apache.org/jira/browse/HBASE-11367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14135735#comment-14135735 ] Andrew Purtell commented on HBASE-11367: I opened HBASE-11992 for backport discussion Pluggable replication endpoint -- Key: HBASE-11367 URL: https://issues.apache.org/jira/browse/HBASE-11367 Project: HBase Issue Type: Sub-task Reporter: Enis Soztutar Assignee: Enis Soztutar Priority: Blocker Fix For: 0.99.0, 2.0.0 Attachments: 0001-11367.patch, hbase-11367_0.98.patch, hbase-11367_v1.patch, hbase-11367_v2.patch, hbase-11367_v3.patch, hbase-11367_v4.patch, hbase-11367_v4.patch, hbase-11367_v5.patch We need a pluggable endpoint for replication for more flexibility. See parent jira for more context. ReplicationSource tails the logs for each peer. This jira introduces ReplicationEndpoint which is customizable per peer. ReplicationEndpoint is run in the same RS process and instantiated per replication peer per region server. Implementations of this interface handle the actual shipping of WAL edits to the remote cluster. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-11367) Pluggable replication endpoint
[ https://issues.apache.org/jira/browse/HBASE-11367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14134572#comment-14134572 ] Enis Soztutar commented on HBASE-11367: --- [~ram_krish] any luck so far? Do you want to create a separate backport issue? I remembered that I have a patch for this that applies on top of 0.98.4, which is backported from the final master patch. Do you want me to attach it? Pluggable replication endpoint -- Key: HBASE-11367 URL: https://issues.apache.org/jira/browse/HBASE-11367 Project: HBase Issue Type: Sub-task Reporter: Enis Soztutar Assignee: Enis Soztutar Priority: Blocker Fix For: 0.99.0, 2.0.0 Attachments: 0001-11367.patch, hbase-11367_v1.patch, hbase-11367_v2.patch, hbase-11367_v3.patch, hbase-11367_v4.patch, hbase-11367_v4.patch, hbase-11367_v5.patch We need a pluggable endpoint for replication for more flexibility. See parent jira for more context. ReplicationSource tails the logs for each peer. This jira introduces ReplicationEndpoint which is customizable per peer. ReplicationEndpoint is run in the same RS process and instantiated per replication peer per region server. Implementations of this interface handle the actual shipping of WAL edits to the remote cluster. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-11367) Pluggable replication endpoint
[ https://issues.apache.org/jira/browse/HBASE-11367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14134958#comment-14134958 ] ramkrishna.s.vasudevan commented on HBASE-11367: [~enis] I have not tried out a patch for 0.98 still. [~apurtell] suggested we could create a separate backport JIRA for doing it in 0.98. If you have a patch, pls attach it to the JIRA. I need couple of days before I start on that. BTW Anoop has given comments on HBASE-11920. I will update that patch and can we commit that to master and branch-1? Pluggable replication endpoint -- Key: HBASE-11367 URL: https://issues.apache.org/jira/browse/HBASE-11367 Project: HBase Issue Type: Sub-task Reporter: Enis Soztutar Assignee: Enis Soztutar Priority: Blocker Fix For: 0.99.0, 2.0.0 Attachments: 0001-11367.patch, hbase-11367_v1.patch, hbase-11367_v2.patch, hbase-11367_v3.patch, hbase-11367_v4.patch, hbase-11367_v4.patch, hbase-11367_v5.patch We need a pluggable endpoint for replication for more flexibility. See parent jira for more context. ReplicationSource tails the logs for each peer. This jira introduces ReplicationEndpoint which is customizable per peer. ReplicationEndpoint is run in the same RS process and instantiated per replication peer per region server. Implementations of this interface handle the actual shipping of WAL edits to the remote cluster. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-11367) Pluggable replication endpoint
[ https://issues.apache.org/jira/browse/HBASE-11367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14126679#comment-14126679 ] ramkrishna.s.vasudevan commented on HBASE-11367: There are few changes to the ReplicationPeer and ReplicationPeers interface though they are marked private is it ok to change the interface in 0.98.7 releases? In general what could be the best policy that could be applied here? May be I can see if there is anyother different way to do it but just asking this because after we back port it in 0.98.7 again in 1.0 it should not have different interface APIs. Pluggable replication endpoint -- Key: HBASE-11367 URL: https://issues.apache.org/jira/browse/HBASE-11367 Project: HBase Issue Type: Sub-task Reporter: Enis Soztutar Assignee: Enis Soztutar Priority: Blocker Fix For: 0.99.0, 2.0.0 Attachments: 0001-11367.patch, hbase-11367_v1.patch, hbase-11367_v2.patch, hbase-11367_v3.patch, hbase-11367_v4.patch, hbase-11367_v4.patch, hbase-11367_v5.patch We need a pluggable endpoint for replication for more flexibility. See parent jira for more context. ReplicationSource tails the logs for each peer. This jira introduces ReplicationEndpoint which is customizable per peer. ReplicationEndpoint is run in the same RS process and instantiated per replication peer per region server. Implementations of this interface handle the actual shipping of WAL edits to the remote cluster. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-11367) Pluggable replication endpoint
[ https://issues.apache.org/jira/browse/HBASE-11367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14127465#comment-14127465 ] Andrew Purtell commented on HBASE-11367: bq. There are few changes to the ReplicationPeer and ReplicationPeers interface though they are marked private is it ok to change the interface in 0.98.7 releases? In general what could be the best policy that could be applied here? The best policy is minimum changes to replication code. Let's not through this backport rewrite 0.98 replication to become branch-1/master replication. Maybe it could be possible to introduce a pluggable replication API to 0.98 that looks really similar to the branch-1/master API without significant code change to replication internals in 0.98. That *could* be acceptable but replication is a fragile area so big change probably means no. I suggest opening a backport JIRA because making a pluggable replication endpoint in 0.98 seems a separate subject now. Pluggable replication endpoint -- Key: HBASE-11367 URL: https://issues.apache.org/jira/browse/HBASE-11367 Project: HBase Issue Type: Sub-task Reporter: Enis Soztutar Assignee: Enis Soztutar Priority: Blocker Fix For: 0.99.0, 2.0.0 Attachments: 0001-11367.patch, hbase-11367_v1.patch, hbase-11367_v2.patch, hbase-11367_v3.patch, hbase-11367_v4.patch, hbase-11367_v4.patch, hbase-11367_v5.patch We need a pluggable endpoint for replication for more flexibility. See parent jira for more context. ReplicationSource tails the logs for each peer. This jira introduces ReplicationEndpoint which is customizable per peer. ReplicationEndpoint is run in the same RS process and instantiated per replication peer per region server. Implementations of this interface handle the actual shipping of WAL edits to the remote cluster. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-11367) Pluggable replication endpoint
[ https://issues.apache.org/jira/browse/HBASE-11367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14127501#comment-14127501 ] Enis Soztutar commented on HBASE-11367: --- bq. Maybe it could be possible to introduce a pluggable replication API to 0.98 that looks really similar to the branch-1/master API without significant code change to replication internals in 0.98 Code-wise, there was some refactoring done for the replication internals, but there should not be any behavioral / semantic changes in this patch. But, agreed that this is better discussed specifically for 0.98 in a separate jira. bq. I suggest opening a backport JIRA because making a pluggable replication endpoint in 0.98 seems a separate subject now. +1. Pluggable replication endpoint -- Key: HBASE-11367 URL: https://issues.apache.org/jira/browse/HBASE-11367 Project: HBase Issue Type: Sub-task Reporter: Enis Soztutar Assignee: Enis Soztutar Priority: Blocker Fix For: 0.99.0, 2.0.0 Attachments: 0001-11367.patch, hbase-11367_v1.patch, hbase-11367_v2.patch, hbase-11367_v3.patch, hbase-11367_v4.patch, hbase-11367_v4.patch, hbase-11367_v5.patch We need a pluggable endpoint for replication for more flexibility. See parent jira for more context. ReplicationSource tails the logs for each peer. This jira introduces ReplicationEndpoint which is customizable per peer. ReplicationEndpoint is run in the same RS process and instantiated per replication peer per region server. Implementations of this interface handle the actual shipping of WAL edits to the remote cluster. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-11367) Pluggable replication endpoint
[ https://issues.apache.org/jira/browse/HBASE-11367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14125413#comment-14125413 ] ramkrishna.s.vasudevan commented on HBASE-11367: [~enis] Do you think it is possible to back port this to 0.98? Pluggable replication endpoint -- Key: HBASE-11367 URL: https://issues.apache.org/jira/browse/HBASE-11367 Project: HBase Issue Type: Sub-task Reporter: Enis Soztutar Assignee: Enis Soztutar Priority: Blocker Fix For: 0.99.0, 2.0.0 Attachments: 0001-11367.patch, hbase-11367_v1.patch, hbase-11367_v2.patch, hbase-11367_v3.patch, hbase-11367_v4.patch, hbase-11367_v4.patch, hbase-11367_v5.patch We need a pluggable endpoint for replication for more flexibility. See parent jira for more context. ReplicationSource tails the logs for each peer. This jira introduces ReplicationEndpoint which is customizable per peer. ReplicationEndpoint is run in the same RS process and instantiated per replication peer per region server. Implementations of this interface handle the actual shipping of WAL edits to the remote cluster. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-11367) Pluggable replication endpoint
[ https://issues.apache.org/jira/browse/HBASE-11367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14125835#comment-14125835 ] Enis Soztutar commented on HBASE-11367: --- I think it should be possible. It is a bit intrusive in the replication code paths with some refactorings, but it should be compatible. Pluggable replication endpoint -- Key: HBASE-11367 URL: https://issues.apache.org/jira/browse/HBASE-11367 Project: HBase Issue Type: Sub-task Reporter: Enis Soztutar Assignee: Enis Soztutar Priority: Blocker Fix For: 0.99.0, 2.0.0 Attachments: 0001-11367.patch, hbase-11367_v1.patch, hbase-11367_v2.patch, hbase-11367_v3.patch, hbase-11367_v4.patch, hbase-11367_v4.patch, hbase-11367_v5.patch We need a pluggable endpoint for replication for more flexibility. See parent jira for more context. ReplicationSource tails the logs for each peer. This jira introduces ReplicationEndpoint which is customizable per peer. ReplicationEndpoint is run in the same RS process and instantiated per replication peer per region server. Implementations of this interface handle the actual shipping of WAL edits to the remote cluster. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-11367) Pluggable replication endpoint
[ https://issues.apache.org/jira/browse/HBASE-11367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14125858#comment-14125858 ] ramkrishna.s.vasudevan commented on HBASE-11367: Let me try it. Pluggable replication endpoint -- Key: HBASE-11367 URL: https://issues.apache.org/jira/browse/HBASE-11367 Project: HBase Issue Type: Sub-task Reporter: Enis Soztutar Assignee: Enis Soztutar Priority: Blocker Fix For: 0.99.0, 2.0.0 Attachments: 0001-11367.patch, hbase-11367_v1.patch, hbase-11367_v2.patch, hbase-11367_v3.patch, hbase-11367_v4.patch, hbase-11367_v4.patch, hbase-11367_v5.patch We need a pluggable endpoint for replication for more flexibility. See parent jira for more context. ReplicationSource tails the logs for each peer. This jira introduces ReplicationEndpoint which is customizable per peer. ReplicationEndpoint is run in the same RS process and instantiated per replication peer per region server. Implementations of this interface handle the actual shipping of WAL edits to the remote cluster. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-11367) Pluggable replication endpoint
[ https://issues.apache.org/jira/browse/HBASE-11367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14061546#comment-14061546 ] Hudson commented on HBASE-11367: FAILURE: Integrated in HBase-1.0 #41 (See [https://builds.apache.org/job/HBase-1.0/41/]) HBASE-11367 Pluggable replication endpoint (enis: rev c76b528b6d5248ceb0ba06aa0238187ef24e5599) * hbase-common/src/main/java/org/apache/hadoop/hbase/HBaseInterfaceAudience.java * hbase-server/src/main/java/org/apache/hadoop/hbase/replication/ReplicationEndpoint.java * hbase-server/src/test/java/org/apache/hadoop/hbase/replication/TestReplicationStateBasic.java * hbase-server/src/main/java/org/apache/hadoop/hbase/replication/master/ReplicationLogCleaner.java * hbase-client/src/main/java/org/apache/hadoop/hbase/client/replication/ReplicationAdmin.java * hbase-server/src/main/java/org/apache/hadoop/hbase/replication/SystemTableWALEntryFilter.java * hbase-server/src/test/java/org/apache/hadoop/hbase/replication/ReplicationSourceDummy.java * hbase-server/src/test/java/org/apache/hadoop/hbase/replication/regionserver/TestReplicationSinkManager.java * hbase-protocol/src/main/protobuf/ZooKeeper.proto * hbase-server/src/test/java/org/apache/hadoop/hbase/replication/TestPerTableCFReplication.java * hbase-server/src/main/java/org/apache/hadoop/hbase/replication/TableCfWALEntryFilter.java * hbase-server/src/main/java/org/apache/hadoop/hbase/mapreduce/replication/VerifyReplication.java * hbase-server/src/test/java/org/apache/hadoop/hbase/replication/TestReplicationEndpoint.java * hbase-client/src/main/java/org/apache/hadoop/hbase/replication/ReplicationPeers.java * hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSource.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/HLog.java * hbase-server/src/main/java/org/apache/hadoop/hbase/replication/ChainWALEntryFilter.java * hbase-client/src/main/java/org/apache/hadoop/hbase/replication/ReplicationPeerZKImpl.java * hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/MetricsSource.java * hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/HBaseInterClusterReplicationEndpoint.java * hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/ZooKeeperProtos.java * hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSourceInterface.java * hbase-client/src/main/java/org/apache/hadoop/hbase/replication/ReplicationPeersZKImpl.java * hbase-server/src/test/java/org/apache/hadoop/hbase/replication/TestReplicationWALEntryFilters.java * hbase-client/src/main/java/org/apache/hadoop/hbase/replication/ReplicationPeerConfig.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/HLogKey.java * hbase-server/src/main/java/org/apache/hadoop/hbase/replication/ScopeWALEntryFilter.java * hbase-server/src/main/java/org/apache/hadoop/hbase/replication/HBaseReplicationEndpoint.java * hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSourceManager.java * hbase-client/src/main/java/org/apache/hadoop/hbase/zookeeper/ZKClusterId.java * hbase-server/src/test/java/org/apache/hadoop/hbase/client/replication/TestReplicationAdmin.java * hbase-server/src/test/java/org/apache/hadoop/hbase/replication/TestReplicationTrackerZKImpl.java * hbase-server/src/main/java/org/apache/hadoop/hbase/replication/BaseReplicationEndpoint.java * hbase-client/src/main/java/org/apache/hadoop/hbase/replication/ReplicationPeer.java * hbase-server/src/main/java/org/apache/hadoop/hbase/replication/WALEntryFilter.java * hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSinkManager.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/WALEdit.java Pluggable replication endpoint -- Key: HBASE-11367 URL: https://issues.apache.org/jira/browse/HBASE-11367 Project: HBase Issue Type: Sub-task Reporter: Enis Soztutar Assignee: Enis Soztutar Priority: Blocker Fix For: 0.99.0, 2.0.0 Attachments: 0001-11367.patch, hbase-11367_v1.patch, hbase-11367_v2.patch, hbase-11367_v3.patch, hbase-11367_v4.patch, hbase-11367_v4.patch, hbase-11367_v5.patch We need a pluggable endpoint for replication for more flexibility. See parent jira for more context. ReplicationSource tails the logs for each peer. This jira introduces ReplicationEndpoint which is customizable per peer. ReplicationEndpoint is run in the same RS process and instantiated per replication peer per region server. Implementations of this interface handle the actual shipping of WAL edits to the remote cluster. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11367) Pluggable replication endpoint
[ https://issues.apache.org/jira/browse/HBASE-11367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14061578#comment-14061578 ] Hudson commented on HBASE-11367: FAILURE: Integrated in HBase-TRUNK #5303 (See [https://builds.apache.org/job/HBase-TRUNK/5303/]) HBASE-11367 Pluggable replication endpoint (enis: rev 463d52d8cf2a87e1f11eb6fabcd0164584e29fbb) * hbase-common/src/main/java/org/apache/hadoop/hbase/HBaseInterfaceAudience.java * hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSinkManager.java * hbase-server/src/main/java/org/apache/hadoop/hbase/replication/TableCfWALEntryFilter.java * hbase-client/src/main/java/org/apache/hadoop/hbase/zookeeper/ZKClusterId.java * hbase-server/src/test/java/org/apache/hadoop/hbase/replication/regionserver/TestReplicationSinkManager.java * hbase-server/src/test/java/org/apache/hadoop/hbase/replication/TestReplicationEndpoint.java * hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSourceManager.java * hbase-client/src/main/java/org/apache/hadoop/hbase/replication/ReplicationPeersZKImpl.java * hbase-client/src/main/java/org/apache/hadoop/hbase/replication/ReplicationPeerConfig.java * hbase-server/src/main/java/org/apache/hadoop/hbase/replication/WALEntryFilter.java * hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSource.java * hbase-server/src/main/java/org/apache/hadoop/hbase/replication/ChainWALEntryFilter.java * hbase-server/src/main/java/org/apache/hadoop/hbase/mapreduce/replication/VerifyReplication.java * hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/HBaseInterClusterReplicationEndpoint.java * hbase-server/src/main/java/org/apache/hadoop/hbase/replication/BaseReplicationEndpoint.java * hbase-server/src/main/java/org/apache/hadoop/hbase/replication/HBaseReplicationEndpoint.java * hbase-client/src/main/java/org/apache/hadoop/hbase/replication/ReplicationPeer.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/HLog.java * hbase-server/src/main/java/org/apache/hadoop/hbase/replication/ReplicationEndpoint.java * hbase-server/src/test/java/org/apache/hadoop/hbase/client/replication/TestReplicationAdmin.java * hbase-client/src/main/java/org/apache/hadoop/hbase/client/replication/ReplicationAdmin.java * hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/MetricsSource.java * hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSourceInterface.java * hbase-server/src/main/java/org/apache/hadoop/hbase/replication/master/ReplicationLogCleaner.java * hbase-client/src/main/java/org/apache/hadoop/hbase/replication/ReplicationPeers.java * hbase-server/src/test/java/org/apache/hadoop/hbase/replication/ReplicationSourceDummy.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/HLogKey.java * hbase-server/src/main/java/org/apache/hadoop/hbase/replication/ScopeWALEntryFilter.java * hbase-server/src/test/java/org/apache/hadoop/hbase/replication/TestReplicationStateBasic.java * hbase-client/src/main/java/org/apache/hadoop/hbase/replication/ReplicationPeerZKImpl.java * hbase-server/src/main/java/org/apache/hadoop/hbase/replication/SystemTableWALEntryFilter.java * hbase-protocol/src/main/protobuf/ZooKeeper.proto * hbase-server/src/test/java/org/apache/hadoop/hbase/replication/TestReplicationTrackerZKImpl.java * hbase-server/src/test/java/org/apache/hadoop/hbase/replication/TestReplicationWALEntryFilters.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/WALEdit.java * hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/ZooKeeperProtos.java * hbase-server/src/test/java/org/apache/hadoop/hbase/replication/TestPerTableCFReplication.java Pluggable replication endpoint -- Key: HBASE-11367 URL: https://issues.apache.org/jira/browse/HBASE-11367 Project: HBase Issue Type: Sub-task Reporter: Enis Soztutar Assignee: Enis Soztutar Priority: Blocker Fix For: 0.99.0, 2.0.0 Attachments: 0001-11367.patch, hbase-11367_v1.patch, hbase-11367_v2.patch, hbase-11367_v3.patch, hbase-11367_v4.patch, hbase-11367_v4.patch, hbase-11367_v5.patch We need a pluggable endpoint for replication for more flexibility. See parent jira for more context. ReplicationSource tails the logs for each peer. This jira introduces ReplicationEndpoint which is customizable per peer. ReplicationEndpoint is run in the same RS process and instantiated per replication peer per region server. Implementations of this interface handle the actual shipping of WAL edits to the remote cluster. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11367) Pluggable replication endpoint
[ https://issues.apache.org/jira/browse/HBASE-11367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14059411#comment-14059411 ] Devaraj Das commented on HBASE-11367: - Looked at the previous version of the patch. Looks pretty good to me. Pluggable replication endpoint -- Key: HBASE-11367 URL: https://issues.apache.org/jira/browse/HBASE-11367 Project: HBase Issue Type: Sub-task Reporter: Enis Soztutar Assignee: Enis Soztutar Priority: Blocker Fix For: 0.99.0 Attachments: 0001-11367.patch, hbase-11367_v1.patch, hbase-11367_v2.patch, hbase-11367_v3.patch, hbase-11367_v4.patch, hbase-11367_v4.patch We need a pluggable endpoint for replication for more flexibility. See parent jira for more context. ReplicationSource tails the logs for each peer. This jira introduces ReplicationEndpoint which is customizable per peer. ReplicationEndpoint is run in the same RS process and instantiated per replication peer per region server. Implementations of this interface handle the actual shipping of WAL edits to the remote cluster. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11367) Pluggable replication endpoint
[
https://issues.apache.org/jira/browse/HBASE-11367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14059544#comment-14059544
]
Hadoop QA commented on HBASE-11367:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12655328/hbase-11367_v5.patch
against trunk revision .
ATTACHMENT ID: 12655328
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 22 new
or modified tests.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:red}-1 findbugs{color}. The patch appears to introduce 4 new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
{color:red}-1 core zombie tests{color}. There are 1 zombie test(s):
at
org.apache.hadoop.hbase.client.TestReplicaWithCluster.testCreateDeleteTable(TestReplicaWithCluster.java:138)
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/10040//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10040//artifact/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10040//artifact/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10040//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10040//artifact/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10040//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10040//artifact/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10040//artifact/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10040//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10040//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/10040//console
This message is automatically generated.
Pluggable replication endpoint
--
Key: HBASE-11367
URL: https://issues.apache.org/jira/browse/HBASE-11367
Project: HBase
Issue Type: Sub-task
Reporter: Enis Soztutar
Assignee: Enis Soztutar
Priority: Blocker
Fix For: 0.99.0
Attachments: 0001-11367.patch, hbase-11367_v1.patch,
hbase-11367_v2.patch, hbase-11367_v3.patch, hbase-11367_v4.patch,
hbase-11367_v4.patch, hbase-11367_v5.patch
We need a pluggable endpoint for replication for more flexibility. See parent
jira for more context.
ReplicationSource tails the logs for each peer. This jira introduces
ReplicationEndpoint which is customizable per peer. ReplicationEndpoint is
run in the same RS process and instantiated per replication peer per region
server. Implementations of this interface handle the actual shipping of WAL
edits to the remote cluster.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11367) Pluggable replication endpoint
[
https://issues.apache.org/jira/browse/HBASE-11367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14058145#comment-14058145
]
Hadoop QA commented on HBASE-11367:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12655119/hbase-11367_v4.patch
against trunk revision .
ATTACHMENT ID: 12655119
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 22 new
or modified tests.
{color:red}-1 patch{color}. The patch command could not apply the patch.
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/10029//console
This message is automatically generated.
Pluggable replication endpoint
--
Key: HBASE-11367
URL: https://issues.apache.org/jira/browse/HBASE-11367
Project: HBase
Issue Type: Sub-task
Reporter: Enis Soztutar
Assignee: Enis Soztutar
Priority: Blocker
Fix For: 0.99.0
Attachments: 0001-11367.patch, hbase-11367_v1.patch,
hbase-11367_v2.patch, hbase-11367_v3.patch, hbase-11367_v4.patch
We need a pluggable endpoint for replication for more flexibility. See parent
jira for more context.
ReplicationSource tails the logs for each peer. This jira introduces
ReplicationEndpoint which is customizable per peer. ReplicationEndpoint is
run in the same RS process and instantiated per replication peer per region
server. Implementations of this interface handle the actual shipping of WAL
edits to the remote cluster.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11367) Pluggable replication endpoint
[
https://issues.apache.org/jira/browse/HBASE-11367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14058249#comment-14058249
]
Hadoop QA commented on HBASE-11367:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12655122/hbase-11367_v4.patch
against trunk revision .
ATTACHMENT ID: 12655122
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 22 new
or modified tests.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:red}-1 findbugs{color}. The patch appears to introduce 4 new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.replication.TestReplicationEndpoint
{color:red}-1 core zombie tests{color}. There are 1 zombie test(s):
at
org.apache.hadoop.hbase.client.TestReplicaWithCluster.testCreateDeleteTable(TestReplicaWithCluster.java:138)
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/10030//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10030//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10030//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10030//artifact/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10030//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10030//artifact/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10030//artifact/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10030//artifact/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10030//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10030//artifact/patchprocess/newPatchFindbugsWarningshbase-client.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/10030//console
This message is automatically generated.
Pluggable replication endpoint
--
Key: HBASE-11367
URL: https://issues.apache.org/jira/browse/HBASE-11367
Project: HBase
Issue Type: Sub-task
Reporter: Enis Soztutar
Assignee: Enis Soztutar
Priority: Blocker
Fix For: 0.99.0
Attachments: 0001-11367.patch, hbase-11367_v1.patch,
hbase-11367_v2.patch, hbase-11367_v3.patch, hbase-11367_v4.patch,
hbase-11367_v4.patch
We need a pluggable endpoint for replication for more flexibility. See parent
jira for more context.
ReplicationSource tails the logs for each peer. This jira introduces
ReplicationEndpoint which is customizable per peer. ReplicationEndpoint is
run in the same RS process and instantiated per replication peer per region
server. Implementations of this interface handle the actual shipping of WAL
edits to the remote cluster.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11367) Pluggable replication endpoint
[ https://issues.apache.org/jira/browse/HBASE-11367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14050679#comment-14050679 ] Enis Soztutar commented on HBASE-11367: --- [~posix4e] thanks for looking. I did not get why you attached a patch? Does this contain v3 patch + your small fixes? The usual flow is to add comments in RB so that the patch author will fix them. Pluggable replication endpoint -- Key: HBASE-11367 URL: https://issues.apache.org/jira/browse/HBASE-11367 Project: HBase Issue Type: Sub-task Reporter: Enis Soztutar Assignee: Enis Soztutar Priority: Blocker Fix For: 0.99.0 Attachments: 0001-11367.patch, hbase-11367_v1.patch, hbase-11367_v2.patch, hbase-11367_v3.patch We need a pluggable endpoint for replication for more flexibility. See parent jira for more context. ReplicationSource tails the logs for each peer. This jira introduces ReplicationEndpoint which is customizable per peer. ReplicationEndpoint is run in the same RS process and instantiated per replication peer per region server. Implementations of this interface handle the actual shipping of WAL edits to the remote cluster. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11367) Pluggable replication endpoint
[ https://issues.apache.org/jira/browse/HBASE-11367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14049409#comment-14049409 ] Alex Newman commented on HBASE-11367: - I found some small nits while looking over review board. Nothing that should hold anything up, I attached a patch. Pluggable replication endpoint -- Key: HBASE-11367 URL: https://issues.apache.org/jira/browse/HBASE-11367 Project: HBase Issue Type: Sub-task Reporter: Enis Soztutar Assignee: Enis Soztutar Priority: Blocker Fix For: 0.99.0 Attachments: 0001-11367.patch, hbase-11367_v1.patch, hbase-11367_v2.patch, hbase-11367_v3.patch We need a pluggable endpoint for replication for more flexibility. See parent jira for more context. ReplicationSource tails the logs for each peer. This jira introduces ReplicationEndpoint which is customizable per peer. ReplicationEndpoint is run in the same RS process and instantiated per replication peer per region server. Implementations of this interface handle the actual shipping of WAL edits to the remote cluster. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11367) Pluggable replication endpoint
[
https://issues.apache.org/jira/browse/HBASE-11367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14049424#comment-14049424
]
Hadoop QA commented on HBASE-11367:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12653465/hbase-11367_v3.patch
against trunk revision .
ATTACHMENT ID: 12653465
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 22 new
or modified tests.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:red}-1 findbugs{color}. The patch appears to introduce 13 new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.master.TestMasterNoCluster
{color:red}-1 core zombie tests{color}. There are 1 zombie test(s):
at
org.apache.hadoop.hbase.client.TestReplicaWithCluster.testChangeTable(TestReplicaWithCluster.java:217)
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/9928//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9928//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9928//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9928//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9928//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9928//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9928//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9928//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9928//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9928//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/9928//console
This message is automatically generated.
Pluggable replication endpoint
--
Key: HBASE-11367
URL: https://issues.apache.org/jira/browse/HBASE-11367
Project: HBase
Issue Type: Sub-task
Reporter: Enis Soztutar
Assignee: Enis Soztutar
Priority: Blocker
Fix For: 0.99.0
Attachments: 0001-11367.patch, hbase-11367_v1.patch,
hbase-11367_v2.patch, hbase-11367_v3.patch
We need a pluggable endpoint for replication for more flexibility. See parent
jira for more context.
ReplicationSource tails the logs for each peer. This jira introduces
ReplicationEndpoint which is customizable per peer. ReplicationEndpoint is
run in the same RS process and instantiated per replication peer per region
server. Implementations of this interface handle the actual shipping of WAL
edits to the remote cluster.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11367) Pluggable replication endpoint
[
https://issues.apache.org/jira/browse/HBASE-11367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14048523#comment-14048523
]
Hadoop QA commented on HBASE-11367:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12653304/hbase-11367_v2.patch
against trunk revision .
ATTACHMENT ID: 12653304
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 22 new
or modified tests.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:red}-1 findbugs{color}. The patch appears to introduce 13 new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 lineLengths{color}. The patch introduces the following lines
longer than 100:
+ public ReplicationPeerZKImpl(Configuration conf, String id,
ReplicationPeerConfig peerConfig) throws ReplicationException {
+// TODO: This should be a PB object rather than a String to be parsed!!
Resolve this in HBASE-11393
+ void addPeer(String peerId, ReplicationPeerConfig peerConfig, String
tableCFs) throws ReplicationException;
+ new java.lang.String[] { Clusterkey,
ReplicationEndpointImpl, Data, Configuration, });
+// send the edits to the endpoint. Will block until the edits are
actually sent and acknowledged
+LOG.warn(Received exception while getting replication peer
config, skipping replay + ex);
+ UUID clusterId, ReplicationEndpoint replicationEndpoint, MetricsSource
metrics) throws IOException {
+tabCFsMap = ReplicationPeerZKImpl.parseTableCFsFromConfig(tab1 ; ;
tab2:cf1 ; tab3:cf1,,cf3 ;);
+tabCFsMap = ReplicationPeerZKImpl.parseTableCFsFromConfig(tab1:tt:cf1 ;
tab2::cf1 ; tab3:cf1,cf3);
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.TestCheckTestClasses
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/9911//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9911//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9911//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9911//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9911//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9911//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9911//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9911//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9911//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9911//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/9911//console
This message is automatically generated.
Pluggable replication endpoint
--
Key: HBASE-11367
URL: https://issues.apache.org/jira/browse/HBASE-11367
Project: HBase
Issue Type: Sub-task
Reporter: Enis Soztutar
Assignee: Enis Soztutar
Priority: Blocker
Fix For: 0.99.0
Attachments: hbase-11367_v1.patch, hbase-11367_v2.patch
We need a pluggable endpoint for replication for more flexibility. See parent
jira for more context.
ReplicationSource tails the logs for each peer. This jira introduces
ReplicationEndpoint which is customizable per peer. ReplicationEndpoint is
run in the same RS process and instantiated per replication peer per region
server. Implementations of this interface handle the actual shipping of WAL
edits to the remote cluster.
--
This message was sent by Atlassian
[jira] [Commented] (HBASE-11367) Pluggable replication endpoint
[
https://issues.apache.org/jira/browse/HBASE-11367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14033280#comment-14033280
]
Hadoop QA commented on HBASE-11367:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12650696/hbase-11367_v1.patch
against trunk revision .
ATTACHMENT ID: 12650696
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 15 new
or modified tests.
{color:red}-1 patch{color}. The patch command could not apply the patch.
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/9782//console
This message is automatically generated.
Pluggable replication endpoint
--
Key: HBASE-11367
URL: https://issues.apache.org/jira/browse/HBASE-11367
Project: HBase
Issue Type: Sub-task
Reporter: Enis Soztutar
Assignee: Enis Soztutar
Priority: Blocker
Fix For: 0.99.0
Attachments: hbase-11367_v1.patch
We need a pluggable endpoint for replication for more flexibility. See parent
jira for more context.
ReplicationSource tails the logs for each peer. This jira introduces
ReplicationEndpoint which is customizable per peer. ReplicationEndpoint is
run in the same RS process and instantiated per replication peer per region
server. Implementations of this interface handle the actual shipping of WAL
edits to the remote cluster.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
