Change in asterixdb[master]: [NO ISSUE][REPL] Exclude Non-Replicated Datasets From Delta ...
Jenkins has posted comments on this change. Change subject: [NO ISSUE][REPL] Exclude Non-Replicated Datasets From Delta Recovery .. Patch Set 1: Build Started https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-cancellation-test/2769/ (6/12) -- To view, visit https://asterix-gerrit.ics.uci.edu/2412 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: comment Gerrit-Change-Id: Ic734af7becf26082e79fae52bd2c01ba567c1c99 Gerrit-PatchSet: 1 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: Murtadha Hubail Gerrit-Reviewer: Jenkins Gerrit-HasComments: No
Change in asterixdb[master]: [NO ISSUE][REPL] Exclude Non-Replicated Datasets From Delta ...
Jenkins has posted comments on this change. Change subject: [NO ISSUE][REPL] Exclude Non-Replicated Datasets From Delta Recovery .. Patch Set 1: Build Started https://asterix-jenkins.ics.uci.edu/job/asterix-verify-storage/3322/ (1/12) -- To view, visit https://asterix-gerrit.ics.uci.edu/2412 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: comment Gerrit-Change-Id: Ic734af7becf26082e79fae52bd2c01ba567c1c99 Gerrit-PatchSet: 1 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: Murtadha Hubail Gerrit-Reviewer: Jenkins Gerrit-HasComments: No
Change in asterixdb[master]: [NO ISSUE][REPL] Exclude Non-Replicated Datasets From Delta ...
Jenkins has posted comments on this change. Change subject: [NO ISSUE][REPL] Exclude Non-Replicated Datasets From Delta Recovery .. Patch Set 1: Build Started https://asterix-jenkins.ics.uci.edu/job/hyracks-gerrit/2702/ (5/12) -- To view, visit https://asterix-gerrit.ics.uci.edu/2412 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: comment Gerrit-Change-Id: Ic734af7becf26082e79fae52bd2c01ba567c1c99 Gerrit-PatchSet: 1 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: Murtadha Hubail Gerrit-Reviewer: Jenkins Gerrit-HasComments: No
Change in asterixdb[master]: [NO ISSUE][REPL] Exclude Non-Replicated Datasets From Delta ...
Jenkins has posted comments on this change. Change subject: [NO ISSUE][REPL] Exclude Non-Replicated Datasets From Delta Recovery .. Patch Set 1: Build Started https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-source-assemblies/3008/ (4/12) -- To view, visit https://asterix-gerrit.ics.uci.edu/2412 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: comment Gerrit-Change-Id: Ic734af7becf26082e79fae52bd2c01ba567c1c99 Gerrit-PatchSet: 1 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: Murtadha Hubail Gerrit-Reviewer: Jenkins Gerrit-HasComments: No
Change in asterixdb[master]: [NO ISSUE][REPL] Exclude Non-Replicated Datasets From Delta ...
Jenkins has posted comments on this change. Change subject: [NO ISSUE][REPL] Exclude Non-Replicated Datasets From Delta Recovery .. Patch Set 1: Build Started https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-sonar/6741/ (3/12) -- To view, visit https://asterix-gerrit.ics.uci.edu/2412 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: comment Gerrit-Change-Id: Ic734af7becf26082e79fae52bd2c01ba567c1c99 Gerrit-PatchSet: 1 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: Murtadha Hubail Gerrit-Reviewer: Jenkins Gerrit-HasComments: No
Change in asterixdb[master]: [NO ISSUE][REPL] Exclude Non-Replicated Datasets From Delta ...
Jenkins has posted comments on this change. Change subject: [NO ISSUE][REPL] Exclude Non-Replicated Datasets From Delta Recovery .. Patch Set 1: Build Started https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-ensure-ancestor/798/ (8/12) -- To view, visit https://asterix-gerrit.ics.uci.edu/2412 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: comment Gerrit-Change-Id: Ic734af7becf26082e79fae52bd2c01ba567c1c99 Gerrit-PatchSet: 1 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: Murtadha Hubail Gerrit-Reviewer: Jenkins Gerrit-HasComments: No
Change in asterixdb[master]: [NO ISSUE][REPL] Exclude Non-Replicated Datasets From Delta ...
Jenkins has posted comments on this change. Change subject: [NO ISSUE][REPL] Exclude Non-Replicated Datasets From Delta Recovery .. Patch Set 1: Build Started https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-notopic/8248/ (2/12) -- To view, visit https://asterix-gerrit.ics.uci.edu/2412 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: comment Gerrit-Change-Id: Ic734af7becf26082e79fae52bd2c01ba567c1c99 Gerrit-PatchSet: 1 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: Murtadha Hubail Gerrit-Reviewer: Jenkins Gerrit-HasComments: No
Change in asterixdb[master]: [NO ISSUE][REPL] Exclude Non-Replicated Datasets From Delta ...
Murtadha Hubail has uploaded a new change for review.
https://asterix-gerrit.ics.uci.edu/2412
Change subject: [NO ISSUE][REPL] Exclude Non-Replicated Datasets From Delta
Recovery
..
[NO ISSUE][REPL] Exclude Non-Replicated Datasets From Delta Recovery
- user model changes: no
- storage format changes: no
- interface changes: no
Details:
- Exclude non-replicated datasets files from
delta recovery.
- Fix used read buffer for large replication
requests.
Change-Id: Ic734af7becf26082e79fae52bd2c01ba567c1c99
---
M
asterixdb/asterix-replication/src/main/java/org/apache/asterix/replication/messaging/PartitionResourcesListTask.java
M
asterixdb/asterix-replication/src/main/java/org/apache/asterix/replication/messaging/ReplicationProtocol.java
M
asterixdb/asterix-replication/src/main/java/org/apache/asterix/replication/sync/ReplicaFilesSynchronizer.java
M
asterixdb/asterix-transactions/src/main/java/org/apache/asterix/transaction/management/resource/PersistentLocalResourceRepository.java
4 files changed, 40 insertions(+), 18 deletions(-)
git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb
refs/changes/12/2412/1
diff --git
a/asterixdb/asterix-replication/src/main/java/org/apache/asterix/replication/messaging/PartitionResourcesListTask.java
b/asterixdb/asterix-replication/src/main/java/org/apache/asterix/replication/messaging/PartitionResourcesListTask.java
index 54d3a02..b2b1ad1 100644
---
a/asterixdb/asterix-replication/src/main/java/org/apache/asterix/replication/messaging/PartitionResourcesListTask.java
+++
b/asterixdb/asterix-replication/src/main/java/org/apache/asterix/replication/messaging/PartitionResourcesListTask.java
@@ -26,9 +26,10 @@
import java.util.stream.Collectors;
import org.apache.asterix.common.api.INcApplicationContext;
-import org.apache.asterix.replication.api.IReplicationWorker;
+import org.apache.asterix.common.replication.IReplicationStrategy;
import org.apache.asterix.common.utils.StoragePathUtil;
import org.apache.asterix.replication.api.IReplicaTask;
+import org.apache.asterix.replication.api.IReplicationWorker;
import
org.apache.asterix.transaction.management.resource.PersistentLocalResourceRepository;
import org.apache.hyracks.api.exceptions.HyracksDataException;
@@ -48,8 +49,10 @@
final PersistentLocalResourceRepository localResourceRepository =
(PersistentLocalResourceRepository)
appCtx.getLocalResourceRepository();
localResourceRepository.cleanup(partition);
-final List partitionResources =
localResourceRepository.getPartitionIndexesFiles(partition).stream()
-
.map(StoragePathUtil::getFileRelativePath).collect(Collectors.toList());
+final IReplicationStrategy replicationStrategy =
appCtx.getReplicationManager().getReplicationStrategy();
+final List partitionResources =
+localResourceRepository.getPartitionReplicatedFiles(partition,
replicationStrategy).stream()
+
.map(StoragePathUtil::getFileRelativePath).collect(Collectors.toList());
final PartitionResourcesListResponse response =
new PartitionResourcesListResponse(partition,
partitionResources);
ReplicationProtocol.sendTo(worker.getChannel(), response,
worker.getReusableBuffer());
diff --git
a/asterixdb/asterix-replication/src/main/java/org/apache/asterix/replication/messaging/ReplicationProtocol.java
b/asterixdb/asterix-replication/src/main/java/org/apache/asterix/replication/messaging/ReplicationProtocol.java
index 280a2d4..41e7d9e 100644
---
a/asterixdb/asterix-replication/src/main/java/org/apache/asterix/replication/messaging/ReplicationProtocol.java
+++
b/asterixdb/asterix-replication/src/main/java/org/apache/asterix/replication/messaging/ReplicationProtocol.java
@@ -72,7 +72,7 @@
final ByteBuffer buf = ensureSize(dataBuffer, requestSize);
// read request
NetworkingUtil.readBytes(socketChannel, buf, requestSize);
-return dataBuffer;
+return buf;
}
public static ReplicationRequestType getRequestType(SocketChannel
socketChannel, ByteBuffer byteBuffer)
@@ -135,6 +135,7 @@
requestBuffer.put(outputStream.getByteArray(), 0,
outputStream.getLength());
requestBuffer.flip();
NetworkingUtil.transferBufferToChannel(channel, requestBuffer);
+channel.socket().getOutputStream().flush();
} catch (IOException e) {
throw new ReplicationException(e);
}
@@ -148,9 +149,9 @@
public static IReplicationMessage readMessage(ReplicationRequestType type,
SocketChannel socketChannel,
ByteBuffer buffer) {
try {
-ReplicationProtocol.readRequest(socketChannel, buffer);
+final ByteBuffer requestBuf =
ReplicationProtocol.readRequest(socketChannel, buffer);
final ByteArrayInputStream input =
-
Change in asterixdb[master]: [NO ISSUE][REPL] Exclude Non-Replicated Datasets From Delta ...
Jenkins has posted comments on this change. Change subject: [NO ISSUE][REPL] Exclude Non-Replicated Datasets From Delta Recovery .. Patch Set 1: Build Started https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-source-format/2749/ (7/12) -- To view, visit https://asterix-gerrit.ics.uci.edu/2412 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: comment Gerrit-Change-Id: Ic734af7becf26082e79fae52bd2c01ba567c1c99 Gerrit-PatchSet: 1 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: Murtadha Hubail Gerrit-Reviewer: Jenkins Gerrit-HasComments: No
Change in asterixdb[master]: [NO ISSUE][REPL] Exclude Non-Replicated Datasets From Delta ...
Anon. E. Moose #1000171 has posted comments on this change. Change subject: [NO ISSUE][REPL] Exclude Non-Replicated Datasets From Delta Recovery .. Patch Set 1: Analytics Compatibility Compilation Successful https://goo.gl/JYtM6c : SUCCESS -- To view, visit https://asterix-gerrit.ics.uci.edu/2412 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: comment Gerrit-Change-Id: Ic734af7becf26082e79fae52bd2c01ba567c1c99 Gerrit-PatchSet: 1 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: Murtadha Hubail Gerrit-Reviewer: Anon. E. Moose #1000171 Gerrit-Reviewer: Jenkins Gerrit-Reviewer: Michael Blow Gerrit-HasComments: No
Change in asterixdb[master]: [NO ISSUE][REPL] Exclude Non-Replicated Datasets From Delta ...
Jenkins has posted comments on this change. Change subject: [NO ISSUE][REPL] Exclude Non-Replicated Datasets From Delta Recovery .. Patch Set 1: Build Started https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-asterix-app-sql-execution/2777/ (9/12) -- To view, visit https://asterix-gerrit.ics.uci.edu/2412 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: comment Gerrit-Change-Id: Ic734af7becf26082e79fae52bd2c01ba567c1c99 Gerrit-PatchSet: 1 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: Murtadha Hubail Gerrit-Reviewer: Anon. E. Moose #1000171 Gerrit-Reviewer: Jenkins Gerrit-Reviewer: Michael Blow Gerrit-HasComments: No
Change in asterixdb[master]: [NO ISSUE][REPL] Exclude Non-Replicated Datasets From Delta ...
Jenkins has posted comments on this change. Change subject: [NO ISSUE][REPL] Exclude Non-Replicated Datasets From Delta Recovery .. Patch Set 1: Integration Tests Started https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-integration-tests/5703/ -- To view, visit https://asterix-gerrit.ics.uci.edu/2412 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: comment Gerrit-Change-Id: Ic734af7becf26082e79fae52bd2c01ba567c1c99 Gerrit-PatchSet: 1 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: Murtadha Hubail Gerrit-Reviewer: Anon. E. Moose #1000171 Gerrit-Reviewer: Jenkins Gerrit-Reviewer: Michael Blow Gerrit-HasComments: No
Change in asterixdb[master]: [NO ISSUE][REPL] Exclude Non-Replicated Datasets From Delta ...
Jenkins has posted comments on this change. Change subject: [NO ISSUE][REPL] Exclude Non-Replicated Datasets From Delta Recovery .. Patch Set 1: Build Started https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-asterix-app/3524/ (10/12) -- To view, visit https://asterix-gerrit.ics.uci.edu/2412 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: comment Gerrit-Change-Id: Ic734af7becf26082e79fae52bd2c01ba567c1c99 Gerrit-PatchSet: 1 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: Murtadha Hubail Gerrit-Reviewer: Anon. E. Moose #1000171 Gerrit-Reviewer: Jenkins Gerrit-Reviewer: Michael Blow Gerrit-HasComments: No
Change in asterixdb[master]: [NO ISSUE][REPL] Exclude Non-Replicated Datasets From Delta ...
Jenkins has posted comments on this change. Change subject: [NO ISSUE][REPL] Exclude Non-Replicated Datasets From Delta Recovery .. Patch Set 1: Build Started https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-verify-no-installer-app/3264/ (11/12) -- To view, visit https://asterix-gerrit.ics.uci.edu/2412 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: comment Gerrit-Change-Id: Ic734af7becf26082e79fae52bd2c01ba567c1c99 Gerrit-PatchSet: 1 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: Murtadha Hubail Gerrit-Reviewer: Anon. E. Moose #1000171 Gerrit-Reviewer: Jenkins Gerrit-Reviewer: Michael Blow Gerrit-HasComments: No
Change in asterixdb[master]: [NO ISSUE][REPL] Exclude Non-Replicated Datasets From Delta ...
Anon. E. Moose #1000171 has posted comments on this change. Change subject: [NO ISSUE][REPL] Exclude Non-Replicated Datasets From Delta Recovery .. Patch Set 1: Contrib-2 Analytics Compatibility Tests Failed https://goo.gl/vF4WZ9 : UNSTABLE -- To view, visit https://asterix-gerrit.ics.uci.edu/2412 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: comment Gerrit-Change-Id: Ic734af7becf26082e79fae52bd2c01ba567c1c99 Gerrit-PatchSet: 1 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: Murtadha Hubail Gerrit-Reviewer: Anon. E. Moose #1000171 Gerrit-Reviewer: Jenkins Gerrit-Reviewer: Michael Blow Gerrit-HasComments: No
Change in asterixdb[master]: [NO ISSUE][REPL] Exclude Non-Replicated Datasets From Delta ...
Jenkins has posted comments on this change. Change subject: [NO ISSUE][REPL] Exclude Non-Replicated Datasets From Delta Recovery .. Patch Set 1: Build Started https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-verify-asterix-app/3289/ (12/12) -- To view, visit https://asterix-gerrit.ics.uci.edu/2412 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: comment Gerrit-Change-Id: Ic734af7becf26082e79fae52bd2c01ba567c1c99 Gerrit-PatchSet: 1 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: Murtadha Hubail Gerrit-Reviewer: Anon. E. Moose #1000171 Gerrit-Reviewer: Jenkins Gerrit-Reviewer: Michael Blow Gerrit-HasComments: No
Change in asterixdb[master]: [NO ISSUE][REPL] Exclude Non-Replicated Datasets From Delta ...
Jenkins has posted comments on this change. Change subject: [NO ISSUE][REPL] Exclude Non-Replicated Datasets From Delta Recovery .. Patch Set 1: Build Started https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-cancellation-test/2771/ (12/12) -- To view, visit https://asterix-gerrit.ics.uci.edu/2412 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: comment Gerrit-Change-Id: Ic734af7becf26082e79fae52bd2c01ba567c1c99 Gerrit-PatchSet: 1 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: Murtadha Hubail Gerrit-Reviewer: Anon. E. Moose #1000171 Gerrit-Reviewer: Jenkins Gerrit-Reviewer: Michael Blow Gerrit-HasComments: No
Change in asterixdb[master]: [NO ISSUE][REPL] Exclude Non-Replicated Datasets From Delta ...
Jenkins has posted comments on this change. Change subject: [NO ISSUE][REPL] Exclude Non-Replicated Datasets From Delta Recovery .. Patch Set 1: Integration-Tests+1 Integration Tests Successful https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-integration-tests/5703/ : SUCCESS -- To view, visit https://asterix-gerrit.ics.uci.edu/2412 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: comment Gerrit-Change-Id: Ic734af7becf26082e79fae52bd2c01ba567c1c99 Gerrit-PatchSet: 1 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: Murtadha Hubail Gerrit-Reviewer: Anon. E. Moose #1000171 Gerrit-Reviewer: Jenkins Gerrit-Reviewer: Michael Blow Gerrit-HasComments: No
Change in asterixdb[master]: [NO ISSUE][REPL] Exclude Non-Replicated Datasets From Delta ...
Jenkins has posted comments on this change. Change subject: [NO ISSUE][REPL] Exclude Non-Replicated Datasets From Delta Recovery .. Patch Set 1: BAD Compatibility Tests Started https://asterix-jenkins.ics.uci.edu/job/asterixbad-compat/2742/ -- To view, visit https://asterix-gerrit.ics.uci.edu/2412 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: comment Gerrit-Change-Id: Ic734af7becf26082e79fae52bd2c01ba567c1c99 Gerrit-PatchSet: 1 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: Murtadha Hubail Gerrit-Reviewer: Anon. E. Moose #1000171 Gerrit-Reviewer: Jenkins Gerrit-Reviewer: Michael Blow Gerrit-HasComments: No
Change in asterixdb[master]: [NO ISSUE][REPL] Exclude Non-Replicated Datasets From Delta ...
Jenkins has posted comments on this change. Change subject: [NO ISSUE][REPL] Exclude Non-Replicated Datasets From Delta Recovery .. Patch Set 1: Contrib+1 BAD Compatibility Tests Successful https://asterix-jenkins.ics.uci.edu/job/asterixbad-compat/2742/ : SUCCESS -- To view, visit https://asterix-gerrit.ics.uci.edu/2412 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: comment Gerrit-Change-Id: Ic734af7becf26082e79fae52bd2c01ba567c1c99 Gerrit-PatchSet: 1 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: Murtadha Hubail Gerrit-Reviewer: Anon. E. Moose #1000171 Gerrit-Reviewer: Jenkins Gerrit-Reviewer: Michael Blow Gerrit-HasComments: No
Change in asterixdb[master]: [NO ISSUE][REPL] Exclude Non-Replicated Datasets From Delta ...
Anon. E. Moose #1000171 has posted comments on this change. Change subject: [NO ISSUE][REPL] Exclude Non-Replicated Datasets From Delta Recovery .. Patch Set 1: Analytics Compatibility Tests Failed https://goo.gl/JEAnJv : UNSTABLE -- To view, visit https://asterix-gerrit.ics.uci.edu/2412 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: comment Gerrit-Change-Id: Ic734af7becf26082e79fae52bd2c01ba567c1c99 Gerrit-PatchSet: 1 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: Murtadha Hubail Gerrit-Reviewer: Anon. E. Moose #1000171 Gerrit-Reviewer: Jenkins Gerrit-Reviewer: Michael Blow Gerrit-HasComments: No
Change in asterixdb[master]: [NO ISSUE][REPL] Exclude Non-Replicated Datasets From Delta ...
Michael Blow has posted comments on this change. Change subject: [NO ISSUE][REPL] Exclude Non-Replicated Datasets From Delta Recovery .. Patch Set 1: Code-Review+2 -- To view, visit https://asterix-gerrit.ics.uci.edu/2412 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: comment Gerrit-Change-Id: Ic734af7becf26082e79fae52bd2c01ba567c1c99 Gerrit-PatchSet: 1 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: Murtadha Hubail Gerrit-Reviewer: Jenkins Gerrit-Reviewer: Michael Blow Gerrit-HasComments: No
Change in asterixdb[master]: [NO ISSUE][REPL] Exclude Non-Replicated Datasets From Delta ...
Murtadha Hubail has submitted this change and it was merged.
Change subject: [NO ISSUE][REPL] Exclude Non-Replicated Datasets From Delta
Recovery
..
[NO ISSUE][REPL] Exclude Non-Replicated Datasets From Delta Recovery
- user model changes: no
- storage format changes: no
- interface changes: no
Details:
- Exclude non-replicated datasets files from
delta recovery.
- Fix used read buffer for large replication
requests.
Change-Id: Ic734af7becf26082e79fae52bd2c01ba567c1c99
Reviewed-on: https://asterix-gerrit.ics.uci.edu/2412
Sonar-Qube: Jenkins
Integration-Tests: Jenkins
Tested-by: Jenkins
Contrib: Jenkins
Reviewed-by: Michael Blow
---
M
asterixdb/asterix-replication/src/main/java/org/apache/asterix/replication/messaging/PartitionResourcesListTask.java
M
asterixdb/asterix-replication/src/main/java/org/apache/asterix/replication/messaging/ReplicationProtocol.java
M
asterixdb/asterix-replication/src/main/java/org/apache/asterix/replication/sync/ReplicaFilesSynchronizer.java
M
asterixdb/asterix-transactions/src/main/java/org/apache/asterix/transaction/management/resource/PersistentLocalResourceRepository.java
4 files changed, 40 insertions(+), 18 deletions(-)
Approvals:
Jenkins: Verified; No violations found; ; Verified
Michael Blow: Looks good to me, approved
diff --git
a/asterixdb/asterix-replication/src/main/java/org/apache/asterix/replication/messaging/PartitionResourcesListTask.java
b/asterixdb/asterix-replication/src/main/java/org/apache/asterix/replication/messaging/PartitionResourcesListTask.java
index 54d3a02..b2b1ad1 100644
---
a/asterixdb/asterix-replication/src/main/java/org/apache/asterix/replication/messaging/PartitionResourcesListTask.java
+++
b/asterixdb/asterix-replication/src/main/java/org/apache/asterix/replication/messaging/PartitionResourcesListTask.java
@@ -26,9 +26,10 @@
import java.util.stream.Collectors;
import org.apache.asterix.common.api.INcApplicationContext;
-import org.apache.asterix.replication.api.IReplicationWorker;
+import org.apache.asterix.common.replication.IReplicationStrategy;
import org.apache.asterix.common.utils.StoragePathUtil;
import org.apache.asterix.replication.api.IReplicaTask;
+import org.apache.asterix.replication.api.IReplicationWorker;
import
org.apache.asterix.transaction.management.resource.PersistentLocalResourceRepository;
import org.apache.hyracks.api.exceptions.HyracksDataException;
@@ -48,8 +49,10 @@
final PersistentLocalResourceRepository localResourceRepository =
(PersistentLocalResourceRepository)
appCtx.getLocalResourceRepository();
localResourceRepository.cleanup(partition);
-final List partitionResources =
localResourceRepository.getPartitionIndexesFiles(partition).stream()
-
.map(StoragePathUtil::getFileRelativePath).collect(Collectors.toList());
+final IReplicationStrategy replicationStrategy =
appCtx.getReplicationManager().getReplicationStrategy();
+final List partitionResources =
+localResourceRepository.getPartitionReplicatedFiles(partition,
replicationStrategy).stream()
+
.map(StoragePathUtil::getFileRelativePath).collect(Collectors.toList());
final PartitionResourcesListResponse response =
new PartitionResourcesListResponse(partition,
partitionResources);
ReplicationProtocol.sendTo(worker.getChannel(), response,
worker.getReusableBuffer());
diff --git
a/asterixdb/asterix-replication/src/main/java/org/apache/asterix/replication/messaging/ReplicationProtocol.java
b/asterixdb/asterix-replication/src/main/java/org/apache/asterix/replication/messaging/ReplicationProtocol.java
index 280a2d4..41e7d9e 100644
---
a/asterixdb/asterix-replication/src/main/java/org/apache/asterix/replication/messaging/ReplicationProtocol.java
+++
b/asterixdb/asterix-replication/src/main/java/org/apache/asterix/replication/messaging/ReplicationProtocol.java
@@ -72,7 +72,7 @@
final ByteBuffer buf = ensureSize(dataBuffer, requestSize);
// read request
NetworkingUtil.readBytes(socketChannel, buf, requestSize);
-return dataBuffer;
+return buf;
}
public static ReplicationRequestType getRequestType(SocketChannel
socketChannel, ByteBuffer byteBuffer)
@@ -135,6 +135,7 @@
requestBuffer.put(outputStream.getByteArray(), 0,
outputStream.getLength());
requestBuffer.flip();
NetworkingUtil.transferBufferToChannel(channel, requestBuffer);
+channel.socket().getOutputStream().flush();
} catch (IOException e) {
throw new ReplicationException(e);
}
@@ -148,9 +149,9 @@
public static IReplicationMessage readMessage(ReplicationRequestType type,
SocketChannel socketChannel,
ByteBuffer buffer) {
try {
-ReplicationProtocol.readRequest(socketChannel, bu
