adoroszlai opened a new pull request #87: HDDS-2273. Avoid buffer copying in GrpcReplicationService URL: https://github.com/apache/hadoop-ozone/pull/87 ## What changes were proposed in this pull request? Use `ByteString.Output` stream instead of `ByteArrayOutputStream`. Its initial size is configured to 1MB (same as previous buffer size), and is flushed when that size is reached. This helps to avoid allocating multiple buffers as well as buffer copy when converting to `ByteString`. https://issues.apache.org/jira/browse/HDDS-2273 ## How was this patch tested? Tested closed container replication manually with a 300MB container. Verified that container is correctly replicated to other datanode. Also verified that flush happens when buffer is full. ``` datanode_1 | - Streaming container data (1) to other datanode datanode_1 | - Sending 1048576 bytes (of type LiteralByteString) for container 1 datanode_1 | - Sending 530637 bytes (of type LiteralByteString) for container 1 datanode_1 | - 1579213 bytes written to the rpc stream from container 1 ... datanode_5 | - Container is downloaded to /tmp/container-copy/container-1.tar.gz datanode_5 | - Container 1 is downloaded, starting to import. datanode_5 | - Container 1 is replicated successfully datanode_5 | - Container 1 is replicated. ```
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
