[
https://issues.apache.org/jira/browse/GIRAPH-1181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16419298#comment-16419298
]
ASF GitHub Bot commented on GIRAPH-1181:
----------------------------------------
Github user yukselakinci commented on a diff in the pull request:
https://github.com/apache/giraph/pull/65#discussion_r178111159
--- Diff:
giraph-core/src/main/java/org/apache/giraph/comm/aggregators/CountingOutputStream.java
---
@@ -36,7 +36,7 @@
* Default constructor
*/
public CountingOutputStream() {
- dataOutput = new ExtendedByteArrayDataOutput();
+ dataOutput = new UnsafeByteArrayOutputStream();
--- End diff --
These changes allow us to write to the underlying buffer without having to
first write to Chunked stream. However, what we write to stream is different
depending on if the temporary Chunked streams is used, since the chunk size is
also written. So we need to be consistent between write and read for kryo
serialization. If the temporary chunk is not used when writing, casting the
byte stream to ordinary ByteArrayInputStream when reading, would cause error
as we assume it used chunked stream. Since we know all our streams derives from
UnsafeByteArray[Input/Outut]Stream, we can safely cast the byte array to
unsafeByteArray stream.
> Changes to ExtendedByteArrayDataOutput
> --------------------------------------
>
> Key: GIRAPH-1181
> URL: https://issues.apache.org/jira/browse/GIRAPH-1181
> Project: Giraph
> Issue Type: Improvement
> Reporter: Yuksel Akinci
> Priority: Major
>
> Adding a new stream type which extends from
> com.esotericsoftware.kryo.io.Input/Output, so that it can be used with kryo
> serializer. It improves the performance of kryo by faster read/writes (unsafe
> IO), and also eliminates the need for interim buffers to convert from
> DataInput and DataOutput.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
