[jira] [Commented] (AVRO-2056) DirectBinaryEncoder Creates Buffer For Each Call To writeDouble
[
https://issues.apache.org/jira/browse/AVRO-2056?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16104567#comment-16104567
]
ASF subversion and git services commented on AVRO-2056:
---
Commit 3801888fa8c9dc96a1613e41cbc33800bc84f1b4 in avro's branch
refs/heads/master from [~belugabehr]
[ https://git-wip-us.apache.org/repos/asf?p=avro.git;h=3801888 ]
AVRO-2056: DirectBinaryEncoder Creates Buffer For Each Call To writeDouble
> DirectBinaryEncoder Creates Buffer For Each Call To writeDouble
> ---
>
> Key: AVRO-2056
> URL: https://issues.apache.org/jira/browse/AVRO-2056
> Project: Avro
> Issue Type: Improvement
> Components: java
>Affects Versions: 1.7.7, 1.8.2
>Reporter: BELUGA BEHR
>Assignee: BELUGA BEHR
>Priority: Minor
> Attachments: AVRO-2056.1.patch
>
>
> Each call to {{writeDouble}} creates a new buffer and promptly throws it away
> even though the class has a re-usable buffer and is used in other methods
> such as {{writeFloat}}. Remove this extra buffer.
> {code:title=org.apache.avro.io.DirectBinaryEncoder}
> // the buffer is used for writing floats, doubles, and large longs.
> private final byte[] buf = new byte[12];
> @Override
> public void writeFloat(float f) throws IOException {
> int len = BinaryData.encodeFloat(f, buf, 0);
> out.write(buf, 0, len);
> }
> @Override
> public void writeDouble(double d) throws IOException {
> byte[] buf = new byte[8];
> int len = BinaryData.encodeDouble(d, buf, 0);
> out.write(buf, 0, len);
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (AVRO-2056) DirectBinaryEncoder Creates Buffer For Each Call To writeDouble
[
https://issues.apache.org/jira/browse/AVRO-2056?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16102845#comment-16102845
]
Gabor Szadovszky commented on AVRO-2056:
[~belugabehr], I've took a look on Perf and I see how it works now. So, there
is no relevant change there, it is more for testing your stuff offline like in
an IDE...
+1
> DirectBinaryEncoder Creates Buffer For Each Call To writeDouble
> ---
>
> Key: AVRO-2056
> URL: https://issues.apache.org/jira/browse/AVRO-2056
> Project: Avro
> Issue Type: Improvement
> Components: java
>Affects Versions: 1.7.7, 1.8.2
>Reporter: BELUGA BEHR
>Assignee: BELUGA BEHR
>Priority: Minor
> Attachments: AVRO-2056.1.patch
>
>
> Each call to {{writeDouble}} creates a new buffer and promptly throws it away
> even though the class has a re-usable buffer and is used in other methods
> such as {{writeFloat}}. Remove this extra buffer.
> {code:title=org.apache.avro.io.DirectBinaryEncoder}
> // the buffer is used for writing floats, doubles, and large longs.
> private final byte[] buf = new byte[12];
> @Override
> public void writeFloat(float f) throws IOException {
> int len = BinaryData.encodeFloat(f, buf, 0);
> out.write(buf, 0, len);
> }
> @Override
> public void writeDouble(double d) throws IOException {
> byte[] buf = new byte[8];
> int len = BinaryData.encodeDouble(d, buf, 0);
> out.write(buf, 0, len);
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (AVRO-2056) DirectBinaryEncoder Creates Buffer For Each Call To writeDouble
[
https://issues.apache.org/jira/browse/AVRO-2056?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16101805#comment-16101805
]
BELUGA BEHR commented on AVRO-2056:
---
[~gszadovszky] I'm sorry, but I do not understand your request to "add the Perf
change." I did not change the Perf test to accomplish the testing other than
un-comment the direct binary encoder:
{code}
Encoder e = encoder_factory.binaryEncoder(out, null);
//Encoder e = encoder_factory.directBinaryEncoder(out, null);
{code}
> DirectBinaryEncoder Creates Buffer For Each Call To writeDouble
> ---
>
> Key: AVRO-2056
> URL: https://issues.apache.org/jira/browse/AVRO-2056
> Project: Avro
> Issue Type: Improvement
> Components: java
>Affects Versions: 1.7.7, 1.8.2
>Reporter: BELUGA BEHR
>Assignee: BELUGA BEHR
>Priority: Minor
> Attachments: AVRO-2056.1.patch
>
>
> Each call to {{writeDouble}} creates a new buffer and promptly throws it away
> even though the class has a re-usable buffer and is used in other methods
> such as {{writeFloat}}. Remove this extra buffer.
> {code:title=org.apache.avro.io.DirectBinaryEncoder}
> // the buffer is used for writing floats, doubles, and large longs.
> private final byte[] buf = new byte[12];
> @Override
> public void writeFloat(float f) throws IOException {
> int len = BinaryData.encodeFloat(f, buf, 0);
> out.write(buf, 0, len);
> }
> @Override
> public void writeDouble(double d) throws IOException {
> byte[] buf = new byte[8];
> int len = BinaryData.encodeDouble(d, buf, 0);
> out.write(buf, 0, len);
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (AVRO-2056) DirectBinaryEncoder Creates Buffer For Each Call To writeDouble
[
https://issues.apache.org/jira/browse/AVRO-2056?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16096959#comment-16096959
]
BELUGA BEHR commented on AVRO-2056:
---
With the patch applied, writing Doubles is between 1% and 2% faster and has the
added benefit of cutting down on garbage collection
{code}
# Buffer Removed
DoubleWrite: 2762 ms 72.407 579.258 200
DoubleWrite: 2786 ms 71.787 574.293 200
DoubleWrite: 2755 ms 72.570 580.561 200
# Buffer Present
DoubleWrite: 2822 ms 70.871 566.965 200
DoubleWrite: 2830 ms 70.667 565.336 200
DoubleWrite: 2807 ms 71.230 569.842 200
{code}
> DirectBinaryEncoder Creates Buffer For Each Call To writeDouble
> ---
>
> Key: AVRO-2056
> URL: https://issues.apache.org/jira/browse/AVRO-2056
> Project: Avro
> Issue Type: Improvement
> Components: java
>Affects Versions: 1.7.7, 1.8.2
>Reporter: BELUGA BEHR
>Assignee: BELUGA BEHR
>Priority: Minor
> Attachments: AVRO-2056.1.patch
>
>
> Each call to {{writeDouble}} creates a new buffer and promptly throws it away
> even though the class has a re-usable buffer and is used in other methods
> such as {{writeFloat}}. Remove this extra buffer.
> {code:title=org.apache.avro.io.DirectBinaryEncoder}
> // the buffer is used for writing floats, doubles, and large longs.
> private final byte[] buf = new byte[12];
> @Override
> public void writeFloat(float f) throws IOException {
> int len = BinaryData.encodeFloat(f, buf, 0);
> out.write(buf, 0, len);
> }
> @Override
> public void writeDouble(double d) throws IOException {
> byte[] buf = new byte[8];
> int len = BinaryData.encodeDouble(d, buf, 0);
> out.write(buf, 0, len);
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (AVRO-2056) DirectBinaryEncoder Creates Buffer For Each Call To writeDouble
[
https://issues.apache.org/jira/browse/AVRO-2056?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16092208#comment-16092208
]
Doug Cutting commented on AVRO-2056:
This may or may not be a mistake. Perhaps the JVM is smart enough to realize
that this array does not escape and stack allocates it or something. We should
benchmark this with Perf.java before changing it. (Perf.java currently only
tests BufferedBinaryEncoder, but it's a one-line change to get it to instead
benchmark DirectBinaryEncoder.) These encoder & decoder methods are among the
most performance-critical parts of Avro and should not be altered without
benchmarking.
> DirectBinaryEncoder Creates Buffer For Each Call To writeDouble
> ---
>
> Key: AVRO-2056
> URL: https://issues.apache.org/jira/browse/AVRO-2056
> Project: Avro
> Issue Type: Improvement
> Components: java
>Affects Versions: 1.7.7, 1.8.2
>Reporter: BELUGA BEHR
>Assignee: BELUGA BEHR
>Priority: Minor
> Attachments: AVRO-2056.1.patch
>
>
> Each call to {{writeDouble}} creates a new buffer and promptly throws it away
> even though the class has a re-usable buffer and is used in other methods
> such as {{writeFloat}}. Remove this extra buffer.
> {code:title=org.apache.avro.io.DirectBinaryEncoder}
> // the buffer is used for writing floats, doubles, and large longs.
> private final byte[] buf = new byte[12];
> @Override
> public void writeFloat(float f) throws IOException {
> int len = BinaryData.encodeFloat(f, buf, 0);
> out.write(buf, 0, len);
> }
> @Override
> public void writeDouble(double d) throws IOException {
> byte[] buf = new byte[8];
> int len = BinaryData.encodeDouble(d, buf, 0);
> out.write(buf, 0, len);
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
