[
https://issues.apache.org/jira/browse/AVRO-1282?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13644024#comment-13644024
]
Leo Romanoff commented on AVRO-1282:
------------------------------------
Scott: I tested with your latest patch and confirm that it brings the
performance back! Very well done. I attach a slightly modified version of your
patch, where I added two more tests to Perf.java. These are the
ReflectFloatTest and ReflectDoubleTest and they do the same as FloatTest and
DoubleTest, but using reflection. If you look at the "M bytes/sec" figures
you'd see that write speed for arrays using reflection is almost the same as
without reflection, which is very good. But reading speed is very slow compared
to non-reflection case, as we know.
Overall, I see speed improvements with factors 3x or 4x for structures and
complex types and much better improvements for arrays, e.g. sometimes 10x or
20x.
With this in mind, I think we are ready for inclusion into a trunk.
> There is still one fundamental flaw: reading blocked encoding is not
> supported (it would trigger an array bounds check
> exception). I have isolated the code that loops and writes arrays, so we can
> add that more easily from here.
I'm not sure if blocked encoding ever worked for Java. If required we can add
it later as you say.
Overall, it was a great joint effort! Thank you very much Scott!!!
> Make use of the sun.misc.Unsafe class during serialization if a JDK supports
> it
> -------------------------------------------------------------------------------
>
> Key: AVRO-1282
> URL: https://issues.apache.org/jira/browse/AVRO-1282
> Project: Avro
> Issue Type: Improvement
> Components: java
> Affects Versions: 1.7.4
> Reporter: Leo Romanoff
> Priority: Minor
> Attachments: AVRO-1282-s1.patch, AVRO-1282-s2.patch,
> AVRO-1282-s3.patch, AVRO-1282-s5.patch, avro-1282-v1.patch,
> avro-1282-v2.patch, avro-1282-v3.patch, avro-1282-v4.patch,
> avro-1282-v5.patch, avro-1282-v6.patch, avro-1282-v7.patch,
> avro-1282-v8.patch, TestUnsafeUtil.java
>
>
> Unsafe can be used to significantly speed up serialization process, if a JDK
> implementation supports java.misc.Unsafe properly. Most JDKs running on PCs
> support it. Some platforms like Android lack a proper support for Unsafe yet.
> There are two possibilities to use Unsafe for serialization:
> 1) Very quick access to the fields of objects. It is way faster than with the
> reflection-based approach using Field.get/set
> 2) Input and Output streams can be using Unsafe to perform very quick
> input/output.
>
> 3) More over, Unsafe makes it possible to serialize to/deserialize from
> off-heap memory directly and very quickly, without any intermediate buffers
> allocated on heap. There is virtually no overhead compared to the usual byte
> arrays.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
