[
https://issues.apache.org/jira/browse/AVRO-1282?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13642048#comment-13642048
]
Leo Romanoff commented on AVRO-1282:
------------------------------------
> We probably should not cache any accessors for GenericRecords (or any
> IndexedRecord for that matter) since they have a fast
> setter / getter already. I'm trying out some changes to my patch to
> incorporate that.
+1 for this. I also came to the same conclusion earlier today. And it took just
2-3 lines to fix it.
Your idea about avoiding iterators is also nice.
Regarding compound keys:
FYI, I experimented with something like this, where I used class or schema as a
key, depending on what is provided as parameters to getOrCreateFieldsArray. It
does not use compound keys and it passes all tests in Avro and IPC, but it
could be that it still does not cover everything.
private static FieldAccessor[] getOrCreateFieldsArray(Class c, Schema schema)
{
Map<Object, FieldAccessor[]> cache = FIELDS_ARRAY_CACHE.get();
Object key = (schema !=null && IndexedRecord.class.isAssignableFrom(c)) ?
schema : c;
FieldAccessor[] fields = cache.get(key);
if (fields == null) {
if (schema == null)
throw new AvroRuntimeException("No schema is known for " + c);
fields = new FieldAccessor[schema.getFields().size()];
cache.put(key, fields);
if (schema != null) {
if(key instanceof Class) {
for (Schema.Field f: schema.getFields()) {
getFieldAccessor(c, f.pos(), f.name());
}
} else {
for (Schema.Field f: schema.getFields()) {
fields [f.pos()] = new GenericRecordAccessor(f.name());
}
}
}
}
return fields;
}
> Make use of the sun.misc.Unsafe class during serialization if a JDK supports
> it
> -------------------------------------------------------------------------------
>
> Key: AVRO-1282
> URL: https://issues.apache.org/jira/browse/AVRO-1282
> Project: Avro
> Issue Type: Improvement
> Components: java
> Affects Versions: 1.7.4
> Reporter: Leo Romanoff
> Priority: Minor
> Attachments: AVRO-1282-s1.patch, avro-1282-v1.patch,
> avro-1282-v2.patch, avro-1282-v3.patch, avro-1282-v4.patch,
> avro-1282-v5.patch, avro-1282-v6.patch, avro-1282-v7.patch,
> avro-1282-v8.patch, TestUnsafeUtil.java
>
>
> Unsafe can be used to significantly speed up serialization process, if a JDK
> implementation supports java.misc.Unsafe properly. Most JDKs running on PCs
> support it. Some platforms like Android lack a proper support for Unsafe yet.
> There are two possibilities to use Unsafe for serialization:
> 1) Very quick access to the fields of objects. It is way faster than with the
> reflection-based approach using Field.get/set
> 2) Input and Output streams can be using Unsafe to perform very quick
> input/output.
>
> 3) More over, Unsafe makes it possible to serialize to/deserialize from
> off-heap memory directly and very quickly, without any intermediate buffers
> allocated on heap. There is virtually no overhead compared to the usual byte
> arrays.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
