lamber-ken commented on a change in pull request #1352: [HUDI-625] Fixing
performance issues around DiskBasedMap & kryo
URL: https://github.com/apache/incubator-hudi/pull/1352#discussion_r383610437
##########
File path:
hudi-common/src/main/java/org/apache/hudi/common/util/SerializationUtils.java
##########
@@ -121,50 +116,16 @@ Object deserialize(byte[] objectData) {
public Kryo newKryo() {
- Kryo kryo = new KryoBase();
+ Kryo kryo = new Kryo();
// ensure that kryo doesn't fail if classes are not registered with kryo.
kryo.setRegistrationRequired(false);
// This would be used for object initialization if nothing else works
out.
- kryo.setInstantiatorStrategy(new
org.objenesis.strategy.StdInstantiatorStrategy());
+ kryo.setInstantiatorStrategy(new Kryo.DefaultInstantiatorStrategy(new
StdInstantiatorStrategy()));
// Handle cases where we may have an odd classloader setup like with
libjars
// for hadoop
kryo.setClassLoader(Thread.currentThread().getContextClassLoader());
return kryo;
}
- private static class KryoBase extends Kryo {
- @Override
- protected Serializer newDefaultSerializer(Class type) {
- final Serializer serializer = super.newDefaultSerializer(type);
- if (serializer instanceof FieldSerializer) {
- final FieldSerializer fieldSerializer = (FieldSerializer) serializer;
- fieldSerializer.setIgnoreSyntheticFields(true);
- }
- return serializer;
- }
-
- @Override
- protected ObjectInstantiator newInstantiator(Class type) {
- return () -> {
- // First try reflectasm - it is fastest way to instantiate an object.
- try {
- final ConstructorAccess access = ConstructorAccess.get(type);
- return access.newInstance();
- } catch (Throwable t) {
- // ignore this exception. We may want to try other way.
- }
- // fall back to java based instantiation.
- try {
- final Constructor constructor = type.getConstructor();
- constructor.setAccessible(true);
- return constructor.newInstance();
- } catch (NoSuchMethodException | IllegalAccessException |
InstantiationException
- | InvocationTargetException e) {
- // ignore this exception. we will fall back to default
instantiation strategy.
- }
- return
super.getInstantiatorStrategy().newInstantiatorOf(type).newInstance();
Review comment:
Get your point, these serializers are not necessary now.
1. Custom serializers are need when there are special requirements for
serialization and deserialization.
2. We use `InstantiatorStrategy`, which is also efficient. kryo-creation [1]
```
kryo.setInstantiatorStrategy(new DefaultInstantiatorStrategy(new
StdInstantiatorStrategy()));
```
3. If we do that currently, it will affect many places.
[1] https://github.com/EsotericSoftware/kryo#object-creation
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
