nastra commented on code in PR #8032:
URL: https://github.com/apache/iceberg/pull/8032#discussion_r1288655052
##########
core/src/main/java/org/apache/iceberg/rest/RESTMetricsReporter.java:
##########
@@ -58,4 +60,11 @@ public void report(MetricsReport report) {
LOG.warn("Failed to report metrics to REST endpoint {}",
metricsEndpoint, e);
}
}
+
+ Object writeReplace() {
+ // fetch the latest headers from the AuthSession and carry them over in a
separate supplier so
+ // that AuthSession doesn't have to be Serializable
Review Comment:
we need to change this for Kryo ser/de, otherwise it will fail with the
below error. This is because we're passing a Lambda with the headers from
`AuthSession` and Kryo ser/des everything in the scope of that Lambda
```
headers (org.apache.iceberg.rest.RESTMetricsReporter)
reporters
(org.apache.iceberg.metrics.MetricsReporters$CompositeMetricsReporter)
metricsReporter (org.apache.iceberg.SerializableTable)
com.esotericsoftware.kryo.KryoException: java.lang.RuntimeException: Could
not serialize lambda
Serialization trace:
headers (org.apache.iceberg.rest.RESTMetricsReporter)
reporters
(org.apache.iceberg.metrics.MetricsReporters$CompositeMetricsReporter)
metricsReporter (org.apache.iceberg.SerializableTable)
at
com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:144)
at
com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:543)
at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:813)
at
com.esotericsoftware.kryo.serializers.CollectionSerializer.read(CollectionSerializer.java:134)
at
com.esotericsoftware.kryo.serializers.CollectionSerializer.read(CollectionSerializer.java:40)
at com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:731)
at
com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:125)
at
com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:543)
at com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:731)
at
com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:125)
at
com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:543)
at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:813)
at
org.apache.iceberg.TestHelpers$KryoHelpers.roundTripSerialize(TestHelpers.java:294)
at
org.apache.iceberg.rest.TestRESTCatalog.testScanReportingOnSerializableTable(TestRESTCatalog.java:2033)
....
Caused by: java.lang.RuntimeException: Could not serialize lambda
at
com.esotericsoftware.kryo.serializers.ClosureSerializer.read(ClosureSerializer.java:78)
at com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:731)
at
com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:125)
... 97 more
Caused by: com.esotericsoftware.kryo.KryoException:
java.lang.UnsupportedOperationException
Serialization trace:
headers (org.apache.iceberg.rest.auth.OAuth2Util$AuthSession)
capturedArgs (java.lang.invoke.SerializedLambda)
at
com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:144)
at
com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:543)
at com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:731)
at
com.esotericsoftware.kryo.serializers.DefaultArraySerializers$ObjectArraySerializer.read(DefaultArraySerializers.java:391)
at
com.esotericsoftware.kryo.serializers.DefaultArraySerializers$ObjectArraySerializer.read(DefaultArraySerializers.java:302)
at com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:731)
at
com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:125)
at
com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:543)
at com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:709)
at
com.esotericsoftware.kryo.serializers.ClosureSerializer.read(ClosureSerializer.java:75)
... 99 more
Caused by: java.lang.UnsupportedOperationException
at
org.apache.iceberg.relocated.com.google.common.collect.ImmutableMap.put(ImmutableMap.java:781)
at
com.esotericsoftware.kryo.serializers.MapSerializer.read(MapSerializer.java:162)
at
com.esotericsoftware.kryo.serializers.MapSerializer.read(MapSerializer.java:39)
at com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:731)
at
com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:125)
```
For standard Java ser/de we're relying on the `writeReplace()` method. I've
updated the comment to make it clear that this is only for standard Java ser/de
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
