Niel Markwick created BEAM-5404:
-----------------------------------
Summary: Inefficient Serialization of Spanner MutationGroup in
pipeline
Key: BEAM-5404
URL: https://issues.apache.org/jira/browse/BEAM-5404
Project: Beam
Issue Type: Bug
Components: io-java-gcp
Affects Versions: 2.6.0, 2.5.0, 2.4.0, 2.3.0
Reporter: Niel Markwick
Assignee: Chamikara Jayalath
The Cloud Spanner connector uses a custom serialization mechanism to convert
MutationGroup objects into a byte array.
This mechanism is very inefficient producing byte arrays approx 10x larger than
simple Java Serialization of the MutationGroup objects, which increases the
resources needed by the connector to ~40x the size of the original mutations.
There are no obvious benefits to using this custom serialization system, as the
objects are deserialized within the pipeline itself.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
