[
https://issues.apache.org/jira/browse/BEAM-5404?focusedWorklogId=145915&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-145915
]
ASF GitHub Bot logged work on BEAM-5404:
----------------------------------------
Author: ASF GitHub Bot
Created on: 20/Sep/18 03:04
Start Date: 20/Sep/18 03:04
Worklog Time Spent: 10m
Work Description: chamikaramj commented on issue #6407: [BEAM-5404] Use
Java serialization for MutationGroup objects.
URL: https://github.com/apache/beam/pull/6407#issuecomment-423024981
Hi Neil,
I'd like someone from Spanner team who is more familiar with this code to
sign off/review this PR. Can you please ask someone from that team to review
this ?
Also, I suggest adding more specific large scale performance benchmarks
related to this change to https://issues.apache.org/jira/browse/BEAM-5404
before changing the default method of serialization.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
-------------------
Worklog Id: (was: 145915)
Time Spent: 20m (was: 10m)
> Inefficient Serialization of Spanner MutationGroup in pipeline
> --------------------------------------------------------------
>
> Key: BEAM-5404
> URL: https://issues.apache.org/jira/browse/BEAM-5404
> Project: Beam
> Issue Type: Bug
> Components: io-java-gcp
> Affects Versions: 2.3.0, 2.4.0, 2.5.0, 2.6.0
> Reporter: Niel Markwick
> Assignee: Chamikara Jayalath
> Priority: Major
> Labels: pull-request-available
> Time Spent: 20m
> Remaining Estimate: 0h
>
> The Cloud Spanner connector uses a custom serialization mechanism to convert
> MutationGroup objects into a byte array.
> This mechanism is very inefficient producing byte arrays approx 10x larger
> than simple Java Serialization of the MutationGroup objects, which increases
> the resources needed by the connector to ~40x the size of the original
> mutations.
> There are no obvious benefits to using this custom serialization system, as
> the objects are deserialized within the pipeline itself.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
