[
https://issues.apache.org/jira/browse/BEAM-7138?focusedWorklogId=236047&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-236047
]
ASF GitHub Bot logged work on BEAM-7138:
----------------------------------------
Author: ASF GitHub Bot
Created on: 01/May/19 23:13
Start Date: 01/May/19 23:13
Worklog Time Spent: 10m
Work Description: lukecwik commented on pull request #8396: [BEAM-7138]
keep Java serialized coder in wire coder construction
URL: https://github.com/apache/beam/pull/8396#discussion_r280248427
##########
File path:
runners/java-fn-execution/src/main/java/org/apache/beam/runners/fnexecution/wire/LengthPrefixUnknownCoders.java
##########
@@ -55,12 +56,16 @@ public static String addLengthPrefixedCoder(
// request to replace the coder with a byte array coder.
// 2) the requested coder is a known coder but not a length prefix coder.
In this case we
// rebuild the coder by recursively length prefixing any unknown
component coders.
- // 3) the requested coder is an unknown coder. In this case we either
wrap the requested coder
+ // 3) the requested coder is a serialized Java coder. In this case we
wrap the coder with
+ // a length prefix coder.
+ // 4) the requested coder is an unknown coder. In this case we either
wrap the requested coder
// with a length prefix coder or replace it with a length prefix byte
array coder.
if (ModelCoders.LENGTH_PREFIX_CODER_URN.equals(urn)) {
return replaceWithByteArrayCoder ? lengthPrefixedByteArrayCoderId :
coderId;
} else if (ModelCoders.urns().contains(urn)) {
return addForModelCoder(coderId, components, replaceWithByteArrayCoder);
+ } else if (CoderTranslation.JAVA_SERIALIZED_CODER_URN.equals(urn)) {
Review comment:
This isn't what your looking for. Runners and SDKs have a duality in the
coder representation. See [Encoding and Decoding Unknown Types](This isn't the
correct place to fix this. It is likely that a portable runner isn't storing
"two" coder representations, one for itself and one for the SDK. See [Encoding
and Decoding Unknown
Types](https://docs.google.com/document/d/1IGduUqmhWDi_69l9nG8kw73HZ5WI5wOps9Tshl5wpQA/edit#heading=h.sh4d5klmtfis).
The SDK may want to represent a type `Foo` but the runner can't encode or
decode this type. The runner instead uses a representation which has the same
wire encoding by using `LengthPrefix<Byte[]>` for its coder while the SDK would
be told to use a `LengthPrefix<Foo>`. Both have the same wire representation.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 236047)
Time Spent: 1h 50m (was: 1h 40m)
> keep Java serialized coder in length-prefixed wire coder construction
> ---------------------------------------------------------------------
>
> Key: BEAM-7138
> URL: https://issues.apache.org/jira/browse/BEAM-7138
> Project: Beam
> Issue Type: Improvement
> Components: java-fn-execution
> Reporter: Heejong Lee
> Assignee: Heejong Lee
> Priority: Major
> Time Spent: 1h 50m
> Remaining Estimate: 0h
>
> don't replace Java serialized coder with byte array coder in length-prefixed
> wire coder construction.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
