[
https://issues.apache.org/jira/browse/BEAM-4689?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Eugene Kirpichov closed BEAM-4689.
----------------------------------
Resolution: Fixed
Fix Version/s: 2.6.0
> Dataflow cannot deserialize SplittableParDo DoFns
> -------------------------------------------------
>
> Key: BEAM-4689
> URL: https://issues.apache.org/jira/browse/BEAM-4689
> Project: Beam
> Issue Type: New Feature
> Components: runner-dataflow
> Reporter: Kenneth Knowles
> Assignee: Eugene Kirpichov
> Priority: Blocker
> Fix For: 2.6.0
>
> Time Spent: 2h
> Remaining Estimate: 0h
>
> The Dataflow postcommit is broken in a way that seems real and user-impacting:
> https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_Gradle/lastCompletedBuild/testReport/junit/org.apache.beam.sdk.transforms/SplittableDoFnTest/testSideInput/
> {code}
> Caused by: java.lang.IllegalArgumentException: unable to deserialize
> Serialized DoFnInfo
> ...
> Caused by: java.io.InvalidClassException:
> org.apache.beam.runners.core.construction.SplittableParDo$RandomUniqueKeyFn;
> local class incompatible: stream classdesc serialVersionUID =
> 6068396661487412884, local class serialVersionUID = -617521663543732196
> {code}
> This means that the worker is using a version of the class from its own
> classpath, not the version from the user's staged pipeline. It implies that
> the worker is not shading runners-core-construction. Because that is where a
> ton of utility DoFns live, it is critical that it be shaded.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
