[ https://issues.apache.org/jira/browse/BEAM-4689?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Eugene Kirpichov closed BEAM-4689. ---------------------------------- Resolution: Fixed Fix Version/s: 2.6.0 > Dataflow cannot deserialize SplittableParDo DoFns > ------------------------------------------------- > > Key: BEAM-4689 > URL: https://issues.apache.org/jira/browse/BEAM-4689 > Project: Beam > Issue Type: New Feature > Components: runner-dataflow > Reporter: Kenneth Knowles > Assignee: Eugene Kirpichov > Priority: Blocker > Fix For: 2.6.0 > > Time Spent: 2h > Remaining Estimate: 0h > > The Dataflow postcommit is broken in a way that seems real and user-impacting: > https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_Gradle/lastCompletedBuild/testReport/junit/org.apache.beam.sdk.transforms/SplittableDoFnTest/testSideInput/ > {code} > Caused by: java.lang.IllegalArgumentException: unable to deserialize > Serialized DoFnInfo > ... > Caused by: java.io.InvalidClassException: > org.apache.beam.runners.core.construction.SplittableParDo$RandomUniqueKeyFn; > local class incompatible: stream classdesc serialVersionUID = > 6068396661487412884, local class serialVersionUID = -617521663543732196 > {code} > This means that the worker is using a version of the class from its own > classpath, not the version from the user's staged pipeline. It implies that > the worker is not shading runners-core-construction. Because that is where a > ton of utility DoFns live, it is critical that it be shaded. -- This message was sent by Atlassian JIRA (v7.6.3#76005)