Hi Juan, it seems that has been introduce by the metrics layer in the core runner API.
Let me check. Regards JB On 30/07/2018 14:47, Juan Carlos Garcia wrote: > Bump! > > Does any of the core-dev roam around here? > > Can someone provide a feedback about BEAM-4597 > <https://issues.apache.org/jira/browse/BEAM-4597> > > Thanks and regards, > > On Thu, Jul 19, 2018 at 3:41 PM, Juan Carlos Garcia <jcgarc...@gmail.com > <mailto:jcgarc...@gmail.com>> wrote: > > Folks, > > Its someone using the SparkRunner out there with the Spark > KryoSerializer ? > > We are being force to use the not so efficient 'JavaSerializer' with > Spark because we face the following exception: > > <exception> > Exception in thread "main" java.lang.RuntimeException: > org.apache.spark.SparkException: Job aborted due to stage failure: > Exception while getting task result: > com.esotericsoftware.kryo.KryoException: Unable to find class: > > org.apache.beam.runners.core.metrics.MetricsContainerImpl$$Lambda$31/1875283985 > Serialization trace: > factory (org.apache.beam.runners.core.metrics.MetricsMap) > counters (org.apache.beam.runners.core.metrics.MetricsContainerImpl) > metricsContainers > (org.apache.beam.runners.core.metrics.MetricsContainerStepMap) > metricsContainers > (org.apache.beam.runners.spark.io.SparkUnboundedSource$Metadata) > at > > org.apache.beam.runners.spark.SparkPipelineResult.runtimeExceptionFrom(SparkPipelineResult.java:55) > at > > org.apache.beam.runners.spark.SparkPipelineResult.beamExceptionFrom(SparkPipelineResult.java:72) > at > > org.apache.beam.runners.spark.SparkPipelineResult.access$000(SparkPipelineResult.java:41) > at > > org.apache.beam.runners.spark.SparkPipelineResult$StreamingMode.stop(SparkPipelineResult.java:163) > at > > org.apache.beam.runners.spark.SparkPipelineResult.offerNewState(SparkPipelineResult.java:198) > at > > org.apache.beam.runners.spark.SparkPipelineResult.waitUntilFinish(SparkPipelineResult.java:101) > at > > org.apache.beam.runners.spark.SparkPipelineResult.waitUntilFinish(SparkPipelineResult.java:87) > at > > org.apache.beam.examples.BugWithKryoOnSpark.main(BugWithKryoOnSpark.java:75) > </exception> > > I created a jira ticket and attached a project example on it, > https://issues.apache.org/jira/browse/BEAM-4597 > <https://issues.apache.org/jira/browse/BEAM-4597> > > Any feedback is appreciated. > > -- > > JC > > > > > -- > > JC > -- Jean-Baptiste Onofré jbono...@apache.org http://blog.nanthrax.net Talend - http://www.talend.com
