I missed the view part ;) Thanks ;)
Regards
JB
On 05/30/2017 07:18 PM, Aviem Zur wrote:
Hi Antony,
Unbounded views are not currently supported in the Spark runner. The following
ticket tracks the progress for adding support for this:
https://issues.apache.org/jira/browse/BEAM-2112
On Tue, May 30, 2017 at 7:11 PM Jean-Baptiste Onofré <[email protected]
<mailto:[email protected]>> wrote:
Hi Antony,
which version of the Spark runner are you using ?
Regards
JB
On 05/30/2017 06:04 PM, Antony Mayi wrote:
> Hi,
>
> started getting following error when running on spark. can anyone see
what I
> could be doing wrong? It only happens with unbounded source. Same
pipeline with
> bounded source runs fine.
>
> thx,
> a.
>
> java.lang.IllegalStateException: No TransformEvaluator registered for
UNBOUNDED
> transform class org.apache.beam.sdk.transforms.View$CreatePCollectionView
> at
>
org.apache.beam.runners.spark.repackaged.com.google.common.base.Preconditions.checkState(Preconditions.java:518)
> at
>
org.apache.beam.runners.spark.translation.streaming.StreamingTransformTranslator$Translator.translateUnbounded(StreamingTransformTranslator.java:529)
> at
>
org.apache.beam.runners.spark.SparkRunner$Evaluator.translate(SparkRunner.java:435)
> at
>
org.apache.beam.runners.spark.SparkRunner$Evaluator.doVisitTransform(SparkRunner.java:405)
> at
>
org.apache.beam.runners.spark.SparkRunner$Evaluator.visitPrimitiveTransform(SparkRunner.java:395)
> at
>
org.apache.beam.sdk.runners.TransformHierarchy$Node.visit(TransformHierarchy.java:491)
> at
>
org.apache.beam.sdk.runners.TransformHierarchy$Node.visit(TransformHierarchy.java:486)
> at
>
org.apache.beam.sdk.runners.TransformHierarchy$Node.visit(TransformHierarchy.java:486)
> at
>
org.apache.beam.sdk.runners.TransformHierarchy$Node.visit(TransformHierarchy.java:486)
> at
>
org.apache.beam.sdk.runners.TransformHierarchy$Node.visit(TransformHierarchy.java:486)
> at
>
org.apache.beam.sdk.runners.TransformHierarchy$Node.access$400(TransformHierarchy.java:235)
> at
org.apache.beam.sdk.runners.TransformHierarchy.visit(TransformHierarchy.java:210)
> at org.apache.beam.sdk.Pipeline.traverseTopologically(Pipeline.java:409)
> at
>
org.apache.beam.runners.spark.translation.streaming.SparkRunnerStreamingContextFactory.call(SparkRunnerStreamingContextFactory.java:88)
> at
>
org.apache.beam.runners.spark.translation.streaming.SparkRunnerStreamingContextFactory.call(SparkRunnerStreamingContextFactory.java:47)
> at
>
org.apache.spark.streaming.api.java.JavaStreamingContext$$anonfun$10.apply(JavaStreamingContext.scala:776)
> at
>
org.apache.spark.streaming.api.java.JavaStreamingContext$$anonfun$10.apply(JavaStreamingContext.scala:775)
> at scala.Option.getOrElse(Option.scala:120)
> at
>
org.apache.spark.streaming.StreamingContext$.getOrCreate(StreamingContext.scala:864)
> at
>
org.apache.spark.streaming.api.java.JavaStreamingContext$.getOrCreate(JavaStreamingContext.scala:775)
> at
>
org.apache.spark.streaming.api.java.JavaStreamingContext.getOrCreate(JavaStreamingContext.scala)
> at org.apache.beam.runners.spark.SparkRunner.run(SparkRunner.java:155)
> at org.apache.beam.runners.spark.SparkRunner.run(SparkRunner.java:85)
> at org.apache.beam.sdk.Pipeline.run(Pipeline.java:295)
> at org.apache.beam.sdk.Pipeline.run(Pipeline.java:281)
>
--
Jean-Baptiste Onofré
[email protected] <mailto:[email protected]>
http://blog.nanthrax.net
Talend - http://www.talend.com
--
Jean-Baptiste Onofré
[email protected]
http://blog.nanthrax.net
Talend - http://www.talend.com
