[ 
https://issues.apache.org/jira/browse/SPARK-2507?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tathagata Das resolved SPARK-2507.
----------------------------------

    Resolution: Fixed

> Compile error of streaming project with 2.0.0-cdh4.6.0
> ------------------------------------------------------
>
>                 Key: SPARK-2507
>                 URL: https://issues.apache.org/jira/browse/SPARK-2507
>             Project: Spark
>          Issue Type: Bug
>          Components: Streaming
>    Affects Versions: 0.9.0, 0.9.1, 1.0.0, 1.0.1, 1.0.2
>         Environment: RedHat 5.3
> 2.0.0-cdh4.6.0
> enable yarn
> java version "1.6.0_45"
>            Reporter: James Z.M. Gao
>            Priority: Minor
>
> Hi,
> When compiling with
> {quote}
>     ./make-distribution.sh --hadoop 2.0.0-cdh4.6.0 --with-yarn --tgz
> {quote}
> I have the following errors on streaming java api:
> {quote}
> Version is 0.9.0-incubating
> Making spark-0.9.0-incubating-hadoop_2.0.0-cdh4.6.0-bin.tar.gz
> Hadoop version set to 2.0.0-cdh4.6.0
> YARN enabled
> [info] Loading project definition from /root/spark-source/project/project
> [info] Loading project definition from /root/spark-source/project
> [info] Set current project to root (in build file:/root/spark-source/)
> [info] Compiling 1 Scala source to 
> /root/spark-source/streaming/target/scala-2.10/classes...
> [error] 
> /root/spark-source/streaming/src/main/scala/org/apache/spark/streaming/api/java/JavaPairDStream.scala:57:
>  type mismatch;
> [error]  found   : org.apache.spark.streaming.dstream.DStream[(K, V)]
> [error]  required: org.apache.spark.streaming.api.java.JavaPairDStream[K,V]
> [error]  Note: implicit method fromPairDStream is not applicable here because 
> it comes after the application point and it lacks an explicit result type
> [error]     dstream.filter((x => f(x).booleanValue()))
> [error]                   ^
> [error] 
> /root/spark-source/streaming/src/main/scala/org/apache/spark/streaming/api/java/JavaPairDStream.scala:60:
>  type mismatch;
> [error]  found   : org.apache.spark.streaming.dstream.DStream[(K, V)]
> [error]  required: org.apache.spark.streaming.api.java.JavaPairDStream[K,V]
> [error]  Note: implicit method fromPairDStream is not applicable here because 
> it comes after the application point and it lacks an explicit result type
> [error]   def cache(): JavaPairDStream[K, V] = dstream.cache()
> [error]                                                     ^
> [error] 
> /root/spark-source/streaming/src/main/scala/org/apache/spark/streaming/api/java/JavaPairDStream.scala:63:
>  type mismatch;
> [error]  found   : org.apache.spark.streaming.dstream.DStream[(K, V)]
> [error]  required: org.apache.spark.streaming.api.java.JavaPairDStream[K,V]
> [error]  Note: implicit method fromPairDStream is not applicable here because 
> it comes after the application point and it lacks an explicit result type
> [error]   def persist(): JavaPairDStream[K, V] = dstream.persist()
> [error]                                                         ^
> ......
> [error] 
> /root/spark-source/streaming/src/main/scala/org/apache/spark/streaming/api/java/JavaPairDStream.scala:669:
>  type mismatch;
> [error]  found   : org.apache.spark.streaming.dstream.DStream[(K, 
> (com.google.common.base.Optional[V], W))]
> [error]  required: 
> org.apache.spark.streaming.api.java.JavaPairDStream[K,(com.google.common.base.Optional[V],
>  W)]
> [error]  Note: implicit method fromPairDStream is not applicable here because 
> it comes after the application point and it lacks an explicit result type
> [error]     joinResult.mapValues{case (v, w) => 
> (JavaUtils.optionToOptional(v), w)}
> [error]                         ^
> [error] 44 errors found
> [error] (streaming/compile:compile) Compilation failed
> {quote}
> Here is a simple PR fix this problem: https://github.com/apache/spark/pull/153



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Reply via email to