Re: Spark 2.0.2 with Kafka source, Error please help!

Thanks Zhu, That was it. Now works great! On Thu, Nov 17, 2016 at 1:07 PM, Shixiong(Ryan) Zhu wrote: > The problem is "optional Gender gender = 3;". The generated class "Gender" > is a trait, and Spark cannot know how to create a trait so it's not > supported. You can

Re: Spark 2.0.2 with Kafka source, Error please help!

The problem is "optional Gender gender = 3;". The generated class "Gender" is a trait, and Spark cannot know how to create a trait so it's not supported. You can define your class which is supported by SQL Encoder, and convert this generated class to the new class in `parseLine`. On Wed, Nov 16,

Re: Spark 2.0.2 with Kafka source, Error please help!

Ryan, I just wanted to provide more info. Here is my .proto file which is the basis for generating the Person class. Thanks. option java_package = "com.example.protos"; enum Gender { MALE = 1; FEMALE = 2; } message Address { optional string street = 1; optional string city = 2; }

Re: Spark 2.0.2 with Kafka source, Error please help!

*Thanks for the response. Following is the Person class..* // Generated by the Scala Plugin for the Protocol Buffer Compiler. // Do not edit! // // Protofile syntax: PROTO2 package com.example.protos.demo @SerialVersionUID(0L) final case class Person( name: scala.Option[String] = None,

Re: Spark 2.0.2 with Kafka source, Error please help!

Could you provide the Person class? On Wed, Nov 16, 2016 at 1:19 PM, shyla deshpande wrote: > I am using 2.11.8. Thanks > > On Wed, Nov 16, 2016 at 1:15 PM, Shixiong(Ryan) Zhu < > shixi...@databricks.com> wrote: > >> Which Scala version are you using? Is it Scala 2.10?

Re: Spark 2.0.2 with Kafka source, Error please help!

I am using 2.11.8. Thanks On Wed, Nov 16, 2016 at 1:15 PM, Shixiong(Ryan) Zhu wrote: > Which Scala version are you using? Is it Scala 2.10? Scala 2.10 has some > known race conditions in reflection and the Scala community doesn't have > plan to fix it

Re: Spark 2.0.2 with Kafka source, Error please help!

Which Scala version are you using? Is it Scala 2.10? Scala 2.10 has some known race conditions in reflection and the Scala community doesn't have plan to fix it ( http://docs.scala-lang.org/overviews/reflection/thread-safety.html) AFAIK, the only way to fix it is upgrading to Scala 2.11. On Wed,

Spark 2.0.2 with Kafka source, Error please help!

I am using protobuf to encode. This may not be related to the new release issue Exception in thread "main" scala.ScalaReflectionException: is not a term at scala.reflect.api.Symbols$SymbolApi$class.asTerm(Symbols.scala:199) at