;
>>> scala> val userDS: Dataset[Teamuser] = newDF.as[Teamuser]
>>> userDS: org.apache.spark.sql.Dataset[Teamuser] = [teamid: string, userid:
>>> string ... 1 more field]
>>>
>>> scala> userDS.show
>>> +--+--+-+
>>> |t
. 1 more field]
>>
>> scala> userDS.show
>> +--+--+-+
>> |teamid|userid| role|
>> +--+------+-----+
>> |t1|u1|role1|
>> +--+--+-+
>>
>>
>> scala> userDS.printSchema
>> root
>> |-- teami
userDS.show
> +--+--+-+
> |teamid|userid| role|
> +--+--+-+
> |t1|u1|role1|
> +--+--+-+
>
>
> scala> userDS.printSchema
> root
> |-- teamid: string (nullable = true)
> |-- userid: string (nullable = true)
> |-- ro
now I get a run time error...
error: Unable to find encoder for type stored in a Dataset. Primitive
types (Int, String, etc) and Product types (case classes) are supported by
importing spark.implicits._ Support for serializing other types will be
added in future releases.
[ERROR] val
true)
Am I missing anything?
Yong
From: shyla deshpande <deshpandesh...@gmail.com>
Sent: Thursday, March 23, 2017 3:49 PM
To: user
Subject: Re: Converting dataframe to dataset question
I realized, my case class was inside the object. It should be defin
I realized, my case class was inside the object. It should be defined
outside the scope of the object. Thanks
On Wed, Mar 22, 2017 at 6:07 PM, shyla deshpande
wrote:
> Why userDS is Dataset[Any], instead of Dataset[Teamuser]? Appreciate your
> help. Thanks
>
>
Why userDS is Dataset[Any], instead of Dataset[Teamuser]? Appreciate
your help. Thanks
val spark = SparkSession
.builder
.config("spark.cassandra.connection.host", cassandrahost)
.appName(getClass.getSimpleName)
.getOrCreate()
import spark.implicits._
val