Re: the scheme in stream reader

2017-06-19 Thread ??????????
;user"<user@spark.apache.org>;"dev"<d...@spark.apache.org>; Subject: Re: the scheme in stream reader The socket source can't know how to parse your data. I think the right thing would be for it to throw an exception saying that you can't set the schema here. Would you m

Re: the scheme in stream reader

2017-06-19 Thread Michael Armbrust
The socket source can't know how to parse your data. I think the right thing would be for it to throw an exception saying that you can't set the schema here. Would you mind opening a JIRA ticket? If you are trying to parse data from something like JSON then you should use from_json` on the

the scheme in stream reader

2017-06-18 Thread ??????????
Hi all, L set the scheme for DataStreamReader but when I print the scheme.It just printed: root |--value:string (nullable=true) My code is val line = ss.readStream.format("socket") .option("ip",xxx) .option("port",xxx) .scheme(StructField("name",StringType??::(StructField("age",