Thank you ver much, Tathagata! El miércoles, 22 de abril de 2015, Tathagata Das <[email protected]> escribió:
> Aaah, that. That is probably a limitation of the SQLContext (cc'ing Yin > for more information). > > > On Wed, Apr 22, 2015 at 7:07 AM, Sergio Jiménez Barrio < > [email protected] > <javascript:_e(%7B%7D,'cvml','[email protected]');>> wrote: > >> Sorry, this is the error: >> >> [error] /home/sergio/Escritorio/hello/streaming.scala:77: Implementation >> restriction: case classes cannot have more than 22 parameters. >> >> >> >> 2015-04-22 16:06 GMT+02:00 Sergio Jiménez Barrio <[email protected] >> <javascript:_e(%7B%7D,'cvml','[email protected]');>>: >> >>> I tried the solution of the guide, but I exceded the size of case class >>> Row: >>> >>> >>> 2015-04-22 15:22 GMT+02:00 Tathagata Das <[email protected] >>> <javascript:_e(%7B%7D,'cvml','[email protected]');>>: >>> >>>> Did you checkout the latest streaming programming guide? >>>> >>>> >>>> http://spark.apache.org/docs/latest/streaming-programming-guide.html#dataframe-and-sql-operations >>>> >>>> You also need to be aware of that to convert json RDDs to dataframe, >>>> sqlContext has to make a pass on the data to learn the schema. This will >>>> fail if a batch has no data. You have to safeguard against that. >>>> >>>> On Wed, Apr 22, 2015 at 6:19 AM, ayan guha <[email protected] >>>> <javascript:_e(%7B%7D,'cvml','[email protected]');>> wrote: >>>> >>>>> What about sqlcontext.createDataframe(rdd)? >>>>> On 22 Apr 2015 23:04, "Sergio Jiménez Barrio" <[email protected] >>>>> <javascript:_e(%7B%7D,'cvml','[email protected]');>> wrote: >>>>> >>>>>> Hi, >>>>>> >>>>>> I am using Kafka with Apache Stream to send JSON to Apache Spark: >>>>>> >>>>>> val messages = KafkaUtils.createDirectStream[String, String, >>>>>> StringDecoder, StringDecoder](ssc, kafkaParams, topicsSet) >>>>>> >>>>>> Now, I want parse the DStream created to DataFrame, but I don't know >>>>>> if Spark 1.3 have some easy way for this. ¿Any suggestion? I can get the >>>>>> message with: >>>>>> >>>>>> val lines = messages.map(_._2) >>>>>> >>>>>> Thank u for all. Sergio J. >>>>>> >>>>>> >>>>>> >>>> >>> >> > -- Atte. Sergio Jiménez
