Convert Dataframe to Dataset in pyspark
In Scala, val ds = sqlContext.read.text("/home/spark/1.6/lines").as[String] what is the equivalent code in pyspark? -- Selvam Raman "லஞ்சம் தவிர்த்து நெஞ்சம் நிமிர்த்து"
Re: Convert Dataframe to Dataset in pyspark
You don't need encoders in python since its all dynamically typed anyway. You can just do the following if you want the data as a string. sqlContext.read.text("/home/spark/1.6/lines").rdd.map(lambda row: row.value) 2017-04-01 5:36 GMT-07:00 Selvam Raman : > In Scala, > val ds = sqlContext.read.text("/home/spark/1.6/lines").as[String] > > what is the equivalent code in pyspark? > > -- > Selvam Raman > "லஞ்சம் தவிர்த்து நெஞ்சம் நிமிர்த்து" >