t;, line 486, in
>> _createFromRDD
>> struct = self._inferSchema(rdd, samplingRatio, names=schema)
>> File "/opt/spark/python/pyspark/sql/session.py", line 466, in
>> _inferSchema
>
Thanks for the reply.
It looks strange that in scala shell I can implement this translation:
scala> sc.parallelize(List(3,2,1,4)).toDF.show
+-+
|value|
+-+
|3|
|2|
|1|
|4|
+-+
But in pyspark i have to write as:
sc.parallelize([3,2,1,4]).map(lambda x:
pt/spark/python/pyspark/sql/session.py", line 486, in
> _createFromRDD
> struct = self._inferSchema(rdd, samplingRatio, names=schema)
>File "/opt/spark/python/pyspark/sql/session.py", line 466, in
> _inferSchema
> schema = _infer_schema(first, names=names)
> F
)
File "/opt/spark/python/pyspark/sql/types.py", line 1067, in
_infer_schema
raise TypeError("Can not infer schema for type: %s" % type(row))
TypeError: Can not infer schema for type:
In my pyspark why this fails? I didnt get the way.
Thanks for helps.
---
