Re: weird issue with sqlContext.createDataFrame - pyspark 1.3.1

2015-10-09 Thread ping yan
Thanks. It does seem like that my pandas installation is corrupted. Thanks! On Fri, Oct 9, 2015 at 11:04 AM, Davies Liu wrote: > Is it possible that you have an very old version of pandas, that does > not have DataFrame (or in different submodule). > > Could you try

Re: weird issue with sqlContext.createDataFrame - pyspark 1.3.1

2015-10-09 Thread Davies Liu
Is it possible that you have an very old version of pandas, that does not have DataFrame (or in different submodule). Could you try this: ``` >>> import pandas >>> pandas.__version__ '0.14.0' ``` On Thu, Oct 8, 2015 at 10:28 PM, ping yan wrote: > I really cannot figure out

weird issue with sqlContext.createDataFrame - pyspark 1.3.1

2015-10-08 Thread ping yan
I really cannot figure out what this is about.. (tried to import pandas, in case that is a dependency, but it didn't help.) >>> from pyspark.sql import SQLContext >>> sqlContext=SQLContext(sc) >>> sqlContext.createDataFrame(l).collect() Traceback (most recent call last): File "", line 1, in