This will be fixed in https://github.com/apache/spark/pull/5230/files
On Fri, Mar 27, 2015 at 9:13 AM, Peter Mac <peter.machar...@noaa.gov> wrote: > I downloaded spark version spark-1.3.0-bin-hadoop2.4. > > When the python version of sql.py is run the following error occurs: > > [root@nde-dev8-template python]# > /root/spark-1.3.0-bin-hadoop2.4/bin/spark-submit sql.py > Spark assembly has been built with Hive, including Datanucleus jars on > classpath > Traceback (most recent call last): > File "/root/spark-1.3.0-bin-hadoop2.4/examples/src/main/python/sql.py", > line 22, in <module> > from pyspark.sql import Row, StructField, StructType, StringType, > IntegerType > ImportError: cannot import name StructField > > ---------------------------------------------------------------------- > The sql.py version, spark-1.2.1-bin-hadoop2.4, does not throw the error: > > [root@nde-dev8-template python]# > /root/spark-1.2.1-bin-hadoop2.4/bin/spark-submit sql.py > Spark assembly has been built with Hive, including Datanucleus jars on > classpath > 15/03/27 14:18:44 WARN NativeCodeLoader: Unable to load native-hadoop > library for your platform... using builtin-java classes where applicable > 15/03/27 14:19:41 WARN ThreadLocalRandom: Failed to generate a seed from > SecureRandom within 3 seconds. Not enough entrophy? > root > |-- age: integer (nullable = true) > |-- name: string (nullable = true) > > root > |-- person_name: string (nullable = false) > |-- person_age: integer (nullable = false) > > root > |-- age: integer (nullable = true) > |-- name: string (nullable = true) > > Justin > > > ------------------------------------------------- > > The OS/JAVA environments are: > > OS: Linux nde-dev8-template 2.6.32-431.17.1.el6.x86_64 #1 SMP Fri Apr 11 > 17:27:00 EDT 2014 x86_64 x86_64 x86_64 GNU/Linux > > JAVA: [root@nde-dev8-template bin]# java -version > java version "1.7.0_51" > Java(TM) SE Runtime Environment (build 1.7.0_51-b13) > Java HotSpot(TM) 64-Bit Server VM (build 24.51-b03, mixed mode) > > The same error occurs when using bin/pyspark shell. > >>>> from pyspark.sql import StructField > Traceback (most recent call last): > File "<stdin>", line 1, in <module> > ImportError: cannot import name StructField > > > --------------------------------------------------- > > Any advice for resolving? Thanks in advance. > > Peter > > > > > > > > -- > View this message in context: > http://apache-spark-user-list.1001560.n3.nabble.com/Python-Example-sql-py-not-working-in-version-spark-1-3-0-bin-hadoop2-4-tp22261.html > Sent from the Apache Spark User List mailing list archive at Nabble.com. > > --------------------------------------------------------------------- > To unsubscribe, e-mail: user-unsubscr...@spark.apache.org > For additional commands, e-mail: user-h...@spark.apache.org > --------------------------------------------------------------------- To unsubscribe, e-mail: user-unsubscr...@spark.apache.org For additional commands, e-mail: user-h...@spark.apache.org
