Tim Ludwinski created SPARK-27712:
-------------------------------------
Summary: createDataFrame() reorders row
Key: SPARK-27712
URL: https://issues.apache.org/jira/browse/SPARK-27712
Project: Spark
Issue Type: Bug
Components: PySpark
Affects Versions: 2.4.0
Environment: emr-5.20.0
PySpark 2.4.0
Python 2.7.15
Reporter: Tim Ludwinski
Executing the following:
{code:java}
my_schema = pyspark.sql.types.StructType([
pyspark.sql.types.StructField("B", pyspark.sql.types.StringType(), True),
pyspark.sql.types.StructField("A", pyspark.sql.types.StringType(), True)
])
spark.createDataFrame(spark.sparkContext.parallelize([pyspark.sql.Row(A="1",
B="2")]), my_schema).collect()
{code}
should produce this:
{code:java}
[Row(A="1", B="2")]
{code}
or this:
{code:java}
[Row(B='2', A='1')]
{code}
but produces this instead:
{code:java}
[Row(B=u'1', A=u'2')]
{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
