Mihaly Hazag created SPARK-33322: ------------------------------------ Summary: Dataframe: data is wrongly presented because of column name Key: SPARK-33322 URL: https://issues.apache.org/jira/browse/SPARK-33322 Project: Spark Issue Type: Bug Components: PySpark Affects Versions: 2.4.5 Reporter: Mihaly Hazag
Consider the code below: `some_text` column got the `some_int` value, while its value is null in the dataframe. !image-2020-11-03-14-42-52-840.png! Renaming the field from `some_text` to `some_apple`, fixed the problem! 🙂 !image-2020-11-03-14-43-13-528.png! Here is the code to reproduce the problem {code:python} from datetime import datetime from pyspark.sql import Row from pyspark.sql.types import StructType, StructField, DateType, StringType, IntegerType schema = StructType( [ StructField('dfdt', DateType(), True), StructField('some_text', StringType(), True), StructField('some_int', IntegerType(), True), ] ) test_df = spark.createDataFrame([ Row(dfdt=datetime.strptime('2020-12-18', '%Y-%m-%d'), some_text='cdsvg', some_int=100) ], schema) display(test_df) {code} -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org