Haejoon Lee created SPARK-46820: ----------------------------------- Summary: Improve error message when createDataFrame have illegal nullable Key: SPARK-46820 URL: https://issues.apache.org/jira/browse/SPARK-46820 Project: Spark Issue Type: Sub-task Components: PySpark Affects Versions: 4.0.0 Reporter: Haejoon Lee
>>> from pyspark.sql.types import StructType, StructField, StringType, >>> IntegerType >>> schema = StructType([ ... StructField("name", StringType(), nullable=True), ... StructField("age", IntegerType(), nullable=False) ... ]) >>> df = spark.createDataFrame([("asd", None])], schema) pyspark.errors.exceptions.base.PySparkValueError: [CANNOT_BE_NONE] Argument `obj` cannot be None. The error message in the example above says "obj", but createDataFrame function has not "obj" reference. We should fix this error message properly. -- This message was sent by Atlassian Jira (v8.20.10#820010) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org