[jira] [Updated] (SPARK-47646) try_to_number fails with NPE for malformed input

2024-03-30 Thread Hyukjin Kwon (Jira) 


 [ 
https://issues.apache.org/jira/browse/SPARK-47646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Hyukjin Kwon updated SPARK-47646:
-
Fix Version/s: 3.4.3

> try_to_number fails with NPE for malformed input
> 
>
> Key: SPARK-47646
> URL: https://issues.apache.org/jira/browse/SPARK-47646
> Project: Spark
>  Issue Type: Bug
>  Components: SQL
>Affects Versions: 4.0.0, 3.5.1
>Reporter: Hyukjin Kwon
>Assignee: Hyukjin Kwon
>Priority: Minor
>  Labels: pull-request-available
> Fix For: 4.0.0, 3.5.2, 3.4.3
>
>
> {code}
> scala> import org.apache.spark.sql.functions._
>  | val df = 
> spark.createDataset(spark.sparkContext.parallelize(Seq("11")))
>  | df.select(try_to_number($"value", lit("$99.99"))).show()
> 24/03/29 16:19:16 ERROR Executor: Exception in task 10.0 in stage 2.0 (TID 15)
> java.lang.NullPointerException: Cannot invoke 
> "org.apache.spark.sql.types.Decimal.toPlainString()" because "" is 
> null
>   at 
> org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIteratorForCodegenStage1.serializefromobject_doConsume_0$(Unknown
>  Source)
>   at 
> org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIteratorForCodegenStage1.processNext(Unknown
>  Source)
>   at 
> org.apache.spark.sql.execution.BufferedRowIterator.hasNext(BufferedRowIterator.java:43)
>   at 
> org.apache.spark.sql.execution.WholeStageCodegenEvaluatorFactory$WholeStageCodegenPartitionEvaluator$$anon$1.hasNext(WholeStageCodegenEvaluatorFactory.scala:50)
>   at 
> org.apache.spark.sql.execution.SparkPlan.$anonfun$getByteArrayRdd$1(SparkPlan.scala:388)
>   at 
> org.apache.spark.rdd.RDD.$anonfun$mapPartitionsInternal$2(RDD.scala:894)
>   at 
> org.apache.spark.rdd.RDD.$anonfun$mapPartitionsInternal$2$adapted(RDD.scala:894)
>   at 
> org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
>   at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:368)
>   at org.apache.spark.rdd.RDD.iterator(RDD.scala:332)
> {code}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org

[jira] [Updated] (SPARK-47646) try_to_number fails with NPE for malformed input

2024-03-29 Thread ASF GitHub Bot (Jira) 


 [ 
https://issues.apache.org/jira/browse/SPARK-47646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

ASF GitHub Bot updated SPARK-47646:
---
Labels: pull-request-available  (was: )

> try_to_number fails with NPE for malformed input
> 
>
> Key: SPARK-47646
> URL: https://issues.apache.org/jira/browse/SPARK-47646
> Project: Spark
>  Issue Type: Bug
>  Components: SQL
>Affects Versions: 4.0.0, 3.5.1
>Reporter: Hyukjin Kwon
>Priority: Minor
>  Labels: pull-request-available
>
> {code}
> scala> import org.apache.spark.sql.functions._
>  | val df = 
> spark.createDataset(spark.sparkContext.parallelize(Seq("11")))
>  | df.select(try_to_number($"value", lit("$99.99"))).show()
> 24/03/29 16:19:16 ERROR Executor: Exception in task 10.0 in stage 2.0 (TID 15)
> java.lang.NullPointerException: Cannot invoke 
> "org.apache.spark.sql.types.Decimal.toPlainString()" because "" is 
> null
>   at 
> org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIteratorForCodegenStage1.serializefromobject_doConsume_0$(Unknown
>  Source)
>   at 
> org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIteratorForCodegenStage1.processNext(Unknown
>  Source)
>   at 
> org.apache.spark.sql.execution.BufferedRowIterator.hasNext(BufferedRowIterator.java:43)
>   at 
> org.apache.spark.sql.execution.WholeStageCodegenEvaluatorFactory$WholeStageCodegenPartitionEvaluator$$anon$1.hasNext(WholeStageCodegenEvaluatorFactory.scala:50)
>   at 
> org.apache.spark.sql.execution.SparkPlan.$anonfun$getByteArrayRdd$1(SparkPlan.scala:388)
>   at 
> org.apache.spark.rdd.RDD.$anonfun$mapPartitionsInternal$2(RDD.scala:894)
>   at 
> org.apache.spark.rdd.RDD.$anonfun$mapPartitionsInternal$2$adapted(RDD.scala:894)
>   at 
> org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
>   at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:368)
>   at org.apache.spark.rdd.RDD.iterator(RDD.scala:332)
> {code}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org