Vladislav Kuzemchik created SPARK-22472: -------------------------------------------
Summary: Datasets generate random values for null primitive types Key: SPARK-22472 URL: https://issues.apache.org/jira/browse/SPARK-22472 Project: Spark Issue Type: Bug Components: SQL Affects Versions: 2.1.1 Reporter: Vladislav Kuzemchik Not sure if it ever were reported. {code} scala> val s = sc.parallelize(Seq[Option[Long]](None,Some(1L),Some(5))).toDF("v") s: org.apache.spark.sql.DataFrame = [v: bigint] scala> s.show(false) +----+ |v | +----+ |null| |1 | |5 | +----+ scala> s.as[Long].map(v => v*2).show(false) +-----+ |value| +-----+ |-2 | |2 | |10 | +-----+ scala> s.select($"v"*2).show(false) +-------+ |(v * 2)| +-------+ |null | |2 | |10 | +-------+ {code} -- This message was sent by Atlassian JIRA (v6.4.14#64029) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org