[ https://issues.apache.org/jira/browse/SPARK-14108?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15212363#comment-15212363 ]
Burak KÖSE commented on SPARK-14108: ------------------------------------ Please give a test case. > calling count() on empty dataframe throws java.util.NoSuchElementException > -------------------------------------------------------------------------- > > Key: SPARK-14108 > URL: https://issues.apache.org/jira/browse/SPARK-14108 > Project: Spark > Issue Type: Bug > Components: SQL > Affects Versions: 1.6.1 > Environment: Tested in Hadoop 2.7.2 EMR 4.x > Reporter: Krishna Shekhram > Priority: Minor > > When calling count() on empty dataframe, then spark code still tries to > iterate through the empty iterator and throws > java.util.NoSuchElementException. > Stacktrace : > java.util.NoSuchElementException: next on empty iterator > at scala.collection.Iterator$$anon$2.next(Iterator.scala:39) > at scala.collection.Iterator$$anon$2.next(Iterator.scala:37) > at > scala.collection.IndexedSeqLike$Elements.next(IndexedSeqLike.scala:64) > at scala.collection.IterableLike$class.head(IterableLike.scala:91) > at > scala.collection.mutable.ArrayOps$ofRef.scala$collection$IndexedSeqOptimized$$super$head(ArrayOps.scala:108) > at > scala.collection.IndexedSeqOptimized$class.head(IndexedSeqOptimized.scala:120) > at scala.collection.mutable.ArrayOps$ofRef.head(ArrayOps.scala:108) > at > org.apache.spark.sql.DataFrame$$anonfun$count$1.apply(DataFrame.scala:1515) > at > org.apache.spark.sql.DataFrame$$anonfun$count$1.apply(DataFrame.scala:1514) > at org.apache.spark.sql.DataFrame.withCallback(DataFrame.scala:2099) > at org.apache.spark.sql.DataFrame.count(DataFrame.scala:1514) > Code Snippet: > This code fails > if(this.df !=null){ > long countOfRows = this.df.count(); > } > If I do this then it works > if(this.df !=null && ! this.df.rdd().isEmpty()){ > long countOfRows = this.df.count(); > } -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org