[ https://issues.apache.org/jira/browse/SPARK-45249?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
kondziolka9ld updated SPARK-45249: ---------------------------------- Description: Hi! I consider why `isCheckpointed` method return `false` - could someone explain it? I would expect `true`. {code:java} Welcome to ____ __ / __/__ ___ _____/ /__ _\ \/ _ \/ _ `/ __/ '_/ /___/ .__/\_,_/_/ /_/\_\ version 3.3.0 /_/ Using Scala version 2.12.15 (OpenJDK 64-Bit Server VM, Java 11.0.13) Type in expressions to have them evaluated. Type :help for more information. scala> sc.setCheckpointDir("file:///tmp/") scala> val df1 = Seq(1,2,3,4).toDF df1: org.apache.spark.sql.DataFrame = [value: int] scala> val df2 = df1.checkpoint() df2: org.apache.spark.sql.Dataset[org.apache.spark.sql.Row] = [value: int] scala> df2.rdd.isCheckpointed res2: Boolean = false // why false?{code} was: Hi! I consider why `isCheckpointed` method return `false` - could someone explain it? I would expect `true`. {code:java} Welcome to ____ __ / __/__ ___ _____/ /__ _\ \/ _ \/ _ `/ __/ '_/ /___/ .__/\_,_/_/ /_/\_\ version 3.3.0 /_/ Using Scala version 2.12.15 (OpenJDK 64-Bit Server VM, Java 11.0.13) Type in expressions to have them evaluated. Type :help for more information.scala> sc.setCheckpointDir("file:///tmp/")scala> val df1 = Seq(1,2,3,4).toDF df1: org.apache.spark.sql.DataFrame = [value: int]scala> val df2 = df1.checkpoint() df2: org.apache.spark.sql.Dataset[org.apache.spark.sql.Row] = [value: int] scala> df2.rdd.isCheckpointed res2: Boolean = false // why false?{code} > Why isCheckpointed returns false while as checkpointing is eager by default? > ---------------------------------------------------------------------------- > > Key: SPARK-45249 > URL: https://issues.apache.org/jira/browse/SPARK-45249 > Project: Spark > Issue Type: Question > Components: Spark Core > Affects Versions: 3.3.0 > Reporter: kondziolka9ld > Priority: Major > > Hi! > I consider why `isCheckpointed` method return `false` - could someone explain > it? I would expect `true`. > {code:java} > Welcome to > ____ __ > / __/__ ___ _____/ /__ > _\ \/ _ \/ _ `/ __/ '_/ > /___/ .__/\_,_/_/ /_/\_\ version 3.3.0 > /_/ > > Using Scala version 2.12.15 (OpenJDK 64-Bit Server VM, Java 11.0.13) > Type in expressions to have them evaluated. > Type :help for more information. > scala> sc.setCheckpointDir("file:///tmp/") > scala> val df1 = Seq(1,2,3,4).toDF > df1: org.apache.spark.sql.DataFrame = [value: int] > scala> val df2 = df1.checkpoint() > df2: org.apache.spark.sql.Dataset[org.apache.spark.sql.Row] = [value: int] > scala> df2.rdd.isCheckpointed > res2: Boolean = false // why false?{code} -- This message was sent by Atlassian Jira (v8.20.10#820010) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org