Nicholas Chammas created SPARK-47190: ----------------------------------------
Summary: Add support for checkpointing to Spark Connect Key: SPARK-47190 URL: https://issues.apache.org/jira/browse/SPARK-47190 Project: Spark Issue Type: Improvement Components: Connect Affects Versions: 4.0.0 Reporter: Nicholas Chammas The {{sparkContext}} that underlies a given {{SparkSession}} is not accessible over Spark Connect. This means you cannot call {{spark.sparkContext.setCheckpointDir(...)}}, which in turn means you cannot checkpoint a DataFrame. We should add support for this somehow to Spark Connect. -- This message was sent by Atlassian Jira (v8.20.10#820010) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org