does).
Thanks,
Assaf.
From: Bernard Jesop [mailto:bernard.je...@gmail.com]
Sent: Thursday, July 13, 2017 6:58 PM
To: Vadim Semenov
Cc: user
Subject: Re: underlying checkpoint
Thank you, one of my mistakes was to think that show() was an action.
2017-07-13 17:52 GMT+02:00 Vadim
Thank you, one of my mistakes was to think that show() was an action.
2017-07-13 17:52 GMT+02:00 Vadim Semenov :
> You need to trigger an action on that rdd to checkpoint it.
>
> ```
> scala>spark.sparkContext.setCheckpointDir(".")
>
> scala>val df =
You need to trigger an action on that rdd to checkpoint it.
```
scala>spark.sparkContext.setCheckpointDir(".")
scala>val df = spark.createDataFrame(List(("Scala", 35), ("Python",
30), ("R", 15), ("Java", 20)))
df: org.apache.spark.sql.DataFrame = [_1: string, _2: int]
scala>
Hi everyone, I just tried this simple program :
* import
org.apache.spark.sql.SparkSession
object CheckpointTest extends App
{
val spark =
SparkSession
.builder()
.appName("Toto")
.getOrCreate()
spark.sparkContext.setCheckpointDir(".")
val df =