My spark streaming checkpoint directory is being written to HDFS with default replication factor of 3.
In my streaming application where I am listening from kafka and setting the
dfs.replication = 2 as below the files are still being written with
replication factor=3
SparkConf sparkConfig = new
SparkConf().setMaster("mymaster").set("spark.hadoop.dfs.replication", "2");
Is there anything else that I need to do ??
Thanks !!!
Abhi
