Hi, I am using spark streaming with 1 minute duration to read data from kafka topic, apply transformations and persist into HDFS.
The application is creating a new directory every 1 minute with many partition files(= nbr of partitions). What parameter should I need to change/configure to persist and create a HDFS directory say *every 30 minutes* instead of duration of the spark streaming application? Any help would be appreciated. Thanks, Naveen
