Spark Streaming | Dynamic Action Support

2022-03-03 Thread Pappu Yadav
Hi, Is there any way I can add/delete actions/jobs dynamically in a running spark streaming job. I will call an API and execute only the configured actions in the system. Eg . In the first batch suppose there are 5 actions in the spark application. Now suppose some configuration is changed and

Spark Structure Streaming | FileStreamSourceLog not deleting list of input files | Spark -2.4.0

2020-04-21 Thread Pappu Yadav
missed something and there is some configuration already present to handle this. * Regards Pappu Yadav