Hi, I am using spark streamming and using the input strategy of watching for files in S3 directories. Using the textFileStream method in the streamming context.
The filename contains relevant for my pipeline manipulation i wonder if there is a more robust way to get this name other than capturing RDD debug information and parse the logs. Thanks, Natu
