Hi , I need to use batch start time in my spark streaming job.
I need the value of batch start time inside one of the functions that is called within a flatmap function in java. Please suggest me how this can be done. I tried to use the StreamingListener class and set the value of a variable inside the onBatchSubmitted function something like this : public void onBatchSubmitted(StreamingListenerBatchSubmitted batchSubmitted) { batchstarttime = batchSubmitted.batchInfo().batchTime().milliseconds(); CommandLineArguments.BATCH_START_TIME = batchstarttime; } But, the issue is that the BATCH_START_TIME set only when the batch starts. I see in the worker logs that BATCH_START_TIME takes the default value and is not set. Please suggest how this can be achieved. BR, Abhi