Hello.
I need to create a custom streaming source by extending *FileStreamSource*.
The idea is to override *commit*, so that processed files (S3 objects in my
case) are renamed to have a certain prefix. However, I don't know how to
use this custom source. Obviously I don't want to compile Spark --
Hi Farshid,
Take a look at this example on github -
https://github.com/hienluu/structured-streaming-sources.
Cheers,
Hien
On Thu, Jul 12, 2018 at 12:52 AM Farshid Zavareh
wrote:
> Hello.
>
> I need to create a custom streaming source by extending *FileStreamSource*.
> The idea is to override
