Hello all, I am running into issues at the moment trying to print my DataStreams to an S3 bucket using writeAsText(“s3://bucket/result.json”) in my Flink job. I used print() on the same DataStream and I see the output I am looking for in standard output. I first confirm that my datastream has data by looking at the standard output, then I cancel my Flink job. After cancelling the job, result.json only gets created in my S3 bucket some of the time. It does not always gets created, but I confirmed that I see my data in standard output.
I understand writeAsText() should be used for debugging purposes only according to Flink’s documentation, but I’m just curious as to why I can’t get writeAsText() to always work every time I cancel my job. Thank you for your help, Michael
