Re: Flink job repeated restart failure

2021-03-26 Thread Arvid Heise
Hi Vinaya, java.io.tmpdir is already the fallback and I'm not aware of another level of fallback. Ensuring java.io.tmpdir is valid is also relevant for some third-party libraries that rely on it (e.g. FileSystem that cache local files). It's good practice to set that appropriately. On Fri, Mar 2

Re: Flink job repeated restart failure

2021-03-25 Thread vinaya
Hi Arvid, Thank you for the suggestion. Indeed, the specified setting was commented out in the Flink configuration (flink-conf.yaml). # io.tmp.dirs: /tmp Is there a fallback (e.g. /tmp) if io.tmp.dirs and System.getProperty("java.io.tmpdir") are both not set? Will configure this setting to a

Re: Flink job repeated restart failure

2021-03-25 Thread Arvid Heise
Hi Vinaya, SpillingAdaptiveSpanningRecordDeserializer tries to create a directory in the temp directory, which you can configure by setting io.tmp.dirs. By default, it's set to System.getProperty("java.io.tmpdir"), which seems to be invalid in your case. (Note that the directory has to exist on th

Flink job repeated restart failure

2021-03-24 Thread VINAYA KUMAR BENDI
Dear all, One of the Flink jobs gave below exception and failed. Several attempts to restart the job resulted in the same exception and the job failed each time. The job started successfully only after changing the file name. Flink Version: 1.11.2 Exception 2021-03-24 20:13:09,288 INFO org.ap