alex-shchetkov commented on issue #2991: URL: https://github.com/apache/iceberg/issues/2991#issuecomment-916134306
I've ran into the same issue as @fcvr1010, where `-Djava.io.tmpdir=/tmp/driver/` did not help at first. However, after I kept re-trying different things on the same cluster, it suddenly worked. Running in cluster mode, the driver node gets assigned somewhat randomly, but when it does, the `/tmp/driver` folder (as specified by `-Djava.io.tmpdir` finally gets created on that node. Once it has been created on all nodes, the job started succeeding every time. I can finally get the jobs to work on an always-on EMR cluster, but that's not ideal at all. I can't get it work right away on a brand new cluster. In my bootstrap script (that runs on all nodes during cluster provisioning) I added a `sudo mkdir /mnt/tmp/driver`, and while I no longer get `java.io.IOException`, the job simply hangs on the last stage every time, never completing. (Which is rather concerning. I'd rather the cluster fail and shut down, instead of wasting money by idling indefinitely) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
