[ https://issues.apache.org/jira/browse/TEZ-4555?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ayush Saxena resolved TEZ-4555. ------------------------------- Fix Version/s: 0.10.4 Resolution: Fixed > Fail fast in LocalClient if the dirs (log, local) haven't been created > ---------------------------------------------------------------------- > > Key: TEZ-4555 > URL: https://issues.apache.org/jira/browse/TEZ-4555 > Project: Apache Tez > Issue Type: Bug > Reporter: László Bodor > Assignee: László Bodor > Priority: Major > Fix For: 0.10.4 > > Time Spent: 0.5h > Remaining Estimate: 0h > > https://github.com/apache/tez/blob/f080031f5c72bc4bfd8090ccdc670bdc0f7fd090/tez-dag/src/main/java/org/apache/tez/client/LocalClient.java#L332-L335 > {code} > Path logDir = new Path(userDir, "localmode-log-dir"); > Path localDir = new Path(userDir, "localmode-local-dir"); > localFs.mkdirs(logDir); > localFs.mkdirs(localDir); > {code} > in case of a non-writable local fs path (/base), this mkdirs silently returns > with false, whereas I can see that it's not writable on my mac: > {code} > mkdir -p /base > mkdir: /base: Read-only file system > {code} > leading to a confusing error message later: > {code} > 2024-04-24T02:03:52,101 ERROR [DAGAppMaster Thread] client.LocalClient: Error > starting DAGAppMaster > java.io.FileNotFoundException: > /base/scratch/laszlobodor/_tez_session_dir/b76689bc-d25e-4d65-a339-44206ff57ce2/.tez/application_1713949431891_0001_wd/tez-conf.pb > (No such file or directory) > at java.io.FileInputStream.open0(Native Method) ~[?:1.8.0_292] > at java.io.FileInputStream.open(FileInputStream.java:195) ~[?:1.8.0_292] > at java.io.FileInputStream.<init>(FileInputStream.java:138) > ~[?:1.8.0_292] > at > org.apache.tez.common.TezUtilsInternal.readUserSpecifiedTezConfiguration(TezUtilsInternal.java:84) > ~[tez-common-0.9.1.2024.0.19.0-3.jar:0.9.1.2024.0.19.0-3] > at > org.apache.tez.client.LocalClient.createDAGAppMaster(LocalClient.java:394) > ~[tez-dag-0.9.1.2024.0.19.0-3.jar:0.9.1.2024.0.19.0-3] > at org.apache.tez.client.LocalClient$1.run(LocalClient.java:357) > [tez-dag-0.9.1.2024.0.19.0-3.jar:0.9.1.2024.0.19.0-3] > at java.lang.Thread.run(Thread.java:748) [?:1.8.0_292] > {code} > actually, the fix should be done in HIVE-28212, but we need to fail fast here > and give a hint to the user about the folder -- This message was sent by Atlassian Jira (v8.20.10#820010)