Dapeng Sun created HIVE-21483: --------------------------------- Summary: HoS would fail when scratch dir is using remote HDFS Key: HIVE-21483 URL: https://issues.apache.org/jira/browse/HIVE-21483 Project: Hive Issue Type: Bug Reporter: Dapeng Sun Assignee: Dapeng Sun
HoS would fail when scratch dir is using remote HDFS: public static URI uploadToHDFS(URI source, HiveConf conf) throws IOException { Path localFile = new Path(source.getPath()); Path remoteFile = new Path(SessionState.get().getSparkSession().getHDFSSessionDir(), getFileName(source)); - FileSystem fileSystem = FileSystem.get(conf); + FileSystem fileSystem = remoteFile.getFileSystem(conf); // Overwrite if the remote file already exists. Whether the file can be added // on executor is up to spark, i.e. spark.files.overwrite fileSystem.copyFromLocalFile(false, true, localFile, remoteFile); Path fullPath = fileSystem.getFileStatus(remoteFile).getPath(); r -- This message was sent by Atlassian JIRA (v7.6.3#76005)