Sahil Takiar created IMPALA-9437: ------------------------------------ Summary: Cannot get Ozone file block size after opening the file Key: IMPALA-9437 URL: https://issues.apache.org/jira/browse/IMPALA-9437 Project: IMPALA Issue Type: Sub-task Reporter: Sahil Takiar
When creating a tmp file on HDFS, {{HdfsTableSink::CreateNewTmpFile}} first opens the file, and then stats the file ({{hdfsGetPathInfo}}) before actually writing any data to the file or closing the file. HDFS seems to allow this behavior. However, Ozone, S3A, and ABFS do not. Impala does this for HDFS in order to get the block size of the opened file. According to {{HdfsTableSink}} it is possible for HDFS to create a block size with a different one than requested by Impala. So in order to track the correct block size for a file, the file needs to be stat'ed after opening it. For S3A and ABFS this isn't a big deal, because they aren't block based filesystem, but Ozone is. So we should investigate the impact of not having this capability and consider adding it to the Ozone client. -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org