Tathagata Das created SPARK-14109:
-------------------------------------
Summary: HDFSMetadataLog throws AbstractFilesSystem exception with
common schemes like s3n
Key: SPARK-14109
URL: https://issues.apache.org/jira/browse/SPARK-14109
Project: Spark
Issue Type: Bug
Components: SQL
Reporter: Tathagata Das
Assignee: Tathagata Das
HDFSMetadataLog uses newer FileContext API to achieve atomic renaming. However,
FileContext implementations may not exist for many scheme for which there may
be FileSystem implementations. In those cases, rather than failing completely,
we should fallback to the FileSystem based implementation, and log warning that
there may be file consistency issues in case the log directory is concurrently
modified.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
