[jira] [Updated] (HDDS-2114) Rename does not preserve non-explicitly created interim directories
[
https://issues.apache.org/jira/browse/HDDS-2114?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nanda kumar updated HDDS-2114:
--
Fix Version/s: 0.4.1
> Rename does not preserve non-explicitly created interim directories
> ---
>
> Key: HDDS-2114
> URL: https://issues.apache.org/jira/browse/HDDS-2114
> Project: Hadoop Distributed Data Store
> Issue Type: Bug
>Reporter: Istvan Fajth
>Assignee: Lokesh Jain
>Priority: Critical
> Labels: pull-request-available
> Fix For: 0.4.1, 0.5.0
>
> Attachments: demonstrative_test.patch
>
> Time Spent: 1.5h
> Remaining Estimate: 0h
>
> I am attaching a patch that adds a test that demonstrates the problem.
> The scenario is coming from the way how Hive implements acid transactions
> with the ORC table format, but the test is redacted to the simplest possible
> code that reproduces the issue.
> The scenario:
> * Given a 3 level directory structure, where the top level directory was
> explicitly created, and the interim directory is implicitly created (for
> example either by creating a file with create("/top/interim/file") or by
> creating a directory with mkdirs("top/interim/dir"))
> * When the leaf is moved out from the implicitly created directory making
> this directory an empty directory
> * Then a FileNotFoundException is thrown when getFileStatus or listStatus is
> called on the interim directory.
> The expected behaviour:
> after the directory is becoming empty, the directory should still be part of
> the file system, moreover an empty FileStatus array should be returned when
> listStatus is called on it, and also a valid FileStatus object should be
> returned when getFileStatus is called on it.
>
>
> As this issue is present with Hive, and as this is how a FileSystem is
> expected to work this seems to be an at least critical issue as I see, please
> feel free to change the priority if needed.
> Also please note that, if the interim directory is explicitly created with
> mkdirs("top/interim") before creating the leaf, then the issue does not
> appear.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDDS-2114) Rename does not preserve non-explicitly created interim directories
[
https://issues.apache.org/jira/browse/HDDS-2114?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mukul Kumar Singh updated HDDS-2114:
Fix Version/s: 0.5.0
Resolution: Fixed
Status: Resolved (was: Patch Available)
Thanks for the contribution [~ljain] and [~pifta]. I have committed this to
trunk.
> Rename does not preserve non-explicitly created interim directories
> ---
>
> Key: HDDS-2114
> URL: https://issues.apache.org/jira/browse/HDDS-2114
> Project: Hadoop Distributed Data Store
> Issue Type: Bug
>Reporter: Istvan Fajth
>Assignee: Lokesh Jain
>Priority: Critical
> Labels: pull-request-available
> Fix For: 0.5.0
>
> Attachments: demonstrative_test.patch
>
> Time Spent: 1h 10m
> Remaining Estimate: 0h
>
> I am attaching a patch that adds a test that demonstrates the problem.
> The scenario is coming from the way how Hive implements acid transactions
> with the ORC table format, but the test is redacted to the simplest possible
> code that reproduces the issue.
> The scenario:
> * Given a 3 level directory structure, where the top level directory was
> explicitly created, and the interim directory is implicitly created (for
> example either by creating a file with create("/top/interim/file") or by
> creating a directory with mkdirs("top/interim/dir"))
> * When the leaf is moved out from the implicitly created directory making
> this directory an empty directory
> * Then a FileNotFoundException is thrown when getFileStatus or listStatus is
> called on the interim directory.
> The expected behaviour:
> after the directory is becoming empty, the directory should still be part of
> the file system, moreover an empty FileStatus array should be returned when
> listStatus is called on it, and also a valid FileStatus object should be
> returned when getFileStatus is called on it.
>
>
> As this issue is present with Hive, and as this is how a FileSystem is
> expected to work this seems to be an at least critical issue as I see, please
> feel free to change the priority if needed.
> Also please note that, if the interim directory is explicitly created with
> mkdirs("top/interim") before creating the leaf, then the issue does not
> appear.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDDS-2114) Rename does not preserve non-explicitly created interim directories
[
https://issues.apache.org/jira/browse/HDDS-2114?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lokesh Jain updated HDDS-2114:
--
Status: Patch Available (was: Open)
> Rename does not preserve non-explicitly created interim directories
> ---
>
> Key: HDDS-2114
> URL: https://issues.apache.org/jira/browse/HDDS-2114
> Project: Hadoop Distributed Data Store
> Issue Type: Bug
>Reporter: Istvan Fajth
>Assignee: Lokesh Jain
>Priority: Critical
> Labels: pull-request-available
> Attachments: demonstrative_test.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> I am attaching a patch that adds a test that demonstrates the problem.
> The scenario is coming from the way how Hive implements acid transactions
> with the ORC table format, but the test is redacted to the simplest possible
> code that reproduces the issue.
> The scenario:
> * Given a 3 level directory structure, where the top level directory was
> explicitly created, and the interim directory is implicitly created (for
> example either by creating a file with create("/top/interim/file") or by
> creating a directory with mkdirs("top/interim/dir"))
> * When the leaf is moved out from the implicitly created directory making
> this directory an empty directory
> * Then a FileNotFoundException is thrown when getFileStatus or listStatus is
> called on the interim directory.
> The expected behaviour:
> after the directory is becoming empty, the directory should still be part of
> the file system, moreover an empty FileStatus array should be returned when
> listStatus is called on it, and also a valid FileStatus object should be
> returned when getFileStatus is called on it.
>
>
> As this issue is present with Hive, and as this is how a FileSystem is
> expected to work this seems to be an at least critical issue as I see, please
> feel free to change the priority if needed.
> Also please note that, if the interim directory is explicitly created with
> mkdirs("top/interim") before creating the leaf, then the issue does not
> appear.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDDS-2114) Rename does not preserve non-explicitly created interim directories
[
https://issues.apache.org/jira/browse/HDDS-2114?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated HDDS-2114:
-
Labels: pull-request-available (was: )
> Rename does not preserve non-explicitly created interim directories
> ---
>
> Key: HDDS-2114
> URL: https://issues.apache.org/jira/browse/HDDS-2114
> Project: Hadoop Distributed Data Store
> Issue Type: Bug
>Reporter: Istvan Fajth
>Assignee: Lokesh Jain
>Priority: Critical
> Labels: pull-request-available
> Attachments: demonstrative_test.patch
>
>
> I am attaching a patch that adds a test that demonstrates the problem.
> The scenario is coming from the way how Hive implements acid transactions
> with the ORC table format, but the test is redacted to the simplest possible
> code that reproduces the issue.
> The scenario:
> * Given a 3 level directory structure, where the top level directory was
> explicitly created, and the interim directory is implicitly created (for
> example either by creating a file with create("/top/interim/file") or by
> creating a directory with mkdirs("top/interim/dir"))
> * When the leaf is moved out from the implicitly created directory making
> this directory an empty directory
> * Then a FileNotFoundException is thrown when getFileStatus or listStatus is
> called on the interim directory.
> The expected behaviour:
> after the directory is becoming empty, the directory should still be part of
> the file system, moreover an empty FileStatus array should be returned when
> listStatus is called on it, and also a valid FileStatus object should be
> returned when getFileStatus is called on it.
>
>
> As this issue is present with Hive, and as this is how a FileSystem is
> expected to work this seems to be an at least critical issue as I see, please
> feel free to change the priority if needed.
> Also please note that, if the interim directory is explicitly created with
> mkdirs("top/interim") before creating the leaf, then the issue does not
> appear.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDDS-2114) Rename does not preserve non-explicitly created interim directories
[
https://issues.apache.org/jira/browse/HDDS-2114?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Istvan Fajth updated HDDS-2114:
---
Attachment: demonstrative_test.patch
> Rename does not preserve non-explicitly created interim directories
> ---
>
> Key: HDDS-2114
> URL: https://issues.apache.org/jira/browse/HDDS-2114
> Project: Hadoop Distributed Data Store
> Issue Type: Bug
>Reporter: Istvan Fajth
>Priority: Critical
> Attachments: demonstrative_test.patch
>
>
> I am attaching a patch that adds a test that demonstrates the problem.
> The scenario is coming from the way how Hive implements acid transactions
> with the ORC table format, but the test is redacted to the simplest possible
> code that reproduces the issue.
> The scenario:
> * Given a 3 level directory structure, where the top level directory was
> explicitly created, and the interim directory is implicitly created (for
> example either by creating a file with create("/top/interim/file") or by
> creating a directory with mkdirs("top/interim/dir"))
> * When the leaf is moved out from the implicitly created directory making
> this directory an empty directory
> * Then a FileNotFoundException is thrown when getFileStatus or listStatus is
> called on the interim directory.
> The expected behaviour:
> after the directory is becoming empty, the directory should still be part of
> the file system, moreover an empty FileStatus array should be returned when
> listStatus is called on it, and also a valid FileStatus object should be
> returned when getFileStatus is called on it.
>
>
> As this issue is present with Hive, and as this is how a FileSystem is
> expected to work this seems to be an at least critical issue as I see, please
> feel free to change the priority if needed.
> Also please note that, if the interim directory is explicitly created with
> mkdirs("top/interim") before creating the leaf, then the issue does not
> appear.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
