[jira] [Updated] (HADOOP-12009) Clarify FileSystem.listStatus() sorting order & fix FileSystemContractBaseTest:testListStatus

[Steve Loughran (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HADOOP-12009?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Steve Loughran updated HADOOP-12009:

Fix Version/s: 2.8.0
  Component/s: fs
   documentation

I've just backported this to 2.8+

> Clarify FileSystem.listStatus() sorting order & fix 
> FileSystemContractBaseTest:testListStatus 
> --
>
> Key: HADOOP-12009
> URL: https://issues.apache.org/jira/browse/HADOOP-12009
> Project: Hadoop Common
>  Issue Type: Improvement
>  Components: documentation, fs, test
>Reporter: Jakob Homan
>Assignee: J.Andreina
>Priority: Minor
> Fix For: 2.8.0, 3.0.0-alpha1
>
> Attachments: HADOOP-12009-003.patch, HADOOP-12009-004.patch, 
> HADOOP-12009.1.patch
>
>
> FileSystem.listStatus does not guarantee that implementations will return 
> sorted entries:
> {code}  /**
>* List the statuses of the files/directories in the given path if the path 
> is
>* a directory.
>* 
>* @param f given path
>* @return the statuses of the files/directories in the given patch
>* @throws FileNotFoundException when the path does not exist;
>* IOException see specific implementation
>*/
>   public abstract FileStatus[] listStatus(Path f) throws 
> FileNotFoundException, 
>  IOException;{code}
> However, FileSystemContractBaseTest, expects the elements to come back sorted:
> {code}Path[] testDirs = { path("/test/hadoop/a"),
> path("/test/hadoop/b"),
> path("/test/hadoop/c/1"), };
>
> // ...
> paths = fs.listStatus(path("/test/hadoop"));
> assertEquals(3, paths.length);
> assertEquals(path("/test/hadoop/a"), paths[0].getPath());
> assertEquals(path("/test/hadoop/b"), paths[1].getPath());
> assertEquals(path("/test/hadoop/c"), paths[2].getPath());{code}
> We should pass this test as long as all the paths are there, regardless of 
> their ordering.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[jira] [Updated] (HADOOP-12009) Clarify FileSystem.listStatus() sorting order & fix FileSystemContractBaseTest:testListStatus

[Jakob Homan (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HADOOP-12009?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jakob Homan updated HADOOP-12009:
-
   Resolution: Fixed
 Hadoop Flags: Reviewed
Fix Version/s: 3.0.0
   Status: Resolved  (was: Patch Available)

I've committed this.  Resolving.  Thanks, J.!

> Clarify FileSystem.listStatus() sorting order & fix 
> FileSystemContractBaseTest:testListStatus 
> --
>
> Key: HADOOP-12009
> URL: https://issues.apache.org/jira/browse/HADOOP-12009
> Project: Hadoop Common
>  Issue Type: Improvement
>  Components: test
>Reporter: Jakob Homan
>Assignee: J.Andreina
>Priority: Minor
> Fix For: 3.0.0
>
> Attachments: HADOOP-12009-003.patch, HADOOP-12009-004.patch, 
> HADOOP-12009.1.patch
>
>
> FileSystem.listStatus does not guarantee that implementations will return 
> sorted entries:
> {code}  /**
>* List the statuses of the files/directories in the given path if the path 
> is
>* a directory.
>* 
>* @param f given path
>* @return the statuses of the files/directories in the given patch
>* @throws FileNotFoundException when the path does not exist;
>* IOException see specific implementation
>*/
>   public abstract FileStatus[] listStatus(Path f) throws 
> FileNotFoundException, 
>  IOException;{code}
> However, FileSystemContractBaseTest, expects the elements to come back sorted:
> {code}Path[] testDirs = { path("/test/hadoop/a"),
> path("/test/hadoop/b"),
> path("/test/hadoop/c/1"), };
>
> // ...
> paths = fs.listStatus(path("/test/hadoop"));
> assertEquals(3, paths.length);
> assertEquals(path("/test/hadoop/a"), paths[0].getPath());
> assertEquals(path("/test/hadoop/b"), paths[1].getPath());
> assertEquals(path("/test/hadoop/c"), paths[2].getPath());{code}
> We should pass this test as long as all the paths are there, regardless of 
> their ordering.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HADOOP-12009) Clarify FileSystem.listStatus() sorting order & fix FileSystemContractBaseTest:testListStatus

[Steve Loughran (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HADOOP-12009?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Steve Loughran updated HADOOP-12009:

Fix Version/s: (was: 2.8.0)
   Status: Patch Available  (was: Reopened)

> Clarify FileSystem.listStatus() sorting order & fix 
> FileSystemContractBaseTest:testListStatus 
> --
>
> Key: HADOOP-12009
> URL: https://issues.apache.org/jira/browse/HADOOP-12009
> Project: Hadoop Common
>  Issue Type: Improvement
>  Components: test
>Reporter: Jakob Homan
>Assignee: J.Andreina
>Priority: Minor
> Attachments: HADOOP-12009-003.patch, HADOOP-12009-004.patch, 
> HADOOP-12009.1.patch
>
>
> FileSystem.listStatus does not guarantee that implementations will return 
> sorted entries:
> {code}  /**
>* List the statuses of the files/directories in the given path if the path 
> is
>* a directory.
>* 
>* @param f given path
>* @return the statuses of the files/directories in the given patch
>* @throws FileNotFoundException when the path does not exist;
>* IOException see specific implementation
>*/
>   public abstract FileStatus[] listStatus(Path f) throws 
> FileNotFoundException, 
>  IOException;{code}
> However, FileSystemContractBaseTest, expects the elements to come back sorted:
> {code}Path[] testDirs = { path("/test/hadoop/a"),
> path("/test/hadoop/b"),
> path("/test/hadoop/c/1"), };
>
> // ...
> paths = fs.listStatus(path("/test/hadoop"));
> assertEquals(3, paths.length);
> assertEquals(path("/test/hadoop/a"), paths[0].getPath());
> assertEquals(path("/test/hadoop/b"), paths[1].getPath());
> assertEquals(path("/test/hadoop/c"), paths[2].getPath());{code}
> We should pass this test as long as all the paths are there, regardless of 
> their ordering.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HADOOP-12009) Clarify FileSystem.listStatus() sorting order & fix FileSystemContractBaseTest:testListStatus

[J.Andreina (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HADOOP-12009?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

J.Andreina updated HADOOP-12009:

Attachment: HADOOP-12009-004.patch

Thanks [~vinayrpet] and [~ste...@apache.org] for correcting me. 
I tried writing testcase , but it fails with localFileSystem. Will figure out 
what can be done. 
Updated the code patch. 
Please review.

> Clarify FileSystem.listStatus() sorting order & fix 
> FileSystemContractBaseTest:testListStatus 
> --
>
> Key: HADOOP-12009
> URL: https://issues.apache.org/jira/browse/HADOOP-12009
> Project: Hadoop Common
>  Issue Type: Improvement
>  Components: test
>Reporter: Jakob Homan
>Assignee: J.Andreina
>Priority: Minor
> Fix For: 2.8.0
>
> Attachments: HADOOP-12009-003.patch, HADOOP-12009-004.patch, 
> HADOOP-12009.1.patch
>
>
> FileSystem.listStatus does not guarantee that implementations will return 
> sorted entries:
> {code}  /**
>* List the statuses of the files/directories in the given path if the path 
> is
>* a directory.
>* 
>* @param f given path
>* @return the statuses of the files/directories in the given patch
>* @throws FileNotFoundException when the path does not exist;
>* IOException see specific implementation
>*/
>   public abstract FileStatus[] listStatus(Path f) throws 
> FileNotFoundException, 
>  IOException;{code}
> However, FileSystemContractBaseTest, expects the elements to come back sorted:
> {code}Path[] testDirs = { path("/test/hadoop/a"),
> path("/test/hadoop/b"),
> path("/test/hadoop/c/1"), };
>
> // ...
> paths = fs.listStatus(path("/test/hadoop"));
> assertEquals(3, paths.length);
> assertEquals(path("/test/hadoop/a"), paths[0].getPath());
> assertEquals(path("/test/hadoop/b"), paths[1].getPath());
> assertEquals(path("/test/hadoop/c"), paths[2].getPath());{code}
> We should pass this test as long as all the paths are there, regardless of 
> their ordering.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HADOOP-12009) Clarify FileSystem.listStatus() sorting order & fix FileSystemContractBaseTest:testListStatus

[Steve Loughran (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HADOOP-12009?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Steve Loughran updated HADOOP-12009:

   Resolution: Fixed
Fix Version/s: 2.8.0
   Status: Resolved  (was: Patch Available)

fixed! thank you for your contribution!

> Clarify FileSystem.listStatus() sorting order & fix 
> FileSystemContractBaseTest:testListStatus 
> --
>
> Key: HADOOP-12009
> URL: https://issues.apache.org/jira/browse/HADOOP-12009
> Project: Hadoop Common
>  Issue Type: Improvement
>  Components: test
>Reporter: Jakob Homan
>Assignee: J.Andreina
>Priority: Minor
> Fix For: 2.8.0
>
> Attachments: HADOOP-12009-003.patch, HADOOP-12009.1.patch
>
>
> FileSystem.listStatus does not guarantee that implementations will return 
> sorted entries:
> {code}  /**
>* List the statuses of the files/directories in the given path if the path 
> is
>* a directory.
>* 
>* @param f given path
>* @return the statuses of the files/directories in the given patch
>* @throws FileNotFoundException when the path does not exist;
>* IOException see specific implementation
>*/
>   public abstract FileStatus[] listStatus(Path f) throws 
> FileNotFoundException, 
>  IOException;{code}
> However, FileSystemContractBaseTest, expects the elements to come back sorted:
> {code}Path[] testDirs = { path("/test/hadoop/a"),
> path("/test/hadoop/b"),
> path("/test/hadoop/c/1"), };
>
> // ...
> paths = fs.listStatus(path("/test/hadoop"));
> assertEquals(3, paths.length);
> assertEquals(path("/test/hadoop/a"), paths[0].getPath());
> assertEquals(path("/test/hadoop/b"), paths[1].getPath());
> assertEquals(path("/test/hadoop/c"), paths[2].getPath());{code}
> We should pass this test as long as all the paths are there, regardless of 
> their ordering.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HADOOP-12009) Clarify FileSystem.listStatus() sorting order & fix FileSystemContractBaseTest:testListStatus

[Steve Loughran (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HADOOP-12009?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Steve Loughran updated HADOOP-12009:

Attachment: HADOOP-12009-003.patch

This is the final patch as applied; adds some  separators in the javadocs & 
updates filesystem.md with the same comment about ordering

> Clarify FileSystem.listStatus() sorting order & fix 
> FileSystemContractBaseTest:testListStatus 
> --
>
> Key: HADOOP-12009
> URL: https://issues.apache.org/jira/browse/HADOOP-12009
> Project: Hadoop Common
>  Issue Type: Improvement
>  Components: test
>Reporter: Jakob Homan
>Assignee: J.Andreina
>Priority: Minor
> Attachments: HADOOP-12009-003.patch, HADOOP-12009.1.patch
>
>
> FileSystem.listStatus does not guarantee that implementations will return 
> sorted entries:
> {code}  /**
>* List the statuses of the files/directories in the given path if the path 
> is
>* a directory.
>* 
>* @param f given path
>* @return the statuses of the files/directories in the given patch
>* @throws FileNotFoundException when the path does not exist;
>* IOException see specific implementation
>*/
>   public abstract FileStatus[] listStatus(Path f) throws 
> FileNotFoundException, 
>  IOException;{code}
> However, FileSystemContractBaseTest, expects the elements to come back sorted:
> {code}Path[] testDirs = { path("/test/hadoop/a"),
> path("/test/hadoop/b"),
> path("/test/hadoop/c/1"), };
>
> // ...
> paths = fs.listStatus(path("/test/hadoop"));
> assertEquals(3, paths.length);
> assertEquals(path("/test/hadoop/a"), paths[0].getPath());
> assertEquals(path("/test/hadoop/b"), paths[1].getPath());
> assertEquals(path("/test/hadoop/c"), paths[2].getPath());{code}
> We should pass this test as long as all the paths are there, regardless of 
> their ordering.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HADOOP-12009) Clarify FileSystem.listStatus() sorting order & fix FileSystemContractBaseTest:testListStatus

[Steve Loughran (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HADOOP-12009?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Steve Loughran updated HADOOP-12009:

Summary: Clarify FileSystem.listStatus() sorting order & fix 
FileSystemContractBaseTest:testListStatus   (was: 
FileSystemContractBaseTest:testListStatus should not assume listStatus returns 
sorted results)

> Clarify FileSystem.listStatus() sorting order & fix 
> FileSystemContractBaseTest:testListStatus 
> --
>
> Key: HADOOP-12009
> URL: https://issues.apache.org/jira/browse/HADOOP-12009
> Project: Hadoop Common
>  Issue Type: Improvement
>  Components: test
>Reporter: Jakob Homan
>Assignee: J.Andreina
>Priority: Minor
> Attachments: HADOOP-12009.1.patch
>
>
> FileSystem.listStatus does not guarantee that implementations will return 
> sorted entries:
> {code}  /**
>* List the statuses of the files/directories in the given path if the path 
> is
>* a directory.
>* 
>* @param f given path
>* @return the statuses of the files/directories in the given patch
>* @throws FileNotFoundException when the path does not exist;
>* IOException see specific implementation
>*/
>   public abstract FileStatus[] listStatus(Path f) throws 
> FileNotFoundException, 
>  IOException;{code}
> However, FileSystemContractBaseTest, expects the elements to come back sorted:
> {code}Path[] testDirs = { path("/test/hadoop/a"),
> path("/test/hadoop/b"),
> path("/test/hadoop/c/1"), };
>
> // ...
> paths = fs.listStatus(path("/test/hadoop"));
> assertEquals(3, paths.length);
> assertEquals(path("/test/hadoop/a"), paths[0].getPath());
> assertEquals(path("/test/hadoop/b"), paths[1].getPath());
> assertEquals(path("/test/hadoop/c"), paths[2].getPath());{code}
> We should pass this test as long as all the paths are there, regardless of 
> their ordering.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)