[jira] [Updated] (HADOOP-9195) Generic Use Date Range PathFilter
[ https://issues.apache.org/jira/browse/HADOOP-9195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Caleb Jones updated HADOOP-9195: Attachment: (was: HADOOP-9195.patch) Generic Use Date Range PathFilter - Key: HADOOP-9195 URL: https://issues.apache.org/jira/browse/HADOOP-9195 Project: Hadoop Common Issue Type: New Feature Components: fs Reporter: Caleb Jones Priority: Minor It would be useful for Hadoop to provide a general purpose date range PathFilter that operates on file mtime. I have implemented one, with tests, and would like to know where best to put it. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9195) Generic Use Date Range PathFilter
[ https://issues.apache.org/jira/browse/HADOOP-9195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Caleb Jones updated HADOOP-9195: Status: Open (was: Patch Available) Generic Use Date Range PathFilter - Key: HADOOP-9195 URL: https://issues.apache.org/jira/browse/HADOOP-9195 Project: Hadoop Common Issue Type: New Feature Components: fs Reporter: Caleb Jones Priority: Minor Attachments: HADOOP-9195.patch It would be useful for Hadoop to provide a general purpose date range PathFilter that operates on file mtime. I have implemented one, with tests, and would like to know where best to put it. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9195) Generic Use Date Range PathFilter
[ https://issues.apache.org/jira/browse/HADOOP-9195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Caleb Jones updated HADOOP-9195: Attachment: HADOOP-9195.patch Generic use date range PathFilter implementation (fixed references to deprecated FileStatus.isDir() and replaced with new FileStatus.isDirectory()). Generic Use Date Range PathFilter - Key: HADOOP-9195 URL: https://issues.apache.org/jira/browse/HADOOP-9195 Project: Hadoop Common Issue Type: New Feature Components: fs Reporter: Caleb Jones Priority: Minor Attachments: HADOOP-9195.patch It would be useful for Hadoop to provide a general purpose date range PathFilter that operates on file mtime. I have implemented one, with tests, and would like to know where best to put it. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9195) Generic Use Date Range PathFilter
[
https://issues.apache.org/jira/browse/HADOOP-9195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13549471#comment-13549471
]
Caleb Jones commented on HADOOP-9195:
-
I'm failing to see how TestZKFailoverController's tests are in any way related
to the mere addition of a new PathFilter in org.apache.hadoop.fs. I've fixed
the compiler warnings though (was referencing deprecated FileStatus.isDir()).
Unless someone can point out how TestZKFailoverController is failing due to
these code changes, I'm going to assume that TestZKFailoverController's tests
are having stability issues:
{noformat}
Error Message
test timed out after 15000 milliseconds
Stacktrace
java.lang.Exception: test timed out after 15000 milliseconds
at java.lang.Object.wait(Native Method)
at
org.apache.hadoop.ha.ZKFailoverController.waitForActiveAttempt(ZKFailoverController.java:458)
at
org.apache.hadoop.ha.ZKFailoverController.doGracefulFailover(ZKFailoverController.java:646)
...
{noformat}
Generic Use Date Range PathFilter
-
Key: HADOOP-9195
URL: https://issues.apache.org/jira/browse/HADOOP-9195
Project: Hadoop Common
Issue Type: New Feature
Components: fs
Reporter: Caleb Jones
Priority: Minor
Attachments: HADOOP-9195.patch
It would be useful for Hadoop to provide a general purpose date range
PathFilter that operates on file mtime. I have implemented one, with tests,
and would like to know where best to put it.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9195) Generic Use Date Range PathFilter
[ https://issues.apache.org/jira/browse/HADOOP-9195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Caleb Jones updated HADOOP-9195: Status: Patch Available (was: Open) Submitting updated patch with compiler warnings fixed. Generic Use Date Range PathFilter - Key: HADOOP-9195 URL: https://issues.apache.org/jira/browse/HADOOP-9195 Project: Hadoop Common Issue Type: New Feature Components: fs Reporter: Caleb Jones Priority: Minor Attachments: HADOOP-9195.patch It would be useful for Hadoop to provide a general purpose date range PathFilter that operates on file mtime. I have implemented one, with tests, and would like to know where best to put it. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9195) Generic Use Date Range PathFilter
[ https://issues.apache.org/jira/browse/HADOOP-9195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Caleb Jones updated HADOOP-9195: Attachment: HADOOP-9195.patch Tweaked it a bit to allow for open-ended ranges. Generic Use Date Range PathFilter - Key: HADOOP-9195 URL: https://issues.apache.org/jira/browse/HADOOP-9195 Project: Hadoop Common Issue Type: New Feature Components: fs Reporter: Caleb Jones Priority: Minor Attachments: HADOOP-9195.patch, HADOOP-9195.patch It would be useful for Hadoop to provide a general purpose date range PathFilter that operates on file mtime. I have implemented one, with tests, and would like to know where best to put it. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9195) Generic Use Date Range PathFilter
[ https://issues.apache.org/jira/browse/HADOOP-9195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Caleb Jones updated HADOOP-9195: Status: Patch Available (was: Open) Generic Use Date Range PathFilter - Key: HADOOP-9195 URL: https://issues.apache.org/jira/browse/HADOOP-9195 Project: Hadoop Common Issue Type: New Feature Components: fs Reporter: Caleb Jones Priority: Minor Attachments: HADOOP-9195.patch, HADOOP-9195.patch It would be useful for Hadoop to provide a general purpose date range PathFilter that operates on file mtime. I have implemented one, with tests, and would like to know where best to put it. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9195) Generic Use Date Range PathFilter
[ https://issues.apache.org/jira/browse/HADOOP-9195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Caleb Jones updated HADOOP-9195: Status: Open (was: Patch Available) Generic Use Date Range PathFilter - Key: HADOOP-9195 URL: https://issues.apache.org/jira/browse/HADOOP-9195 Project: Hadoop Common Issue Type: New Feature Components: fs Reporter: Caleb Jones Priority: Minor Attachments: HADOOP-9195.patch, HADOOP-9195.patch It would be useful for Hadoop to provide a general purpose date range PathFilter that operates on file mtime. I have implemented one, with tests, and would like to know where best to put it. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HADOOP-9195) Generic Use Date Range PathFilter
Caleb Jones created HADOOP-9195: --- Summary: Generic Use Date Range PathFilter Key: HADOOP-9195 URL: https://issues.apache.org/jira/browse/HADOOP-9195 Project: Hadoop Common Issue Type: New Feature Components: fs Reporter: Caleb Jones Priority: Minor It would be useful for Hadoop to provide a general purpose date range PathFilter that operates on file mtime. I have implemented one, with tests, and would like to know where best to put it. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9195) Generic Use Date Range PathFilter
[ https://issues.apache.org/jira/browse/HADOOP-9195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13549367#comment-13549367 ] Caleb Jones commented on HADOOP-9195: - I'll attach a patch. Generic Use Date Range PathFilter - Key: HADOOP-9195 URL: https://issues.apache.org/jira/browse/HADOOP-9195 Project: Hadoop Common Issue Type: New Feature Components: fs Reporter: Caleb Jones Priority: Minor It would be useful for Hadoop to provide a general purpose date range PathFilter that operates on file mtime. I have implemented one, with tests, and would like to know where best to put it. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9195) Generic Use Date Range PathFilter
[ https://issues.apache.org/jira/browse/HADOOP-9195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Caleb Jones updated HADOOP-9195: Attachment: HADOOP-9195.patch Attached is a patch for this implementation along with unit tests. I reviewed http://wiki.apache.org/hadoop/HowToContribute and structured the code accordingly. Generic Use Date Range PathFilter - Key: HADOOP-9195 URL: https://issues.apache.org/jira/browse/HADOOP-9195 Project: Hadoop Common Issue Type: New Feature Components: fs Reporter: Caleb Jones Priority: Minor Attachments: HADOOP-9195.patch It would be useful for Hadoop to provide a general purpose date range PathFilter that operates on file mtime. I have implemented one, with tests, and would like to know where best to put it. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8989) hadoop dfs -find feature
[ https://issues.apache.org/jira/browse/HADOOP-8989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13549411#comment-13549411 ] Caleb Jones commented on HADOOP-8989: - I've submitted a patch for HADOOP-9195 (DateTimePathFilter based on mtime) which may have some functionality similar to this. hadoop dfs -find feature Key: HADOOP-8989 URL: https://issues.apache.org/jira/browse/HADOOP-8989 Project: Hadoop Common Issue Type: New Feature Reporter: Marco Nicosia Assignee: Jonathan Allen Attachments: HADOOP-8989.patch, HADOOP-8989.patch, HADOOP-8989.patch, HADOOP-8989.patch, HADOOP-8989.patch, HADOOP-8989.patch, HADOOP-8989.patch Both sysadmins and users make frequent use of the unix 'find' command, but Hadoop has no correlate. Without this, users are writing scripts which make heavy use of hadoop dfs -lsr, and implementing find one-offs. I think hdfs -lsr is somewhat taxing on the NameNode, and a really slow experience on the client side. Possibly an in-NameNode find operation would be only a bit more taxing on the NameNode, but significantly faster from the client's point of view? The minimum set of options I can think of which would make a Hadoop find command generally useful is (in priority order): * -type (file or directory, for now) * -atime/-ctime-mtime (... and -creationtime?) (both + and - arguments) * -print0 (for piping to xargs -0) * -depth * -owner/-group (and -nouser/-nogroup) * -name (allowing for shell pattern, or even regex?) * -perm * -size One possible special case, but could possibly be really cool if it ran from within the NameNode: * -delete The hadoop dfs -lsr | hadoop dfs -rm cycle is really, really slow. Lower priority, some people do use operators, mostly to execute -or searches such as: * find / \(-nouser -or -nogroup\) Finally, I thought I'd include a link to the [Posix spec for find|http://www.opengroup.org/onlinepubs/009695399/utilities/find.html] -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9195) Generic Use Date Range PathFilter
[ https://issues.apache.org/jira/browse/HADOOP-9195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Caleb Jones updated HADOOP-9195: Status: Patch Available (was: Open) Generic Use Date Range PathFilter - Key: HADOOP-9195 URL: https://issues.apache.org/jira/browse/HADOOP-9195 Project: Hadoop Common Issue Type: New Feature Components: fs Reporter: Caleb Jones Priority: Minor Attachments: HADOOP-9195.patch It would be useful for Hadoop to provide a general purpose date range PathFilter that operates on file mtime. I have implemented one, with tests, and would like to know where best to put it. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
