[jira] [Updated] (HADOOP-9483) winutils support for readlink command
[ https://issues.apache.org/jira/browse/HADOOP-9483?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suresh Srinivas updated HADOOP-9483: Target Version/s: (was: 3.0.0) Fix Version/s: (was: 3.0.0) 2.0.5-beta I merged the patch to branch-2. winutils support for readlink command - Key: HADOOP-9483 URL: https://issues.apache.org/jira/browse/HADOOP-9483 Project: Hadoop Common Issue Type: Improvement Components: util Affects Versions: 3.0.0 Reporter: Chris Nauroth Assignee: Arpit Agarwal Fix For: 2.0.5-beta Attachments: HADOOP-9483.003.patch, HADOOP-9483.004.patch, HADOOP-9483.005.patch, HADOOP-9483.007.patch, HADOOP-9483.patch, HADOOP-9483.patch The current codebase relies on the Unix readlink command to determine the target of a symlink on the local file system. winutils currently does not support this functionality on Windows. Adding the command to winutils will prevent the need to use GnuWin32 or Cygwin for readlink support. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9483) winutils support for readlink command
[ https://issues.apache.org/jira/browse/HADOOP-9483?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arpit Agarwal updated HADOOP-9483: -- Attachment: HADOOP-9483.005.patch Good catch, fixed! winutils support for readlink command - Key: HADOOP-9483 URL: https://issues.apache.org/jira/browse/HADOOP-9483 Project: Hadoop Common Issue Type: Improvement Components: util Affects Versions: 3.0.0 Reporter: Chris Nauroth Assignee: Arpit Agarwal Attachments: HADOOP-9483.003.patch, HADOOP-9483.004.patch, HADOOP-9483.005.patch, HADOOP-9483.patch, HADOOP-9483.patch The current codebase relies on the Unix readlink command to determine the target of a symlink on the local file system. winutils currently does not support this functionality on Windows. Adding the command to winutils will prevent the need to use GnuWin32 or Cygwin for readlink support. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9483) winutils support for readlink command
[ https://issues.apache.org/jira/browse/HADOOP-9483?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arpit Agarwal updated HADOOP-9483: -- Attachment: HADOOP-9483.006.patch Make TestWinUtils changes consistent with HADOOP-9043 to keep the rebase trivial. winutils support for readlink command - Key: HADOOP-9483 URL: https://issues.apache.org/jira/browse/HADOOP-9483 Project: Hadoop Common Issue Type: Improvement Components: util Affects Versions: 3.0.0 Reporter: Chris Nauroth Assignee: Arpit Agarwal Attachments: HADOOP-9483.003.patch, HADOOP-9483.004.patch, HADOOP-9483.005.patch, HADOOP-9483.006.patch, HADOOP-9483.patch, HADOOP-9483.patch The current codebase relies on the Unix readlink command to determine the target of a symlink on the local file system. winutils currently does not support this functionality on Windows. Adding the command to winutils will prevent the need to use GnuWin32 or Cygwin for readlink support. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9483) winutils support for readlink command
[ https://issues.apache.org/jira/browse/HADOOP-9483?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arpit Agarwal updated HADOOP-9483: -- Attachment: (was: HADOOP-9483.006.patch) winutils support for readlink command - Key: HADOOP-9483 URL: https://issues.apache.org/jira/browse/HADOOP-9483 Project: Hadoop Common Issue Type: Improvement Components: util Affects Versions: 3.0.0 Reporter: Chris Nauroth Assignee: Arpit Agarwal Attachments: HADOOP-9483.003.patch, HADOOP-9483.004.patch, HADOOP-9483.005.patch, HADOOP-9483.patch, HADOOP-9483.patch The current codebase relies on the Unix readlink command to determine the target of a symlink on the local file system. winutils currently does not support this functionality on Windows. Adding the command to winutils will prevent the need to use GnuWin32 or Cygwin for readlink support. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9483) winutils support for readlink command
[ https://issues.apache.org/jira/browse/HADOOP-9483?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arpit Agarwal updated HADOOP-9483: -- Status: Patch Available (was: Open) winutils support for readlink command - Key: HADOOP-9483 URL: https://issues.apache.org/jira/browse/HADOOP-9483 Project: Hadoop Common Issue Type: Improvement Components: util Affects Versions: 3.0.0 Reporter: Chris Nauroth Assignee: Arpit Agarwal Attachments: HADOOP-9483.003.patch, HADOOP-9483.004.patch, HADOOP-9483.005.patch, HADOOP-9483.patch, HADOOP-9483.patch The current codebase relies on the Unix readlink command to determine the target of a symlink on the local file system. winutils currently does not support this functionality on Windows. Adding the command to winutils will prevent the need to use GnuWin32 or Cygwin for readlink support. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9483) winutils support for readlink command
[ https://issues.apache.org/jira/browse/HADOOP-9483?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arpit Agarwal updated HADOOP-9483: -- Status: Open (was: Patch Available) winutils support for readlink command - Key: HADOOP-9483 URL: https://issues.apache.org/jira/browse/HADOOP-9483 Project: Hadoop Common Issue Type: Improvement Components: util Affects Versions: 3.0.0 Reporter: Chris Nauroth Assignee: Arpit Agarwal Attachments: HADOOP-9483.003.patch, HADOOP-9483.004.patch, HADOOP-9483.005.patch, HADOOP-9483.patch, HADOOP-9483.patch The current codebase relies on the Unix readlink command to determine the target of a symlink on the local file system. winutils currently does not support this functionality on Windows. Adding the command to winutils will prevent the need to use GnuWin32 or Cygwin for readlink support. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9483) winutils support for readlink command
[ https://issues.apache.org/jira/browse/HADOOP-9483?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arpit Agarwal updated HADOOP-9483: -- Attachment: HADOOP-9483.007.patch Rebased patch. winutils support for readlink command - Key: HADOOP-9483 URL: https://issues.apache.org/jira/browse/HADOOP-9483 Project: Hadoop Common Issue Type: Improvement Components: util Affects Versions: 3.0.0 Reporter: Chris Nauroth Assignee: Arpit Agarwal Attachments: HADOOP-9483.003.patch, HADOOP-9483.004.patch, HADOOP-9483.005.patch, HADOOP-9483.007.patch, HADOOP-9483.patch, HADOOP-9483.patch The current codebase relies on the Unix readlink command to determine the target of a symlink on the local file system. winutils currently does not support this functionality on Windows. Adding the command to winutils will prevent the need to use GnuWin32 or Cygwin for readlink support. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9483) winutils support for readlink command
[ https://issues.apache.org/jira/browse/HADOOP-9483?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suresh Srinivas updated HADOOP-9483: Resolution: Fixed Fix Version/s: 3.0.0 Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) +1 for the patch. Thank you Arpit. Thanks to Chris and Ivan for the reviews. winutils support for readlink command - Key: HADOOP-9483 URL: https://issues.apache.org/jira/browse/HADOOP-9483 Project: Hadoop Common Issue Type: Improvement Components: util Affects Versions: 3.0.0 Reporter: Chris Nauroth Assignee: Arpit Agarwal Fix For: 3.0.0 Attachments: HADOOP-9483.003.patch, HADOOP-9483.004.patch, HADOOP-9483.005.patch, HADOOP-9483.007.patch, HADOOP-9483.patch, HADOOP-9483.patch The current codebase relies on the Unix readlink command to determine the target of a symlink on the local file system. winutils currently does not support this functionality on Windows. Adding the command to winutils will prevent the need to use GnuWin32 or Cygwin for readlink support. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9483) winutils support for readlink command
[
https://issues.apache.org/jira/browse/HADOOP-9483?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Arpit Agarwal updated HADOOP-9483:
--
Attachment: HADOOP-9483.003.patch
Thanks for the quick review Ivan!
I have addressed all your feedback except for the below.
{quote}
2. To be able to handle input paths longer then MAX_PATH I think you'll have to
add a long path prefix before calling into CreateFile API. We already have an
API available in winutils to convert to a long path, check what we do on other
places.
{quote}
I am not sure it is possible to get a path longer than MAX_PATH as input to
Readlink since winutils is invoked via the shell. The situation you have
handled elsewhere is when the path is passed in via JNI. Please let me know if
I am wrong.
{quote}
Should this be if argc != 2?
{quote}
Deliberate for consistency with POSIX readlink. All extra arguments are ignored.
{quote}
I would also rather use a winerror code dword throughout the function as an
indicator that something failed (instead of the 'succeeded' boolean) and at the
function end check if it is equal to ERROR_SUCCESS or not
{quote}
It seems to complicates the state, especially since we don't really care about
the exact error code for most Win32 calls here. However I don't feel too
strongly about it so let me know what you think.
{quote}
9. Should we integrate the readlink functionality with RawLocalFs in this
Jira?
{quote}
That is intended to be fixed in HADOOP-9527.
winutils support for readlink command
-
Key: HADOOP-9483
URL: https://issues.apache.org/jira/browse/HADOOP-9483
Project: Hadoop Common
Issue Type: Improvement
Components: util
Affects Versions: 3.0.0
Reporter: Chris Nauroth
Assignee: Arpit Agarwal
Attachments: HADOOP-9483.003.patch, HADOOP-9483.patch,
HADOOP-9483.patch
The current codebase relies on the Unix readlink command to determine the
target of a symlink on the local file system. winutils currently does not
support this functionality on Windows. Adding the command to winutils will
prevent the need to use GnuWin32 or Cygwin for readlink support.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9483) winutils support for readlink command
[
https://issues.apache.org/jira/browse/HADOOP-9483?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Arpit Agarwal updated HADOOP-9483:
--
Attachment: HADOOP-9483.004.patch
Addressed your comments.
{quote}
10. I see what you're trying to do with the for loop. However, this won't work
when you get ERROR_MORE_DATA back, right? I say, let's just allocate a 1024
size array on stack, and if the this is not enough, have the function fail. If
in the future this turns out to be a problem, we can fix it (what is very
unlikely to happen). Make sense?
{quote}
It works when the return code is ERROR_MORE_DATA. It can be confirmed with a
buffer larger than sizeof(REPARSE_DATA_BUFFER) but not large enough for the
path strings. Since we are handling long paths it makes all the more sense to
handle this case. :)
winutils support for readlink command
-
Key: HADOOP-9483
URL: https://issues.apache.org/jira/browse/HADOOP-9483
Project: Hadoop Common
Issue Type: Improvement
Components: util
Affects Versions: 3.0.0
Reporter: Chris Nauroth
Assignee: Arpit Agarwal
Attachments: HADOOP-9483.003.patch, HADOOP-9483.004.patch,
HADOOP-9483.patch, HADOOP-9483.patch
The current codebase relies on the Unix readlink command to determine the
target of a symlink on the local file system. winutils currently does not
support this functionality on Windows. Adding the command to winutils will
prevent the need to use GnuWin32 or Cygwin for readlink support.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9483) winutils support for readlink command
[ https://issues.apache.org/jira/browse/HADOOP-9483?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Nauroth updated HADOOP-9483: -- Assignee: Arpit Agarwal Arpit, thanks for the additional research on the APIs. It makes sense. I haven't started any code for this yet, so please feel free to start. I've reassigned the issue to you. winutils support for readlink command - Key: HADOOP-9483 URL: https://issues.apache.org/jira/browse/HADOOP-9483 Project: Hadoop Common Issue Type: Improvement Components: util Affects Versions: 3.0.0 Reporter: Chris Nauroth Assignee: Arpit Agarwal The current codebase relies on the Unix readlink command to determine the target of a symlink on the local file system. winutils currently does not support this functionality on Windows. Adding the command to winutils will prevent the need to use GnuWin32 or Cygwin for readlink support. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9483) winutils support for readlink command
[ https://issues.apache.org/jira/browse/HADOOP-9483?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arpit Agarwal updated HADOOP-9483: -- Attachment: HADOOP-9483.patch Attaching a patch for winutils readlink. I used the FSCTL to get the closest behavior to POSIX readlink. The initial call to GetFileInformationByHandle is not required as the results of the FSCTL are sufficient to distinguish between sym links and other file types. Chris's initial research was very helpful. winutils support for readlink command - Key: HADOOP-9483 URL: https://issues.apache.org/jira/browse/HADOOP-9483 Project: Hadoop Common Issue Type: Improvement Components: util Affects Versions: 3.0.0 Reporter: Chris Nauroth Assignee: Arpit Agarwal Attachments: HADOOP-9483.patch The current codebase relies on the Unix readlink command to determine the target of a symlink on the local file system. winutils currently does not support this functionality on Windows. Adding the command to winutils will prevent the need to use GnuWin32 or Cygwin for readlink support. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9483) winutils support for readlink command
[ https://issues.apache.org/jira/browse/HADOOP-9483?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arpit Agarwal updated HADOOP-9483: -- Status: Patch Available (was: Open) winutils support for readlink command - Key: HADOOP-9483 URL: https://issues.apache.org/jira/browse/HADOOP-9483 Project: Hadoop Common Issue Type: Improvement Components: util Affects Versions: 3.0.0 Reporter: Chris Nauroth Assignee: Arpit Agarwal Attachments: HADOOP-9483.patch, HADOOP-9483.patch The current codebase relies on the Unix readlink command to determine the target of a symlink on the local file system. winutils currently does not support this functionality on Windows. Adding the command to winutils will prevent the need to use GnuWin32 or Cygwin for readlink support. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9483) winutils support for readlink command
[ https://issues.apache.org/jira/browse/HADOOP-9483?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Nauroth updated HADOOP-9483: -- Summary: winutils support for readlink command (was: winutils support fo readlink command) winutils support for readlink command - Key: HADOOP-9483 URL: https://issues.apache.org/jira/browse/HADOOP-9483 Project: Hadoop Common Issue Type: Improvement Components: util Affects Versions: 3.0.0 Reporter: Chris Nauroth The current codebase relies on the Unix readlink command to determine the target of a symlink on the local file system. winutils currently does not support this functionality on Windows. Adding the command to winutils will prevent the need to use GnuWin32 or Cygwin for readlink support. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
