[jira] [Updated] (MAPREDUCE-1744) DistributedCache creates its own FileSytem instance when adding a file/archive to the path
[ https://issues.apache.org/jira/browse/MAPREDUCE-1744?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alejandro Abdelnur updated MAPREDUCE-1744: -- Resolution: Fixed Fix Version/s: 1.0.0 0.23.1 Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) > DistributedCache creates its own FileSytem instance when adding a > file/archive to the path > -- > > Key: MAPREDUCE-1744 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-1744 > Project: Hadoop Map/Reduce > Issue Type: Bug >Reporter: Dick King >Assignee: Dick King > Fix For: 0.23.1, 1.0.0 > > Attachments: BZ-3503564--2010-05-06.patch, > MAPREDUCE-1744-0.23-trunk.patch, MAPREDUCE-1744-0.23-trunk.patch, > MAPREDUCE-1744.patch, h1744.patch, mapred-1744-1.patch, mapred-1744-2.patch, > mapred-1744-3.patch, mapred-1744.patch > > > According to the contract of {{UserGroupInformation.doAs()}} the only > required operations within the {{doAs()}} block are the > creation of a {{JobClient}} or getting a {{FileSystem}} . > The {{DistributedCache.add(File/Archive)ToClasspath()}} methods create a > {{FileSystem}} instance outside of the {{doAs()}} block, > this {{FileSystem}} instance is not in the scope of the proxy user but of the > superuser and permissions may make the method > fail. > One option is to overload the methods above to receive a filesystem. > Another option is to do obtain the {{FileSystem}} within a {{doAs()}} block, > for this it would be required to have the proxy > user set in the passed configuration. > The second option seems nicer, but I don't know if the proxy user is as a > property in the jobconf. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-1744) DistributedCache creates its own FileSytem instance when adding a file/archive to the path
[ https://issues.apache.org/jira/browse/MAPREDUCE-1744?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alejandro Abdelnur updated MAPREDUCE-1744: -- Attachment: MAPREDUCE-1744-0.23-trunk.patch updated patch removing dup @deprecated and using file/archive.getFileSystem(conf) instead of FileSystem.get(conf). > DistributedCache creates its own FileSytem instance when adding a > file/archive to the path > -- > > Key: MAPREDUCE-1744 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-1744 > Project: Hadoop Map/Reduce > Issue Type: Bug >Reporter: Dick King >Assignee: Alejandro Abdelnur > Attachments: BZ-3503564--2010-05-06.patch, > MAPREDUCE-1744-0.23-trunk.patch, MAPREDUCE-1744-0.23-trunk.patch, > MAPREDUCE-1744.patch, h1744.patch, mapred-1744-1.patch, mapred-1744-2.patch, > mapred-1744-3.patch, mapred-1744.patch > > > According to the contract of {{UserGroupInformation.doAs()}} the only > required operations within the {{doAs()}} block are the > creation of a {{JobClient}} or getting a {{FileSystem}} . > The {{DistributedCache.add(File/Archive)ToClasspath()}} methods create a > {{FileSystem}} instance outside of the {{doAs()}} block, > this {{FileSystem}} instance is not in the scope of the proxy user but of the > superuser and permissions may make the method > fail. > One option is to overload the methods above to receive a filesystem. > Another option is to do obtain the {{FileSystem}} within a {{doAs()}} block, > for this it would be required to have the proxy > user set in the passed configuration. > The second option seems nicer, but I don't know if the proxy user is as a > property in the jobconf. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-1744) DistributedCache creates its own FileSytem instance when adding a file/archive to the path
[ https://issues.apache.org/jira/browse/MAPREDUCE-1744?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alejandro Abdelnur updated MAPREDUCE-1744: -- Status: Patch Available (was: Reopened) > DistributedCache creates its own FileSytem instance when adding a > file/archive to the path > -- > > Key: MAPREDUCE-1744 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-1744 > Project: Hadoop Map/Reduce > Issue Type: Bug >Reporter: Dick King >Assignee: Alejandro Abdelnur > Attachments: BZ-3503564--2010-05-06.patch, > MAPREDUCE-1744-0.23-trunk.patch, MAPREDUCE-1744.patch, h1744.patch, > mapred-1744-1.patch, mapred-1744-2.patch, mapred-1744-3.patch, > mapred-1744.patch > > > According to the contract of {{UserGroupInformation.doAs()}} the only > required operations within the {{doAs()}} block are the > creation of a {{JobClient}} or getting a {{FileSystem}} . > The {{DistributedCache.add(File/Archive)ToClasspath()}} methods create a > {{FileSystem}} instance outside of the {{doAs()}} block, > this {{FileSystem}} instance is not in the scope of the proxy user but of the > superuser and permissions may make the method > fail. > One option is to overload the methods above to receive a filesystem. > Another option is to do obtain the {{FileSystem}} within a {{doAs()}} block, > for this it would be required to have the proxy > user set in the passed configuration. > The second option seems nicer, but I don't know if the proxy user is as a > property in the jobconf. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-1744) DistributedCache creates its own FileSytem instance when adding a file/archive to the path
[ https://issues.apache.org/jira/browse/MAPREDUCE-1744?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alejandro Abdelnur updated MAPREDUCE-1744: -- Attachment: MAPREDUCE-1744-0.23-trunk.patch Patch ported to trunk & 0.23. Patch cannot use original testcase as it is no more. Piggybacking in an existing testcase . > DistributedCache creates its own FileSytem instance when adding a > file/archive to the path > -- > > Key: MAPREDUCE-1744 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-1744 > Project: Hadoop Map/Reduce > Issue Type: Bug >Reporter: Dick King >Assignee: Alejandro Abdelnur > Attachments: BZ-3503564--2010-05-06.patch, > MAPREDUCE-1744-0.23-trunk.patch, MAPREDUCE-1744.patch, h1744.patch, > mapred-1744-1.patch, mapred-1744-2.patch, mapred-1744-3.patch, > mapred-1744.patch > > > According to the contract of {{UserGroupInformation.doAs()}} the only > required operations within the {{doAs()}} block are the > creation of a {{JobClient}} or getting a {{FileSystem}} . > The {{DistributedCache.add(File/Archive)ToClasspath()}} methods create a > {{FileSystem}} instance outside of the {{doAs()}} block, > this {{FileSystem}} instance is not in the scope of the proxy user but of the > superuser and permissions may make the method > fail. > One option is to overload the methods above to receive a filesystem. > Another option is to do obtain the {{FileSystem}} within a {{doAs()}} block, > for this it would be required to have the proxy > user set in the passed configuration. > The second option seems nicer, but I don't know if the proxy user is as a > property in the jobconf. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (MAPREDUCE-1744) DistributedCache creates its own FileSytem instance when adding a file/archive to the path
[ https://issues.apache.org/jira/browse/MAPREDUCE-1744?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krishna Ramachandran updated MAPREDUCE-1744: Status: Resolved (was: Patch Available) Resolution: Invalid Marking it as invalid should be addressed/fixed by https://issues.apache.org/jira/browse/MAPREDUCE-950 > DistributedCache creates its own FileSytem instance when adding a > file/archive to the path > -- > > Key: MAPREDUCE-1744 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-1744 > Project: Hadoop Map/Reduce > Issue Type: Bug >Reporter: Dick King >Assignee: Krishna Ramachandran > Attachments: BZ-3503564--2010-05-06.patch, h1744.patch, > mapred-1744-1.patch, mapred-1744-2.patch, mapred-1744-3.patch, > mapred-1744.patch, MAPREDUCE-1744.patch > > > According to the contract of {{UserGroupInformation.doAs()}} the only > required operations within the {{doAs()}} block are the > creation of a {{JobClient}} or getting a {{FileSystem}} . > The {{DistributedCache.add(File/Archive)ToClasspath()}} methods create a > {{FileSystem}} instance outside of the {{doAs()}} block, > this {{FileSystem}} instance is not in the scope of the proxy user but of the > superuser and permissions may make the method > fail. > One option is to overload the methods above to receive a filesystem. > Another option is to do obtain the {{FileSystem}} within a {{doAs()}} block, > for this it would be required to have the proxy > user set in the passed configuration. > The second option seems nicer, but I don't know if the proxy user is as a > property in the jobconf. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-1744) DistributedCache creates its own FileSytem instance when adding a file/archive to the path
[ https://issues.apache.org/jira/browse/MAPREDUCE-1744?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krishna Ramachandran updated MAPREDUCE-1744: Status: Patch Available (was: Open) this is an important fix for oozie Fix this or address 923 > DistributedCache creates its own FileSytem instance when adding a > file/archive to the path > -- > > Key: MAPREDUCE-1744 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-1744 > Project: Hadoop Map/Reduce > Issue Type: Bug >Reporter: Dick King >Assignee: Krishna Ramachandran > Attachments: BZ-3503564--2010-05-06.patch, h1744.patch, > mapred-1744-1.patch, mapred-1744-2.patch, mapred-1744-3.patch, > mapred-1744.patch, MAPREDUCE-1744.patch > > > According to the contract of {{UserGroupInformation.doAs()}} the only > required operations within the {{doAs()}} block are the > creation of a {{JobClient}} or getting a {{FileSystem}} . > The {{DistributedCache.add(File/Archive)ToClasspath()}} methods create a > {{FileSystem}} instance outside of the {{doAs()}} block, > this {{FileSystem}} instance is not in the scope of the proxy user but of the > superuser and permissions may make the method > fail. > One option is to overload the methods above to receive a filesystem. > Another option is to do obtain the {{FileSystem}} within a {{doAs()}} block, > for this it would be required to have the proxy > user set in the passed configuration. > The second option seems nicer, but I don't know if the proxy user is as a > property in the jobconf. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-1744) DistributedCache creates its own FileSytem instance when adding a file/archive to the path
[ https://issues.apache.org/jira/browse/MAPREDUCE-1744?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krishna Ramachandran updated MAPREDUCE-1744: Attachment: mapred-1744-3.patch After further discussion with Arun, it has been decided to put this on hold this until we reconcile with mapred-950 Just for completeness, I modified the existing APIs, add*toClassPath(Path ...) to be compatible > DistributedCache creates its own FileSytem instance when adding a > file/archive to the path > -- > > Key: MAPREDUCE-1744 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-1744 > Project: Hadoop Map/Reduce > Issue Type: Bug >Reporter: Dick King >Assignee: Krishna Ramachandran > Attachments: BZ-3503564--2010-05-06.patch, h1744.patch, > mapred-1744-1.patch, mapred-1744-2.patch, mapred-1744-3.patch, > mapred-1744.patch, MAPREDUCE-1744.patch > > > According to the contract of {{UserGroupInformation.doAs()}} the only > required operations within the {{doAs()}} block are the > creation of a {{JobClient}} or getting a {{FileSystem}} . > The {{DistributedCache.add(File/Archive)ToClasspath()}} methods create a > {{FileSystem}} instance outside of the {{doAs()}} block, > this {{FileSystem}} instance is not in the scope of the proxy user but of the > superuser and permissions may make the method > fail. > One option is to overload the methods above to receive a filesystem. > Another option is to do obtain the {{FileSystem}} within a {{doAs()}} block, > for this it would be required to have the proxy > user set in the passed configuration. > The second option seems nicer, but I don't know if the proxy user is as a > property in the jobconf. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-1744) DistributedCache creates its own FileSytem instance when adding a file/archive to the path
[ https://issues.apache.org/jira/browse/MAPREDUCE-1744?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun C Murthy updated MAPREDUCE-1744: - Status: Open (was: Patch Available) Krishna, you have made an incompatible change to Job.add(File,Archive)ToClassPath by changing it's signature (exceptions). Rather than calling cluster.getFileSystem you should use Path.getFileSystem(conf) on the path there. > DistributedCache creates its own FileSytem instance when adding a > file/archive to the path > -- > > Key: MAPREDUCE-1744 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-1744 > Project: Hadoop Map/Reduce > Issue Type: Bug >Reporter: Dick King >Assignee: Krishna Ramachandran > Attachments: BZ-3503564--2010-05-06.patch, h1744.patch, > mapred-1744-1.patch, mapred-1744-2.patch, mapred-1744.patch, > MAPREDUCE-1744.patch > > > According to the contract of {{UserGroupInformation.doAs()}} the only > required operations within the {{doAs()}} block are the > creation of a {{JobClient}} or getting a {{FileSystem}} . > The {{DistributedCache.add(File/Archive)ToClasspath()}} methods create a > {{FileSystem}} instance outside of the {{doAs()}} block, > this {{FileSystem}} instance is not in the scope of the proxy user but of the > superuser and permissions may make the method > fail. > One option is to overload the methods above to receive a filesystem. > Another option is to do obtain the {{FileSystem}} within a {{doAs()}} block, > for this it would be required to have the proxy > user set in the passed configuration. > The second option seems nicer, but I don't know if the proxy user is as a > property in the jobconf. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-1744) DistributedCache creates its own FileSytem instance when adding a file/archive to the path
[ https://issues.apache.org/jira/browse/MAPREDUCE-1744?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krishna Ramachandran updated MAPREDUCE-1744: Attachment: mapred-1744-2.patch restored * @deprecated Use {...@link Job# > DistributedCache creates its own FileSytem instance when adding a > file/archive to the path > -- > > Key: MAPREDUCE-1744 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-1744 > Project: Hadoop Map/Reduce > Issue Type: Bug >Reporter: Dick King >Assignee: Krishna Ramachandran > Attachments: BZ-3503564--2010-05-06.patch, h1744.patch, > mapred-1744-1.patch, mapred-1744-2.patch, mapred-1744.patch, > MAPREDUCE-1744.patch > > > According to the contract of {{UserGroupInformation.doAs()}} the only > required operations within the {{doAs()}} block are the > creation of a {{JobClient}} or getting a {{FileSystem}} . > The {{DistributedCache.add(File/Archive)ToClasspath()}} methods create a > {{FileSystem}} instance outside of the {{doAs()}} block, > this {{FileSystem}} instance is not in the scope of the proxy user but of the > superuser and permissions may make the method > fail. > One option is to overload the methods above to receive a filesystem. > Another option is to do obtain the {{FileSystem}} within a {{doAs()}} block, > for this it would be required to have the proxy > user set in the passed configuration. > The second option seems nicer, but I don't know if the proxy user is as a > property in the jobconf. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-1744) DistributedCache creates its own FileSytem instance when adding a file/archive to the path
[ https://issues.apache.org/jira/browse/MAPREDUCE-1744?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krishna Ramachandran updated MAPREDUCE-1744: Attachment: mapred-1744-1.patch Changes since Chris's last review Removed deprecated string and restored correct Strings (bad copy paste from an earlier release caused this) TestCommandLineJobSubmission failure is not probably related to this patch - it has random failures with or without this patch Though it passed with patch [junit] Running org.apache.hadoop.mapred.TestCommandLineJobSubmission [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 56.406 sec I get this error on a fresh checkout from trunk testshell/ExternalMapReduce java.lang.NoClassDefFoundError: testshell/ExternalMapReduce at org.apache.hadoop.mapred.TestCommandLineJobSubmission.testJobShell(TestCommandLineJobSubmission.java:122) Caused by: java.lang.ClassNotFoundException: testshell.ExternalMapReduce at java.net.URLClassLoader$1.run(URLClassLoader.java:200) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(URLClassLoader.java:188) at java.lang.ClassLoader.loadClass(ClassLoader.java:307) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301) at java.lang.ClassLoader.loadClass(ClassLoader.java:252) at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:320) > DistributedCache creates its own FileSytem instance when adding a > file/archive to the path > -- > > Key: MAPREDUCE-1744 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-1744 > Project: Hadoop Map/Reduce > Issue Type: Bug >Reporter: Dick King >Assignee: Krishna Ramachandran > Attachments: BZ-3503564--2010-05-06.patch, h1744.patch, > mapred-1744-1.patch, mapred-1744.patch, MAPREDUCE-1744.patch > > > According to the contract of {{UserGroupInformation.doAs()}} the only > required operations within the {{doAs()}} block are the > creation of a {{JobClient}} or getting a {{FileSystem}} . > The {{DistributedCache.add(File/Archive)ToClasspath()}} methods create a > {{FileSystem}} instance outside of the {{doAs()}} block, > this {{FileSystem}} instance is not in the scope of the proxy user but of the > superuser and permissions may make the method > fail. > One option is to overload the methods above to receive a filesystem. > Another option is to do obtain the {{FileSystem}} within a {{doAs()}} block, > for this it would be required to have the proxy > user set in the passed configuration. > The second option seems nicer, but I don't know if the proxy user is as a > property in the jobconf. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-1744) DistributedCache creates its own FileSytem instance when adding a file/archive to the path
[ https://issues.apache.org/jira/browse/MAPREDUCE-1744?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krishna Ramachandran updated MAPREDUCE-1744: Status: Patch Available (was: Open) Patch revised > DistributedCache creates its own FileSytem instance when adding a > file/archive to the path > -- > > Key: MAPREDUCE-1744 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-1744 > Project: Hadoop Map/Reduce > Issue Type: Bug >Reporter: Dick King >Assignee: Krishna Ramachandran > Attachments: BZ-3503564--2010-05-06.patch, h1744.patch, > mapred-1744-1.patch, mapred-1744.patch, MAPREDUCE-1744.patch > > > According to the contract of {{UserGroupInformation.doAs()}} the only > required operations within the {{doAs()}} block are the > creation of a {{JobClient}} or getting a {{FileSystem}} . > The {{DistributedCache.add(File/Archive)ToClasspath()}} methods create a > {{FileSystem}} instance outside of the {{doAs()}} block, > this {{FileSystem}} instance is not in the scope of the proxy user but of the > superuser and permissions may make the method > fail. > One option is to overload the methods above to receive a filesystem. > Another option is to do obtain the {{FileSystem}} within a {{doAs()}} block, > for this it would be required to have the proxy > user set in the passed configuration. > The second option seems nicer, but I don't know if the proxy user is as a > property in the jobconf. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-1744) DistributedCache creates its own FileSytem instance when adding a file/archive to the path
[ https://issues.apache.org/jira/browse/MAPREDUCE-1744?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Douglas updated MAPREDUCE-1744: - Status: Open (was: Patch Available) * {{MRJobConfig}} literals- {{CLASSPATH_FILES}} and {{CLASSPATH_ARCHIVES}}- have been replaced with deprecated String literals * The patch appears to cause {{TestCommandLineJobSubmission}} to fail * The patch changes the separator to be {{System.getProperty("path.separator")}}, but the cache property is parsed using {{StringUtils::getStringCollection}} that is hard-coded to split on the {{,}} character. > DistributedCache creates its own FileSytem instance when adding a > file/archive to the path > -- > > Key: MAPREDUCE-1744 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-1744 > Project: Hadoop Map/Reduce > Issue Type: Bug >Reporter: Dick King >Assignee: Krishna Ramachandran > Attachments: BZ-3503564--2010-05-06.patch, h1744.patch, > mapred-1744.patch, MAPREDUCE-1744.patch > > > According to the contract of {{UserGroupInformation.doAs()}} the only > required operations within the {{doAs()}} block are the > creation of a {{JobClient}} or getting a {{FileSystem}} . > The {{DistributedCache.add(File/Archive)ToClasspath()}} methods create a > {{FileSystem}} instance outside of the {{doAs()}} block, > this {{FileSystem}} instance is not in the scope of the proxy user but of the > superuser and permissions may make the method > fail. > One option is to overload the methods above to receive a filesystem. > Another option is to do obtain the {{FileSystem}} within a {{doAs()}} block, > for this it would be required to have the proxy > user set in the passed configuration. > The second option seems nicer, but I don't know if the proxy user is as a > property in the jobconf. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-1744) DistributedCache creates its own FileSytem instance when adding a file/archive to the path
[ https://issues.apache.org/jira/browse/MAPREDUCE-1744?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krishna Ramachandran updated MAPREDUCE-1744: Status: Patch Available (was: Open) > DistributedCache creates its own FileSytem instance when adding a > file/archive to the path > -- > > Key: MAPREDUCE-1744 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-1744 > Project: Hadoop Map/Reduce > Issue Type: Bug >Reporter: Dick King >Assignee: Krishna Ramachandran > Attachments: BZ-3503564--2010-05-06.patch, h1744.patch, > mapred-1744.patch, MAPREDUCE-1744.patch > > > According to the contract of {{UserGroupInformation.doAs()}} the only > required operations within the {{doAs()}} block are the > creation of a {{JobClient}} or getting a {{FileSystem}} . > The {{DistributedCache.add(File/Archive)ToClasspath()}} methods create a > {{FileSystem}} instance outside of the {{doAs()}} block, > this {{FileSystem}} instance is not in the scope of the proxy user but of the > superuser and permissions may make the method > fail. > One option is to overload the methods above to receive a filesystem. > Another option is to do obtain the {{FileSystem}} within a {{doAs()}} block, > for this it would be required to have the proxy > user set in the passed configuration. > The second option seems nicer, but I don't know if the proxy user is as a > property in the jobconf. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-1744) DistributedCache creates its own FileSytem instance when adding a file/archive to the path
[ https://issues.apache.org/jira/browse/MAPREDUCE-1744?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krishna Ramachandran updated MAPREDUCE-1744: Status: In Progress (was: Patch Available) > DistributedCache creates its own FileSytem instance when adding a > file/archive to the path > -- > > Key: MAPREDUCE-1744 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-1744 > Project: Hadoop Map/Reduce > Issue Type: Bug >Reporter: Dick King >Assignee: Krishna Ramachandran > Attachments: BZ-3503564--2010-05-06.patch, h1744.patch, > mapred-1744.patch, MAPREDUCE-1744.patch > > > According to the contract of {{UserGroupInformation.doAs()}} the only > required operations within the {{doAs()}} block are the > creation of a {{JobClient}} or getting a {{FileSystem}} . > The {{DistributedCache.add(File/Archive)ToClasspath()}} methods create a > {{FileSystem}} instance outside of the {{doAs()}} block, > this {{FileSystem}} instance is not in the scope of the proxy user but of the > superuser and permissions may make the method > fail. > One option is to overload the methods above to receive a filesystem. > Another option is to do obtain the {{FileSystem}} within a {{doAs()}} block, > for this it would be required to have the proxy > user set in the passed configuration. > The second option seems nicer, but I don't know if the proxy user is as a > property in the jobconf. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-1744) DistributedCache creates its own FileSytem instance when adding a file/archive to the path
[ https://issues.apache.org/jira/browse/MAPREDUCE-1744?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krishna Ramachandran updated MAPREDUCE-1744: Status: Patch Available (was: Open) Assignee: Krishna Ramachandran > DistributedCache creates its own FileSytem instance when adding a > file/archive to the path > -- > > Key: MAPREDUCE-1744 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-1744 > Project: Hadoop Map/Reduce > Issue Type: Bug >Reporter: Dick King >Assignee: Krishna Ramachandran > Attachments: BZ-3503564--2010-05-06.patch, h1744.patch, > mapred-1744.patch, MAPREDUCE-1744.patch > > > According to the contract of {{UserGroupInformation.doAs()}} the only > required operations within the {{doAs()}} block are the > creation of a {{JobClient}} or getting a {{FileSystem}} . > The {{DistributedCache.add(File/Archive)ToClasspath()}} methods create a > {{FileSystem}} instance outside of the {{doAs()}} block, > this {{FileSystem}} instance is not in the scope of the proxy user but of the > superuser and permissions may make the method > fail. > One option is to overload the methods above to receive a filesystem. > Another option is to do obtain the {{FileSystem}} within a {{doAs()}} block, > for this it would be required to have the proxy > user set in the passed configuration. > The second option seems nicer, but I don't know if the proxy user is as a > property in the jobconf. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-1744) DistributedCache creates its own FileSytem instance when adding a file/archive to the path
[ https://issues.apache.org/jira/browse/MAPREDUCE-1744?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dick King updated MAPREDUCE-1744: - Status: Open (was: Patch Available) > DistributedCache creates its own FileSytem instance when adding a > file/archive to the path > -- > > Key: MAPREDUCE-1744 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-1744 > Project: Hadoop Map/Reduce > Issue Type: Bug >Reporter: Dick King > Attachments: BZ-3503564--2010-05-06.patch, h1744.patch, > mapred-1744.patch, MAPREDUCE-1744.patch > > > According to the contract of {{UserGroupInformation.doAs()}} the only > required operations within the {{doAs()}} block are the > creation of a {{JobClient}} or getting a {{FileSystem}} . > The {{DistributedCache.add(File/Archive)ToClasspath()}} methods create a > {{FileSystem}} instance outside of the {{doAs()}} block, > this {{FileSystem}} instance is not in the scope of the proxy user but of the > superuser and permissions may make the method > fail. > One option is to overload the methods above to receive a filesystem. > Another option is to do obtain the {{FileSystem}} within a {{doAs()}} block, > for this it would be required to have the proxy > user set in the passed configuration. > The second option seems nicer, but I don't know if the proxy user is as a > property in the jobconf. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-1744) DistributedCache creates its own FileSytem instance when adding a file/archive to the path
[ https://issues.apache.org/jira/browse/MAPREDUCE-1744?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krishna Ramachandran updated MAPREDUCE-1744: Attachment: mapred-1744.patch amendment to existing patch > DistributedCache creates its own FileSytem instance when adding a > file/archive to the path > -- > > Key: MAPREDUCE-1744 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-1744 > Project: Hadoop Map/Reduce > Issue Type: Bug >Reporter: Dick King > Attachments: BZ-3503564--2010-05-06.patch, h1744.patch, > mapred-1744.patch, MAPREDUCE-1744.patch > > > According to the contract of {{UserGroupInformation.doAs()}} the only > required operations within the {{doAs()}} block are the > creation of a {{JobClient}} or getting a {{FileSystem}} . > The {{DistributedCache.add(File/Archive)ToClasspath()}} methods create a > {{FileSystem}} instance outside of the {{doAs()}} block, > this {{FileSystem}} instance is not in the scope of the proxy user but of the > superuser and permissions may make the method > fail. > One option is to overload the methods above to receive a filesystem. > Another option is to do obtain the {{FileSystem}} within a {{doAs()}} block, > for this it would be required to have the proxy > user set in the passed configuration. > The second option seems nicer, but I don't know if the proxy user is as a > property in the jobconf. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-1744) DistributedCache creates its own FileSytem instance when adding a file/archive to the path
[ https://issues.apache.org/jira/browse/MAPREDUCE-1744?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Douglas updated MAPREDUCE-1744: - Status: Open (was: Patch Available) > DistributedCache creates its own FileSytem instance when adding a > file/archive to the path > -- > > Key: MAPREDUCE-1744 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-1744 > Project: Hadoop Map/Reduce > Issue Type: Bug >Reporter: Dick King > Attachments: BZ-3503564--2010-05-06.patch, h1744.patch, > MAPREDUCE-1744.patch > > > According to the contract of {{UserGroupInformation.doAs()}} the only > required operations within the {{doAs()}} block are the > creation of a {{JobClient}} or getting a {{FileSystem}} . > The {{DistributedCache.add(File/Archive)ToClasspath()}} methods create a > {{FileSystem}} instance outside of the {{doAs()}} block, > this {{FileSystem}} instance is not in the scope of the proxy user but of the > superuser and permissions may make the method > fail. > One option is to overload the methods above to receive a filesystem. > Another option is to do obtain the {{FileSystem}} within a {{doAs()}} block, > for this it would be required to have the proxy > user set in the passed configuration. > The second option seems nicer, but I don't know if the proxy user is as a > property in the jobconf. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-1744) DistributedCache creates its own FileSytem instance when adding a file/archive to the path
[ https://issues.apache.org/jira/browse/MAPREDUCE-1744?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Douglas updated MAPREDUCE-1744: - Status: Patch Available (was: Open) > DistributedCache creates its own FileSytem instance when adding a > file/archive to the path > -- > > Key: MAPREDUCE-1744 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-1744 > Project: Hadoop Map/Reduce > Issue Type: Bug >Reporter: Dick King > Attachments: BZ-3503564--2010-05-06.patch, h1744.patch, > MAPREDUCE-1744.patch > > > According to the contract of {{UserGroupInformation.doAs()}} the only > required operations within the {{doAs()}} block are the > creation of a {{JobClient}} or getting a {{FileSystem}} . > The {{DistributedCache.add(File/Archive)ToClasspath()}} methods create a > {{FileSystem}} instance outside of the {{doAs()}} block, > this {{FileSystem}} instance is not in the scope of the proxy user but of the > superuser and permissions may make the method > fail. > One option is to overload the methods above to receive a filesystem. > Another option is to do obtain the {{FileSystem}} within a {{doAs()}} block, > for this it would be required to have the proxy > user set in the passed configuration. > The second option seems nicer, but I don't know if the proxy user is as a > property in the jobconf. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-1744) DistributedCache creates its own FileSytem instance when adding a file/archive to the path
[ https://issues.apache.org/jira/browse/MAPREDUCE-1744?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krishna Ramachandran updated MAPREDUCE-1744: Attachment: h1744.patch Forward port from y-20.1xx Modified DistributedCache - Add file/archive to class path and to cache. New parameter fs should be obtained within a doAs block test-patch ran successfully > DistributedCache creates its own FileSytem instance when adding a > file/archive to the path > -- > > Key: MAPREDUCE-1744 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-1744 > Project: Hadoop Map/Reduce > Issue Type: Bug >Reporter: Dick King > Attachments: BZ-3503564--2010-05-06.patch, h1744.patch, > MAPREDUCE-1744.patch > > > According to the contract of {{UserGroupInformation.doAs()}} the only > required operations within the {{doAs()}} block are the > creation of a {{JobClient}} or getting a {{FileSystem}} . > The {{DistributedCache.add(File/Archive)ToClasspath()}} methods create a > {{FileSystem}} instance outside of the {{doAs()}} block, > this {{FileSystem}} instance is not in the scope of the proxy user but of the > superuser and permissions may make the method > fail. > One option is to overload the methods above to receive a filesystem. > Another option is to do obtain the {{FileSystem}} within a {{doAs()}} block, > for this it would be required to have the proxy > user set in the passed configuration. > The second option seems nicer, but I don't know if the proxy user is as a > property in the jobconf. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-1744) DistributedCache creates its own FileSytem instance when adding a file/archive to the path
[ https://issues.apache.org/jira/browse/MAPREDUCE-1744?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krishna Ramachandran updated MAPREDUCE-1744: Status: Patch Available (was: Open) Modified DistributedCache methods to add a file to classpath and cache. New parameter fs should be obtained within a doAs block. Forward port from 20.1xx test-patch ran successfully > DistributedCache creates its own FileSytem instance when adding a > file/archive to the path > -- > > Key: MAPREDUCE-1744 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-1744 > Project: Hadoop Map/Reduce > Issue Type: Bug >Reporter: Dick King > Attachments: BZ-3503564--2010-05-06.patch, MAPREDUCE-1744.patch > > > According to the contract of {{UserGroupInformation.doAs()}} the only > required operations within the {{doAs()}} block are the > creation of a {{JobClient}} or getting a {{FileSystem}} . > The {{DistributedCache.add(File/Archive)ToClasspath()}} methods create a > {{FileSystem}} instance outside of the {{doAs()}} block, > this {{FileSystem}} instance is not in the scope of the proxy user but of the > superuser and permissions may make the method > fail. > One option is to overload the methods above to receive a filesystem. > Another option is to do obtain the {{FileSystem}} within a {{doAs()}} block, > for this it would be required to have the proxy > user set in the passed configuration. > The second option seems nicer, but I don't know if the proxy user is as a > property in the jobconf. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-1744) DistributedCache creates its own FileSytem instance when adding a file/archive to the path
[ https://issues.apache.org/jira/browse/MAPREDUCE-1744?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun C Murthy updated MAPREDUCE-1744: - Attachment: MAPREDUCE-1744.patch Updated patch on Dick's behalf, fixed a couple of test-cases to use the new apis as opposed to the deprecated ones. > DistributedCache creates its own FileSytem instance when adding a > file/archive to the path > -- > > Key: MAPREDUCE-1744 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-1744 > Project: Hadoop Map/Reduce > Issue Type: Bug >Reporter: Dick King > Attachments: BZ-3503564--2010-05-06.patch, MAPREDUCE-1744.patch > > > According to the contract of {{UserGroupInformation.doAs()}} the only > required operations within the {{doAs()}} block are the > creation of a {{JobClient}} or getting a {{FileSystem}} . > The {{DistributedCache.add(File/Archive)ToClasspath()}} methods create a > {{FileSystem}} instance outside of the {{doAs()}} block, > this {{FileSystem}} instance is not in the scope of the proxy user but of the > superuser and permissions may make the method > fail. > One option is to overload the methods above to receive a filesystem. > Another option is to do obtain the {{FileSystem}} within a {{doAs()}} block, > for this it would be required to have the proxy > user set in the passed configuration. > The second option seems nicer, but I don't know if the proxy user is as a > property in the jobconf. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-1744) DistributedCache creates its own FileSytem instance when adding a file/archive to the path
[ https://issues.apache.org/jira/browse/MAPREDUCE-1744?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dick King updated MAPREDUCE-1744: - Attachment: BZ-3503564--2010-05-06.patch This is a patch for this issue. We use the file system argument version. This patch applies to a different release of hadoop, and must be forward ported before we submit it. However, we do license it. > DistributedCache creates its own FileSytem instance when adding a > file/archive to the path > -- > > Key: MAPREDUCE-1744 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-1744 > Project: Hadoop Map/Reduce > Issue Type: Bug >Reporter: Dick King > Attachments: BZ-3503564--2010-05-06.patch > > > According to the contract of {{UserGroupInformation.doAs()}} the only > required operations within the {{doAs()}} block are the > creation of a {{JobClient}} or getting a {{FileSystem}} . > The {{DistributedCache.add(File/Archive)ToClasspath()}} methods create a > {{FileSystem}} instance outside of the {{doAs()}} block, > this {{FileSystem}} instance is not in the scope of the proxy user but of the > superuser and permissions may make the method > fail. > One option is to overload the methods above to receive a filesystem. > Another option is to do obtain the {{FileSystem}} within a {{doAs()}} block, > for this it would be required to have the proxy > user set in the passed configuration. > The second option seems nicer, but I don't know if the proxy user is as a > property in the jobconf. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.