[jira] [Updated] (MAPREDUCE-5217) DistCp fails when launched by Oozie in a secure cluster
[ https://issues.apache.org/jira/browse/MAPREDUCE-5217?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siddharth Seth updated MAPREDUCE-5217: -- Affects Version/s: (was: 2.0.4-alpha) (was: trunk) DistCp fails when launched by Oozie in a secure cluster --- Key: MAPREDUCE-5217 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5217 Project: Hadoop Map/Reduce Issue Type: Bug Components: distcp, security Affects Versions: 1.1.2 Environment: Hadoop secure cluster Reporter: Venkat Ranganathan Assignee: Venkat Ranganathan Attachments: MAPREDUCE-5217-branch-1.patch, MAPREDUCE-5217-branch-1.patch.2, MAPREDUCE-5217-branch-1.patch.3, MAPREDUCE-5217-trunk.patch, MAPREDUCE-5217-trunk.patch, MAPREDUCE-5217-trunk.patch.2, MAPREDUCE-5217-trunk.patch.3 As mentioned in MAPREDUCE-4324, Oozie has the following boilerplate code in in the main launcher for Pig, Hive, MR and Sqoop actions. if (System.getenv(HADOOP_TOKEN_FILE_LOCATION) != null) { jobConf.set(mapreduce.job.credentials.binary, System.getenv(HADOOP_TOKEN_FILE_LOCATION)); } For Java action, which does not have a main launcher in oozie, the above codecan be added by the user as the user purportedly has the code that is launched. But for DistCp action, the user has no such luxury. The solution attempted in MAPREDUCE-4324 would have helped DistCp, but it was not implemented as it would break MAPREDUCE-3727. So, we have to fix DistCp and add the same boilerplate code so that DistCp action can be launched by Oozie in a secure cluster. The code added checks for an System env. variable to be set which is not typically set in normal command line execution of DistCp, DistCp runs fine with commnad line usage both in secure and non-secure cluster. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-5217) DistCp fails when launched by Oozie in a secure cluster
[ https://issues.apache.org/jira/browse/MAPREDUCE-5217?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siddharth Seth updated MAPREDUCE-5217: -- Resolution: Fixed Fix Version/s: 1.3.0 Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Committed to branch-1. DistCp fails when launched by Oozie in a secure cluster --- Key: MAPREDUCE-5217 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5217 Project: Hadoop Map/Reduce Issue Type: Bug Components: distcp, security Affects Versions: 1.1.2 Environment: Hadoop secure cluster Reporter: Venkat Ranganathan Assignee: Venkat Ranganathan Fix For: 1.3.0 Attachments: MAPREDUCE-5217-branch-1.patch, MAPREDUCE-5217-branch-1.patch.2, MAPREDUCE-5217-branch-1.patch.3, MAPREDUCE-5217-trunk.patch, MAPREDUCE-5217-trunk.patch, MAPREDUCE-5217-trunk.patch.2, MAPREDUCE-5217-trunk.patch.3 As mentioned in MAPREDUCE-4324, Oozie has the following boilerplate code in in the main launcher for Pig, Hive, MR and Sqoop actions. if (System.getenv(HADOOP_TOKEN_FILE_LOCATION) != null) { jobConf.set(mapreduce.job.credentials.binary, System.getenv(HADOOP_TOKEN_FILE_LOCATION)); } For Java action, which does not have a main launcher in oozie, the above codecan be added by the user as the user purportedly has the code that is launched. But for DistCp action, the user has no such luxury. The solution attempted in MAPREDUCE-4324 would have helped DistCp, but it was not implemented as it would break MAPREDUCE-3727. So, we have to fix DistCp and add the same boilerplate code so that DistCp action can be launched by Oozie in a secure cluster. The code added checks for an System env. variable to be set which is not typically set in normal command line execution of DistCp, DistCp runs fine with commnad line usage both in secure and non-secure cluster. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-5217) DistCp fails when launched by Oozie in a secure cluster
[ https://issues.apache.org/jira/browse/MAPREDUCE-5217?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Venkat Ranganathan updated MAPREDUCE-5217: -- Attachment: MAPREDUCE-5217-branch-1.patch.3 Patch for branch1 DistCp fails when launched by Oozie in a secure cluster --- Key: MAPREDUCE-5217 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5217 Project: Hadoop Map/Reduce Issue Type: Bug Components: distcp, security Affects Versions: trunk, 1.1.2, 2.0.4-alpha Environment: Hadoop secure cluster Reporter: Venkat Ranganathan Assignee: Venkat Ranganathan Attachments: MAPREDUCE-5217-branch-1.patch, MAPREDUCE-5217-branch-1.patch.2, MAPREDUCE-5217-branch-1.patch.3, MAPREDUCE-5217-trunk.patch, MAPREDUCE-5217-trunk.patch, MAPREDUCE-5217-trunk.patch.2, MAPREDUCE-5217-trunk.patch.3 As mentioned in MAPREDUCE-4324, Oozie has the following boilerplate code in in the main launcher for Pig, Hive, MR and Sqoop actions. if (System.getenv(HADOOP_TOKEN_FILE_LOCATION) != null) { jobConf.set(mapreduce.job.credentials.binary, System.getenv(HADOOP_TOKEN_FILE_LOCATION)); } For Java action, which does not have a main launcher in oozie, the above codecan be added by the user as the user purportedly has the code that is launched. But for DistCp action, the user has no such luxury. The solution attempted in MAPREDUCE-4324 would have helped DistCp, but it was not implemented as it would break MAPREDUCE-3727. So, we have to fix DistCp and add the same boilerplate code so that DistCp action can be launched by Oozie in a secure cluster. The code added checks for an System env. variable to be set which is not typically set in normal command line execution of DistCp, DistCp runs fine with commnad line usage both in secure and non-secure cluster. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-5217) DistCp fails when launched by Oozie in a secure cluster
[ https://issues.apache.org/jira/browse/MAPREDUCE-5217?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jitendra Nath Pandey updated MAPREDUCE-5217: Affects Version/s: 2.0.4-alpha DistCp fails when launched by Oozie in a secure cluster --- Key: MAPREDUCE-5217 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5217 Project: Hadoop Map/Reduce Issue Type: Bug Components: distcp, security Affects Versions: trunk, 1.1.2, 2.0.4-alpha Environment: Hadoop secure cluster Reporter: Venkat Ranganathan Assignee: Venkat Ranganathan Attachments: MAPREDUCE-5217-branch-1.patch, MAPREDUCE-5217-trunk.patch, MAPREDUCE-5217-trunk.patch As mentioned in MAPREDUCE-4324, Oozie has the following boilerplate code in in the main launcher for Pig, Hive, MR and Sqoop actions. if (System.getenv(HADOOP_TOKEN_FILE_LOCATION) != null) { jobConf.set(mapreduce.job.credentials.binary, System.getenv(HADOOP_TOKEN_FILE_LOCATION)); } For Java action, which does not have a main launcher in oozie, the above codecan be added by the user as the user purportedly has the code that is launched. But for DistCp action, the user has no such luxury. The solution attempted in MAPREDUCE-4324 would have helped DistCp, but it was not implemented as it would break MAPREDUCE-3727. So, we have to fix DistCp and add the same boilerplate code so that DistCp action can be launched by Oozie in a secure cluster. The code added checks for an System env. variable to be set which is not typically set in normal command line execution of DistCp, DistCp runs fine with commnad line usage both in secure and non-secure cluster. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-5217) DistCp fails when launched by Oozie in a secure cluster
[ https://issues.apache.org/jira/browse/MAPREDUCE-5217?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Venkat Ranganathan updated MAPREDUCE-5217: -- Attachment: MAPREDUCE-5217-trunk.patch.2 DistCp fails when launched by Oozie in a secure cluster --- Key: MAPREDUCE-5217 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5217 Project: Hadoop Map/Reduce Issue Type: Bug Components: distcp, security Affects Versions: trunk, 1.1.2, 2.0.4-alpha Environment: Hadoop secure cluster Reporter: Venkat Ranganathan Assignee: Venkat Ranganathan Attachments: MAPREDUCE-5217-branch-1.patch, MAPREDUCE-5217-trunk.patch, MAPREDUCE-5217-trunk.patch, MAPREDUCE-5217-trunk.patch.2 As mentioned in MAPREDUCE-4324, Oozie has the following boilerplate code in in the main launcher for Pig, Hive, MR and Sqoop actions. if (System.getenv(HADOOP_TOKEN_FILE_LOCATION) != null) { jobConf.set(mapreduce.job.credentials.binary, System.getenv(HADOOP_TOKEN_FILE_LOCATION)); } For Java action, which does not have a main launcher in oozie, the above codecan be added by the user as the user purportedly has the code that is launched. But for DistCp action, the user has no such luxury. The solution attempted in MAPREDUCE-4324 would have helped DistCp, but it was not implemented as it would break MAPREDUCE-3727. So, we have to fix DistCp and add the same boilerplate code so that DistCp action can be launched by Oozie in a secure cluster. The code added checks for an System env. variable to be set which is not typically set in normal command line execution of DistCp, DistCp runs fine with commnad line usage both in secure and non-secure cluster. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-5217) DistCp fails when launched by Oozie in a secure cluster
[ https://issues.apache.org/jira/browse/MAPREDUCE-5217?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Venkat Ranganathan updated MAPREDUCE-5217: -- Attachment: MAPREDUCE-5217-branch-1.patch.2 branch-1 patch after addressing Jitendra's comments DistCp fails when launched by Oozie in a secure cluster --- Key: MAPREDUCE-5217 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5217 Project: Hadoop Map/Reduce Issue Type: Bug Components: distcp, security Affects Versions: trunk, 1.1.2, 2.0.4-alpha Environment: Hadoop secure cluster Reporter: Venkat Ranganathan Assignee: Venkat Ranganathan Attachments: MAPREDUCE-5217-branch-1.patch, MAPREDUCE-5217-branch-1.patch.2, MAPREDUCE-5217-trunk.patch, MAPREDUCE-5217-trunk.patch, MAPREDUCE-5217-trunk.patch.2 As mentioned in MAPREDUCE-4324, Oozie has the following boilerplate code in in the main launcher for Pig, Hive, MR and Sqoop actions. if (System.getenv(HADOOP_TOKEN_FILE_LOCATION) != null) { jobConf.set(mapreduce.job.credentials.binary, System.getenv(HADOOP_TOKEN_FILE_LOCATION)); } For Java action, which does not have a main launcher in oozie, the above codecan be added by the user as the user purportedly has the code that is launched. But for DistCp action, the user has no such luxury. The solution attempted in MAPREDUCE-4324 would have helped DistCp, but it was not implemented as it would break MAPREDUCE-3727. So, we have to fix DistCp and add the same boilerplate code so that DistCp action can be launched by Oozie in a secure cluster. The code added checks for an System env. variable to be set which is not typically set in normal command line execution of DistCp, DistCp runs fine with commnad line usage both in secure and non-secure cluster. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-5217) DistCp fails when launched by Oozie in a secure cluster
[ https://issues.apache.org/jira/browse/MAPREDUCE-5217?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Venkat Ranganathan updated MAPREDUCE-5217: -- Attachment: MAPREDUCE-5217-trunk.patch.3 DistCp fails when launched by Oozie in a secure cluster --- Key: MAPREDUCE-5217 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5217 Project: Hadoop Map/Reduce Issue Type: Bug Components: distcp, security Affects Versions: trunk, 1.1.2, 2.0.4-alpha Environment: Hadoop secure cluster Reporter: Venkat Ranganathan Assignee: Venkat Ranganathan Attachments: MAPREDUCE-5217-branch-1.patch, MAPREDUCE-5217-branch-1.patch.2, MAPREDUCE-5217-trunk.patch, MAPREDUCE-5217-trunk.patch, MAPREDUCE-5217-trunk.patch.2, MAPREDUCE-5217-trunk.patch.3 As mentioned in MAPREDUCE-4324, Oozie has the following boilerplate code in in the main launcher for Pig, Hive, MR and Sqoop actions. if (System.getenv(HADOOP_TOKEN_FILE_LOCATION) != null) { jobConf.set(mapreduce.job.credentials.binary, System.getenv(HADOOP_TOKEN_FILE_LOCATION)); } For Java action, which does not have a main launcher in oozie, the above codecan be added by the user as the user purportedly has the code that is launched. But for DistCp action, the user has no such luxury. The solution attempted in MAPREDUCE-4324 would have helped DistCp, but it was not implemented as it would break MAPREDUCE-3727. So, we have to fix DistCp and add the same boilerplate code so that DistCp action can be launched by Oozie in a secure cluster. The code added checks for an System env. variable to be set which is not typically set in normal command line execution of DistCp, DistCp runs fine with commnad line usage both in secure and non-secure cluster. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-5217) DistCp fails when launched by Oozie in a secure cluster
[ https://issues.apache.org/jira/browse/MAPREDUCE-5217?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Venkat Ranganathan updated MAPREDUCE-5217: -- Attachment: MAPREDUCE-5217.patch DistCp fails when launched by Oozie in a secure cluster --- Key: MAPREDUCE-5217 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5217 Project: Hadoop Map/Reduce Issue Type: Bug Components: distcp Affects Versions: 1.1.2 Environment: Hadoop secure cluster Reporter: Venkat Ranganathan Assignee: Venkat Ranganathan Attachments: MAPREDUCE-5217.patch As mentioned in MAPREDUCE-4324, Oozie has the following boilerplate code in in the main launcher for Pig, Hive, MR and Sqoop actions. if (System.getenv(HADOOP_TOKEN_FILE_LOCATION) != null) { jobConf.set(mapreduce.job.credentials.binary, System.getenv(HADOOP_TOKEN_FILE_LOCATION)); } For Java action, which does not have a main launcher in oozie, the above codecan be added by the user as the user purportedly has the code that is launched. But for DistCp action, the user has no such luxury. The solution attempted in MAPREDUCE-4324 would have helped DistCp, but it was not implemented as it would break MAPREDUCE-3727. So, we have to fix DistCp and add the same boilerplate code so that DistCp action can be launched by Oozie in a secure cluster. The code added checks for an System env. variable to be set which is not typically set in normal command line execution of DistCp, DistCp runs fine with commnad line usage both in secure and non-secure cluster. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-5217) DistCp fails when launched by Oozie in a secure cluster
[ https://issues.apache.org/jira/browse/MAPREDUCE-5217?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Venkat Ranganathan updated MAPREDUCE-5217: -- Status: Patch Available (was: Open) DistCp fails when launched by Oozie in a secure cluster --- Key: MAPREDUCE-5217 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5217 Project: Hadoop Map/Reduce Issue Type: Bug Components: distcp Affects Versions: 1.1.2 Environment: Hadoop secure cluster Reporter: Venkat Ranganathan Assignee: Venkat Ranganathan Attachments: MAPREDUCE-5217.patch As mentioned in MAPREDUCE-4324, Oozie has the following boilerplate code in in the main launcher for Pig, Hive, MR and Sqoop actions. if (System.getenv(HADOOP_TOKEN_FILE_LOCATION) != null) { jobConf.set(mapreduce.job.credentials.binary, System.getenv(HADOOP_TOKEN_FILE_LOCATION)); } For Java action, which does not have a main launcher in oozie, the above codecan be added by the user as the user purportedly has the code that is launched. But for DistCp action, the user has no such luxury. The solution attempted in MAPREDUCE-4324 would have helped DistCp, but it was not implemented as it would break MAPREDUCE-3727. So, we have to fix DistCp and add the same boilerplate code so that DistCp action can be launched by Oozie in a secure cluster. The code added checks for an System env. variable to be set which is not typically set in normal command line execution of DistCp, DistCp runs fine with commnad line usage both in secure and non-secure cluster. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-5217) DistCp fails when launched by Oozie in a secure cluster
[ https://issues.apache.org/jira/browse/MAPREDUCE-5217?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Venkat Ranganathan updated MAPREDUCE-5217: -- Status: Open (was: Patch Available) Uploading trunk patch DistCp fails when launched by Oozie in a secure cluster --- Key: MAPREDUCE-5217 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5217 Project: Hadoop Map/Reduce Issue Type: Bug Components: distcp Affects Versions: 1.1.2 Environment: Hadoop secure cluster Reporter: Venkat Ranganathan Assignee: Venkat Ranganathan Attachments: MAPREDUCE-5217.patch As mentioned in MAPREDUCE-4324, Oozie has the following boilerplate code in in the main launcher for Pig, Hive, MR and Sqoop actions. if (System.getenv(HADOOP_TOKEN_FILE_LOCATION) != null) { jobConf.set(mapreduce.job.credentials.binary, System.getenv(HADOOP_TOKEN_FILE_LOCATION)); } For Java action, which does not have a main launcher in oozie, the above codecan be added by the user as the user purportedly has the code that is launched. But for DistCp action, the user has no such luxury. The solution attempted in MAPREDUCE-4324 would have helped DistCp, but it was not implemented as it would break MAPREDUCE-3727. So, we have to fix DistCp and add the same boilerplate code so that DistCp action can be launched by Oozie in a secure cluster. The code added checks for an System env. variable to be set which is not typically set in normal command line execution of DistCp, DistCp runs fine with commnad line usage both in secure and non-secure cluster. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-5217) DistCp fails when launched by Oozie in a secure cluster
[ https://issues.apache.org/jira/browse/MAPREDUCE-5217?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Venkat Ranganathan updated MAPREDUCE-5217: -- Component/s: security DistCp fails when launched by Oozie in a secure cluster --- Key: MAPREDUCE-5217 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5217 Project: Hadoop Map/Reduce Issue Type: Bug Components: distcp, security Affects Versions: 1.1.2 Environment: Hadoop secure cluster Reporter: Venkat Ranganathan Assignee: Venkat Ranganathan As mentioned in MAPREDUCE-4324, Oozie has the following boilerplate code in in the main launcher for Pig, Hive, MR and Sqoop actions. if (System.getenv(HADOOP_TOKEN_FILE_LOCATION) != null) { jobConf.set(mapreduce.job.credentials.binary, System.getenv(HADOOP_TOKEN_FILE_LOCATION)); } For Java action, which does not have a main launcher in oozie, the above codecan be added by the user as the user purportedly has the code that is launched. But for DistCp action, the user has no such luxury. The solution attempted in MAPREDUCE-4324 would have helped DistCp, but it was not implemented as it would break MAPREDUCE-3727. So, we have to fix DistCp and add the same boilerplate code so that DistCp action can be launched by Oozie in a secure cluster. The code added checks for an System env. variable to be set which is not typically set in normal command line execution of DistCp, DistCp runs fine with commnad line usage both in secure and non-secure cluster. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-5217) DistCp fails when launched by Oozie in a secure cluster
[ https://issues.apache.org/jira/browse/MAPREDUCE-5217?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Venkat Ranganathan updated MAPREDUCE-5217: -- Attachment: (was: MAPREDUCE-5217.patch) DistCp fails when launched by Oozie in a secure cluster --- Key: MAPREDUCE-5217 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5217 Project: Hadoop Map/Reduce Issue Type: Bug Components: distcp, security Affects Versions: 1.1.2 Environment: Hadoop secure cluster Reporter: Venkat Ranganathan Assignee: Venkat Ranganathan As mentioned in MAPREDUCE-4324, Oozie has the following boilerplate code in in the main launcher for Pig, Hive, MR and Sqoop actions. if (System.getenv(HADOOP_TOKEN_FILE_LOCATION) != null) { jobConf.set(mapreduce.job.credentials.binary, System.getenv(HADOOP_TOKEN_FILE_LOCATION)); } For Java action, which does not have a main launcher in oozie, the above codecan be added by the user as the user purportedly has the code that is launched. But for DistCp action, the user has no such luxury. The solution attempted in MAPREDUCE-4324 would have helped DistCp, but it was not implemented as it would break MAPREDUCE-3727. So, we have to fix DistCp and add the same boilerplate code so that DistCp action can be launched by Oozie in a secure cluster. The code added checks for an System env. variable to be set which is not typically set in normal command line execution of DistCp, DistCp runs fine with commnad line usage both in secure and non-secure cluster. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-5217) DistCp fails when launched by Oozie in a secure cluster
[ https://issues.apache.org/jira/browse/MAPREDUCE-5217?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Venkat Ranganathan updated MAPREDUCE-5217: -- Attachment: MAPREDUCE-5217-trunk.patch DistCp fails when launched by Oozie in a secure cluster --- Key: MAPREDUCE-5217 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5217 Project: Hadoop Map/Reduce Issue Type: Bug Components: distcp, security Affects Versions: 1.1.2 Environment: Hadoop secure cluster Reporter: Venkat Ranganathan Assignee: Venkat Ranganathan Attachments: MAPREDUCE-5217-trunk.patch As mentioned in MAPREDUCE-4324, Oozie has the following boilerplate code in in the main launcher for Pig, Hive, MR and Sqoop actions. if (System.getenv(HADOOP_TOKEN_FILE_LOCATION) != null) { jobConf.set(mapreduce.job.credentials.binary, System.getenv(HADOOP_TOKEN_FILE_LOCATION)); } For Java action, which does not have a main launcher in oozie, the above codecan be added by the user as the user purportedly has the code that is launched. But for DistCp action, the user has no such luxury. The solution attempted in MAPREDUCE-4324 would have helped DistCp, but it was not implemented as it would break MAPREDUCE-3727. So, we have to fix DistCp and add the same boilerplate code so that DistCp action can be launched by Oozie in a secure cluster. The code added checks for an System env. variable to be set which is not typically set in normal command line execution of DistCp, DistCp runs fine with commnad line usage both in secure and non-secure cluster. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-5217) DistCp fails when launched by Oozie in a secure cluster
[ https://issues.apache.org/jira/browse/MAPREDUCE-5217?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Venkat Ranganathan updated MAPREDUCE-5217: -- Affects Version/s: trunk Status: Patch Available (was: Open) The same issue is in trunk DistCp fails when launched by Oozie in a secure cluster --- Key: MAPREDUCE-5217 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5217 Project: Hadoop Map/Reduce Issue Type: Bug Components: distcp, security Affects Versions: 1.1.2, trunk Environment: Hadoop secure cluster Reporter: Venkat Ranganathan Assignee: Venkat Ranganathan Attachments: MAPREDUCE-5217-trunk.patch As mentioned in MAPREDUCE-4324, Oozie has the following boilerplate code in in the main launcher for Pig, Hive, MR and Sqoop actions. if (System.getenv(HADOOP_TOKEN_FILE_LOCATION) != null) { jobConf.set(mapreduce.job.credentials.binary, System.getenv(HADOOP_TOKEN_FILE_LOCATION)); } For Java action, which does not have a main launcher in oozie, the above codecan be added by the user as the user purportedly has the code that is launched. But for DistCp action, the user has no such luxury. The solution attempted in MAPREDUCE-4324 would have helped DistCp, but it was not implemented as it would break MAPREDUCE-3727. So, we have to fix DistCp and add the same boilerplate code so that DistCp action can be launched by Oozie in a secure cluster. The code added checks for an System env. variable to be set which is not typically set in normal command line execution of DistCp, DistCp runs fine with commnad line usage both in secure and non-secure cluster. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-5217) DistCp fails when launched by Oozie in a secure cluster
[ https://issues.apache.org/jira/browse/MAPREDUCE-5217?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Venkat Ranganathan updated MAPREDUCE-5217: -- Status: Open (was: Patch Available) Uploaded wrong patch DistCp fails when launched by Oozie in a secure cluster --- Key: MAPREDUCE-5217 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5217 Project: Hadoop Map/Reduce Issue Type: Bug Components: distcp, security Affects Versions: 1.1.2, trunk Environment: Hadoop secure cluster Reporter: Venkat Ranganathan Assignee: Venkat Ranganathan Attachments: MAPREDUCE-5217-trunk.patch, MAPREDUCE-5217-trunk.patch As mentioned in MAPREDUCE-4324, Oozie has the following boilerplate code in in the main launcher for Pig, Hive, MR and Sqoop actions. if (System.getenv(HADOOP_TOKEN_FILE_LOCATION) != null) { jobConf.set(mapreduce.job.credentials.binary, System.getenv(HADOOP_TOKEN_FILE_LOCATION)); } For Java action, which does not have a main launcher in oozie, the above codecan be added by the user as the user purportedly has the code that is launched. But for DistCp action, the user has no such luxury. The solution attempted in MAPREDUCE-4324 would have helped DistCp, but it was not implemented as it would break MAPREDUCE-3727. So, we have to fix DistCp and add the same boilerplate code so that DistCp action can be launched by Oozie in a secure cluster. The code added checks for an System env. variable to be set which is not typically set in normal command line execution of DistCp, DistCp runs fine with commnad line usage both in secure and non-secure cluster. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-5217) DistCp fails when launched by Oozie in a secure cluster
[ https://issues.apache.org/jira/browse/MAPREDUCE-5217?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Venkat Ranganathan updated MAPREDUCE-5217: -- Attachment: MAPREDUCE-5217-trunk.patch DistCp fails when launched by Oozie in a secure cluster --- Key: MAPREDUCE-5217 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5217 Project: Hadoop Map/Reduce Issue Type: Bug Components: distcp, security Affects Versions: trunk, 1.1.2 Environment: Hadoop secure cluster Reporter: Venkat Ranganathan Assignee: Venkat Ranganathan Attachments: MAPREDUCE-5217-trunk.patch, MAPREDUCE-5217-trunk.patch As mentioned in MAPREDUCE-4324, Oozie has the following boilerplate code in in the main launcher for Pig, Hive, MR and Sqoop actions. if (System.getenv(HADOOP_TOKEN_FILE_LOCATION) != null) { jobConf.set(mapreduce.job.credentials.binary, System.getenv(HADOOP_TOKEN_FILE_LOCATION)); } For Java action, which does not have a main launcher in oozie, the above codecan be added by the user as the user purportedly has the code that is launched. But for DistCp action, the user has no such luxury. The solution attempted in MAPREDUCE-4324 would have helped DistCp, but it was not implemented as it would break MAPREDUCE-3727. So, we have to fix DistCp and add the same boilerplate code so that DistCp action can be launched by Oozie in a secure cluster. The code added checks for an System env. variable to be set which is not typically set in normal command line execution of DistCp, DistCp runs fine with commnad line usage both in secure and non-secure cluster. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-5217) DistCp fails when launched by Oozie in a secure cluster
[ https://issues.apache.org/jira/browse/MAPREDUCE-5217?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Venkat Ranganathan updated MAPREDUCE-5217: -- Status: Patch Available (was: Open) DistCp fails when launched by Oozie in a secure cluster --- Key: MAPREDUCE-5217 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5217 Project: Hadoop Map/Reduce Issue Type: Bug Components: distcp, security Affects Versions: 1.1.2, trunk Environment: Hadoop secure cluster Reporter: Venkat Ranganathan Assignee: Venkat Ranganathan Attachments: MAPREDUCE-5217-trunk.patch, MAPREDUCE-5217-trunk.patch As mentioned in MAPREDUCE-4324, Oozie has the following boilerplate code in in the main launcher for Pig, Hive, MR and Sqoop actions. if (System.getenv(HADOOP_TOKEN_FILE_LOCATION) != null) { jobConf.set(mapreduce.job.credentials.binary, System.getenv(HADOOP_TOKEN_FILE_LOCATION)); } For Java action, which does not have a main launcher in oozie, the above codecan be added by the user as the user purportedly has the code that is launched. But for DistCp action, the user has no such luxury. The solution attempted in MAPREDUCE-4324 would have helped DistCp, but it was not implemented as it would break MAPREDUCE-3727. So, we have to fix DistCp and add the same boilerplate code so that DistCp action can be launched by Oozie in a secure cluster. The code added checks for an System env. variable to be set which is not typically set in normal command line execution of DistCp, DistCp runs fine with commnad line usage both in secure and non-secure cluster. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-5217) DistCp fails when launched by Oozie in a secure cluster
[ https://issues.apache.org/jira/browse/MAPREDUCE-5217?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Venkat Ranganathan updated MAPREDUCE-5217: -- Attachment: MAPREDUCE-5217-branch-1.patch Patch for branch-1 DistCp fails when launched by Oozie in a secure cluster --- Key: MAPREDUCE-5217 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5217 Project: Hadoop Map/Reduce Issue Type: Bug Components: distcp, security Affects Versions: trunk, 1.1.2 Environment: Hadoop secure cluster Reporter: Venkat Ranganathan Assignee: Venkat Ranganathan Attachments: MAPREDUCE-5217-branch-1.patch, MAPREDUCE-5217-trunk.patch, MAPREDUCE-5217-trunk.patch As mentioned in MAPREDUCE-4324, Oozie has the following boilerplate code in in the main launcher for Pig, Hive, MR and Sqoop actions. if (System.getenv(HADOOP_TOKEN_FILE_LOCATION) != null) { jobConf.set(mapreduce.job.credentials.binary, System.getenv(HADOOP_TOKEN_FILE_LOCATION)); } For Java action, which does not have a main launcher in oozie, the above codecan be added by the user as the user purportedly has the code that is launched. But for DistCp action, the user has no such luxury. The solution attempted in MAPREDUCE-4324 would have helped DistCp, but it was not implemented as it would break MAPREDUCE-3727. So, we have to fix DistCp and add the same boilerplate code so that DistCp action can be launched by Oozie in a secure cluster. The code added checks for an System env. variable to be set which is not typically set in normal command line execution of DistCp, DistCp runs fine with commnad line usage both in secure and non-secure cluster. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira