[jira] [Commented] (OOZIE-3567) Oozie ShellAction should support absolute bash file path
[ https://issues.apache.org/jira/browse/OOZIE-3567?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16990986#comment-16990986 ] Roey Shem Tov commented on OOZIE-3567: -- [~asalamon74] any suggestions ? > Oozie ShellAction should support absolute bash file path > > > Key: OOZIE-3567 > URL: https://issues.apache.org/jira/browse/OOZIE-3567 > Project: Oozie > Issue Type: Improvement > Components: action >Affects Versions: 4.2.0, 5.0.0, 5.1.0 >Reporter: Roey Shem Tov >Priority: Major > Labels: patch-available > Fix For: 5.3.0 > > Attachments: > 0001-OOZIE-3567-Oozie-ShellAction-should-support-absolute.patch > > > as related in OOZIE-3320 im opening a new PR (me from different user), still > can`t execute shell from mount at excpected. > > File is not checked to be existed on resolveExecutable, because mount can be > at the workers but not at the oozie servers (very common use). -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (OOZIE-3567) Oozie ShellAction should support absolute bash file path
[ https://issues.apache.org/jira/browse/OOZIE-3567?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16989874#comment-16989874 ] Roey Shem Tov commented on OOZIE-3567: -- [~asalamon74] can help here why patch is failing? i upload a new one. > Oozie ShellAction should support absolute bash file path > > > Key: OOZIE-3567 > URL: https://issues.apache.org/jira/browse/OOZIE-3567 > Project: Oozie > Issue Type: Improvement > Components: action >Affects Versions: 4.2.0, 5.0.0, 5.1.0 >Reporter: Roey Shem Tov >Priority: Major > Labels: patch-available > Fix For: 5.3.0 > > Attachments: > 0001-OOZIE-3567-Oozie-ShellAction-should-support-absolute.patch > > > as related in OOZIE-3320 im opening a new PR (me from different user), still > can`t execute shell from mount at excpected. > > File is not checked to be existed on resolveExecutable, because mount can be > at the workers but not at the oozie servers (very common use). -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (OOZIE-3567) Oozie ShellAction should support absolute bash file path
[ https://issues.apache.org/jira/browse/OOZIE-3567?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Roey Shem Tov updated OOZIE-3567: - Attachment: 0001-OOZIE-3567-Oozie-ShellAction-should-support-absolute.patch > Oozie ShellAction should support absolute bash file path > > > Key: OOZIE-3567 > URL: https://issues.apache.org/jira/browse/OOZIE-3567 > Project: Oozie > Issue Type: Improvement > Components: action >Affects Versions: 4.2.0, 5.0.0, 5.1.0 >Reporter: Roey Shem Tov >Priority: Major > Labels: patch-available > Fix For: 5.3.0 > > Attachments: > 0001-OOZIE-3567-Oozie-ShellAction-should-support-absolute.patch > > > as related in OOZIE-3320 im opening a new PR (me from different user), still > can`t execute shell from mount at excpected. > > File is not checked to be existed on resolveExecutable, because mount can be > at the workers but not at the oozie servers (very common use). -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (OOZIE-3567) Oozie ShellAction should support absolute bash file path
[ https://issues.apache.org/jira/browse/OOZIE-3567?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Roey Shem Tov updated OOZIE-3567: - Attachment: (was: [OOZIE-3567]_Oozie_ShellAction_should_support_absolute_bash_file_path.patch) > Oozie ShellAction should support absolute bash file path > > > Key: OOZIE-3567 > URL: https://issues.apache.org/jira/browse/OOZIE-3567 > Project: Oozie > Issue Type: Improvement > Components: action >Affects Versions: 4.2.0, 5.0.0, 5.1.0 >Reporter: Roey Shem Tov >Priority: Major > Labels: patch-available > Fix For: 5.3.0 > > > as related in OOZIE-3320 im opening a new PR (me from different user), still > can`t execute shell from mount at excpected. > > File is not checked to be existed on resolveExecutable, because mount can be > at the workers but not at the oozie servers (very common use). -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (OOZIE-3320) Oozie ShellAction should support absolute bash file path
[
https://issues.apache.org/jira/browse/OOZIE-3320?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16989811#comment-16989811
]
Roey Shem Tov commented on OOZIE-3320:
--
Can you close the issue since i open it at OOZIE-3567?
> Oozie ShellAction should support absolute bash file path
>
>
> Key: OOZIE-3320
> URL: https://issues.apache.org/jira/browse/OOZIE-3320
> Project: Oozie
> Issue Type: Improvement
> Components: action
>Affects Versions: 5.0.0, 4.3.1
>Reporter: Roey Shem Tov
>Assignee: Roey Shem Tov
>Priority: Major
> Labels: patch
> Attachments: OOZIE-3086.patch, OOZIE-3320.patch
>
>
> bash files that saved on shared mount, cannot be execute by the ShellAction
> in proper way.
> Example:
> Worker-1,Worker-2,Worker-3 have shared mount /mnt/hadoop
> on /mnt/hadoop there is a file script.sh
> Right now there is two options to submit it using ShellAction:
> # Upload it to hdfs, add it as a file and submit script.sh
> # use bash as exec and file location (/mnt/hadoop/script.sh) as argument
> (e.g bash>/exec>/mnt/hadoop/script.sh
> Best option is that the command will support shared mounted file :
> /mnt/hadoop/script.sh
>
> This code is taking only the file name instead it full path:
> {code:java}
> String exec = actionXml.getChild("exec", ns).getTextTrim();
> String execName = new Path(exec).getName();
> actionConf.set(ShellMain.CONF_OOZIE_SHELL_EXEC, execName);
> {code}
>
> Best option to support shared mount file is to support file:// starting for
> bash files that are local (or shared by mount), e.g:
> {code:java}
> String exec = actionXml.getChild("exec", ns).getTextTrim(); String execName;
> String localFilePrefix = "file://";
> // When exec starts with 'file://' refer it as local file.
> if (exec.startsWith(localFilePrefix))
> execName = exec.substring(localFilePrefix.length());
> else execName = new Path(exec).getName();
> actionConf.set(ShellMain.CONF_OOZIE_SHELL_EXEC, execName);
> {code}
>
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (OOZIE-3567) Oozie ShellAction should support absolute bash file path
[ https://issues.apache.org/jira/browse/OOZIE-3567?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Roey Shem Tov updated OOZIE-3567: - Flags: Patch (was: Important) Labels: patch-available (was: ) > Oozie ShellAction should support absolute bash file path > > > Key: OOZIE-3567 > URL: https://issues.apache.org/jira/browse/OOZIE-3567 > Project: Oozie > Issue Type: Improvement > Components: action >Affects Versions: 4.2.0, 5.0.0, 5.1.0 >Reporter: Roey Shem Tov >Priority: Major > Labels: patch-available > Fix For: 5.3.0 > > Attachments: > [OOZIE-3567]_Oozie_ShellAction_should_support_absolute_bash_file_path.patch > > > as related in OOZIE-3320 im opening a new PR (me from different user), still > can`t execute shell from mount at excpected. > > File is not checked to be existed on resolveExecutable, because mount can be > at the workers but not at the oozie servers (very common use). -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (OOZIE-3567) Oozie ShellAction should support absolute bash file path
[ https://issues.apache.org/jira/browse/OOZIE-3567?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Roey Shem Tov updated OOZIE-3567: - Attachment: (was: [OOZIE-3567]_Oozie_ShellAction_should_support_absolute_bash_file_path.patch) > Oozie ShellAction should support absolute bash file path > > > Key: OOZIE-3567 > URL: https://issues.apache.org/jira/browse/OOZIE-3567 > Project: Oozie > Issue Type: Improvement > Components: action >Affects Versions: 4.2.0, 5.0.0, 5.1.0 >Reporter: Roey Shem Tov >Priority: Major > Fix For: 5.3.0 > > Attachments: > [OOZIE-3567]_Oozie_ShellAction_should_support_absolute_bash_file_path.patch > > > as related in OOZIE-3320 im opening a new PR (me from different user), still > can`t execute shell from mount at excpected. > > File is not checked to be existed on resolveExecutable, because mount can be > at the workers but not at the oozie servers (very common use). -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Comment Edited] (OOZIE-3567) Oozie ShellAction should support absolute bash file path
[ https://issues.apache.org/jira/browse/OOZIE-3567?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16989799#comment-16989799 ] Roey Shem Tov edited comment on OOZIE-3567 at 12/6/19 2:16 PM: --- [~asalamon74] No different at all, I opened OOZIE-3320 a year ago(from another user) and saw right now it wasn`t resolved, so i prefer to resolve it from here (i changed my main user at jira+github). was (Author: roeyshemtov): [~asalamon74] No different at all, I opened OOZIE-3320 (from another user) and saw right now this wasn`t resolved, so i prefer to keep it from here (i changed main user at jira+github). > Oozie ShellAction should support absolute bash file path > > > Key: OOZIE-3567 > URL: https://issues.apache.org/jira/browse/OOZIE-3567 > Project: Oozie > Issue Type: Improvement > Components: action >Affects Versions: 4.2.0, 5.0.0, 5.1.0 >Reporter: Roey Shem Tov >Priority: Major > Fix For: 5.3.0 > > Attachments: > [OOZIE-3567]_Oozie_ShellAction_should_support_absolute_bash_file_path.patch > > > as related in OOZIE-3320 im opening a new PR (me from different user), still > can`t execute shell from mount at excpected. > > File is not checked to be existed on resolveExecutable, because mount can be > at the workers but not at the oozie servers (very common use). -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (OOZIE-3567) Oozie ShellAction should support absolute bash file path
[ https://issues.apache.org/jira/browse/OOZIE-3567?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16989799#comment-16989799 ] Roey Shem Tov commented on OOZIE-3567: -- [~asalamon74] No different at all, I opened OOZIE-3320 (from another user) and saw right now this wasn`t resolved, so i prefer to keep it from here (i changed main user at jira+github). > Oozie ShellAction should support absolute bash file path > > > Key: OOZIE-3567 > URL: https://issues.apache.org/jira/browse/OOZIE-3567 > Project: Oozie > Issue Type: Improvement > Components: action >Affects Versions: 4.2.0, 5.0.0, 5.1.0 >Reporter: Roey Shem Tov >Priority: Major > Fix For: 5.3.0 > > Attachments: > [OOZIE-3567]_Oozie_ShellAction_should_support_absolute_bash_file_path.patch > > > as related in OOZIE-3320 im opening a new PR (me from different user), still > can`t execute shell from mount at excpected. > > File is not checked to be existed on resolveExecutable, because mount can be > at the workers but not at the oozie servers (very common use). -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (OOZIE-3567) Oozie ShellAction should support absolute bash file path
[ https://issues.apache.org/jira/browse/OOZIE-3567?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Roey Shem Tov updated OOZIE-3567: - Attachment: [OOZIE-3567]_Oozie_ShellAction_should_support_absolute_bash_file_path.patch External issue URL: https://github.com/apache/oozie/pull/47/files Description: as related in OOZIE-3320 im opening a new PR (me from different user), still can`t execute shell from mount at excpected. File is not checked to be existed on resolveExecutable, because mount can be at the workers but not at the oozie servers (very common use). was:as related in OOZIE-3320 im opening a new PR (me from different user), still can`t execute shell from mount at excpected. > Oozie ShellAction should support absolute bash file path > > > Key: OOZIE-3567 > URL: https://issues.apache.org/jira/browse/OOZIE-3567 > Project: Oozie > Issue Type: Improvement > Components: action >Affects Versions: 4.2.0, 5.0.0, 5.1.0 >Reporter: Roey Shem Tov >Priority: Major > Fix For: 5.3.0 > > Attachments: > [OOZIE-3567]_Oozie_ShellAction_should_support_absolute_bash_file_path.patch > > > as related in OOZIE-3320 im opening a new PR (me from different user), still > can`t execute shell from mount at excpected. > > File is not checked to be existed on resolveExecutable, because mount can be > at the workers but not at the oozie servers (very common use). -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (OOZIE-3567) Oozie ShellAction should support absolute bash file path
Roey Shem Tov created OOZIE-3567: Summary: Oozie ShellAction should support absolute bash file path Key: OOZIE-3567 URL: https://issues.apache.org/jira/browse/OOZIE-3567 Project: Oozie Issue Type: Improvement Components: action Affects Versions: 5.1.0, 5.0.0, 4.2.0 Reporter: Roey Shem Tov Fix For: 5.3.0 as related in OOZIE-3320 im opening a new PR (me from different user), still can`t execute shell from mount at excpected. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (OOZIE-3320) Oozie ShellAction should support absolute bash file path
[
https://issues.apache.org/jira/browse/OOZIE-3320?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16580024#comment-16580024
]
Roey Shem Tov commented on OOZIE-3320:
--
[~andras.piros] any suggestion?
> Oozie ShellAction should support absolute bash file path
>
>
> Key: OOZIE-3320
> URL: https://issues.apache.org/jira/browse/OOZIE-3320
> Project: Oozie
> Issue Type: Improvement
> Components: action
>Affects Versions: 5.0.0, 4.3.1
>Reporter: Roey Shem Tov
>Assignee: Roey Shem Tov
>Priority: Major
> Labels: patch
> Attachments: OOZIE-3086.patch, OOZIE-3320.patch
>
>
> bash files that saved on shared mount, cannot be execute by the ShellAction
> in proper way.
> Example:
> Worker-1,Worker-2,Worker-3 have shared mount /mnt/hadoop
> on /mnt/hadoop there is a file script.sh
> Right now there is two options to submit it using ShellAction:
> # Upload it to hdfs, add it as a file and submit script.sh
> # use bash as exec and file location (/mnt/hadoop/script.sh) as argument
> (e.g bash>/exec>/mnt/hadoop/script.sh
> Best option is that the command will support shared mounted file :
> /mnt/hadoop/script.sh
>
> This code is taking only the file name instead it full path:
> {code:java}
> String exec = actionXml.getChild("exec", ns).getTextTrim();
> String execName = new Path(exec).getName();
> actionConf.set(ShellMain.CONF_OOZIE_SHELL_EXEC, execName);
> {code}
>
> Best option to support shared mount file is to support file:// starting for
> bash files that are local (or shared by mount), e.g:
> {code:java}
> String exec = actionXml.getChild("exec", ns).getTextTrim(); String execName;
> String localFilePrefix = "file://";
> // When exec starts with 'file://' refer it as local file.
> if (exec.startsWith(localFilePrefix))
> execName = exec.substring(localFilePrefix.length());
> else execName = new Path(exec).getName();
> actionConf.set(ShellMain.CONF_OOZIE_SHELL_EXEC, execName);
> {code}
>
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OOZIE-3320) Oozie ShellAction should support absolute bash file path
[
https://issues.apache.org/jira/browse/OOZIE-3320?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16575076#comment-16575076
]
Roey Shem Tov commented on OOZIE-3320:
--
Upload new patch.
According to [~andras.piros], i extract the code to outside method, and
surround {{if-else}} with brackets,
but the DG_ShellActionExtension.twiki looks fine to me, because according to
the twiki:
{code:java}
The =exec= element must contain the path of the Shell command to
execute.
{code}
it say you must put the path to the shell comand (or out bash file in this
case),
so what should i change in that?
> Oozie ShellAction should support absolute bash file path
>
>
> Key: OOZIE-3320
> URL: https://issues.apache.org/jira/browse/OOZIE-3320
> Project: Oozie
> Issue Type: Improvement
> Components: action
>Affects Versions: 5.0.0, 4.3.1
>Reporter: Roey Shem Tov
>Assignee: Roey Shem Tov
>Priority: Major
> Labels: patch
> Attachments: OOZIE-3086.patch, OOZIE-3320.patch
>
>
> bash files that saved on shared mount, cannot be execute by the ShellAction
> in proper way.
> Example:
> Worker-1,Worker-2,Worker-3 have shared mount /mnt/hadoop
> on /mnt/hadoop there is a file script.sh
> Right now there is two options to submit it using ShellAction:
> # Upload it to hdfs, add it as a file and submit script.sh
> # use bash as exec and file location (/mnt/hadoop/script.sh) as argument
> (e.g bash>/exec>/mnt/hadoop/script.sh
> Best option is that the command will support shared mounted file :
> /mnt/hadoop/script.sh
>
> This code is taking only the file name instead it full path:
> {code:java}
> String exec = actionXml.getChild("exec", ns).getTextTrim();
> String execName = new Path(exec).getName();
> actionConf.set(ShellMain.CONF_OOZIE_SHELL_EXEC, execName);
> {code}
>
> Best option to support shared mount file is to support file:// starting for
> bash files that are local (or shared by mount), e.g:
> {code:java}
> String exec = actionXml.getChild("exec", ns).getTextTrim(); String execName;
> String localFilePrefix = "file://";
> // When exec starts with 'file://' refer it as local file.
> if (exec.startsWith(localFilePrefix))
> execName = exec.substring(localFilePrefix.length());
> else execName = new Path(exec).getName();
> actionConf.set(ShellMain.CONF_OOZIE_SHELL_EXEC, execName);
> {code}
>
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (OOZIE-3320) Oozie ShellAction should support absolute bash file path
[
https://issues.apache.org/jira/browse/OOZIE-3320?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Roey Shem Tov updated OOZIE-3320:
-
Attachment: OOZIE-3320.patch
> Oozie ShellAction should support absolute bash file path
>
>
> Key: OOZIE-3320
> URL: https://issues.apache.org/jira/browse/OOZIE-3320
> Project: Oozie
> Issue Type: Improvement
> Components: action
>Affects Versions: 5.0.0, 4.3.1
>Reporter: Roey Shem Tov
>Assignee: Roey Shem Tov
>Priority: Major
> Labels: patch
> Attachments: OOZIE-3086.patch, OOZIE-3320.patch
>
>
> bash files that saved on shared mount, cannot be execute by the ShellAction
> in proper way.
> Example:
> Worker-1,Worker-2,Worker-3 have shared mount /mnt/hadoop
> on /mnt/hadoop there is a file script.sh
> Right now there is two options to submit it using ShellAction:
> # Upload it to hdfs, add it as a file and submit script.sh
> # use bash as exec and file location (/mnt/hadoop/script.sh) as argument
> (e.g bash>/exec>/mnt/hadoop/script.sh
> Best option is that the command will support shared mounted file :
> /mnt/hadoop/script.sh
>
> This code is taking only the file name instead it full path:
> {code:java}
> String exec = actionXml.getChild("exec", ns).getTextTrim();
> String execName = new Path(exec).getName();
> actionConf.set(ShellMain.CONF_OOZIE_SHELL_EXEC, execName);
> {code}
>
> Best option to support shared mount file is to support file:// starting for
> bash files that are local (or shared by mount), e.g:
> {code:java}
> String exec = actionXml.getChild("exec", ns).getTextTrim(); String execName;
> String localFilePrefix = "file://";
> // When exec starts with 'file://' refer it as local file.
> if (exec.startsWith(localFilePrefix))
> execName = exec.substring(localFilePrefix.length());
> else execName = new Path(exec).getName();
> actionConf.set(ShellMain.CONF_OOZIE_SHELL_EXEC, execName);
> {code}
>
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (OOZIE-3320) Oozie ShellAction should support absolute bash file path
Roey Shem Tov created OOZIE-3320:
Summary: Oozie ShellAction should support absolute bash file path
Key: OOZIE-3320
URL: https://issues.apache.org/jira/browse/OOZIE-3320
Project: Oozie
Issue Type: Improvement
Components: action
Affects Versions: 4.3.1, 5.0.0
Reporter: Roey Shem Tov
Attachments: OOZIE-3086.patch
bash files that saved on shared mount, cannot be execute by the ShellAction in
proper way.
Example:
Worker-1,Worker-2,Worker-3 have shared mount /mnt/hadoop
on /mnt/hadoop there is a file script.sh
Right now there is two options to submit it using ShellAction:
# Upload it to hdfs, add it as a file and submit script.sh
# use bash as exec and file location (/mnt/hadoop/script.sh) as argument (e.g
bash>/exec>/mnt/hadoop/script.sh
Best option is that the command will support shared mounted file :
/mnt/hadoop/script.sh
This code is taking only the file name instead it full path:
{code:java}
String exec = actionXml.getChild("exec", ns).getTextTrim();
String execName = new Path(exec).getName();
actionConf.set(ShellMain.CONF_OOZIE_SHELL_EXEC, execName);
{code}
Best option to support shared mount file is to support file:// starting for
bash files that are local (or shared by mount), e.g:
{code:java}
String exec = actionXml.getChild("exec", ns).getTextTrim(); String execName;
String localFilePrefix = "file://";
// When exec starts with 'file://' refer it as local file.
if (exec.startsWith(localFilePrefix))
execName = exec.substring(localFilePrefix.length());
else execName = new Path(exec).getName();
actionConf.set(ShellMain.CONF_OOZIE_SHELL_EXEC, execName);
{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
