Failed: OOZIE-3320 PreCommit Build #742
Jira: https://issues.apache.org/jira/browse/OOZIE-3320 Build: https://builds.apache.org/job/PreCommit-OOZIE-Build/742/ ### ## LAST 100 LINES OF THE CONSOLE ### [...truncated 453.07 KB...] [INFO] Apache Oozie Docs .. SUCCESS [ 9.326 s] [INFO] Apache Oozie WebApp SUCCESS [ 31.366 s] [INFO] Apache Oozie Tools . SUCCESS [ 1.364 s] [INFO] Apache Oozie MiniOozie . SUCCESS [ 1.314 s] [INFO] Apache Oozie Fluent Job Client . SUCCESS [ 0.222 s] [INFO] Apache Oozie Server SUCCESS [ 3.575 s] [INFO] Apache Oozie Distro SUCCESS [ 54.044 s] [INFO] Apache Oozie ZooKeeper Security Tests 5.1.0-SNAPSHOT SUCCESS [ 2.321 s] [INFO] [INFO] BUILD SUCCESS [INFO] [INFO] Total time: 03:31 min [INFO] Finished at: 2018-08-09T17:43:10Z [INFO] Reports Running test-patch task CLEAN Running test-patch task RAW_PATCH_ANALYSIS Running test-patch task RAT Running test-patch task JAVADOC Running test-patch task COMPILE Running test-patch task FINDBUGS_DIFF [TRACE] Downloading FindBugs diff JAR from https://repo1.maven.org/maven2/me/andrz/findbugs/findbugs-diff/0.1.0/findbugs-diff-0.1.0-all.jar [TRACE] FindBugs diff JAR downloaded grep: /home/jenkins/jenkins-slave/workspace/PreCommit-OOZIE-Build/bin/findbugs-diff-0.1.0-all.jar.md5sum: No such file or directory [TRACE] FindBugs diff JAR checked, is safe to use [TRACE] Performing FindBugs diffs [TRACE] FindBugs diffs performed [TRACE] Checking FindBugs diffs and creating reports [INFO] There are no new bugs found totally]. [TRACE] FindBugs diffs checked and reports created [TRACE] Summary file size is 1236 bytes [TRACE] Full summary file size is 184 bytes [TRACE] File [/home/jenkins/jenkins-slave/workspace/PreCommit-OOZIE-Build/test-patch/tmp/FINDBUGS_DIFF/diff/findbugs-diff-0.1.0-all.jar] removed [TRACE] File [/home/jenkins/jenkins-slave/workspace/PreCommit-OOZIE-Build/test-patch/tmp/FINDBUGS_DIFF/diff/findbugs-diff-0.1.0-all.jar.md5sum] removed Running test-patch task BACKWARDS_COMPATIBILITY Running test-patch task TESTS Running test-patch task DISTRO Testing JIRA OOZIE-3320 Cleaning local git workspace +1 PATCH_APPLIES +1 CLEAN -1 RAW_PATCH_ANALYSIS +1 the patch does not introduce any @author tags +1 the patch does not introduce any tabs +1 the patch does not introduce any trailing spaces +1 the patch does not introduce any line longer than 132 -1 the patch does not add/modify any testcase +1 RAT +1 the patch does not seem to introduce new RAT warnings +1 JAVADOC +1 JAVADOC +1 the patch does not seem to introduce new Javadoc warning(s) +1 the patch does not seem to introduce new Javadoc error(s) ERROR: the current HEAD has 2 Javadoc error(s) +1 COMPILE +1 HEAD compiles +1 patch compiles +1 the patch does not seem to introduce new javac warnings +1 There are no new bugs found in total. +1 BACKWARDS_COMPATIBILITY +1 the patch does not change any JPA Entity/Colum/Basic/Lob/Transient annotations +1 the patch does not modify JPA files +1 TESTS Tests run: 2918 Tests failed at first run: TestCoordActionsKillXCommand#testActionKillCommandActionNumbers For the complete list of flaky tests, see TEST-SUMMARY-FULL files. +1 DISTRO +1 distro tarball builds with the patch -1 Overall result, please check the reported -1(s) The full output of the test-patch run is available at https://builds.apache.org/job/PreCommit-OOZIE-Build/742/ Adding comment to JIRA % Total% Received % Xferd Average Speed TimeTime Time Current Dload Upload Total SpentLeft Speed 0 00 00 0 0 0 --:--:-- --:--:-- --:--:-- 0 0 00 00 0 0 0 --:--:-- --:--:-- --:--:-- 0100 55060 3441 100 2065 4101 2461 --:--:-- --:--:-- --:--:-- 4101
[jira] [Commented] (OOZIE-3320) Oozie ShellAction should support absolute bash file path
[
https://issues.apache.org/jira/browse/OOZIE-3320?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16575193#comment-16575193
]
Hadoop QA commented on OOZIE-3320:
--
Testing JIRA OOZIE-3320
Cleaning local git workspace
{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:red}-1 RAW_PATCH_ANALYSIS{color}
.{color:green}+1{color} the patch does not introduce any @author tags
.{color:green}+1{color} the patch does not introduce any tabs
.{color:green}+1{color} the patch does not introduce any trailing spaces
.{color:green}+1{color} the patch does not introduce any line longer than
132
.{color:red}-1{color} the patch does not add/modify any testcase
{color:green}+1 RAT{color}
.{color:green}+1{color} the patch does not seem to introduce new RAT
warnings
{color:green}+1 JAVADOC{color}
{color:green}+1 JAVADOC{color}
.{color:green}+1{color} the patch does not seem to introduce new Javadoc
warning(s)
.{color:green}+1{color} the patch does not seem to introduce new Javadoc
error(s)
.{color:red}ERROR{color}: the current HEAD has 2 Javadoc error(s)
{color:green}+1 COMPILE{color}
.{color:green}+1{color} HEAD compiles
.{color:green}+1{color} patch compiles
.{color:green}+1{color} the patch does not seem to introduce new javac
warnings
{color:green}+1{color} There are no new bugs found in total.
{color:green}+1 BACKWARDS_COMPATIBILITY{color}
.{color:green}+1{color} the patch does not change any JPA
Entity/Colum/Basic/Lob/Transient annotations
.{color:green}+1{color} the patch does not modify JPA files
{color:green}+1 TESTS{color}
.Tests run: 2918
.{color:orange}Tests failed at first run:{color}
TestCoordActionsKillXCommand#testActionKillCommandActionNumbers
.For the complete list of flaky tests, see TEST-SUMMARY-FULL files.
{color:green}+1 DISTRO{color}
.{color:green}+1{color} distro tarball builds with the patch
{color:red}*-1 Overall result, please check the reported -1(s)*{color}
The full output of the test-patch run is available at
. https://builds.apache.org/job/PreCommit-OOZIE-Build/742/
> Oozie ShellAction should support absolute bash file path
>
>
> Key: OOZIE-3320
> URL: https://issues.apache.org/jira/browse/OOZIE-3320
> Project: Oozie
> Issue Type: Improvement
> Components: action
>Affects Versions: 5.0.0, 4.3.1
>Reporter: Roey Shem Tov
>Assignee: Roey Shem Tov
>Priority: Major
> Labels: patch
> Attachments: OOZIE-3086.patch, OOZIE-3320.patch
>
>
> bash files that saved on shared mount, cannot be execute by the ShellAction
> in proper way.
> Example:
> Worker-1,Worker-2,Worker-3 have shared mount /mnt/hadoop
> on /mnt/hadoop there is a file script.sh
> Right now there is two options to submit it using ShellAction:
> # Upload it to hdfs, add it as a file and submit script.sh
> # use bash as exec and file location (/mnt/hadoop/script.sh) as argument
> (e.g bash>/exec>/mnt/hadoop/script.sh
> Best option is that the command will support shared mounted file :
> /mnt/hadoop/script.sh
>
> This code is taking only the file name instead it full path:
> {code:java}
> String exec = actionXml.getChild("exec", ns).getTextTrim();
> String execName = new Path(exec).getName();
> actionConf.set(ShellMain.CONF_OOZIE_SHELL_EXEC, execName);
> {code}
>
> Best option to support shared mount file is to support file:// starting for
> bash files that are local (or shared by mount), e.g:
> {code:java}
> String exec = actionXml.getChild("exec", ns).getTextTrim(); String execName;
> String localFilePrefix = "file://";
> // When exec starts with 'file://' refer it as local file.
> if (exec.startsWith(localFilePrefix))
> execName = exec.substring(localFilePrefix.length());
> else execName = new Path(exec).getName();
> actionConf.set(ShellMain.CONF_OOZIE_SHELL_EXEC, execName);
> {code}
>
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OOZIE-3320) Oozie ShellAction should support absolute bash file path
[
https://issues.apache.org/jira/browse/OOZIE-3320?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16575076#comment-16575076
]
Roey Shem Tov commented on OOZIE-3320:
--
Upload new patch.
According to [~andras.piros], i extract the code to outside method, and
surround {{if-else}} with brackets,
but the DG_ShellActionExtension.twiki looks fine to me, because according to
the twiki:
{code:java}
The =exec= element must contain the path of the Shell command to
execute.
{code}
it say you must put the path to the shell comand (or out bash file in this
case),
so what should i change in that?
> Oozie ShellAction should support absolute bash file path
>
>
> Key: OOZIE-3320
> URL: https://issues.apache.org/jira/browse/OOZIE-3320
> Project: Oozie
> Issue Type: Improvement
> Components: action
>Affects Versions: 5.0.0, 4.3.1
>Reporter: Roey Shem Tov
>Assignee: Roey Shem Tov
>Priority: Major
> Labels: patch
> Attachments: OOZIE-3086.patch, OOZIE-3320.patch
>
>
> bash files that saved on shared mount, cannot be execute by the ShellAction
> in proper way.
> Example:
> Worker-1,Worker-2,Worker-3 have shared mount /mnt/hadoop
> on /mnt/hadoop there is a file script.sh
> Right now there is two options to submit it using ShellAction:
> # Upload it to hdfs, add it as a file and submit script.sh
> # use bash as exec and file location (/mnt/hadoop/script.sh) as argument
> (e.g bash>/exec>/mnt/hadoop/script.sh
> Best option is that the command will support shared mounted file :
> /mnt/hadoop/script.sh
>
> This code is taking only the file name instead it full path:
> {code:java}
> String exec = actionXml.getChild("exec", ns).getTextTrim();
> String execName = new Path(exec).getName();
> actionConf.set(ShellMain.CONF_OOZIE_SHELL_EXEC, execName);
> {code}
>
> Best option to support shared mount file is to support file:// starting for
> bash files that are local (or shared by mount), e.g:
> {code:java}
> String exec = actionXml.getChild("exec", ns).getTextTrim(); String execName;
> String localFilePrefix = "file://";
> // When exec starts with 'file://' refer it as local file.
> if (exec.startsWith(localFilePrefix))
> execName = exec.substring(localFilePrefix.length());
> else execName = new Path(exec).getName();
> actionConf.set(ShellMain.CONF_OOZIE_SHELL_EXEC, execName);
> {code}
>
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OOZIE-3320) Oozie ShellAction should support absolute bash file path
[
https://issues.apache.org/jira/browse/OOZIE-3320?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16575072#comment-16575072
]
Hadoop QA commented on OOZIE-3320:
--
PreCommit-OOZIE-Build started
> Oozie ShellAction should support absolute bash file path
>
>
> Key: OOZIE-3320
> URL: https://issues.apache.org/jira/browse/OOZIE-3320
> Project: Oozie
> Issue Type: Improvement
> Components: action
>Affects Versions: 5.0.0, 4.3.1
>Reporter: Roey Shem Tov
>Assignee: Roey Shem Tov
>Priority: Major
> Labels: patch
> Attachments: OOZIE-3086.patch, OOZIE-3320.patch
>
>
> bash files that saved on shared mount, cannot be execute by the ShellAction
> in proper way.
> Example:
> Worker-1,Worker-2,Worker-3 have shared mount /mnt/hadoop
> on /mnt/hadoop there is a file script.sh
> Right now there is two options to submit it using ShellAction:
> # Upload it to hdfs, add it as a file and submit script.sh
> # use bash as exec and file location (/mnt/hadoop/script.sh) as argument
> (e.g bash>/exec>/mnt/hadoop/script.sh
> Best option is that the command will support shared mounted file :
> /mnt/hadoop/script.sh
>
> This code is taking only the file name instead it full path:
> {code:java}
> String exec = actionXml.getChild("exec", ns).getTextTrim();
> String execName = new Path(exec).getName();
> actionConf.set(ShellMain.CONF_OOZIE_SHELL_EXEC, execName);
> {code}
>
> Best option to support shared mount file is to support file:// starting for
> bash files that are local (or shared by mount), e.g:
> {code:java}
> String exec = actionXml.getChild("exec", ns).getTextTrim(); String execName;
> String localFilePrefix = "file://";
> // When exec starts with 'file://' refer it as local file.
> if (exec.startsWith(localFilePrefix))
> execName = exec.substring(localFilePrefix.length());
> else execName = new Path(exec).getName();
> actionConf.set(ShellMain.CONF_OOZIE_SHELL_EXEC, execName);
> {code}
>
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (OOZIE-3320) Oozie ShellAction should support absolute bash file path
[
https://issues.apache.org/jira/browse/OOZIE-3320?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Roey Shem Tov updated OOZIE-3320:
-
Attachment: OOZIE-3320.patch
> Oozie ShellAction should support absolute bash file path
>
>
> Key: OOZIE-3320
> URL: https://issues.apache.org/jira/browse/OOZIE-3320
> Project: Oozie
> Issue Type: Improvement
> Components: action
>Affects Versions: 5.0.0, 4.3.1
>Reporter: Roey Shem Tov
>Assignee: Roey Shem Tov
>Priority: Major
> Labels: patch
> Attachments: OOZIE-3086.patch, OOZIE-3320.patch
>
>
> bash files that saved on shared mount, cannot be execute by the ShellAction
> in proper way.
> Example:
> Worker-1,Worker-2,Worker-3 have shared mount /mnt/hadoop
> on /mnt/hadoop there is a file script.sh
> Right now there is two options to submit it using ShellAction:
> # Upload it to hdfs, add it as a file and submit script.sh
> # use bash as exec and file location (/mnt/hadoop/script.sh) as argument
> (e.g bash>/exec>/mnt/hadoop/script.sh
> Best option is that the command will support shared mounted file :
> /mnt/hadoop/script.sh
>
> This code is taking only the file name instead it full path:
> {code:java}
> String exec = actionXml.getChild("exec", ns).getTextTrim();
> String execName = new Path(exec).getName();
> actionConf.set(ShellMain.CONF_OOZIE_SHELL_EXEC, execName);
> {code}
>
> Best option to support shared mount file is to support file:// starting for
> bash files that are local (or shared by mount), e.g:
> {code:java}
> String exec = actionXml.getChild("exec", ns).getTextTrim(); String execName;
> String localFilePrefix = "file://";
> // When exec starts with 'file://' refer it as local file.
> if (exec.startsWith(localFilePrefix))
> execName = exec.substring(localFilePrefix.length());
> else execName = new Path(exec).getName();
> actionConf.set(ShellMain.CONF_OOZIE_SHELL_EXEC, execName);
> {code}
>
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OOZIE-3324) Cannot compile with findbugs check
[
https://issues.apache.org/jira/browse/OOZIE-3324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16575002#comment-16575002
]
Andras Salamon commented on OOZIE-3324:
---
Jenkins comment is missing but here is the build info:
[https://builds.apache.org/job/PreCommit-OOZIE-Build/740/]
> Cannot compile with findbugs check
> --
>
> Key: OOZIE-3324
> URL: https://issues.apache.org/jira/browse/OOZIE-3324
> Project: Oozie
> Issue Type: Bug
>Affects Versions: 5.1.0
>Reporter: Andras Salamon
>Assignee: Andras Salamon
>Priority: Critical
> Attachments: OOZIE-3324-1.patch
>
>
> Latest snapshot compilation fails because of missing findbugs-filter.xml file:
> {noformat}
> $ mvn clean install -DskipTests
> [INFO]
>
> [INFO] BUILD FAILURE
> [INFO]
>
> [INFO] Total time: 11.852 s
> [INFO] Finished at: 2018-08-09T09:11:41+02:00
> [INFO]
>
> [ERROR] Could not find resource
> '/Users/andrassalamon/src/oozie/fluent-job/fluent-job-api/findbugs-filter.xml'.
> -> [Help 1]{noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OOZIE-3325) Oozie running shell script times wrong, but hadoopjob running ok
[ https://issues.apache.org/jira/browse/OOZIE-3325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16574699#comment-16574699 ] pharrell.wang commented on OOZIE-3325: -- Job Configuration： end_date 2028-08-15T14:36+0800 wf_application_path hdfs://emr-cluster:8020/user/bjwwyl/hue/workspaces/workflow-1533625234.69/hue-oozie-1533625234.69 oozie.use.system.libpath True dryrun False security_enabled False oozie.coord.action.nominal_time 153371700 coord.nominal.time 2018-08-08 16:30:00.0 user.name bjwwyl oozie.coord.application.path hdfs://emr-cluster:8020/user/hue/oozie/deployments/_bjwwyl_-oozie-367-1533710641.93 mapreduce.job.user.name bjwwyl coord.id 208-180724190149594-oozie-oozi-C@2 oozie.wf.application.path hdfs://emr-cluster:8020/user/bjwwyl/hue/workspaces/workflow-1533625234.69/hue-oozie-1533625234.69 coord.name vpn_hour_schedule jobTracker rm1 hue-id-c 367 nameNode hdfs://emr-cluster:8020 start_date 2018-08-08T15:00+0800 > Oozie running shell script times wrong, but hadoopjob running ok > > > Key: OOZIE-3325 > URL: https://issues.apache.org/jira/browse/OOZIE-3325 > Project: Oozie > Issue Type: Bug >Reporter: pharrell.wang >Priority: Major > > **Log as follows > 2018-08-08 16:38:39,461 WARN SshActionExecutor:523 - > SERVER[wwyl-02p10-hadoopclient] USER[bjwwyl] GROUP[-] TOKEN[] > APP[vpn_hour_shell] JOB[212-180724190149594-oozie-oozi-W] > ACTION[212-180724190149594-oozie-oozi-W@ssh-abd3] Execution data not set > 2018-08-08 16:38:39,466 WARN CoordActionCheckXCommand:523 - > SERVER[wwyl-02p10-hadoopclient] USER[bjwwyl] GROUP[-] TOKEN[] > APP[vpn_hour_shell] JOB[208-180724190149594-oozie-oozi-C] > ACTION[208-180724190149594-oozie-oozi-C@2] Unexpected workflow > 212-180724190149594-oozie-oozi-W STATUS RUNNING > 2018-08-08 16:38:49,461 WARN SshActionExecutor:523 - > SERVER[wwyl-02p10-hadoopclient] USER[bjwwyl] GROUP[-] TOKEN[] > APP[vpn_hour_shell] JOB[212-180724190149594-oozie-oozi-W] > ACTION[212-180724190149594-oozie-oozi-W@ssh-abd3] Execution data not set > 2018-08-08 16:38:49,466 WARN CoordActionCheckXCommand:523 - > SERVER[wwyl-02p10-hadoopclient] USER[bjwwyl] GROUP[-] TOKEN[] > APP[vpn_hour_shell] JOB[208-180724190149594-oozie-oozi-C] > ACTION[208-180724190149594-oozie-oozi-C@2] Unexpected workflow > 212-180724190149594-oozie-oozi-W STATUS RUNNING > 2018-08-08 16:38:59,451 WARN SshActionExecutor:523 - > SERVER[wwyl-02p10-hadoopclient] USER[bjwwyl] GROUP[-] TOKEN[] > APP[vpn_hour_shell] JOB[212-180724190149594-oozie-oozi-W] > ACTION[212-180724190149594-oozie-oozi-W@ssh-abd3] Execution data not set > 2018-08-08 16:38:59,457 WARN CoordActionCheckXCommand:523 - > SERVER[wwyl-02p10-hadoopclient] USER[bjwwyl] GROUP[-] TOKEN[] > APP[vpn_hour_shell] JOB[208-180724190149594-oozie-oozi-C] > ACTION[208-180724190149594-oozie-oozi-C@2] Unexpected workflow > 212-180724190149594-oozie-oozi-W STATUS RUNNING > 2018-08-08 16:39:09,466 WARN SshActionExecutor:523 - > SERVER[wwyl-02p10-hadoopclient] USER[bjwwyl] GROUP[-] TOKEN[] > APP[vpn_hour_shell] JOB[212-180724190149594-oozie-oozi-W] > ACTION[212-180724190149594-oozie-oozi-W@ssh-abd3] Execution data not set > 2018-08-08 16:39:09,471 WARN CoordActionCheckXCommand:523 - > SERVER[wwyl-02p10-hadoopclient] USER[bjwwyl] GROUP[-] TOKEN[] > APP[vpn_hour_shell] JOB[208-180724190149594-oozie-oozi-C] > ACTION[208-180724190149594-oozie-oozi-C@2] Unexpected workflow > 212-180724190149594-oozie-oozi-W STATUS RUNNING > 2018-08-08 16:39:19,467 WARN SshActionExecutor:523 - > SERVER[wwyl-02p10-hadoopclient] USER[bjwwyl] GROUP[-] TOKEN[] > APP[vpn_hour_shell] JOB[212-180724190149594-oozie-oozi-W] > ACTION[212-180724190149594-oozie-oozi-W@ssh-abd3] Execution data not set > 2018-08-08 16:39:19,472 WARN CoordActionCheckXCommand:523 - > SERVER[wwyl-02p10-hadoopclient] USER[bjwwyl] GROUP[-] TOKEN[] > APP[vpn_hour_shell] JOB[208-180724190149594-oozie-oozi-C] > ACTION[208-180724190149594-oozie-oozi-C@2] Unexpected workflow > 212-180724190149594-oozie-oozi-W STATUS RUNNING > 2018-08-08 16:39:29,468 WARN SshActionExecutor:523 - > SERVER[wwyl-02p10-hadoopclient] USER[bjwwyl] GROUP[-] TOKEN[] > APP[vpn_hour_shell] JOB[212-180724190149594-oozie-oozi-W] > ACTION[212-180724190149594-oozie-oozi-W@ssh-abd3] Execution data not set > 2018-08-08 16:39:29,473 WARN CoordActionCheckXCommand:523 - > SERVER[wwyl-02p10-hadoopclient] USER[bjwwyl] GROUP[-] TOKEN[] > APP[vpn_hour_shell] JOB[208-180724190149594-oozie-oozi-C] > ACTION[208-180724190149594-oozie-oozi-C@2] Unexpected workflow > 212-180724190149594-oozie-oozi-W STATUS
[jira] [Created] (OOZIE-3325) Oozie running shell script times wrong, but hadoopjob running ok
pharrell.wang created OOZIE-3325: Summary: Oozie running shell script times wrong, but hadoopjob running ok Key: OOZIE-3325 URL: https://issues.apache.org/jira/browse/OOZIE-3325 Project: Oozie Issue Type: Bug Reporter: pharrell.wang **Log as follows 2018-08-08 16:38:39,461 WARN SshActionExecutor:523 - SERVER[wwyl-02p10-hadoopclient] USER[bjwwyl] GROUP[-] TOKEN[] APP[vpn_hour_shell] JOB[212-180724190149594-oozie-oozi-W] ACTION[212-180724190149594-oozie-oozi-W@ssh-abd3] Execution data not set 2018-08-08 16:38:39,466 WARN CoordActionCheckXCommand:523 - SERVER[wwyl-02p10-hadoopclient] USER[bjwwyl] GROUP[-] TOKEN[] APP[vpn_hour_shell] JOB[208-180724190149594-oozie-oozi-C] ACTION[208-180724190149594-oozie-oozi-C@2] Unexpected workflow 212-180724190149594-oozie-oozi-W STATUS RUNNING 2018-08-08 16:38:49,461 WARN SshActionExecutor:523 - SERVER[wwyl-02p10-hadoopclient] USER[bjwwyl] GROUP[-] TOKEN[] APP[vpn_hour_shell] JOB[212-180724190149594-oozie-oozi-W] ACTION[212-180724190149594-oozie-oozi-W@ssh-abd3] Execution data not set 2018-08-08 16:38:49,466 WARN CoordActionCheckXCommand:523 - SERVER[wwyl-02p10-hadoopclient] USER[bjwwyl] GROUP[-] TOKEN[] APP[vpn_hour_shell] JOB[208-180724190149594-oozie-oozi-C] ACTION[208-180724190149594-oozie-oozi-C@2] Unexpected workflow 212-180724190149594-oozie-oozi-W STATUS RUNNING 2018-08-08 16:38:59,451 WARN SshActionExecutor:523 - SERVER[wwyl-02p10-hadoopclient] USER[bjwwyl] GROUP[-] TOKEN[] APP[vpn_hour_shell] JOB[212-180724190149594-oozie-oozi-W] ACTION[212-180724190149594-oozie-oozi-W@ssh-abd3] Execution data not set 2018-08-08 16:38:59,457 WARN CoordActionCheckXCommand:523 - SERVER[wwyl-02p10-hadoopclient] USER[bjwwyl] GROUP[-] TOKEN[] APP[vpn_hour_shell] JOB[208-180724190149594-oozie-oozi-C] ACTION[208-180724190149594-oozie-oozi-C@2] Unexpected workflow 212-180724190149594-oozie-oozi-W STATUS RUNNING 2018-08-08 16:39:09,466 WARN SshActionExecutor:523 - SERVER[wwyl-02p10-hadoopclient] USER[bjwwyl] GROUP[-] TOKEN[] APP[vpn_hour_shell] JOB[212-180724190149594-oozie-oozi-W] ACTION[212-180724190149594-oozie-oozi-W@ssh-abd3] Execution data not set 2018-08-08 16:39:09,471 WARN CoordActionCheckXCommand:523 - SERVER[wwyl-02p10-hadoopclient] USER[bjwwyl] GROUP[-] TOKEN[] APP[vpn_hour_shell] JOB[208-180724190149594-oozie-oozi-C] ACTION[208-180724190149594-oozie-oozi-C@2] Unexpected workflow 212-180724190149594-oozie-oozi-W STATUS RUNNING 2018-08-08 16:39:19,467 WARN SshActionExecutor:523 - SERVER[wwyl-02p10-hadoopclient] USER[bjwwyl] GROUP[-] TOKEN[] APP[vpn_hour_shell] JOB[212-180724190149594-oozie-oozi-W] ACTION[212-180724190149594-oozie-oozi-W@ssh-abd3] Execution data not set 2018-08-08 16:39:19,472 WARN CoordActionCheckXCommand:523 - SERVER[wwyl-02p10-hadoopclient] USER[bjwwyl] GROUP[-] TOKEN[] APP[vpn_hour_shell] JOB[208-180724190149594-oozie-oozi-C] ACTION[208-180724190149594-oozie-oozi-C@2] Unexpected workflow 212-180724190149594-oozie-oozi-W STATUS RUNNING 2018-08-08 16:39:29,468 WARN SshActionExecutor:523 - SERVER[wwyl-02p10-hadoopclient] USER[bjwwyl] GROUP[-] TOKEN[] APP[vpn_hour_shell] JOB[212-180724190149594-oozie-oozi-W] ACTION[212-180724190149594-oozie-oozi-W@ssh-abd3] Execution data not set 2018-08-08 16:39:29,473 WARN CoordActionCheckXCommand:523 - SERVER[wwyl-02p10-hadoopclient] USER[bjwwyl] GROUP[-] TOKEN[] APP[vpn_hour_shell] JOB[208-180724190149594-oozie-oozi-C] ACTION[208-180724190149594-oozie-oozi-C@2] Unexpected workflow 212-180724190149594-oozie-oozi-W STATUS RUNNING 2018-08-08 16:39:39,456 WARN SshActionExecutor:523 - SERVER[wwyl-02p10-hadoopclient] USER[bjwwyl] GROUP[-] TOKEN[] APP[vpn_hour_shell] JOB[212-180724190149594-oozie-oozi-W] ACTION[212-180724190149594-oozie-oozi-W@ssh-abd3] Execution data not set 2018-08-08 16:39:39,461 WARN CoordActionCheckXCommand:523 - SERVER[wwyl-02p10-hadoopclient] USER[bjwwyl] GROUP[-] TOKEN[] APP[vpn_hour_shell] JOB[208-180724190149594-oozie-oozi-C] ACTION[208-180724190149594-oozie-oozi-C@2] Unexpected workflow 212-180724190149594-oozie-oozi-W STATUS RUNNING 2018-08-08 16:39:49,837 WARN ActionCheckXCommand:523 - SERVER[wwyl-02p10-hadoopclient] USER[bjwwyl] GROUP[-] TOKEN[] APP[vpn_hour_shell] JOB[212-180724190149594-oozie-oozi-W] ACTION[212-180724190149594-oozie-oozi-W@ssh-abd3] Exception while executing check(). Error Code [ERR_UNKNOWN_ERROR], Message[ERR_UNKNOWN_ERROR: unknown error] org.apache.oozie.action.ActionExecutorException: ERR_UNKNOWN_ERROR: unknown error at org.apache.oozie.action.ssh.SshActionExecutor.check(SshActionExecutor.java:156) at org.apache.oozie.command.wf.ActionCheckXCommand.execute(ActionCheckXCommand.java:182) at
Re: Review Request 67999: Parsing sharelib timestamps is not threadsafe
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/67999/#review207018 --- Ship it! Ship It! - András Piros On Aug. 9, 2018, 8:11 a.m., Denes Bodo wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/67999/ > --- > > (Updated Aug. 9, 2018, 8:11 a.m.) > > > Review request for oozie and András Piros. > > > Bugs: OOZIE-3304 > https://issues.apache.org/jira/browse/OOZIE-3304 > > > Repository: oozie-git > > > Description > --- > > In rare cases the fNumberFormatException occurs while parsing latest sharelib > directory names > > > Diffs > - > > core/src/main/java/org/apache/oozie/service/ShareLibService.java a901567 > core/src/test/java/org/apache/oozie/service/TestHAShareLibService.java > 5087505 > core/src/test/java/org/apache/oozie/service/TestShareLibService.java > d244166 > > > Diff: https://reviews.apache.org/r/67999/diff/3/ > > > Testing > --- > > > Thanks, > > Denes Bodo > >
[jira] [Commented] (OOZIE-3304) Parsing sharelib timestamps is not threadsafe
[
https://issues.apache.org/jira/browse/OOZIE-3304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16574624#comment-16574624
]
Andras Piros commented on OOZIE-3304:
-
[~dionusos] [~matijhs] thanks for the contribution! +1
> Parsing sharelib timestamps is not threadsafe
> -
>
> Key: OOZIE-3304
> URL: https://issues.apache.org/jira/browse/OOZIE-3304
> Project: Oozie
> Issue Type: Bug
> Components: core
>Affects Versions: 5.0.0b1, 4.3.1
>Reporter: Denes Bodo
>Assignee: Denes Bodo
>Priority: Critical
> Labels: usability
> Attachments: OOZIE-3304-001.patch, OOZIE-3304-002.patch,
> OOZIE-3304-003.patch, OOZIE-3304-004.patch, OOZIE-3304-005.patch,
> OOZIE-3304-006.patch
>
>
> In rare cases the following Exception can be read in log files when an action
> fails:
> {code:java}
> org.apache.oozie.action.ActionExecutorException: NumberFormatException:
> multiple points
> at
> org.apache.oozie.action.ActionExecutor.convertException(ActionExecutor.java:446)
> at
> org.apache.oozie.action.hadoop.JavaActionExecutor.submitLauncher(JavaActionExecutor.java:1271)
> at
> org.apache.oozie.action.hadoop.JavaActionExecutor.start(JavaActionExecutor.java:1472)
> at
> org.apache.oozie.command.wf.ActionStartXCommand.execute(ActionStartXCommand.java:234)
> at
> org.apache.oozie.command.wf.ActionStartXCommand.execute(ActionStartXCommand.java:65)
> at org.apache.oozie.command.XCommand.call(XCommand.java:287)
> at
> org.apache.oozie.service.CallableQueueService$CompositeCallable.call(CallableQueueService.java:332)
> at
> org.apache.oozie.service.CallableQueueService$CompositeCallable.call(CallableQueueService.java:261)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> org.apache.oozie.service.CallableQueueService$CallableWrapper.run(CallableQueueService.java:179)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: java.lang.NumberFormatException: multiple points
> at
> sun.misc.FloatingDecimal.readJavaFormatString(FloatingDecimal.java:1890)
> at sun.misc.FloatingDecimal.parseDouble(FloatingDecimal.java:110)
> at java.lang.Double.parseDouble(Double.java:538)
> at java.text.DigitList.getDouble(DigitList.java:169)
> at java.text.DecimalFormat.parse(DecimalFormat.java:2056)
> at java.text.SimpleDateFormat.subParse(SimpleDateFormat.java:2160)
> at java.text.SimpleDateFormat.parse(SimpleDateFormat.java:1514)
> at java.text.DateFormat.parse(DateFormat.java:364)
> at
> org.apache.oozie.service.ShareLibService.getLatestLibPath(ShareLibService.java:727)
> at
> org.apache.oozie.service.ShareLibService.getShareLibRootPath(ShareLibService.java:595)
> at
> org.apache.oozie.action.hadoop.JavaActionExecutor.getSharelibRoot(JavaActionExecutor.java:1297)
> at
> org.apache.oozie.action.hadoop.JavaActionExecutor.initShareLibExcluder(JavaActionExecutor.java:858)
> at
> org.apache.oozie.action.hadoop.JavaActionExecutor.setLibFilesArchives(JavaActionExecutor.java:866)
> at
> org.apache.oozie.action.hadoop.JavaActionExecutor.submitLauncher(JavaActionExecutor.java:1156)
> ... 11 more
> {code}
> or
> {code:java}
> 2018-07-12 04:48:52,649 WARN ForkedActionStartXCommand:523 -
> SERVER[ctr-e138-1518143905142-410551-01-03.hwx.site] USER[user] GROUP[-]
> TOKEN[] APP[demo-wf] JOB[023-180712043119670-oozie-oozi-W]
> ACTION[023-180712043119670-oozie-oozi-W@streaming-node] Error starting
> action [streaming-node]. ErrorType [ERROR], ErrorCode
> [NumberFormatException], Message [NumberFormatException: For input string: ""]
> org.apache.oozie.action.ActionExecutorException: NumberFormatException: For
> input string: ""
> at
> org.apache.oozie.action.ActionExecutor.convertException(ActionExecutor.java:446)
> at
> org.apache.oozie.action.hadoop.JavaActionExecutor.submitLauncher(JavaActionExecutor.java:1271)
> at
> org.apache.oozie.action.hadoop.JavaActionExecutor.start(JavaActionExecutor.java:1472)
> at
> org.apache.oozie.command.wf.ActionStartXCommand.execute(ActionStartXCommand.java:234)
> at
> org.apache.oozie.command.wf.ForkedActionStartXCommand.execute(ForkedActionStartXCommand.java:41)
> at
> org.apache.oozie.command.wf.ForkedActionStartXCommand.execute(ForkedActionStartXCommand.java:30)
> at org.apache.oozie.command.XCommand.call(XCommand.java:287)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
>
[jira] [Commented] (OOZIE-3304) Parsing sharelib timestamps is not threadsafe
[
https://issues.apache.org/jira/browse/OOZIE-3304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16574594#comment-16574594
]
Hadoop QA commented on OOZIE-3304:
--
Testing JIRA OOZIE-3304
Cleaning local git workspace
{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:green}+1 RAW_PATCH_ANALYSIS{color}
.{color:green}+1{color} the patch does not introduce any @author tags
.{color:green}+1{color} the patch does not introduce any tabs
.{color:green}+1{color} the patch does not introduce any trailing spaces
.{color:green}+1{color} the patch does not introduce any line longer than
132
.{color:green}+1{color} the patch adds/modifies 2 testcase(s)
{color:green}+1 RAT{color}
.{color:green}+1{color} the patch does not seem to introduce new RAT
warnings
{color:green}+1 JAVADOC{color}
{color:green}+1 JAVADOC{color}
.{color:green}+1{color} the patch does not seem to introduce new Javadoc
warning(s)
.{color:green}+1{color} the patch does not seem to introduce new Javadoc
error(s)
.{color:red}ERROR{color}: the current HEAD has 2 Javadoc error(s)
{color:green}+1 COMPILE{color}
.{color:green}+1{color} HEAD compiles
.{color:green}+1{color} patch compiles
.{color:green}+1{color} the patch does not seem to introduce new javac
warnings
{color:green}+1{color} There are no new bugs found in total.
{color:green}+1 BACKWARDS_COMPATIBILITY{color}
.{color:green}+1{color} the patch does not change any JPA
Entity/Colum/Basic/Lob/Transient annotations
.{color:green}+1{color} the patch does not modify JPA files
{color:green}+1 TESTS{color}
.Tests run: 2918
.{color:orange}Tests failed at first run:{color}
TestCoordActionsKillXCommand#testActionKillCommandActionNumbers
TestStatusTransitService#testCoordStatusTransitWithLock
.For the complete list of flaky tests, see TEST-SUMMARY-FULL files.
{color:green}+1 DISTRO{color}
.{color:green}+1{color} distro tarball builds with the patch
{color:green}*+1 Overall result, good!, no -1s*{color}
The full output of the test-patch run is available at
. https://builds.apache.org/job/PreCommit-OOZIE-Build/741/
> Parsing sharelib timestamps is not threadsafe
> -
>
> Key: OOZIE-3304
> URL: https://issues.apache.org/jira/browse/OOZIE-3304
> Project: Oozie
> Issue Type: Bug
> Components: core
>Affects Versions: 5.0.0b1, 4.3.1
>Reporter: Denes Bodo
>Assignee: Denes Bodo
>Priority: Critical
> Labels: usability
> Attachments: OOZIE-3304-001.patch, OOZIE-3304-002.patch,
> OOZIE-3304-003.patch, OOZIE-3304-004.patch, OOZIE-3304-005.patch,
> OOZIE-3304-006.patch
>
>
> In rare cases the following Exception can be read in log files when an action
> fails:
> {code:java}
> org.apache.oozie.action.ActionExecutorException: NumberFormatException:
> multiple points
> at
> org.apache.oozie.action.ActionExecutor.convertException(ActionExecutor.java:446)
> at
> org.apache.oozie.action.hadoop.JavaActionExecutor.submitLauncher(JavaActionExecutor.java:1271)
> at
> org.apache.oozie.action.hadoop.JavaActionExecutor.start(JavaActionExecutor.java:1472)
> at
> org.apache.oozie.command.wf.ActionStartXCommand.execute(ActionStartXCommand.java:234)
> at
> org.apache.oozie.command.wf.ActionStartXCommand.execute(ActionStartXCommand.java:65)
> at org.apache.oozie.command.XCommand.call(XCommand.java:287)
> at
> org.apache.oozie.service.CallableQueueService$CompositeCallable.call(CallableQueueService.java:332)
> at
> org.apache.oozie.service.CallableQueueService$CompositeCallable.call(CallableQueueService.java:261)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> org.apache.oozie.service.CallableQueueService$CallableWrapper.run(CallableQueueService.java:179)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: java.lang.NumberFormatException: multiple points
> at
> sun.misc.FloatingDecimal.readJavaFormatString(FloatingDecimal.java:1890)
> at sun.misc.FloatingDecimal.parseDouble(FloatingDecimal.java:110)
> at java.lang.Double.parseDouble(Double.java:538)
> at java.text.DigitList.getDouble(DigitList.java:169)
> at java.text.DecimalFormat.parse(DecimalFormat.java:2056)
> at java.text.SimpleDateFormat.subParse(SimpleDateFormat.java:2160)
> at java.text.SimpleDateFormat.parse(SimpleDateFormat.java:1514)
> at java.text.DateFormat.parse(DateFormat.java:364)
> at
>
Success: OOZIE-3304 PreCommit Build #741
Jira: https://issues.apache.org/jira/browse/OOZIE-3304 Build: https://builds.apache.org/job/PreCommit-OOZIE-Build/741/ ### ## LAST 100 LINES OF THE CONSOLE ### [...truncated 458.49 KB...] [INFO] Apache Oozie Docs .. SUCCESS [ 10.115 s] [INFO] Apache Oozie WebApp SUCCESS [ 31.744 s] [INFO] Apache Oozie Tools . SUCCESS [ 1.523 s] [INFO] Apache Oozie MiniOozie . SUCCESS [ 1.463 s] [INFO] Apache Oozie Fluent Job Client . SUCCESS [ 0.256 s] [INFO] Apache Oozie Server SUCCESS [ 3.664 s] [INFO] Apache Oozie Distro SUCCESS [01:04 min] [INFO] Apache Oozie ZooKeeper Security Tests 5.1.0-SNAPSHOT SUCCESS [ 2.591 s] [INFO] [INFO] BUILD SUCCESS [INFO] [INFO] Total time: 03:57 min [INFO] Finished at: 2018-08-09T09:56:53Z [INFO] Reports Running test-patch task CLEAN Running test-patch task RAW_PATCH_ANALYSIS Running test-patch task RAT Running test-patch task JAVADOC Running test-patch task COMPILE Running test-patch task FINDBUGS_DIFF [TRACE] Downloading FindBugs diff JAR from https://repo1.maven.org/maven2/me/andrz/findbugs/findbugs-diff/0.1.0/findbugs-diff-0.1.0-all.jar [TRACE] FindBugs diff JAR downloaded grep: /home/jenkins/jenkins-slave/workspace/PreCommit-OOZIE-Build/bin/findbugs-diff-0.1.0-all.jar.md5sum: No such file or directory [TRACE] FindBugs diff JAR checked, is safe to use [TRACE] Performing FindBugs diffs [TRACE] FindBugs diffs performed [TRACE] Checking FindBugs diffs and creating reports [INFO] There are no new bugs found totally]. [TRACE] FindBugs diffs checked and reports created [TRACE] Summary file size is 1235 bytes [TRACE] Full summary file size is 184 bytes [TRACE] File [/home/jenkins/jenkins-slave/workspace/PreCommit-OOZIE-Build/test-patch/tmp/FINDBUGS_DIFF/diff/findbugs-diff-0.1.0-all.jar] removed [TRACE] File [/home/jenkins/jenkins-slave/workspace/PreCommit-OOZIE-Build/test-patch/tmp/FINDBUGS_DIFF/diff/findbugs-diff-0.1.0-all.jar.md5sum] removed Running test-patch task BACKWARDS_COMPATIBILITY Running test-patch task TESTS Running test-patch task DISTRO Testing JIRA OOZIE-3304 Cleaning local git workspace +1 PATCH_APPLIES +1 CLEAN +1 RAW_PATCH_ANALYSIS +1 the patch does not introduce any @author tags +1 the patch does not introduce any tabs +1 the patch does not introduce any trailing spaces +1 the patch does not introduce any line longer than 132 +1 the patch adds/modifies 2 testcase(s) +1 RAT +1 the patch does not seem to introduce new RAT warnings +1 JAVADOC +1 JAVADOC +1 the patch does not seem to introduce new Javadoc warning(s) +1 the patch does not seem to introduce new Javadoc error(s) ERROR: the current HEAD has 2 Javadoc error(s) +1 COMPILE +1 HEAD compiles +1 patch compiles +1 the patch does not seem to introduce new javac warnings +1 There are no new bugs found in total. +1 BACKWARDS_COMPATIBILITY +1 the patch does not change any JPA Entity/Colum/Basic/Lob/Transient annotations +1 the patch does not modify JPA files +1 TESTS Tests run: 2918 Tests failed at first run: TestCoordActionsKillXCommand#testActionKillCommandActionNumbers TestStatusTransitService#testCoordStatusTransitWithLock For the complete list of flaky tests, see TEST-SUMMARY-FULL files. +1 DISTRO +1 distro tarball builds with the patch +1 Overall result, good!, no -1s The full output of the test-patch run is available at https://builds.apache.org/job/PreCommit-OOZIE-Build/741/ Adding comment to JIRA % Total% Received % Xferd Average Speed TimeTime Time Current Dload Upload Total SpentLeft Speed 0 00 00 0 0 0 --:--:-- --:--:-- --:--:-- 0 0 00 00 0 0 0 --:--:-- --:--:-- --:--:-- 0100 55860 3481 100 2105 4248 2569 --:--:-- --:--:-- --:--:-- 4245
[jira] [Commented] (OOZIE-3319) Log SSH action callback error output
[
https://issues.apache.org/jira/browse/OOZIE-3319?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16574564#comment-16574564
]
Attila Sasvari commented on OOZIE-3319:
---
[~ericlin] thanks for the patch!
Remarks:
- If we redirect the standard output of the callback command to stdout, it may
interfere with the actual command's standard output; is not desirable. See
https://github.com/apache/oozie/blob/e229e4dbb42b7bc0fdb866ccc530b533a2b2008b/core/src/main/resources/ssh-wrapper.sh#L41
{code}
if $cmnd "$@" >>${stdout} 2>>${stderr}; then
{code}
- I checked the new wrapper: upon a successful execution, the stderr file will
contain
{code}
% Total% Received % Xferd Average Speed TimeTime Time Current
Dload Upload Total SpentLeft Speed
^M 0 00 00 0 0 0 --:--:-- --:--:-- --:--:--
0^M100 50 0 100 5 0691 --:--:-- --:--:-- --:--:--
714
{code}
- Upon failure:
{code}
Not able to perform callback operation.
Callback command: curl
http://localhost:21000/oozie/callback?id=000-180809004852667-oozie-oozi-W@ssh-6185=OK
--data-binary
@/root/oozie-oozi/000-180809004852667-oozie-oozi-W.bak/ssh-6185--ssh/20907.000-180809004852667-oozie-oozi-W@ssh-6185@0.stdout
--request POST --header content-type:text/plain
{code}
I would suggest to only log the callback command in the failure case, and do
not redirect stdout.
- Did you consider to add a new test case to TestSshActionExecutor with an
invalid callback URL?
> Log SSH action callback error output
>
>
> Key: OOZIE-3319
> URL: https://issues.apache.org/jira/browse/OOZIE-3319
> Project: Oozie
> Issue Type: Improvement
> Components: core
>Affects Versions: trunk
>Reporter: Eric Lin
>Assignee: Eric Lin
>Priority: Minor
> Attachments: OOZIE-3319.patch
>
>
> When SSH callback fails, there is no error message logged to tell us what was
> going on
> {code}
> if $callback; then
> echo "Callback operation successful."
> else
> echo "Not able to perform callback operation." >>${stderr}
> fi
> {code}
> The only message showed in the remote machine's log was "Not able to perform
> callback operation." which is useless.
> Request to log the stderr out of the $callback action to stderr log as well.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
Failed: OOZIE-3324 PreCommit Build #740
Jira: https://issues.apache.org/jira/browse/OOZIE-3324 Build: https://builds.apache.org/job/PreCommit-OOZIE-Build/740/ ### ## LAST 100 LINES OF THE CONSOLE ### [...truncated 1.10 MB...] In BundleJobBean.java: org.apache.oozie.BundleJobBean.pcIsDetached() has Boolean return type and returns explicit null -1 There are [74] new bugs found below threshold in [tools] that must be fixed, listing only the first [5] ones. You can find the FindBugs diff here (look for the red and orange ones): tools/findbugs-new.html The top [5] most important FindBugs errors are: At OozieDBCLI.java:[line 562]: Found reliance on default encoding in org.apache.oozie.tools.OozieDBCLI.convertClobToBlobInMysql(String, Connection): new java.io.FileWriter(String, boolean) At OozieDBCLI.java:[line 591]: Found reliance on default encoding in org.apache.oozie.tools.OozieDBCLI.convertClobToBlobInPostgres(String, Connection, String): new java.io.FileWriter(String, boolean) At OozieDBCLI.java:[line 1034]: Found reliance on default encoding in org.apache.oozie.tools.OozieDBCLI.createOozieSysTable(String, boolean, String): new java.io.FileWriter(String, boolean) At OozieDBCLI.java:[line 810]: Found reliance on default encoding in org.apache.oozie.tools.OozieDBCLI.ddlTweaks(String, boolean): new java.io.FileWriter(String, boolean) At OozieDBCLI.java:[line 770]: Found reliance on default encoding in org.apache.oozie.tools.OozieDBCLI.ddlTweaksFor50(String, boolean): new java.io.FileWriter(String, boolean) 0 There are [2] new bugs found in [fluent-job/fluent-job-api] that would be nice to have fixed. You can find the FindBugs diff here: fluent-job/fluent-job-api/findbugs-new.html -1 There are [1] new bugs found below threshold in [server] that must be fixed. You can find the FindBugs diff here (look for the red and orange ones): server/findbugs-new.html The most important FindBugs errors are: At JspHandlerProvider.java:[line 43]: java/io/File.init(Ljava/lang/String;Ljava/lang/String;)V reads a file whose location might be specified by user input At JspHandlerProvider.java:[line 43] +1 There are no new bugs found in [docs]. -1 There are [4] new bugs found below threshold in [sharelib/hive2] that must be fixed. You can find the FindBugs diff here (look for the red and orange ones): sharelib/hive2/findbugs-new.html The most important FindBugs errors are: At Hive2Main.java:[line 271]: org.apache.oozie.action.hadoop.Hive2Main.runBeeline(String[], String) invokes System.exit(...), which shuts down the entire virtual machine At Hive2Main.java:[line 279]: java/io/FileInputStream.init(Ljava/lang/String;)V reads a file whose location might be specified by user input At Hive2Main.java:[line 165]: At Hive2Main.java:[line 146] At Hive2Main.java:[line 149]: java/io/File.init(Ljava/lang/String;)V reads a file whose location might be specified by user input java/io/FileOutputStream.init(Ljava/lang/String;)V writes to a file whose location might be specified by user input: At Hive2Main.java:[line 146] -1 There are [3] new bugs found below threshold in [sharelib/pig] that must be fixed. You can find the FindBugs diff here (look for the red and orange ones): sharelib/pig/findbugs-new.html The most important FindBugs errors are: At PigMain.java:[line 258]: java/io/File.init(Ljava/lang/String;)V reads a file whose location might be specified by user input At PigMain.java:[line 141]: At PigMain.java:[line 131] At PigMain.java:[line 245]: At PigMain.java:[line 199] At PigMain.java:[line 137]: java/io/File.init(Ljava/lang/String;)V reads a file whose location might be specified by user input java/io/File.init(Ljava/lang/String;)V reads a file whose location might be specified by user input: At PigMain.java:[line 131] +1 There are no new bugs found in [sharelib/streaming]. -1 There are [2] new bugs found below threshold in [sharelib/hive] that must be fixed. You can find the FindBugs diff here (look for the red and orange ones): sharelib/hive/findbugs-new.html The most important FindBugs errors are: At HiveMain.java:[line 317]: java/io/FileInputStream.init(Ljava/lang/String;)V reads a file whose location might be specified by user input At HiveMain.java:[line 226]: At HiveMain.java:[line 207] At HiveMain.java:[line 210]: java/io/File.init(Ljava/lang/String;)V reads a file whose location might be specified by user input At HiveMain.java:[line 207] +1 There are no new bugs found in [sharelib/hcatalog]. +1 There are no new bugs found in [sharelib/sqoop]. -1 There are [26] new bugs found below threshold in [sharelib/oozie] that must be fixed, listing only the first [5] ones. You can find the FindBugs diff here (look for the red and orange ones): sharelib/oozie/findbugs-new.html The top [5] most important FindBugs errors are: At ActionStats.java:[line 37]: Unwritten
[jira] [Commented] (OOZIE-3304) Parsing sharelib timestamps is not threadsafe
[
https://issues.apache.org/jira/browse/OOZIE-3304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16574468#comment-16574468
]
Hadoop QA commented on OOZIE-3304:
--
PreCommit-OOZIE-Build started
> Parsing sharelib timestamps is not threadsafe
> -
>
> Key: OOZIE-3304
> URL: https://issues.apache.org/jira/browse/OOZIE-3304
> Project: Oozie
> Issue Type: Bug
> Components: core
>Affects Versions: 5.0.0b1, 4.3.1
>Reporter: Denes Bodo
>Assignee: Denes Bodo
>Priority: Critical
> Labels: usability
> Attachments: OOZIE-3304-001.patch, OOZIE-3304-002.patch,
> OOZIE-3304-003.patch, OOZIE-3304-004.patch, OOZIE-3304-005.patch,
> OOZIE-3304-006.patch
>
>
> In rare cases the following Exception can be read in log files when an action
> fails:
> {code:java}
> org.apache.oozie.action.ActionExecutorException: NumberFormatException:
> multiple points
> at
> org.apache.oozie.action.ActionExecutor.convertException(ActionExecutor.java:446)
> at
> org.apache.oozie.action.hadoop.JavaActionExecutor.submitLauncher(JavaActionExecutor.java:1271)
> at
> org.apache.oozie.action.hadoop.JavaActionExecutor.start(JavaActionExecutor.java:1472)
> at
> org.apache.oozie.command.wf.ActionStartXCommand.execute(ActionStartXCommand.java:234)
> at
> org.apache.oozie.command.wf.ActionStartXCommand.execute(ActionStartXCommand.java:65)
> at org.apache.oozie.command.XCommand.call(XCommand.java:287)
> at
> org.apache.oozie.service.CallableQueueService$CompositeCallable.call(CallableQueueService.java:332)
> at
> org.apache.oozie.service.CallableQueueService$CompositeCallable.call(CallableQueueService.java:261)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> org.apache.oozie.service.CallableQueueService$CallableWrapper.run(CallableQueueService.java:179)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: java.lang.NumberFormatException: multiple points
> at
> sun.misc.FloatingDecimal.readJavaFormatString(FloatingDecimal.java:1890)
> at sun.misc.FloatingDecimal.parseDouble(FloatingDecimal.java:110)
> at java.lang.Double.parseDouble(Double.java:538)
> at java.text.DigitList.getDouble(DigitList.java:169)
> at java.text.DecimalFormat.parse(DecimalFormat.java:2056)
> at java.text.SimpleDateFormat.subParse(SimpleDateFormat.java:2160)
> at java.text.SimpleDateFormat.parse(SimpleDateFormat.java:1514)
> at java.text.DateFormat.parse(DateFormat.java:364)
> at
> org.apache.oozie.service.ShareLibService.getLatestLibPath(ShareLibService.java:727)
> at
> org.apache.oozie.service.ShareLibService.getShareLibRootPath(ShareLibService.java:595)
> at
> org.apache.oozie.action.hadoop.JavaActionExecutor.getSharelibRoot(JavaActionExecutor.java:1297)
> at
> org.apache.oozie.action.hadoop.JavaActionExecutor.initShareLibExcluder(JavaActionExecutor.java:858)
> at
> org.apache.oozie.action.hadoop.JavaActionExecutor.setLibFilesArchives(JavaActionExecutor.java:866)
> at
> org.apache.oozie.action.hadoop.JavaActionExecutor.submitLauncher(JavaActionExecutor.java:1156)
> ... 11 more
> {code}
> or
> {code:java}
> 2018-07-12 04:48:52,649 WARN ForkedActionStartXCommand:523 -
> SERVER[ctr-e138-1518143905142-410551-01-03.hwx.site] USER[user] GROUP[-]
> TOKEN[] APP[demo-wf] JOB[023-180712043119670-oozie-oozi-W]
> ACTION[023-180712043119670-oozie-oozi-W@streaming-node] Error starting
> action [streaming-node]. ErrorType [ERROR], ErrorCode
> [NumberFormatException], Message [NumberFormatException: For input string: ""]
> org.apache.oozie.action.ActionExecutorException: NumberFormatException: For
> input string: ""
> at
> org.apache.oozie.action.ActionExecutor.convertException(ActionExecutor.java:446)
> at
> org.apache.oozie.action.hadoop.JavaActionExecutor.submitLauncher(JavaActionExecutor.java:1271)
> at
> org.apache.oozie.action.hadoop.JavaActionExecutor.start(JavaActionExecutor.java:1472)
> at
> org.apache.oozie.command.wf.ActionStartXCommand.execute(ActionStartXCommand.java:234)
> at
> org.apache.oozie.command.wf.ForkedActionStartXCommand.execute(ForkedActionStartXCommand.java:41)
> at
> org.apache.oozie.command.wf.ForkedActionStartXCommand.execute(ForkedActionStartXCommand.java:30)
> at org.apache.oozie.command.XCommand.call(XCommand.java:287)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
>
Re: Review Request 67999: Parsing sharelib timestamps is not threadsafe
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/67999/ --- (Updated Aug. 9, 2018, 10:11 a.m.) Review request for oozie and András Piros. Changes --- Eliminated too long line Bugs: OOZIE-3304 https://issues.apache.org/jira/browse/OOZIE-3304 Repository: oozie-git Description --- In rare cases the fNumberFormatException occurs while parsing latest sharelib directory names Diffs (updated) - core/src/main/java/org/apache/oozie/service/ShareLibService.java a901567 core/src/test/java/org/apache/oozie/service/TestHAShareLibService.java 5087505 core/src/test/java/org/apache/oozie/service/TestShareLibService.java d244166 Diff: https://reviews.apache.org/r/67999/diff/3/ Changes: https://reviews.apache.org/r/67999/diff/2-3/ Testing --- Thanks, Denes Bodo
[jira] [Updated] (OOZIE-3304) Parsing sharelib timestamps is not threadsafe
[
https://issues.apache.org/jira/browse/OOZIE-3304?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Denes Bodo updated OOZIE-3304:
--
Attachment: OOZIE-3304-006.patch
> Parsing sharelib timestamps is not threadsafe
> -
>
> Key: OOZIE-3304
> URL: https://issues.apache.org/jira/browse/OOZIE-3304
> Project: Oozie
> Issue Type: Bug
> Components: core
>Affects Versions: 5.0.0b1, 4.3.1
>Reporter: Denes Bodo
>Assignee: Denes Bodo
>Priority: Critical
> Labels: usability
> Attachments: OOZIE-3304-001.patch, OOZIE-3304-002.patch,
> OOZIE-3304-003.patch, OOZIE-3304-004.patch, OOZIE-3304-005.patch,
> OOZIE-3304-006.patch
>
>
> In rare cases the following Exception can be read in log files when an action
> fails:
> {code:java}
> org.apache.oozie.action.ActionExecutorException: NumberFormatException:
> multiple points
> at
> org.apache.oozie.action.ActionExecutor.convertException(ActionExecutor.java:446)
> at
> org.apache.oozie.action.hadoop.JavaActionExecutor.submitLauncher(JavaActionExecutor.java:1271)
> at
> org.apache.oozie.action.hadoop.JavaActionExecutor.start(JavaActionExecutor.java:1472)
> at
> org.apache.oozie.command.wf.ActionStartXCommand.execute(ActionStartXCommand.java:234)
> at
> org.apache.oozie.command.wf.ActionStartXCommand.execute(ActionStartXCommand.java:65)
> at org.apache.oozie.command.XCommand.call(XCommand.java:287)
> at
> org.apache.oozie.service.CallableQueueService$CompositeCallable.call(CallableQueueService.java:332)
> at
> org.apache.oozie.service.CallableQueueService$CompositeCallable.call(CallableQueueService.java:261)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> org.apache.oozie.service.CallableQueueService$CallableWrapper.run(CallableQueueService.java:179)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: java.lang.NumberFormatException: multiple points
> at
> sun.misc.FloatingDecimal.readJavaFormatString(FloatingDecimal.java:1890)
> at sun.misc.FloatingDecimal.parseDouble(FloatingDecimal.java:110)
> at java.lang.Double.parseDouble(Double.java:538)
> at java.text.DigitList.getDouble(DigitList.java:169)
> at java.text.DecimalFormat.parse(DecimalFormat.java:2056)
> at java.text.SimpleDateFormat.subParse(SimpleDateFormat.java:2160)
> at java.text.SimpleDateFormat.parse(SimpleDateFormat.java:1514)
> at java.text.DateFormat.parse(DateFormat.java:364)
> at
> org.apache.oozie.service.ShareLibService.getLatestLibPath(ShareLibService.java:727)
> at
> org.apache.oozie.service.ShareLibService.getShareLibRootPath(ShareLibService.java:595)
> at
> org.apache.oozie.action.hadoop.JavaActionExecutor.getSharelibRoot(JavaActionExecutor.java:1297)
> at
> org.apache.oozie.action.hadoop.JavaActionExecutor.initShareLibExcluder(JavaActionExecutor.java:858)
> at
> org.apache.oozie.action.hadoop.JavaActionExecutor.setLibFilesArchives(JavaActionExecutor.java:866)
> at
> org.apache.oozie.action.hadoop.JavaActionExecutor.submitLauncher(JavaActionExecutor.java:1156)
> ... 11 more
> {code}
> or
> {code:java}
> 2018-07-12 04:48:52,649 WARN ForkedActionStartXCommand:523 -
> SERVER[ctr-e138-1518143905142-410551-01-03.hwx.site] USER[user] GROUP[-]
> TOKEN[] APP[demo-wf] JOB[023-180712043119670-oozie-oozi-W]
> ACTION[023-180712043119670-oozie-oozi-W@streaming-node] Error starting
> action [streaming-node]. ErrorType [ERROR], ErrorCode
> [NumberFormatException], Message [NumberFormatException: For input string: ""]
> org.apache.oozie.action.ActionExecutorException: NumberFormatException: For
> input string: ""
> at
> org.apache.oozie.action.ActionExecutor.convertException(ActionExecutor.java:446)
> at
> org.apache.oozie.action.hadoop.JavaActionExecutor.submitLauncher(JavaActionExecutor.java:1271)
> at
> org.apache.oozie.action.hadoop.JavaActionExecutor.start(JavaActionExecutor.java:1472)
> at
> org.apache.oozie.command.wf.ActionStartXCommand.execute(ActionStartXCommand.java:234)
> at
> org.apache.oozie.command.wf.ForkedActionStartXCommand.execute(ForkedActionStartXCommand.java:41)
> at
> org.apache.oozie.command.wf.ForkedActionStartXCommand.execute(ForkedActionStartXCommand.java:30)
> at org.apache.oozie.command.XCommand.call(XCommand.java:287)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> org.apache.oozie.service.CallableQueueService$CallableWrapper.run(CallableQueueService.java:179)
> at
>
[jira] [Commented] (OOZIE-3324) Cannot compile with findbugs check
[
https://issues.apache.org/jira/browse/OOZIE-3324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16574430#comment-16574430
]
Hadoop QA commented on OOZIE-3324:
--
PreCommit-OOZIE-Build started
> Cannot compile with findbugs check
> --
>
> Key: OOZIE-3324
> URL: https://issues.apache.org/jira/browse/OOZIE-3324
> Project: Oozie
> Issue Type: Bug
>Affects Versions: 5.1.0
>Reporter: Andras Salamon
>Assignee: Andras Salamon
>Priority: Critical
> Attachments: OOZIE-3324-1.patch
>
>
> Latest snapshot compilation fails because of missing findbugs-filter.xml file:
> {noformat}
> $ mvn clean install -DskipTests
> [INFO]
>
> [INFO] BUILD FAILURE
> [INFO]
>
> [INFO] Total time: 11.852 s
> [INFO] Finished at: 2018-08-09T09:11:41+02:00
> [INFO]
>
> [ERROR] Could not find resource
> '/Users/andrassalamon/src/oozie/fluent-job/fluent-job-api/findbugs-filter.xml'.
> -> [Help 1]{noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (OOZIE-3314) Remove findbugs-filter.xml and convert its contents to annotations
[ https://issues.apache.org/jira/browse/OOZIE-3314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16571621#comment-16571621 ] Andras Salamon edited comment on OOZIE-3314 at 8/9/18 7:49 AM: --- [~andras.piros] Created a review board for the request: [https://reviews.apache.org/r/68255/] was (Author: asalamon74): [~andras.piros] Created a review board for the request: [https://reviews.apache.org/r/68255/|http://example.com/] > Remove findbugs-filter.xml and convert its contents to annotations > -- > > Key: OOZIE-3314 > URL: https://issues.apache.org/jira/browse/OOZIE-3314 > Project: Oozie > Issue Type: Bug > Components: core >Reporter: Peter Bacsko >Assignee: Andras Salamon >Priority: Major > Fix For: 5.1.0 > > Attachments: OOZIE-3314-1.patch, OOZIE-3314-2.patch > > > In oozie-core, we have a file called "findbugs-filter.xml" which tells > findbugs that it shluld ignore certain problems in a couple of classes. > However if we try to compile a sub-module or run findbugs directly (let's say > in a sharelib project), the build will fail because it won't be able to open > findbugs.xml. It's not straightforward how to define the path of the XML in a > way that it makes this xml accessible regardless of what module you're > compiling. > It's better to just convert its contents to annotations - especially since we > already use this method too. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OOZIE-3324) Cannot compile with findbugs check
[
https://issues.apache.org/jira/browse/OOZIE-3324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16574394#comment-16574394
]
Andras Salamon commented on OOZIE-3324:
---
The OOZIE-3314 patch changed the content of the findbugs-filter.xml file and
moved it to the fluent-job/fluent-job-api subdirectory. The commit only changed
the content but kept the file at the original directory.
> Cannot compile with findbugs check
> --
>
> Key: OOZIE-3324
> URL: https://issues.apache.org/jira/browse/OOZIE-3324
> Project: Oozie
> Issue Type: Bug
>Affects Versions: 5.1.0
>Reporter: Andras Salamon
>Assignee: Andras Salamon
>Priority: Critical
>
> Latest snapshot compilation fails because of missing findbugs-filter.xml file:
> {noformat}
> $ mvn clean install -DskipTests
> [INFO]
>
> [INFO] BUILD FAILURE
> [INFO]
>
> [INFO] Total time: 11.852 s
> [INFO] Finished at: 2018-08-09T09:11:41+02:00
> [INFO]
>
> [ERROR] Could not find resource
> '/Users/andrassalamon/src/oozie/fluent-job/fluent-job-api/findbugs-filter.xml'.
> -> [Help 1]{noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (OOZIE-3324) Cannot compile with findbugs check
Andras Salamon created OOZIE-3324:
-
Summary: Cannot compile with findbugs check
Key: OOZIE-3324
URL: https://issues.apache.org/jira/browse/OOZIE-3324
Project: Oozie
Issue Type: Bug
Affects Versions: 5.1.0
Reporter: Andras Salamon
Assignee: Andras Salamon
Latest snapshot compilation fails because of missing findbugs-filter.xml file:
{noformat}
$ mvn clean install -DskipTests
[INFO]
[INFO] BUILD FAILURE
[INFO]
[INFO] Total time: 11.852 s
[INFO] Finished at: 2018-08-09T09:11:41+02:00
[INFO]
[ERROR] Could not find resource
'/Users/andrassalamon/src/oozie/fluent-job/fluent-job-api/findbugs-filter.xml'.
-> [Help 1]{noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Resolved] (OOZIE-3086) Oozie shell action ignores absolute path
[
https://issues.apache.org/jira/browse/OOZIE-3086?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Attila Sasvari resolved OOZIE-3086.
---
Resolution: Duplicate
> Oozie shell action ignores absolute path
>
>
> Key: OOZIE-3086
> URL: https://issues.apache.org/jira/browse/OOZIE-3086
> Project: Oozie
> Issue Type: Bug
> Components: action
>Affects Versions: 3.1.3, 3.2.0, 3.3.0, trunk, 3.3.1, 3.3.2, 4.0.0, 4.1.0,
> 4.0.1, 4.2.0, 4.3.0
>Reporter: Attila Sasvari
>Priority: Major
>
> [ShellActionExecutor's setupActionConf() method|
> https://github.com/apache/oozie/blob/883c7556d2f302261eae5aec6e323a5b1de04f74/core/src/main/java/org/apache/oozie/action/hadoop/ShellActionExecutor.java#L57]
> retrieves only the *basename* for the shell script specified by the user.
> {code}
> Configuration setupActionConf(Configuration actionConf, Context context,
> Element actionXml, Path appPath)
> {
> ...
> String exec = actionXml.getChild("exec", ns).getTextTrim();
> String execName = new Path(exec).getName();
> actionConf.set(ShellMain.CONF_OOZIE_SHELL_EXEC, execName);
> {code}
> As a result, {{ShellMain}} will
> [execute|https://github.com/apache/oozie/blob/883c7556d2f302261eae5aec6e323a5b1de04f74/sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ShellMain.java#L356]
> the default command available in $PATH.
> {code}
> protected String getExec(Configuration actionConf) {
> String exec = actionConf.get(CONF_OOZIE_SHELL_EXEC);
> {code}
> Consequences:
> - This behaviour makes it also hard/impossible to use custom executable at a
> given location (that is available on each node where a shell action can run,
> for example a specific python version).
> - Even if a user specifies a wrong path (e.g. a non-existing {{/usr/bn/id}})
> on purpose to force failure of a shell action, the command might succeed with
> no apparent reason.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OOZIE-3320) Oozie ShellAction should support absolute bash file path
[
https://issues.apache.org/jira/browse/OOZIE-3320?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16574357#comment-16574357
]
Attila Sasvari commented on OOZIE-3320:
---
Thanks [~roeyshemtov24]. It duplicates
[OOZIE-3086|https://issues.apache.org/jira/browse/OOZIE-3086].
> Oozie ShellAction should support absolute bash file path
>
>
> Key: OOZIE-3320
> URL: https://issues.apache.org/jira/browse/OOZIE-3320
> Project: Oozie
> Issue Type: Improvement
> Components: action
>Affects Versions: 5.0.0, 4.3.1
>Reporter: Roey Shem Tov
>Assignee: Roey Shem Tov
>Priority: Major
> Labels: patch
> Attachments: OOZIE-3086.patch
>
>
> bash files that saved on shared mount, cannot be execute by the ShellAction
> in proper way.
> Example:
> Worker-1,Worker-2,Worker-3 have shared mount /mnt/hadoop
> on /mnt/hadoop there is a file script.sh
> Right now there is two options to submit it using ShellAction:
> # Upload it to hdfs, add it as a file and submit script.sh
> # use bash as exec and file location (/mnt/hadoop/script.sh) as argument
> (e.g bash>/exec>/mnt/hadoop/script.sh
> Best option is that the command will support shared mounted file :
> /mnt/hadoop/script.sh
>
> This code is taking only the file name instead it full path:
> {code:java}
> String exec = actionXml.getChild("exec", ns).getTextTrim();
> String execName = new Path(exec).getName();
> actionConf.set(ShellMain.CONF_OOZIE_SHELL_EXEC, execName);
> {code}
>
> Best option to support shared mount file is to support file:// starting for
> bash files that are local (or shared by mount), e.g:
> {code:java}
> String exec = actionXml.getChild("exec", ns).getTextTrim(); String execName;
> String localFilePrefix = "file://";
> // When exec starts with 'file://' refer it as local file.
> if (exec.startsWith(localFilePrefix))
> execName = exec.substring(localFilePrefix.length());
> else execName = new Path(exec).getName();
> actionConf.set(ShellMain.CONF_OOZIE_SHELL_EXEC, execName);
> {code}
>
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] Subscription: Oozie Patch Available
Issue Subscription Filter: Oozie Patch Available (95 issues) Subscriber: ooziedaily Key Summary OOZIE-3320 Oozie ShellAction should support absolute bash file path https://issues.apache.org/jira/browse/OOZIE-3320 OOZIE-3319 Log SSH action callback error output https://issues.apache.org/jira/browse/OOZIE-3319 OOZIE-3318 Fix Javadoc check in the pre-commit https://issues.apache.org/jira/browse/OOZIE-3318 OOZIE-3304 Parsing sharelib timestamps is not threadsafe https://issues.apache.org/jira/browse/OOZIE-3304 OOZIE-3301 Update NOTICE file https://issues.apache.org/jira/browse/OOZIE-3301 OOZIE-3298 OYA: external ID is not filled properly and failing MR job is treated as SUCCEEDED https://issues.apache.org/jira/browse/OOZIE-3298 OOZIE-3274 Remove slf4j https://issues.apache.org/jira/browse/OOZIE-3274 OOZIE-3266 Coord action rerun support RERUN_SKIP_NODES option https://issues.apache.org/jira/browse/OOZIE-3266 OOZIE-3265 properties RERUN_FAIL_NODES and RERUN_SKIP_NODES should be able to appear together https://issues.apache.org/jira/browse/OOZIE-3265 OOZIE-3256 refactor OozieCLI class https://issues.apache.org/jira/browse/OOZIE-3256 OOZIE-3249 [tools] Instrumentation log parser https://issues.apache.org/jira/browse/OOZIE-3249 OOZIE-3229 Improved filtering options in V2SLAServlet https://issues.apache.org/jira/browse/OOZIE-3229 OOZIE-3218 Oozie Sqoop action with command splits the select clause into multiple parts due to delimiter being space https://issues.apache.org/jira/browse/OOZIE-3218 OOZIE-3199 Let system property restriction configurable https://issues.apache.org/jira/browse/OOZIE-3199 OOZIE-3196 Authorization: restrict world readability by user https://issues.apache.org/jira/browse/OOZIE-3196 OOZIE-3194 Oozie should set proper permissions to sharelib after upload https://issues.apache.org/jira/browse/OOZIE-3194 OOZIE-3186 Oozie is unable to use configuration linked using jceks://file/... https://issues.apache.org/jira/browse/OOZIE-3186 OOZIE-3179 Adding a configurable config-default.xml location to a workflow https://issues.apache.org/jira/browse/OOZIE-3179 OOZIE-3170 Oozie Diagnostic Bundle tool fails with NPE due to missing service class https://issues.apache.org/jira/browse/OOZIE-3170 OOZIE-3160 PriorityDelayQueue put()/take() can cause significant CPU load due to busy waiting https://issues.apache.org/jira/browse/OOZIE-3160 OOZIE-3135 Configure log4j2 in SqoopMain https://issues.apache.org/jira/browse/OOZIE-3135 OOZIE-3091 Oozie Sqoop Avro Import fails with "java.lang.NoClassDefFoundError: org/apache/avro/mapred/AvroWrapper" https://issues.apache.org/jira/browse/OOZIE-3091 OOZIE-3071 Oozie 4.3 Spark sharelib ueses a different version of commons-lang3 than Spark 2.2.0 https://issues.apache.org/jira/browse/OOZIE-3071 OOZIE-3063 Sanitizing variables that are part of openjpa.ConnectionProperties https://issues.apache.org/jira/browse/OOZIE-3063 OOZIE-3062 Set HADOOP_CONF_DIR for spark action https://issues.apache.org/jira/browse/OOZIE-3062 OOZIE-3061 Kill only those child jobs which are not already killed https://issues.apache.org/jira/browse/OOZIE-3061 OOZIE-2952 Fix Findbugs warnings in oozie-sharelib-oozie https://issues.apache.org/jira/browse/OOZIE-2952 OOZIE-2949 Escape quotes whitespaces in Sqoop field https://issues.apache.org/jira/browse/OOZIE-2949 OOZIE-2927 Append new line character for Hive2 query using query tag https://issues.apache.org/jira/browse/OOZIE-2927 OOZIE-2877 Oozie Git Action https://issues.apache.org/jira/browse/OOZIE-2877 OOZIE-2834 ParameterVerifier logging non-useful warning for workflow definition https://issues.apache.org/jira/browse/OOZIE-2834 OOZIE-2833 when using uber mode the regex pattern used in the extractHeapSizeMB method does not allow heap sizes specified in bytes. https://issues.apache.org/jira/browse/OOZIE-2833 OOZIE-2812 SparkConfigurationService should support loading configurations from multiple Spark versions https://issues.apache.org/jira/browse/OOZIE-2812 OOZIE-2795 Create lib directory or symlink for Oozie CLI during packaging https://issues.apache.org/jira/browse/OOZIE-2795 OOZIE-2784 Include WEEK as a parameter in the Coordinator Expression Language Evaulator https://issues.apache.org/jira/browse/OOZIE-2784 OOZIE-2779 Mask Hive2 action Beeline JDBC password https://issues.apache.org/jira/browse/OOZIE-2779 OOZIE-2736 Reduce the number of threads during test execution https://issues.apache.org/jira/browse/OOZIE-2736 OOZIE-2734 Switch docs from twiki to markdown
