Failed: OOZIE-2877 PreCommit Build #599
Jira: https://issues.apache.org/jira/browse/OOZIE-2877 Build: https://builds.apache.org/job/PreCommit-OOZIE-Build/599/ ### ## LAST 100 LINES OF THE CONSOLE ### [...truncated 1.72 MB...] [ERROR] There are [4] new bugs found below threshold in [sharelib/git]. [DEBUG] You can find the FindBugs diff here (look for the red and orange ones): sharelib/git/findbugs-new.html [TRACE] New XMLLib present, calling 'xmllint --xpath' to get bug instance counts [DEBUG] There are no new bugs found in [sharelib/distcp]. [TRACE] New XMLLib present, calling 'xmllint --xpath' to get bug instance counts [DEBUG] There are no new bugs found in [sharelib/spark]. [TRACE] New XMLLib present, calling 'xmllint --xpath' to get bug instance counts [DEBUG] There are no new bugs found in [client]. [ERROR] There are [4] new bugs found below threshold in total that must be fixed. [TRACE] FindBugs diffs checked and reports created [TRACE] Summary file size is 3245 bytes [TRACE] Full summary file size is 2375 bytes [TRACE] File [/home/jenkins/jenkins-slave/workspace/PreCommit-OOZIE-Build/test-patch/tmp/FINDBUGS_DIFF/diff/findbugs-diff-0.1.0-all.jar] removed [TRACE] File [/home/jenkins/jenkins-slave/workspace/PreCommit-OOZIE-Build/test-patch/tmp/FINDBUGS_DIFF/diff/findbugs-diff-0.1.0-all.jar.md5sum] removed Running test-patch task BACKWARDS_COMPATIBILITY Running test-patch task TESTS Running test-patch task DISTRO Testing JIRA OOZIE-2877 Cleaning local git workspace +1 PATCH_APPLIES +1 CLEAN -1 RAW_PATCH_ANALYSIS +1 the patch does not introduce any @author tags -1 the patch contains 10 line(s) with tabs -1 the patch contains 10 line(s) with trailing spaces +1 the patch does not introduce any line longer than 132 +1 the patch adds/modifies 3 testcase(s) +1 RAT +1 the patch does not seem to introduce new RAT warnings +1 JAVADOC +1 JAVADOC +1 the patch does not seem to introduce new Javadoc warning(s) +1 the patch does not seem to introduce new Javadoc error(s) ERROR: the current HEAD has 2 Javadoc error(s) +1 COMPILE +1 HEAD compiles +1 patch compiles +1 the patch does not seem to introduce new javac warnings -1 There are [4] new bugs found below threshold in total that must be fixed. +1 There are no new bugs found in [examples]. +1 There are no new bugs found in [webapp]. +1 There are no new bugs found in [core]. +1 There are no new bugs found in [tools]. +1 There are no new bugs found in [server]. +1 There are no new bugs found in [docs]. +1 There are no new bugs found in [sharelib/hive2]. +1 There are no new bugs found in [sharelib/pig]. +1 There are no new bugs found in [sharelib/streaming]. +1 There are no new bugs found in [sharelib/hive]. +1 There are no new bugs found in [sharelib/hcatalog]. +1 There are no new bugs found in [sharelib/sqoop]. +1 There are no new bugs found in [sharelib/oozie]. -1 There are [4] new bugs found below threshold in [sharelib/git] that must be fixed. You can find the FindBugs diff here (look for the red and orange ones): sharelib/git/findbugs-new.html The most important FindBugs errors are: At GitMain.java:[line 151]: java/io/File.(Ljava/lang/String;)V reads a file whose location might be specified by user input At GitMain.java:[line 151]: At GitMain.java:[line 145] At GitMain.java:[line 134]: java/io/File. (Ljava/lang/String;)V reads a file whose location might be specified by user input Possible null pointer dereference in org.apache.oozie.action.hadoop.GitOperations.cloneRepoToFS(Path) due to return value of called method: At GitMain.java:[line 134] Known null at GitOperations.java:[line 151]: Dereferenced at GitOperations.java:[line 151] +1 There are no new bugs found in [sharelib/distcp]. +1 There are no new bugs found in [sharelib/spark]. +1 There are no new bugs found in [client]. +1 BACKWARDS_COMPATIBILITY +1 the patch does not change any JPA Entity/Colum/Basic/Lob/Transient annotations +1 the patch does not modify JPA files +1 TESTS Tests run: 2135 +1 DISTRO +1 distro tarball builds with the patch -1 Overall result, please check the reported -1(s) The full output of the test-patch run is available at https://builds.apache.org/job/PreCommit-OOZIE-Build/599/ Adding comment to JIRA % Total% Received % Xferd Average Speed TimeTime Time Current Dload Upload Total SpentLeft Speed 0 00 00 0 0 0 --:--:-- --:--:-- --:--:-- 0 0 00 00 0 0 0 --:--:-- --:--:-- --:--:-- 0{"self":"https://issues.apache.org/jira/rest/api/2/issue/13068659/comment/16499833","id":"16499833","author":{"self":"https://issues.apache.org/jira/rest/api/2/user?username=hadoopqa","name":"hadoopqa","
[jira] [Commented] (OOZIE-2877) Oozie Git Action
[
https://issues.apache.org/jira/browse/OOZIE-2877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16499833#comment-16499833
]
Hadoop QA commented on OOZIE-2877:
--
Testing JIRA OOZIE-2877
Cleaning local git workspace
{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:red}-1 RAW_PATCH_ANALYSIS{color}
.{color:green}+1{color} the patch does not introduce any @author tags
.{color:red}-1{color} the patch contains 10 line(s) with tabs
.{color:red}-1{color} the patch contains 10 line(s) with trailing spaces
.{color:green}+1{color} the patch does not introduce any line longer than
132
.{color:green}+1{color} the patch adds/modifies 3 testcase(s)
{color:green}+1 RAT{color}
.{color:green}+1{color} the patch does not seem to introduce new RAT
warnings
{color:green}+1 JAVADOC{color}
{color:green}+1 JAVADOC{color}
.{color:green}+1{color} the patch does not seem to introduce new Javadoc
warning(s)
.{color:green}+1{color} the patch does not seem to introduce new Javadoc
error(s)
.{color:red}ERROR{color}: the current HEAD has 2 Javadoc error(s)
{color:green}+1 COMPILE{color}
.{color:green}+1{color} HEAD compiles
.{color:green}+1{color} patch compiles
.{color:green}+1{color} the patch does not seem to introduce new javac
warnings
{color:red}-1{color} There are [4] new bugs found below threshold in total that
must be fixed.
. {color:green}+1{color} There are no new bugs found in [examples].
. {color:green}+1{color} There are no new bugs found in [webapp].
. {color:green}+1{color} There are no new bugs found in [core].
. {color:green}+1{color} There are no new bugs found in [tools].
. {color:green}+1{color} There are no new bugs found in [server].
. {color:green}+1{color} There are no new bugs found in [docs].
. {color:green}+1{color} There are no new bugs found in [sharelib/hive2].
. {color:green}+1{color} There are no new bugs found in [sharelib/pig].
. {color:green}+1{color} There are no new bugs found in [sharelib/streaming].
. {color:green}+1{color} There are no new bugs found in [sharelib/hive].
. {color:green}+1{color} There are no new bugs found in [sharelib/hcatalog].
. {color:green}+1{color} There are no new bugs found in [sharelib/sqoop].
. {color:green}+1{color} There are no new bugs found in [sharelib/oozie].
. {color:red}-1{color} There are [4] new bugs found below threshold in
[sharelib/git] that must be fixed.
. You can find the FindBugs diff here (look for the red and orange ones):
sharelib/git/findbugs-new.html
. The most important FindBugs errors are:
. At GitMain.java:[line 151]: java/io/File.(Ljava/lang/String;)V
reads a file whose location might be specified by user input
. At GitMain.java:[line 151]: At GitMain.java:[line 145]
. At GitMain.java:[line 134]: java/io/File.(Ljava/lang/String;)V
reads a file whose location might be specified by user input
. Possible null pointer dereference in
org.apache.oozie.action.hadoop.GitOperations.cloneRepoToFS(Path) due to return
value of called method: At GitMain.java:[line 134]
. Known null at GitOperations.java:[line 151]: Dereferenced at
GitOperations.java:[line 151]
. {color:green}+1{color} There are no new bugs found in [sharelib/distcp].
. {color:green}+1{color} There are no new bugs found in [sharelib/spark].
. {color:green}+1{color} There are no new bugs found in [client].
{color:green}+1 BACKWARDS_COMPATIBILITY{color}
.{color:green}+1{color} the patch does not change any JPA
Entity/Colum/Basic/Lob/Transient annotations
.{color:green}+1{color} the patch does not modify JPA files
{color:green}+1 TESTS{color}
.Tests run: 2135
{color:green}+1 DISTRO{color}
.{color:green}+1{color} distro tarball builds with the patch
{color:red}*-1 Overall result, please check the reported -1(s)*{color}
The full output of the test-patch run is available at
. https://builds.apache.org/job/PreCommit-OOZIE-Build/599/
> Oozie Git Action
>
>
> Key: OOZIE-2877
> URL: https://issues.apache.org/jira/browse/OOZIE-2877
> Project: Oozie
> Issue Type: Sub-task
> Components: action
>Reporter: Clay B.
>Assignee: Clay B.
>Priority: Major
> Labels: action
> Fix For: trunk
>
> Attachments: 0001-OOZIE-2877-Oozie-Git-Action.patch,
> 0002-OOZIE-2877-Oozie-Git-Action.patch,
> 0003-OOZIE-2877-Oozie-Git-Action.patch,
> 0004-OOZIE-2877-Oozie-Git-Action.patch,
> 0005-OOZIE-2877-Oozie-Git-Action.patch,
> 0006-OOZIE-2877-Oozie-Git-Action.patch,
> 0007-OOZIE-2877-Oozie-Git-Action.patch,
> 0008-OOZIE-2877-Oozie-Git-Action.patch,
> 0009-OOZIE-2877-Oozie-Git-Action.patch, OOZIE-2877.010.patch,
> OOZIE-2877.011.patch, OOZIE-2877.012.patch
>
>
> To aide in deploying ASCII artifacts to clusters, l
[jira] [Commented] (OOZIE-3227) Eliminate duplicated dependencies from distributed cache
[
https://issues.apache.org/jira/browse/OOZIE-3227?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16499841#comment-16499841
]
Hadoop QA commented on OOZIE-3227:
--
PreCommit-OOZIE-Build started
> Eliminate duplicated dependencies from distributed cache
>
>
> Key: OOZIE-3227
> URL: https://issues.apache.org/jira/browse/OOZIE-3227
> Project: Oozie
> Issue Type: Sub-task
> Components: core
>Affects Versions: 5.0.0, 4.3.1
>Reporter: Denes Bodo
>Assignee: Denes Bodo
>Priority: Major
> Attachments: OOZIE-3227_001.patch, OOZIE-3227_002.patch
>
>
> Using Hadoop 3 it is not allowed to have multiple dependencies with same file
> names on the list of *mapreduce.job.cache.files*.
> The issue occurs when I have the same file name on multiple sharelib folders
> and/or my application's lib folder. This can be avoided but not easy all the
> time.
> I suggest to remove the duplicates from this list.
> A quick workaround for the source code in JavaActionExecutor is like:
> {code}
> removeDuplicatedDependencies(launcherJobConf,
> "mapreduce.job.cache.files");
> removeDuplicatedDependencies(launcherJobConf,
> "mapreduce.job.cache.archives");
> ..
> private void removeDuplicatedDependencies(JobConf conf, String key) {
> final Map nameToPath = new HashMap<>();
> StringBuilder uniqList = new StringBuilder();
> for(String dependency: conf.get(key).split(",")) {
> final String[] arr = dependency.split("/");
> final String dependencyName = arr[arr.length - 1];
> if(nameToPath.containsKey(dependencyName)) {
> LOG.warn(dependencyName + " [" + dependency + "] is already
> defined in " + key + ". Skipping...");
> } else {
> nameToPath.put(dependencyName, dependency);
> uniqList.append(dependency).append(",");
> }
> }
> uniqList.setLength(uniqList.length() - 1);
> conf.set(key, uniqList.toString());
> }
> {code}
> Other way is to eliminate the deprecated
> *org.apache.hadoop.filecache.DistributedCache*.
> I am going to have a deeper understanding how we should use distributed cache
> and all the comments are welcome.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OOZIE-3270) Upgrade Derby to 10.14.1.0
[ https://issues.apache.org/jira/browse/OOZIE-3270?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16499868#comment-16499868 ] Peter Cseh commented on OOZIE-3270: --- Although there are no junit test failures I'm having difficulties with running spark actions after the derby bump. Please hold with this patch > Upgrade Derby to 10.14.1.0 > --- > > Key: OOZIE-3270 > URL: https://issues.apache.org/jira/browse/OOZIE-3270 > Project: Oozie > Issue Type: Improvement >Reporter: Peter Cseh >Assignee: Peter Cseh >Priority: Major > Attachments: OOZIE-3270.01.patch, OOZIE-3270.02.patch > > > We should upgrade Derby to 10.14.1.0 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OOZIE-3156) SSH action status turns OK wrongly when failed to connect to host
[
https://issues.apache.org/jira/browse/OOZIE-3156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16499871#comment-16499871
]
Andras Piros commented on OOZIE-3156:
-
[~txsing] good idea! While you're at it, you could modify the signature of the
method to be extracted to cover rest of {{TestSshActionExecutor}}. Thanks!
Hint: in patch v4, {{DG_SshActionExtension.twiki#31}} is a line longer than 132
chars, violating {{RAW_PATCH_ANALYSIS}} pre-commit check.
> SSH action status turns OK wrongly when failed to connect to host
> -
>
> Key: OOZIE-3156
> URL: https://issues.apache.org/jira/browse/OOZIE-3156
> Project: Oozie
> Issue Type: Bug
> Components: action
>Affects Versions: 5.0.0
>Reporter: TIAN XING
>Assignee: TIAN XING
>Priority: Major
> Attachments: OOZIE-3156-v1.patch, OOZIE-3156-v2.patch,
> OOZIE-3156-v3.patch, OOZIE-3156-v4.patch, ssh-check-bug.patch
>
>
> When {{check()}} method of {{SshActionExecutor}} gets invoked, oozie will ssh
> connect to the host and check whether the pid of the process that ssh action
> started is still there (by checking the returned value of command "{{ssh
> ps -p }}" ) to determine whether ssh action completes or not.
> However, we found cases where oozie fails to connect to host during action
> status check (e.g., the host is under heavy load, or network is bad etc.).
> In such cases, the return value of command "{{ssh ps -p }}"
> will be 255 (ssh command exits with the exit status of the remote command or
> with 255 if an error occurred.).
> According the current logic of method {{getActionStatus()}} in
> {{SshActionExecutor}}, the action status will be determined as OK which may
> not be correct.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (OOZIE-3218) Oozie Sqoop action with command splits the select clause into multiple parts due to delimiter being space
[
https://issues.apache.org/jira/browse/OOZIE-3218?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andras Piros updated OOZIE-3218:
Attachment: OOZIE-3218-3.patch
> Oozie Sqoop action with command splits the select clause into multiple parts
> due to delimiter being space
> -
>
> Key: OOZIE-3218
> URL: https://issues.apache.org/jira/browse/OOZIE-3218
> Project: Oozie
> Issue Type: Bug
> Components: action, workflow
>Affects Versions: 3.3.2, 4.1.0, 4.2.0, 4.3.0, 5.0.0
> Environment: Hortonworks Hadoop HDP-2.6.4.x release
> oozie admin -version: Oozie server build version: 4.2.0.2.6.4.0-91
>Reporter: Mahesh Balakrishnan
>Assignee: Mahesh Balakrishnan
>Priority: Major
> Attachments: OOZIE-3218-2.patch, OOZIE-3218-3.patch, OOZIE-3218.patch
>
>
> When running a Oozie Sqoop action which has command with {{--query}} in place
> the query is split into multiple parts causing {{"Unrecognized argument:"}}
> and in-turn fails.
> {code:xml}
>
> ${resourceManager}
> ${nameNode}
> import --verbose --connect jdbc:mysql://test.openstacklocal/db
> --query select * from abc where $CONDITIONS --username test --password test
> --driver com.mysql.jdbc.Driver -m 1
>
>
> {code}
>
> Oozie Launcher logs:
> {noformat}
> Sqoop command arguments :
> import
> --verbose
> --connect
> jdbc:mysql://test.openstacklocal/db
> --query
> "select
> *
> from
> abc
> where
> $CONDITIONS"
> --username
> hive
> --password
>
> --driver
> com.mysql.jdbc.Driver
> -m
> 1
> Fetching child yarn jobs
> tag id : oozie-a1bbe03a0983b9e822d12ae7bb269ee3
> 2791 [main] INFO org.apache.hadoop.yarn.client.RMProxy - Connecting to
> ResourceManager at hdp263-3.openstacklocal/172.26.105.248:8050
> Child yarn jobs are found -
> =
> >>> Invoking Sqoop command line now >>>
> 3172 [main] WARN org.apache.sqoop.tool.SqoopTool - $SQOOP_CONF_DIR has not
> been set in the environment. Cannot check for additional configuration.
> 3172 [main] WARN org.apache.sqoop.tool.SqoopTool - $SQOOP_CONF_DIR has not
> been set in the environment. Cannot check for additional configuration.
> 3218 [main] INFO org.apache.sqoop.Sqoop - Running Sqoop version:
> 1.4.6.2.6.4.0-91
> 3218 [main] INFO org.apache.sqoop.Sqoop - Running Sqoop version:
> 1.4.6.2.6.4.0-91
> 3287 [main] DEBUG org.apache.sqoop.tool.BaseSqoopTool - Enabled debug
> logging.
> 3287 [main] DEBUG org.apache.sqoop.tool.BaseSqoopTool - Enabled debug
> logging.
> 3288 [main] ERROR org.apache.sqoop.tool.BaseSqoopTool - Error parsing
> arguments for import:
> 3288 [main] ERROR org.apache.sqoop.tool.BaseSqoopTool - Error parsing
> arguments for import:
> 3288 [main] ERROR org.apache.sqoop.tool.BaseSqoopTool - Unrecognized
> argument: *
> 3288 [main] ERROR org.apache.sqoop.tool.BaseSqoopTool - Unrecognized
> argument: *
> 3288 [main] ERROR org.apache.sqoop.tool.BaseSqoopTool - Unrecognized
> argument: from
> 3288 [main] ERROR org.apache.sqoop.tool.BaseSqoopTool - Unrecognized
> argument: from
> 3288 [main] ERROR org.apache.sqoop.tool.BaseSqoopTool - Unrecognized
> argument: abc
> 3288 [main] ERROR org.apache.sqoop.tool.BaseSqoopTool - Unrecognized
> argument: abc
> 3288 [main] ERROR org.apache.sqoop.tool.BaseSqoopTool - Unrecognized
> argument: where
> 3288 [main] ERROR org.apache.sqoop.tool.BaseSqoopTool - Unrecognized
> argument: where
> 3288 [main] ERROR org.apache.sqoop.tool.BaseSqoopTool - Unrecognized
> argument: $CONDITIONS"
> 3288 [main] ERROR org.apache.sqoop.tool.BaseSqoopTool - Unrecognized
> argument: $CONDITIONS"
> 3288 [main] ERROR org.apache.sqoop.tool.BaseSqoopTool - Unrecognized
> argument: --username
> 3288 [main] ERROR org.apache.sqoop.tool.BaseSqoopTool - Unrecognized
> argument: --username
> 3288 [main] ERROR org.apache.sqoop.tool.BaseSqoopTool - Unrecognized
> argument: abc
> 3288 [main] ERROR org.apache.sqoop.tool.BaseSqoopTool - Unrecognized
> argument: abc
> 3288 [main] ERROR org.apache.sqoop.tool.BaseSqoopTool - Unrecognized
> argument: --password
> 3288 [main] ERROR org.apache.sqoop.tool.BaseSqoopTool - Unrecognized
> argument: --password
> 3288 [main] ERROR org.apache.sqoop.tool.BaseSqoopTool - Unrecognized
> argument: abc
> 3288 [main] ERROR org.apache.sqoop.tool.BaseSqoopTool - Unrecognized
> argument: abc
> 3288 [main] ERROR org.apache.sqoop.tool.BaseSqoopTool - Unrecognized
> argument: --driver
> 3288 [main] ERROR org.apache.sqoop.tool.BaseSqoopTool - Unrecognized
> argument: --driver
> 3288 [main] ERROR org.apache.sqoop.tool.BaseSqoopTool - Unrecognized
> argument: com.mysql.jdbc.Driver
> 3288 [main] ERROR org.apache.sqoop.tool.BaseS
[jira] [Commented] (OOZIE-3218) Oozie Sqoop action with command splits the select clause into multiple parts due to delimiter being space
[
https://issues.apache.org/jira/browse/OOZIE-3218?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16499892#comment-16499892
]
Hadoop QA commented on OOZIE-3218:
--
PreCommit-OOZIE-Build started
> Oozie Sqoop action with command splits the select clause into multiple parts
> due to delimiter being space
> -
>
> Key: OOZIE-3218
> URL: https://issues.apache.org/jira/browse/OOZIE-3218
> Project: Oozie
> Issue Type: Bug
> Components: action, workflow
>Affects Versions: 3.3.2, 4.1.0, 4.2.0, 4.3.0, 5.0.0
> Environment: Hortonworks Hadoop HDP-2.6.4.x release
> oozie admin -version: Oozie server build version: 4.2.0.2.6.4.0-91
>Reporter: Mahesh Balakrishnan
>Assignee: Mahesh Balakrishnan
>Priority: Major
> Attachments: OOZIE-3218-2.patch, OOZIE-3218-3.patch, OOZIE-3218.patch
>
>
> When running a Oozie Sqoop action which has command with {{--query}} in place
> the query is split into multiple parts causing {{"Unrecognized argument:"}}
> and in-turn fails.
> {code:xml}
>
> ${resourceManager}
> ${nameNode}
> import --verbose --connect jdbc:mysql://test.openstacklocal/db
> --query select * from abc where $CONDITIONS --username test --password test
> --driver com.mysql.jdbc.Driver -m 1
>
>
> {code}
>
> Oozie Launcher logs:
> {noformat}
> Sqoop command arguments :
> import
> --verbose
> --connect
> jdbc:mysql://test.openstacklocal/db
> --query
> "select
> *
> from
> abc
> where
> $CONDITIONS"
> --username
> hive
> --password
>
> --driver
> com.mysql.jdbc.Driver
> -m
> 1
> Fetching child yarn jobs
> tag id : oozie-a1bbe03a0983b9e822d12ae7bb269ee3
> 2791 [main] INFO org.apache.hadoop.yarn.client.RMProxy - Connecting to
> ResourceManager at hdp263-3.openstacklocal/172.26.105.248:8050
> Child yarn jobs are found -
> =
> >>> Invoking Sqoop command line now >>>
> 3172 [main] WARN org.apache.sqoop.tool.SqoopTool - $SQOOP_CONF_DIR has not
> been set in the environment. Cannot check for additional configuration.
> 3172 [main] WARN org.apache.sqoop.tool.SqoopTool - $SQOOP_CONF_DIR has not
> been set in the environment. Cannot check for additional configuration.
> 3218 [main] INFO org.apache.sqoop.Sqoop - Running Sqoop version:
> 1.4.6.2.6.4.0-91
> 3218 [main] INFO org.apache.sqoop.Sqoop - Running Sqoop version:
> 1.4.6.2.6.4.0-91
> 3287 [main] DEBUG org.apache.sqoop.tool.BaseSqoopTool - Enabled debug
> logging.
> 3287 [main] DEBUG org.apache.sqoop.tool.BaseSqoopTool - Enabled debug
> logging.
> 3288 [main] ERROR org.apache.sqoop.tool.BaseSqoopTool - Error parsing
> arguments for import:
> 3288 [main] ERROR org.apache.sqoop.tool.BaseSqoopTool - Error parsing
> arguments for import:
> 3288 [main] ERROR org.apache.sqoop.tool.BaseSqoopTool - Unrecognized
> argument: *
> 3288 [main] ERROR org.apache.sqoop.tool.BaseSqoopTool - Unrecognized
> argument: *
> 3288 [main] ERROR org.apache.sqoop.tool.BaseSqoopTool - Unrecognized
> argument: from
> 3288 [main] ERROR org.apache.sqoop.tool.BaseSqoopTool - Unrecognized
> argument: from
> 3288 [main] ERROR org.apache.sqoop.tool.BaseSqoopTool - Unrecognized
> argument: abc
> 3288 [main] ERROR org.apache.sqoop.tool.BaseSqoopTool - Unrecognized
> argument: abc
> 3288 [main] ERROR org.apache.sqoop.tool.BaseSqoopTool - Unrecognized
> argument: where
> 3288 [main] ERROR org.apache.sqoop.tool.BaseSqoopTool - Unrecognized
> argument: where
> 3288 [main] ERROR org.apache.sqoop.tool.BaseSqoopTool - Unrecognized
> argument: $CONDITIONS"
> 3288 [main] ERROR org.apache.sqoop.tool.BaseSqoopTool - Unrecognized
> argument: $CONDITIONS"
> 3288 [main] ERROR org.apache.sqoop.tool.BaseSqoopTool - Unrecognized
> argument: --username
> 3288 [main] ERROR org.apache.sqoop.tool.BaseSqoopTool - Unrecognized
> argument: --username
> 3288 [main] ERROR org.apache.sqoop.tool.BaseSqoopTool - Unrecognized
> argument: abc
> 3288 [main] ERROR org.apache.sqoop.tool.BaseSqoopTool - Unrecognized
> argument: abc
> 3288 [main] ERROR org.apache.sqoop.tool.BaseSqoopTool - Unrecognized
> argument: --password
> 3288 [main] ERROR org.apache.sqoop.tool.BaseSqoopTool - Unrecognized
> argument: --password
> 3288 [main] ERROR org.apache.sqoop.tool.BaseSqoopTool - Unrecognized
> argument: abc
> 3288 [main] ERROR org.apache.sqoop.tool.BaseSqoopTool - Unrecognized
> argument: abc
> 3288 [main] ERROR org.apache.sqoop.tool.BaseSqoopTool - Unrecognized
> argument: --driver
> 3288 [main] ERROR org.apache.sqoop.tool.BaseSqoopTool - Unrecognized
> argument: --driver
> 3288 [main] ERROR org.apache.sqoop.tool.BaseSqoopTool - Unrecognized
> argument: com.mysql.jdbc.Driver
Success: OOZIE-3227 PreCommit Build #600
Jira: https://issues.apache.org/jira/browse/OOZIE-3227
Build: https://builds.apache.org/job/PreCommit-OOZIE-Build/600/
###
## LAST 100 LINES OF THE CONSOLE
###
[...truncated 1.71 MB...]
[DEBUG] There are no new bugs found in [sharelib/pig].
[TRACE] New XMLLib present, calling 'xmllint --xpath' to get bug instance counts
[DEBUG] There are no new bugs found in [sharelib/streaming].
[TRACE] New XMLLib present, calling 'xmllint --xpath' to get bug instance counts
[DEBUG] There are no new bugs found in [sharelib/hive].
[TRACE] New XMLLib present, calling 'xmllint --xpath' to get bug instance counts
[DEBUG] There are no new bugs found in [sharelib/hcatalog].
[TRACE] New XMLLib present, calling 'xmllint --xpath' to get bug instance counts
[DEBUG] There are no new bugs found in [sharelib/sqoop].
[TRACE] New XMLLib present, calling 'xmllint --xpath' to get bug instance counts
[DEBUG] There are no new bugs found in [sharelib/oozie].
[TRACE] New XMLLib present, calling 'xmllint --xpath' to get bug instance counts
[DEBUG] There are no new bugs found in [sharelib/distcp].
[TRACE] New XMLLib present, calling 'xmllint --xpath' to get bug instance counts
[DEBUG] There are no new bugs found in [sharelib/spark].
[TRACE] New XMLLib present, calling 'xmllint --xpath' to get bug instance counts
[DEBUG] There are no new bugs found in [client].
[INFO] There are no new bugs found totally].
[TRACE] FindBugs diffs checked and reports created
[TRACE] Summary file size is 2365 bytes
[TRACE] Full summary file size is 1314 bytes
[TRACE] File
[/home/jenkins/jenkins-slave/workspace/PreCommit-OOZIE-Build/test-patch/tmp/FINDBUGS_DIFF/diff/findbugs-diff-0.1.0-all.jar]
removed
[TRACE] File
[/home/jenkins/jenkins-slave/workspace/PreCommit-OOZIE-Build/test-patch/tmp/FINDBUGS_DIFF/diff/findbugs-diff-0.1.0-all.jar.md5sum]
removed
Running test-patch task BACKWARDS_COMPATIBILITY
Running test-patch task TESTS
Running test-patch task DISTRO
Testing JIRA OOZIE-3227
Cleaning local git workspace
+1 PATCH_APPLIES
+1 CLEAN
+1 RAW_PATCH_ANALYSIS
+1 the patch does not introduce any @author tags
+1 the patch does not introduce any tabs
+1 the patch does not introduce any trailing spaces
+1 the patch does not introduce any line longer than 132
+1 the patch adds/modifies 2 testcase(s)
+1 RAT
+1 the patch does not seem to introduce new RAT warnings
+1 JAVADOC
+1 JAVADOC
+1 the patch does not seem to introduce new Javadoc warning(s)
+1 the patch does not seem to introduce new Javadoc error(s)
ERROR: the current HEAD has 2 Javadoc error(s)
+1 COMPILE
+1 HEAD compiles
+1 patch compiles
+1 the patch does not seem to introduce new javac warnings
+1 There are no new bugs found in total.
+1 There are no new bugs found in [examples].
+1 There are no new bugs found in [webapp].
+1 There are no new bugs found in [core].
+1 There are no new bugs found in [tools].
+1 There are no new bugs found in [server].
+1 There are no new bugs found in [docs].
+1 There are no new bugs found in [sharelib/hive2].
+1 There are no new bugs found in [sharelib/pig].
+1 There are no new bugs found in [sharelib/streaming].
+1 There are no new bugs found in [sharelib/hive].
+1 There are no new bugs found in [sharelib/hcatalog].
+1 There are no new bugs found in [sharelib/sqoop].
+1 There are no new bugs found in [sharelib/oozie].
+1 There are no new bugs found in [sharelib/distcp].
+1 There are no new bugs found in [sharelib/spark].
+1 There are no new bugs found in [client].
+1 BACKWARDS_COMPATIBILITY
+1 the patch does not change any JPA Entity/Colum/Basic/Lob/Transient
annotations
+1 the patch does not modify JPA files
+1 TESTS
Tests run: 2146
+1 DISTRO
+1 distro tarball builds with the patch
+1 Overall result, good!, no -1s
The full output of the test-patch run is available at
https://builds.apache.org/job/PreCommit-OOZIE-Build/600/
Adding comment to JIRA
% Total% Received % Xferd Average Speed TimeTime Time Current
Dload Upload Total SpentLeft Speed
0 00 00 0 0 0 --:--:-- --:--:-- --:--:--
0100 30020 0 100 3002 0 3269 --:--:-- --:--:-- --:--:--
3266{"self":"https://issues.apache.org/jira/rest/api/2/issue/13154507/comment/16499906","id":"16499906","author":{"self":"https://issues.apache.org/jira/rest/api/2/user?username=hadoopqa","name":"hadoopqa","key":"hadoopqa","emailAddress":"blackhole
at hadoop dot apache dot
org","avatarUrls":{"48x48":"https://issues.apache.org/jira/secure/useravatar?ownerId=hadoopqa&avatarId=10393","24x24":"https://issues.apache.org/jira/secure/useravatar?size=small&ownerId=hadoopqa&avatarId=10393","16x16":"https://issues.apache.org/jira/secure/useravatar?size=xs
[jira] [Commented] (OOZIE-3227) Eliminate duplicated dependencies from distributed cache
[
https://issues.apache.org/jira/browse/OOZIE-3227?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16499906#comment-16499906
]
Hadoop QA commented on OOZIE-3227:
--
Testing JIRA OOZIE-3227
Cleaning local git workspace
{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:green}+1 RAW_PATCH_ANALYSIS{color}
.{color:green}+1{color} the patch does not introduce any @author tags
.{color:green}+1{color} the patch does not introduce any tabs
.{color:green}+1{color} the patch does not introduce any trailing spaces
.{color:green}+1{color} the patch does not introduce any line longer than
132
.{color:green}+1{color} the patch adds/modifies 2 testcase(s)
{color:green}+1 RAT{color}
.{color:green}+1{color} the patch does not seem to introduce new RAT
warnings
{color:green}+1 JAVADOC{color}
{color:green}+1 JAVADOC{color}
.{color:green}+1{color} the patch does not seem to introduce new Javadoc
warning(s)
.{color:green}+1{color} the patch does not seem to introduce new Javadoc
error(s)
.{color:red}ERROR{color}: the current HEAD has 2 Javadoc error(s)
{color:green}+1 COMPILE{color}
.{color:green}+1{color} HEAD compiles
.{color:green}+1{color} patch compiles
.{color:green}+1{color} the patch does not seem to introduce new javac
warnings
{color:green}+1{color} There are no new bugs found in total.
. {color:green}+1{color} There are no new bugs found in [examples].
. {color:green}+1{color} There are no new bugs found in [webapp].
. {color:green}+1{color} There are no new bugs found in [core].
. {color:green}+1{color} There are no new bugs found in [tools].
. {color:green}+1{color} There are no new bugs found in [server].
. {color:green}+1{color} There are no new bugs found in [docs].
. {color:green}+1{color} There are no new bugs found in [sharelib/hive2].
. {color:green}+1{color} There are no new bugs found in [sharelib/pig].
. {color:green}+1{color} There are no new bugs found in [sharelib/streaming].
. {color:green}+1{color} There are no new bugs found in [sharelib/hive].
. {color:green}+1{color} There are no new bugs found in [sharelib/hcatalog].
. {color:green}+1{color} There are no new bugs found in [sharelib/sqoop].
. {color:green}+1{color} There are no new bugs found in [sharelib/oozie].
. {color:green}+1{color} There are no new bugs found in [sharelib/distcp].
. {color:green}+1{color} There are no new bugs found in [sharelib/spark].
. {color:green}+1{color} There are no new bugs found in [client].
{color:green}+1 BACKWARDS_COMPATIBILITY{color}
.{color:green}+1{color} the patch does not change any JPA
Entity/Colum/Basic/Lob/Transient annotations
.{color:green}+1{color} the patch does not modify JPA files
{color:green}+1 TESTS{color}
.Tests run: 2146
{color:green}+1 DISTRO{color}
.{color:green}+1{color} distro tarball builds with the patch
{color:green}*+1 Overall result, good!, no -1s*{color}
The full output of the test-patch run is available at
. https://builds.apache.org/job/PreCommit-OOZIE-Build/600/
> Eliminate duplicated dependencies from distributed cache
>
>
> Key: OOZIE-3227
> URL: https://issues.apache.org/jira/browse/OOZIE-3227
> Project: Oozie
> Issue Type: Sub-task
> Components: core
>Affects Versions: 5.0.0, 4.3.1
>Reporter: Denes Bodo
>Assignee: Denes Bodo
>Priority: Major
> Attachments: OOZIE-3227_001.patch, OOZIE-3227_002.patch
>
>
> Using Hadoop 3 it is not allowed to have multiple dependencies with same file
> names on the list of *mapreduce.job.cache.files*.
> The issue occurs when I have the same file name on multiple sharelib folders
> and/or my application's lib folder. This can be avoided but not easy all the
> time.
> I suggest to remove the duplicates from this list.
> A quick workaround for the source code in JavaActionExecutor is like:
> {code}
> removeDuplicatedDependencies(launcherJobConf,
> "mapreduce.job.cache.files");
> removeDuplicatedDependencies(launcherJobConf,
> "mapreduce.job.cache.archives");
> ..
> private void removeDuplicatedDependencies(JobConf conf, String key) {
> final Map nameToPath = new HashMap<>();
> StringBuilder uniqList = new StringBuilder();
> for(String dependency: conf.get(key).split(",")) {
> final String[] arr = dependency.split("/");
> final String dependencyName = arr[arr.length - 1];
> if(nameToPath.containsKey(dependencyName)) {
> LOG.warn(dependencyName + " [" + dependency + "] is already
> defined in " + key + ". Skipping...");
> } else {
> nameToPath.put(dependencyName, dependency);
> uniqList.append(de
[jira] [Commented] (OOZIE-2877) Oozie Git Action
[
https://issues.apache.org/jira/browse/OOZIE-2877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16499919#comment-16499919
]
Andras Piros commented on OOZIE-2877:
-
[~gezapeti] answering some of your comments:
* credential files are stored on the HDFS path potentially only readable by the
executing user of the Git action. It's stored in a temporary subfolder of the
current working directory (YARN NodeManager container folder) also only
readable by that user, and will possibly be deleted by YARN NodeManager after
the action has been run. So I think we're clean here
* {{destination-uri}} is checked via calls to *{{FileSystem#get()}}* and
[*{{FileSystem#makeQualified()}}*|https://github.com/apache/hadoop/blob/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileSystem.java#L587-L590]
* {{job-tracker}} appearances substituted by {{resource-manager}} in latest
patch
[~clayb] many thanks for your extended efforts on the feature! The only
remaining thingies I see at the moment are:
* addressing remaining pre-commit issues:
{noformat}
-1 RAW_PATCH_ANALYSIS
. -1 the patch contains 10 line(s) with tabs
. -1 the patch contains 10 line(s) with trailing spaces
-1 There are [4] new bugs found below threshold in total that must be fixed.
. -1 There are [4] new bugs found below threshold in [sharelib/git] that must
be fixed.
. You can find the FindBugs diff here (look for the red and orange ones):
sharelib/git/findbugs-new.html
. The most important FindBugs errors are:
. At GitMain.java:[line 151]: java/io/File.(Ljava/lang/String;)V reads a
file whose location might be specified by user input
. At GitMain.java:[line 151]: At GitMain.java:[line 145]
. At GitMain.java:[line 134]: java/io/File.(Ljava/lang/String;)V reads a
file whose location might be specified by user input
. Possible null pointer dereference in
org.apache.oozie.action.hadoop.GitOperations.cloneRepoToFS(Path) due to return
value of called method: At GitMain.java:[line 134]
. Known null at GitOperations.java:[line 151]: Dereferenced at
GitOperations.java:[line 151]
{noformat}
* substituting {{examples/src/main/apps/git/workflow.xml}}'s {{git-uri}} value
with {{https://github.com/apache/oozie}}
* substituting {{TestGitActionExecutor}}'s {{git-uri}} value with
{{https://github.com/klout/scoozie}} so that integration / JUnit test will not
timeout on a machine that has mediocre internet connection
Let me know whether you need my help while finishing.
> Oozie Git Action
>
>
> Key: OOZIE-2877
> URL: https://issues.apache.org/jira/browse/OOZIE-2877
> Project: Oozie
> Issue Type: Sub-task
> Components: action
>Reporter: Clay B.
>Assignee: Clay B.
>Priority: Major
> Labels: action
> Fix For: trunk
>
> Attachments: 0001-OOZIE-2877-Oozie-Git-Action.patch,
> 0002-OOZIE-2877-Oozie-Git-Action.patch,
> 0003-OOZIE-2877-Oozie-Git-Action.patch,
> 0004-OOZIE-2877-Oozie-Git-Action.patch,
> 0005-OOZIE-2877-Oozie-Git-Action.patch,
> 0006-OOZIE-2877-Oozie-Git-Action.patch,
> 0007-OOZIE-2877-Oozie-Git-Action.patch,
> 0008-OOZIE-2877-Oozie-Git-Action.patch,
> 0009-OOZIE-2877-Oozie-Git-Action.patch, OOZIE-2877.010.patch,
> OOZIE-2877.011.patch, OOZIE-2877.012.patch
>
>
> To aide in deploying ASCII artifacts to clusters, let's provide a tie-in for
> a source-code management system. Git would be my preferred choice.
> Ideally, this could handle a user's key material e.g. for an ssh key to pull
> down from a secured repository. This would free users from handling their own
> key staging and clean-up on YARN nodes and only require them to store the key
> secured in HDFS.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (OOZIE-3227) Eliminate duplicate dependencies when using Hadoop 3 DistributedCache
[
https://issues.apache.org/jira/browse/OOZIE-3227?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andras Piros updated OOZIE-3227:
Summary: Eliminate duplicate dependencies when using Hadoop 3
DistributedCache (was: Eliminate duplicated dependencies from distributed
cache)
> Eliminate duplicate dependencies when using Hadoop 3 DistributedCache
> -
>
> Key: OOZIE-3227
> URL: https://issues.apache.org/jira/browse/OOZIE-3227
> Project: Oozie
> Issue Type: Sub-task
> Components: core
>Affects Versions: 5.0.0, 4.3.1
>Reporter: Denes Bodo
>Assignee: Denes Bodo
>Priority: Major
> Attachments: OOZIE-3227_001.patch, OOZIE-3227_002.patch
>
>
> Using Hadoop 3 it is not allowed to have multiple dependencies with same file
> names on the list of *mapreduce.job.cache.files*.
> The issue occurs when I have the same file name on multiple sharelib folders
> and/or my application's lib folder. This can be avoided but not easy all the
> time.
> I suggest to remove the duplicates from this list.
> A quick workaround for the source code in JavaActionExecutor is like:
> {code}
> removeDuplicatedDependencies(launcherJobConf,
> "mapreduce.job.cache.files");
> removeDuplicatedDependencies(launcherJobConf,
> "mapreduce.job.cache.archives");
> ..
> private void removeDuplicatedDependencies(JobConf conf, String key) {
> final Map nameToPath = new HashMap<>();
> StringBuilder uniqList = new StringBuilder();
> for(String dependency: conf.get(key).split(",")) {
> final String[] arr = dependency.split("/");
> final String dependencyName = arr[arr.length - 1];
> if(nameToPath.containsKey(dependencyName)) {
> LOG.warn(dependencyName + " [" + dependency + "] is already
> defined in " + key + ". Skipping...");
> } else {
> nameToPath.put(dependencyName, dependency);
> uniqList.append(dependency).append(",");
> }
> }
> uniqList.setLength(uniqList.length() - 1);
> conf.set(key, uniqList.toString());
> }
> {code}
> Other way is to eliminate the deprecated
> *org.apache.hadoop.filecache.DistributedCache*.
> I am going to have a deeper understanding how we should use distributed cache
> and all the comments are welcome.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (OOZIE-3156) SSH action status turns OK wrongly when failed to connect to host
[
https://issues.apache.org/jira/browse/OOZIE-3156?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
TIAN XING updated OOZIE-3156:
-
Attachment: OOZIE-3156-v5.patch
> SSH action status turns OK wrongly when failed to connect to host
> -
>
> Key: OOZIE-3156
> URL: https://issues.apache.org/jira/browse/OOZIE-3156
> Project: Oozie
> Issue Type: Bug
> Components: action
>Affects Versions: 5.0.0
>Reporter: TIAN XING
>Assignee: TIAN XING
>Priority: Major
> Attachments: OOZIE-3156-v1.patch, OOZIE-3156-v2.patch,
> OOZIE-3156-v3.patch, OOZIE-3156-v4.patch, OOZIE-3156-v5.patch,
> ssh-check-bug.patch
>
>
> When {{check()}} method of {{SshActionExecutor}} gets invoked, oozie will ssh
> connect to the host and check whether the pid of the process that ssh action
> started is still there (by checking the returned value of command "{{ssh
> ps -p }}" ) to determine whether ssh action completes or not.
> However, we found cases where oozie fails to connect to host during action
> status check (e.g., the host is under heavy load, or network is bad etc.).
> In such cases, the return value of command "{{ssh ps -p }}"
> will be 255 (ssh command exits with the exit status of the remote command or
> with 255 if an error occurred.).
> According the current logic of method {{getActionStatus()}} in
> {{SshActionExecutor}}, the action status will be determined as OK which may
> not be correct.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OOZIE-3156) SSH action status turns OK wrongly when failed to connect to host
[
https://issues.apache.org/jira/browse/OOZIE-3156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16499939#comment-16499939
]
Hadoop QA commented on OOZIE-3156:
--
PreCommit-OOZIE-Build started
> SSH action status turns OK wrongly when failed to connect to host
> -
>
> Key: OOZIE-3156
> URL: https://issues.apache.org/jira/browse/OOZIE-3156
> Project: Oozie
> Issue Type: Bug
> Components: action
>Affects Versions: 5.0.0
>Reporter: TIAN XING
>Assignee: TIAN XING
>Priority: Major
> Attachments: OOZIE-3156-v1.patch, OOZIE-3156-v2.patch,
> OOZIE-3156-v3.patch, OOZIE-3156-v4.patch, OOZIE-3156-v5.patch,
> ssh-check-bug.patch
>
>
> When {{check()}} method of {{SshActionExecutor}} gets invoked, oozie will ssh
> connect to the host and check whether the pid of the process that ssh action
> started is still there (by checking the returned value of command "{{ssh
> ps -p }}" ) to determine whether ssh action completes or not.
> However, we found cases where oozie fails to connect to host during action
> status check (e.g., the host is under heavy load, or network is bad etc.).
> In such cases, the return value of command "{{ssh ps -p }}"
> will be 255 (ssh command exits with the exit status of the remote command or
> with 255 if an error occurred.).
> According the current logic of method {{getActionStatus()}} in
> {{SshActionExecutor}}, the action status will be determined as OK which may
> not be correct.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OOZIE-3156) SSH action status turns OK wrongly when failed to connect to host
[
https://issues.apache.org/jira/browse/OOZIE-3156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16499941#comment-16499941
]
Andras Piros commented on OOZIE-3156:
-
[~txsing] thanks for the contribution! +1 for patch v5 (pending Jenkins)
> SSH action status turns OK wrongly when failed to connect to host
> -
>
> Key: OOZIE-3156
> URL: https://issues.apache.org/jira/browse/OOZIE-3156
> Project: Oozie
> Issue Type: Bug
> Components: action
>Affects Versions: 5.0.0
>Reporter: TIAN XING
>Assignee: TIAN XING
>Priority: Major
> Attachments: OOZIE-3156-v1.patch, OOZIE-3156-v2.patch,
> OOZIE-3156-v3.patch, OOZIE-3156-v4.patch, OOZIE-3156-v5.patch,
> ssh-check-bug.patch
>
>
> When {{check()}} method of {{SshActionExecutor}} gets invoked, oozie will ssh
> connect to the host and check whether the pid of the process that ssh action
> started is still there (by checking the returned value of command "{{ssh
> ps -p }}" ) to determine whether ssh action completes or not.
> However, we found cases where oozie fails to connect to host during action
> status check (e.g., the host is under heavy load, or network is bad etc.).
> In such cases, the return value of command "{{ssh ps -p }}"
> will be 255 (ssh command exits with the exit status of the remote command or
> with 255 if an error occurred.).
> According the current logic of method {{getActionStatus()}} in
> {{SshActionExecutor}}, the action status will be determined as OK which may
> not be correct.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
Failed: OOZIE-3218 PreCommit Build #601
Jira: https://issues.apache.org/jira/browse/OOZIE-3218
Build: https://builds.apache.org/job/PreCommit-OOZIE-Build/601/
###
## LAST 100 LINES OF THE CONSOLE
###
[...truncated 1.72 MB...]
[TRACE] New XMLLib present, calling 'xmllint --xpath' to get bug instance counts
[DEBUG] There are no new bugs found in [sharelib/distcp].
[TRACE] New XMLLib present, calling 'xmllint --xpath' to get bug instance counts
[DEBUG] There are no new bugs found in [sharelib/spark].
[TRACE] New XMLLib present, calling 'xmllint --xpath' to get bug instance counts
[DEBUG] There are no new bugs found in [client].
[INFO] There are no new bugs found totally].
[TRACE] FindBugs diffs checked and reports created
[TRACE] Summary file size is 2365 bytes
[TRACE] Full summary file size is 1314 bytes
[TRACE] File
[/home/jenkins/jenkins-slave/workspace/PreCommit-OOZIE-Build@2/test-patch/tmp/FINDBUGS_DIFF/diff/findbugs-diff-0.1.0-all.jar]
removed
[TRACE] File
[/home/jenkins/jenkins-slave/workspace/PreCommit-OOZIE-Build@2/test-patch/tmp/FINDBUGS_DIFF/diff/findbugs-diff-0.1.0-all.jar.md5sum]
removed
Running test-patch task BACKWARDS_COMPATIBILITY
Running test-patch task TESTS
Running test-patch task DISTRO
Testing JIRA OOZIE-3218
Cleaning local git workspace
+1 PATCH_APPLIES
+1 CLEAN
+1 RAW_PATCH_ANALYSIS
+1 the patch does not introduce any @author tags
+1 the patch does not introduce any tabs
+1 the patch does not introduce any trailing spaces
+1 the patch does not introduce any line longer than 132
+1 the patch adds/modifies 1 testcase(s)
+1 RAT
+1 the patch does not seem to introduce new RAT warnings
+1 JAVADOC
+1 JAVADOC
+1 the patch does not seem to introduce new Javadoc warning(s)
+1 the patch does not seem to introduce new Javadoc error(s)
ERROR: the current HEAD has 2 Javadoc error(s)
+1 COMPILE
+1 HEAD compiles
+1 patch compiles
+1 the patch does not seem to introduce new javac warnings
+1 There are no new bugs found in total.
+1 There are no new bugs found in [examples].
+1 There are no new bugs found in [webapp].
+1 There are no new bugs found in [core].
+1 There are no new bugs found in [tools].
+1 There are no new bugs found in [server].
+1 There are no new bugs found in [docs].
+1 There are no new bugs found in [sharelib/hive2].
+1 There are no new bugs found in [sharelib/pig].
+1 There are no new bugs found in [sharelib/streaming].
+1 There are no new bugs found in [sharelib/hive].
+1 There are no new bugs found in [sharelib/hcatalog].
+1 There are no new bugs found in [sharelib/sqoop].
+1 There are no new bugs found in [sharelib/oozie].
+1 There are no new bugs found in [sharelib/distcp].
+1 There are no new bugs found in [sharelib/spark].
+1 There are no new bugs found in [client].
+1 BACKWARDS_COMPATIBILITY
+1 the patch does not change any JPA Entity/Colum/Basic/Lob/Transient
annotations
+1 the patch does not modify JPA files
-1 TESTS
Tests run: 2132
Tests failed: 1
Tests errors: 0
The patch failed the following testcases:
testSqoopActionWithRedundantPrefix(org.apache.oozie.action.hadoop.TestSqoopActionExecutor)
Tests failing with errors:
+1 DISTRO
+1 distro tarball builds with the patch
-1 Overall result, please check the reported -1(s)
The full output of the test-patch run is available at
https://builds.apache.org/job/PreCommit-OOZIE-Build/601/
Adding comment to JIRA
% Total% Received % Xferd Average Speed TimeTime Time Current
Dload Upload Total SpentLeft Speed
0 00 00 0 0 0 --:--:-- --:--:-- --:--:--
0100 32430 0 100 3243 0 4474 --:--:-- --:--:-- --:--:--
4473{"self":"https://issues.apache.org/jira/rest/api/2/issue/13152080/comment/16499985","id":"16499985","author":{"self":"https://issues.apache.org/jira/rest/api/2/user?username=hadoopqa","name":"hadoopqa","key":"hadoopqa","emailAddress":"blackhole
at hadoop dot apache dot
org","avatarUrls":{"48x48":"https://issues.apache.org/jira/secure/useravatar?ownerId=hadoopqa&avatarId=10393","24x24":"https://issues.apache.org/jira/secure/useravatar?size=small&ownerId=hadoopqa&avatarId=10393","16x16":"https://issues.apache.org/jira/secure/useravatar?size=xsmall&ownerId=hadoopqa&avatarId=10393","32x32":"https://issues.apache.org/jira/secure/useravatar?size=medium&ownerId=hadoopqa&avatarId=10393"},"displayName":"Hadoop
QA","active":true,"timeZone":"Etc/UTC"},"body":"\nTesting JIRA
OOZIE-3218\n\nCleaning local git
workspace\n\n\n\n{color:green}+1
PATCH_APPLIES{color}\n{color:green}+1 CLEAN{color}\n{color:green}+1
RAW_PATCH_ANALYSIS{color}\n.{color:green}+1{color} the patch does not
introduce any @author tags\n.{color:green}+
[jira] [Commented] (OOZIE-3218) Oozie Sqoop action with command splits the select clause into multiple parts due to delimiter being space
[
https://issues.apache.org/jira/browse/OOZIE-3218?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16499985#comment-16499985
]
Hadoop QA commented on OOZIE-3218:
--
Testing JIRA OOZIE-3218
Cleaning local git workspace
{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:green}+1 RAW_PATCH_ANALYSIS{color}
.{color:green}+1{color} the patch does not introduce any @author tags
.{color:green}+1{color} the patch does not introduce any tabs
.{color:green}+1{color} the patch does not introduce any trailing spaces
.{color:green}+1{color} the patch does not introduce any line longer than
132
.{color:green}+1{color} the patch adds/modifies 1 testcase(s)
{color:green}+1 RAT{color}
.{color:green}+1{color} the patch does not seem to introduce new RAT
warnings
{color:green}+1 JAVADOC{color}
{color:green}+1 JAVADOC{color}
.{color:green}+1{color} the patch does not seem to introduce new Javadoc
warning(s)
.{color:green}+1{color} the patch does not seem to introduce new Javadoc
error(s)
.{color:red}ERROR{color}: the current HEAD has 2 Javadoc error(s)
{color:green}+1 COMPILE{color}
.{color:green}+1{color} HEAD compiles
.{color:green}+1{color} patch compiles
.{color:green}+1{color} the patch does not seem to introduce new javac
warnings
{color:green}+1{color} There are no new bugs found in total.
. {color:green}+1{color} There are no new bugs found in [examples].
. {color:green}+1{color} There are no new bugs found in [webapp].
. {color:green}+1{color} There are no new bugs found in [core].
. {color:green}+1{color} There are no new bugs found in [tools].
. {color:green}+1{color} There are no new bugs found in [server].
. {color:green}+1{color} There are no new bugs found in [docs].
. {color:green}+1{color} There are no new bugs found in [sharelib/hive2].
. {color:green}+1{color} There are no new bugs found in [sharelib/pig].
. {color:green}+1{color} There are no new bugs found in [sharelib/streaming].
. {color:green}+1{color} There are no new bugs found in [sharelib/hive].
. {color:green}+1{color} There are no new bugs found in [sharelib/hcatalog].
. {color:green}+1{color} There are no new bugs found in [sharelib/sqoop].
. {color:green}+1{color} There are no new bugs found in [sharelib/oozie].
. {color:green}+1{color} There are no new bugs found in [sharelib/distcp].
. {color:green}+1{color} There are no new bugs found in [sharelib/spark].
. {color:green}+1{color} There are no new bugs found in [client].
{color:green}+1 BACKWARDS_COMPATIBILITY{color}
.{color:green}+1{color} the patch does not change any JPA
Entity/Colum/Basic/Lob/Transient annotations
.{color:green}+1{color} the patch does not modify JPA files
{color:red}-1 TESTS{color}
.Tests run: 2132
.Tests failed: 1
.Tests errors: 0
.The patch failed the following testcases:
testSqoopActionWithRedundantPrefix(org.apache.oozie.action.hadoop.TestSqoopActionExecutor)
.Tests failing with errors:
{color:green}+1 DISTRO{color}
.{color:green}+1{color} distro tarball builds with the patch
{color:red}*-1 Overall result, please check the reported -1(s)*{color}
The full output of the test-patch run is available at
. https://builds.apache.org/job/PreCommit-OOZIE-Build/601/
> Oozie Sqoop action with command splits the select clause into multiple parts
> due to delimiter being space
> -
>
> Key: OOZIE-3218
> URL: https://issues.apache.org/jira/browse/OOZIE-3218
> Project: Oozie
> Issue Type: Bug
> Components: action, workflow
>Affects Versions: 3.3.2, 4.1.0, 4.2.0, 4.3.0, 5.0.0
> Environment: Hortonworks Hadoop HDP-2.6.4.x release
> oozie admin -version: Oozie server build version: 4.2.0.2.6.4.0-91
>Reporter: Mahesh Balakrishnan
>Assignee: Mahesh Balakrishnan
>Priority: Major
> Attachments: OOZIE-3218-2.patch, OOZIE-3218-3.patch, OOZIE-3218.patch
>
>
> When running a Oozie Sqoop action which has command with {{--query}} in place
> the query is split into multiple parts causing {{"Unrecognized argument:"}}
> and in-turn fails.
> {code:xml}
>
> ${resourceManager}
> ${nameNode}
> import --verbose --connect jdbc:mysql://test.openstacklocal/db
> --query select * from abc where $CONDITIONS --username test --password test
> --driver com.mysql.jdbc.Driver -m 1
>
>
> {code}
>
> Oozie Launcher logs:
> {noformat}
> Sqoop command arguments :
> import
> --verbose
> --connect
> jdbc:mysql://test.openstacklocal/db
> --query
> "select
> *
> from
> abc
> where
> $CONDITIONS"
> --username
> hive
> --password
>
> --driver
> com.mysql.jdbc.Driver
> -m
> 1
> Fetching chil
[jira] [Created] (OOZIE-3273) FsAction should fail on retry if destination path exists
Shubham created OOZIE-3273:
--
Summary: FsAction should fail on retry if destination path exists
Key: OOZIE-3273
URL: https://issues.apache.org/jira/browse/OOZIE-3273
Project: Oozie
Issue Type: Bug
Reporter: Shubham
This FsAction fails with error code FS008 if the source files already exist in
target folder.
The expected behavior should be that Oozie will try this action once again
after 1 minute, and marked the action as failed because the error is still
there.
However, Oozie marks the action as success on retry. (we didn't clean up the
target folder)
Logs:
{code}
2018-05-15 00:08:05,187 INFO ActionStartXCommand:520 - SERVER[mn2.sf.priv]
USER[hive] GROUP[-] TOKEN[] APP[sample-wf]
JOB[061-180514024838863-oozie-oozi-W]
ACTION[061-180514024838863-oozie-oozi-W@loading] Start action
[061-180514024838863-oozie-oozi-W@loading] with user-retry state :
userRetryCount [0], userRetryMax [2], userRetryInterval [1]
2018-05-15 00:08:05,201 WARN ActionStartXCommand:523 - SERVER[mn2.sf.priv]
USER[hive] GROUP[-] TOKEN[] APP[sample-wf]
JOB[061-180514024838863-oozie-oozi-W]
ACTION[061-180514024838863-oozie-oozi-W@loading] Error starting action
[load-staging]. ErrorType [ERROR], ErrorCode [FS008], Message [FS008: move,
could not move
[hdfs://nn:8020/user/hive/audit/data/ingestion/USER_ACCOUNT_AF_A/1522284431816-2018-03-28_1747_11.816-PT2M10.096S-TEST.0-19462_24325-67401946-8fcf-4940-91ec-063016a5da48.avro]
to [hdfs://nn:8020/user/hive/audit/data/staging/USER_ACCOUNT_AF_A]]
org.apache.oozie.action.ActionExecutorException: FS008: move, could not move
[hdfs://nn:8020/user/hive/audit/data/ingestion/SAMPLE_WF/1522284431816-2018-03-28_1747_11.816-PT2M10.096S-TEST.0-19462_24325-67401946-8fcf-4940-91ec-063016a5da48.avro]
to [hdfs://nn:8020/user/hive/audit/data/staging/USER_ACCOUNT_AF_A]
at
org.apache.oozie.action.hadoop.FsActionExecutor.move(FsActionExecutor.java:509)
at
org.apache.oozie.action.hadoop.FsActionExecutor.doOperations(FsActionExecutor.java:199)
at
org.apache.oozie.action.hadoop.FsActionExecutor.start(FsActionExecutor.java:609)
at
org.apache.oozie.command.wf.ActionStartXCommand.execute(ActionStartXCommand.java:234)
at
org.apache.oozie.command.wf.ActionStartXCommand.execute(ActionStartXCommand.java:65)
at org.apache.oozie.command.XCommand.call(XCommand.java:287)
at
org.apache.oozie.service.CallableQueueService$CompositeCallable.call(CallableQueueService.java:331)
at
org.apache.oozie.service.CallableQueueService$CompositeCallable.call(CallableQueueService.java:260)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
org.apache.oozie.service.CallableQueueService$CallableWrapper.run(CallableQueueService.java:178)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
2018-05-15 00:08:05,202 WARN ActionStartXCommand:523 - SERVER[mn2.sf.priv]
USER[hive] GROUP[-] TOKEN[] APP[sample-wf]
JOB[061-180514024838863-oozie-oozi-W]
ACTION[061-180514024838863-oozie-oozi-W@loading] Setting Action Status to
[DONE]
2018-05-15 00:08:05,202 INFO ActionStartXCommand:520 - SERVER[mn2.sf.priv]
USER[hive] GROUP[-] TOKEN[] APP[sample-wf]
JOB[061-180514024838863-oozie-oozi-W]
ACTION[061-180514024838863-oozie-oozi-W@loading] Preparing retry this
action [061-180514024838863-oozie-oozi-W@loading], errorCode [FS008],
userRetryCount [0], userRetryMax [2], userRetryInterval [1]
2018-05-15 00:09:05,254 INFO ActionStartXCommand:520 - SERVER[mn2.sf.priv]
USER[hive] GROUP[-] TOKEN[] APP[sample-wf]
JOB[061-180514024838863-oozie-oozi-W]
ACTION[061-180514024838863-oozie-oozi-W@loading] Start action
[061-180514024838863-oozie-oozi-W@loading] with user-retry state :
userRetryCount [1], userRetryMax [2], userRetryInterval [1]
2018-05-15 00:09:05,276 INFO ActionStartXCommand:520 - SERVER[mn2.sf.priv]
USER[hive] GROUP[-] TOKEN[] APP[sample-wf]
JOB[061-180514024838863-oozie-oozi-W]
ACTION[061-180514024838863-oozie-oozi-W@loading]
[***061-180514024838863-oozie-oozi-W@loading***]Action status=DONE
2018-05-15 00:09:05,277 INFO ActionStartXCommand:520 - SERVER[mn2.sf.priv]
USER[hive] GROUP[-] TOKEN[] APP[sample-wf]
JOB[061-180514024838863-oozie-oozi-W]
ACTION[061-180514024838863-oozie-oozi-W@loading]
[***061-180514024838863-oozie-oozi-W@loading***]Action updated in DB!
2018-05-15 00:09:05,314 INFO ActionStartXCommand:520 - SERVER[mn2.sf.priv]
USER[hive] GROUP[-] TOKEN[] APP[sample-wf]
JOB[061-180514024838863-oozie-oozi-W]
ACTION[061-180514024838863-oozie-oozi-W@end] Start action
[061-180514024838863-oozie-oozi-W@end] with user-retry state :
userRetryCount [0], userRetryMax [0], userRetryInterval [10]
2018-05-15 00:09:05,314 INFO ActionStartXCo
[jira] [Updated] (OOZIE-3273) FsAction should fail on retry if destination path exists
[
https://issues.apache.org/jira/browse/OOZIE-3273?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andras Piros updated OOZIE-3273:
Description:
This FsAction fails with error code FS008 if the source files already exist in
target folder.
The expected behavior should be that Oozie will try this action once again
after 1 minute, and marked the action as failed because the error is still
there.
However, Oozie marks the action as success on retry. (we didn't clean up the
target folder)
Logs:
{code}
2018-05-15 00:08:05,187 INFO ActionStartXCommand:520 - SERVER[mn2.sf.priv]
USER[hive] GROUP[-] TOKEN[] APP[sample-wf]
JOB[061-180514024838863-oozie-oozi-W]
ACTION[061-180514024838863-oozie-oozi-W@loading] Start action
[061-180514024838863-oozie-oozi-W@loading] with user-retry state :
userRetryCount [0], userRetryMax [2], userRetryInterval [1]
2018-05-15 00:08:05,201 WARN ActionStartXCommand:523 - SERVER[mn2.sf.priv]
USER[hive] GROUP[-] TOKEN[] APP[sample-wf]
JOB[061-180514024838863-oozie-oozi-W]
ACTION[061-180514024838863-oozie-oozi-W@loading] Error starting action
[load-staging]. ErrorType [ERROR], ErrorCode [FS008], Message [FS008: move,
could not move
[hdfs://nn:8020/user/hive/audit/data/ingestion/USER_ACCOUNT_AF_A/1522284431816-2018-03-28_1747_11.816-PT2M10.096S-TEST.0-19462_24325-67401946-8fcf-4940-91ec-063016a5da48.avro]
to [hdfs://nn:8020/user/hive/audit/data/staging/USER_ACCOUNT_AF_A]]
org.apache.oozie.action.ActionExecutorException: FS008: move, could not move
[hdfs://nn:8020/user/hive/audit/data/ingestion/SAMPLE_WF/1522284431816-2018-03-28_1747_11.816-PT2M10.096S-TEST.0-19462_24325-67401946-8fcf-4940-91ec-063016a5da48.avro]
to [hdfs://nn:8020/user/hive/audit/data/staging/USER_ACCOUNT_AF_A]
at
org.apache.oozie.action.hadoop.FsActionExecutor.move(FsActionExecutor.java:509)
at
org.apache.oozie.action.hadoop.FsActionExecutor.doOperations(FsActionExecutor.java:199)
at
org.apache.oozie.action.hadoop.FsActionExecutor.start(FsActionExecutor.java:609)
at
org.apache.oozie.command.wf.ActionStartXCommand.execute(ActionStartXCommand.java:234)
at
org.apache.oozie.command.wf.ActionStartXCommand.execute(ActionStartXCommand.java:65)
at org.apache.oozie.command.XCommand.call(XCommand.java:287)
at
org.apache.oozie.service.CallableQueueService$CompositeCallable.call(CallableQueueService.java:331)
at
org.apache.oozie.service.CallableQueueService$CompositeCallable.call(CallableQueueService.java:260)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
org.apache.oozie.service.CallableQueueService$CallableWrapper.run(CallableQueueService.java:178)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
2018-05-15 00:08:05,202 WARN ActionStartXCommand:523 - SERVER[mn2.sf.priv]
USER[hive] GROUP[-] TOKEN[] APP[sample-wf]
JOB[061-180514024838863-oozie-oozi-W]
ACTION[061-180514024838863-oozie-oozi-W@loading] Setting Action Status to
[DONE]
2018-05-15 00:08:05,202 INFO ActionStartXCommand:520 - SERVER[mn2.sf.priv]
USER[hive] GROUP[-] TOKEN[] APP[sample-wf]
JOB[061-180514024838863-oozie-oozi-W]
ACTION[061-180514024838863-oozie-oozi-W@loading] Preparing retry this
action [061-180514024838863-oozie-oozi-W@loading], errorCode [FS008],
userRetryCount [0], userRetryMax [2], userRetryInterval [1]
2018-05-15 00:09:05,254 INFO ActionStartXCommand:520 - SERVER[mn2.sf.priv]
USER[hive] GROUP[-] TOKEN[] APP[sample-wf]
JOB[061-180514024838863-oozie-oozi-W]
ACTION[061-180514024838863-oozie-oozi-W@loading] Start action
[061-180514024838863-oozie-oozi-W@loading] with user-retry state :
userRetryCount [1], userRetryMax [2], userRetryInterval [1]
2018-05-15 00:09:05,276 INFO ActionStartXCommand:520 - SERVER[mn2.sf.priv]
USER[hive] GROUP[-] TOKEN[] APP[sample-wf]
JOB[061-180514024838863-oozie-oozi-W]
ACTION[061-180514024838863-oozie-oozi-W@loading]
[***061-180514024838863-oozie-oozi-W@loading***]Action status=DONE
2018-05-15 00:09:05,277 INFO ActionStartXCommand:520 - SERVER[mn2.sf.priv]
USER[hive] GROUP[-] TOKEN[] APP[sample-wf]
JOB[061-180514024838863-oozie-oozi-W]
ACTION[061-180514024838863-oozie-oozi-W@loading]
[***061-180514024838863-oozie-oozi-W@loading***]Action updated in DB!
2018-05-15 00:09:05,314 INFO ActionStartXCommand:520 - SERVER[mn2.sf.priv]
USER[hive] GROUP[-] TOKEN[] APP[sample-wf]
JOB[061-180514024838863-oozie-oozi-W]
ACTION[061-180514024838863-oozie-oozi-W@end] Start action
[061-180514024838863-oozie-oozi-W@end] with user-retry state :
userRetryCount [0], userRetryMax [0], userRetryInterval [10]
2018-05-15 00:09:05,314 INFO ActionStartXCommand:520 - SERVER[mn2.sf.priv]
USER[hive] GROUP[-] TOKEN[] APP[sample-wf]
JOB[061-180514024838863-oozie-oozi-
[jira] [Commented] (OOZIE-3156) SSH action status turns OK wrongly when failed to connect to host
[
https://issues.apache.org/jira/browse/OOZIE-3156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16500082#comment-16500082
]
Hadoop QA commented on OOZIE-3156:
--
Testing JIRA OOZIE-3156
Cleaning local git workspace
{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:red}-1 RAW_PATCH_ANALYSIS{color}
.{color:green}+1{color} the patch does not introduce any @author tags
.{color:green}+1{color} the patch does not introduce any tabs
.{color:red}-1{color} the patch contains 1 line(s) with trailing spaces
.{color:green}+1{color} the patch does not introduce any line longer than
132
.{color:green}+1{color} the patch adds/modifies 1 testcase(s)
{color:green}+1 RAT{color}
.{color:green}+1{color} the patch does not seem to introduce new RAT
warnings
{color:green}+1 JAVADOC{color}
{color:green}+1 JAVADOC{color}
.{color:green}+1{color} the patch does not seem to introduce new Javadoc
warning(s)
.{color:green}+1{color} the patch does not seem to introduce new Javadoc
error(s)
.{color:red}ERROR{color}: the current HEAD has 2 Javadoc error(s)
{color:green}+1 COMPILE{color}
.{color:green}+1{color} HEAD compiles
.{color:green}+1{color} patch compiles
.{color:green}+1{color} the patch does not seem to introduce new javac
warnings
{color:green}+1{color} There are no new bugs found in total.
. {color:green}+1{color} There are no new bugs found in [webapp].
. {color:green}+1{color} There are no new bugs found in [core].
. {color:green}+1{color} There are no new bugs found in [tools].
. {color:green}+1{color} There are no new bugs found in [sharelib/hive2].
. {color:green}+1{color} There are no new bugs found in [sharelib/distcp].
. {color:green}+1{color} There are no new bugs found in [sharelib/hcatalog].
. {color:green}+1{color} There are no new bugs found in [sharelib/streaming].
. {color:green}+1{color} There are no new bugs found in [sharelib/sqoop].
. {color:green}+1{color} There are no new bugs found in [sharelib/oozie].
. {color:green}+1{color} There are no new bugs found in [sharelib/pig].
. {color:green}+1{color} There are no new bugs found in [sharelib/hive].
. {color:green}+1{color} There are no new bugs found in [sharelib/spark].
. {color:green}+1{color} There are no new bugs found in [client].
. {color:green}+1{color} There are no new bugs found in [examples].
. {color:green}+1{color} There are no new bugs found in [docs].
. {color:green}+1{color} There are no new bugs found in [server].
{color:green}+1 BACKWARDS_COMPATIBILITY{color}
.{color:green}+1{color} the patch does not change any JPA
Entity/Colum/Basic/Lob/Transient annotations
.{color:green}+1{color} the patch does not modify JPA files
{color:red}-1 TESTS{color}
.Tests run: 2146
.Tests failed: 0
.Tests errors: 1
.The patch failed the following testcases:
.Tests failing with errors:
testNewUsingACLs(org.apache.oozie.util.TestZKUtilsWithSecurity)
.{color:orange}Tests failed at first run:{color}
TestCoordActionsKillXCommand#testActionKillCommandActionNumbers
.For the complete list of flaky tests, see TEST-SUMMARY-FULL files.
{color:green}+1 DISTRO{color}
.{color:green}+1{color} distro tarball builds with the patch
{color:red}*-1 Overall result, please check the reported -1(s)*{color}
The full output of the test-patch run is available at
. https://builds.apache.org/job/PreCommit-OOZIE-Build/602/
> SSH action status turns OK wrongly when failed to connect to host
> -
>
> Key: OOZIE-3156
> URL: https://issues.apache.org/jira/browse/OOZIE-3156
> Project: Oozie
> Issue Type: Bug
> Components: action
>Affects Versions: 5.0.0
>Reporter: TIAN XING
>Assignee: TIAN XING
>Priority: Major
> Attachments: OOZIE-3156-v1.patch, OOZIE-3156-v2.patch,
> OOZIE-3156-v3.patch, OOZIE-3156-v4.patch, OOZIE-3156-v5.patch,
> ssh-check-bug.patch
>
>
> When {{check()}} method of {{SshActionExecutor}} gets invoked, oozie will ssh
> connect to the host and check whether the pid of the process that ssh action
> started is still there (by checking the returned value of command "{{ssh
> ps -p }}" ) to determine whether ssh action completes or not.
> However, we found cases where oozie fails to connect to host during action
> status check (e.g., the host is under heavy load, or network is bad etc.).
> In such cases, the return value of command "{{ssh ps -p }}"
> will be 255 (ssh command exits with the exit status of the remote command or
> with 255 if an error occurred.).
> According the current logic of method {{getActionStatus()}} in
> {{SshActionExecutor}}, the action status will be determined as OK which may
> not be correct.
--
This
Failed: OOZIE-3156 PreCommit Build #602
Jira: https://issues.apache.org/jira/browse/OOZIE-3156
Build: https://builds.apache.org/job/PreCommit-OOZIE-Build/602/
###
## LAST 100 LINES OF THE CONSOLE
###
[...truncated 1.73 MB...]
[DEBUG] There are no new bugs found in [docs].
[TRACE] New XMLLib present, calling 'xmllint --xpath' to get bug instance counts
[DEBUG] There are no new bugs found in [server].
[INFO] There are no new bugs found totally].
[TRACE] FindBugs diffs checked and reports created
[TRACE] Summary file size is 2362 bytes
[TRACE] Full summary file size is 1314 bytes
[TRACE] File
[/home/jenkins/jenkins-slave/workspace/PreCommit-OOZIE-Build/test-patch/tmp/FINDBUGS_DIFF/diff/findbugs-diff-0.1.0-all.jar]
removed
[TRACE] File
[/home/jenkins/jenkins-slave/workspace/PreCommit-OOZIE-Build/test-patch/tmp/FINDBUGS_DIFF/diff/findbugs-diff-0.1.0-all.jar.md5sum]
removed
Running test-patch task BACKWARDS_COMPATIBILITY
Running test-patch task TESTS
Running test-patch task DISTRO
Testing JIRA OOZIE-3156
Cleaning local git workspace
+1 PATCH_APPLIES
+1 CLEAN
-1 RAW_PATCH_ANALYSIS
+1 the patch does not introduce any @author tags
+1 the patch does not introduce any tabs
-1 the patch contains 1 line(s) with trailing spaces
+1 the patch does not introduce any line longer than 132
+1 the patch adds/modifies 1 testcase(s)
+1 RAT
+1 the patch does not seem to introduce new RAT warnings
+1 JAVADOC
+1 JAVADOC
+1 the patch does not seem to introduce new Javadoc warning(s)
+1 the patch does not seem to introduce new Javadoc error(s)
ERROR: the current HEAD has 2 Javadoc error(s)
+1 COMPILE
+1 HEAD compiles
+1 patch compiles
+1 the patch does not seem to introduce new javac warnings
+1 There are no new bugs found in total.
+1 There are no new bugs found in [webapp].
+1 There are no new bugs found in [core].
+1 There are no new bugs found in [tools].
+1 There are no new bugs found in [sharelib/hive2].
+1 There are no new bugs found in [sharelib/distcp].
+1 There are no new bugs found in [sharelib/hcatalog].
+1 There are no new bugs found in [sharelib/streaming].
+1 There are no new bugs found in [sharelib/sqoop].
+1 There are no new bugs found in [sharelib/oozie].
+1 There are no new bugs found in [sharelib/pig].
+1 There are no new bugs found in [sharelib/hive].
+1 There are no new bugs found in [sharelib/spark].
+1 There are no new bugs found in [client].
+1 There are no new bugs found in [examples].
+1 There are no new bugs found in [docs].
+1 There are no new bugs found in [server].
+1 BACKWARDS_COMPATIBILITY
+1 the patch does not change any JPA Entity/Colum/Basic/Lob/Transient
annotations
+1 the patch does not modify JPA files
-1 TESTS
Tests run: 2146
Tests failed: 0
Tests errors: 1
The patch failed the following testcases:
Tests failing with errors:
testNewUsingACLs(org.apache.oozie.util.TestZKUtilsWithSecurity)
Tests failed at first run:
TestCoordActionsKillXCommand#testActionKillCommandActionNumbers
For the complete list of flaky tests, see TEST-SUMMARY-FULL files.
+1 DISTRO
+1 distro tarball builds with the patch
-1 Overall result, please check the reported -1(s)
The full output of the test-patch run is available at
https://builds.apache.org/job/PreCommit-OOZIE-Build/602/
Adding comment to JIRA
% Total% Received % Xferd Average Speed TimeTime Time Current
Dload Upload Total SpentLeft Speed
0 00 00 0 0 0 --:--:-- --:--:-- --:--:-- 0
0 00 00 0 0 0 --:--:-- --:--:-- --:--:--
0{"self":"https://issues.apache.org/jira/rest/api/2/issue/13130096/comment/16500082","id":"16500082","author":{"self":"https://issues.apache.org/jira/rest/api/2/user?username=hadoopqa","name":"hadoopqa","key":"hadoopqa","emailAddress":"blackhole
at hadoop dot apache dot
org","avatarUrls":{"48x48":"https://issues.apache.org/jira/secure/useravatar?ownerId=hadoopqa&avatarId=10393","24x24":"https://issues.apache.org/jira/secure/useravatar?size=small&ownerId=hadoopqa&avatarId=10393","16x16":"https://issues.apache.org/jira/secure/useravatar?size=xsmall&ownerId=hadoopqa&avatarId=10393","32x32":"https://issues.apache.org/jira/secure/useravatar?size=medium&ownerId=hadoopqa&avatarId=10393"},"displayName":"Hadoop
QA","active":true,"timeZone":"Etc/UTC"},"body":"\nTesting JIRA
OOZIE-3156\n\nCleaning local git
workspace\n\n\n\n{color:green}+1
PATCH_APPLIES{color}\n{color:green}+1 CLEAN{color}\n{color:red}-1
RAW_PATCH_ANALYSIS{color}\n.{color:green}+1{color} the patch does not
introduce any @author tags\n.{color:green}+1{color} the patch does not
introduce any tabs\n.{color:red}-1{color} the patch contains 1
[jira] [Updated] (OOZIE-3156) SSH action status turns OK wrongly when failed to connect to host
[
https://issues.apache.org/jira/browse/OOZIE-3156?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
TIAN XING updated OOZIE-3156:
-
Attachment: OOZIE-3156-v6.patch
> SSH action status turns OK wrongly when failed to connect to host
> -
>
> Key: OOZIE-3156
> URL: https://issues.apache.org/jira/browse/OOZIE-3156
> Project: Oozie
> Issue Type: Bug
> Components: action
>Affects Versions: 5.0.0
>Reporter: TIAN XING
>Assignee: TIAN XING
>Priority: Major
> Attachments: OOZIE-3156-v1.patch, OOZIE-3156-v2.patch,
> OOZIE-3156-v3.patch, OOZIE-3156-v4.patch, OOZIE-3156-v5.patch,
> OOZIE-3156-v6.patch, ssh-check-bug.patch
>
>
> When {{check()}} method of {{SshActionExecutor}} gets invoked, oozie will ssh
> connect to the host and check whether the pid of the process that ssh action
> started is still there (by checking the returned value of command "{{ssh
> ps -p }}" ) to determine whether ssh action completes or not.
> However, we found cases where oozie fails to connect to host during action
> status check (e.g., the host is under heavy load, or network is bad etc.).
> In such cases, the return value of command "{{ssh ps -p }}"
> will be 255 (ssh command exits with the exit status of the remote command or
> with 255 if an error occurred.).
> According the current logic of method {{getActionStatus()}} in
> {{SshActionExecutor}}, the action status will be determined as OK which may
> not be correct.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OOZIE-3156) SSH action status turns OK wrongly when failed to connect to host
[
https://issues.apache.org/jira/browse/OOZIE-3156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16500161#comment-16500161
]
Hadoop QA commented on OOZIE-3156:
--
PreCommit-OOZIE-Build started
> SSH action status turns OK wrongly when failed to connect to host
> -
>
> Key: OOZIE-3156
> URL: https://issues.apache.org/jira/browse/OOZIE-3156
> Project: Oozie
> Issue Type: Bug
> Components: action
>Affects Versions: 5.0.0
>Reporter: TIAN XING
>Assignee: TIAN XING
>Priority: Major
> Attachments: OOZIE-3156-v1.patch, OOZIE-3156-v2.patch,
> OOZIE-3156-v3.patch, OOZIE-3156-v4.patch, OOZIE-3156-v5.patch,
> OOZIE-3156-v6.patch, ssh-check-bug.patch
>
>
> When {{check()}} method of {{SshActionExecutor}} gets invoked, oozie will ssh
> connect to the host and check whether the pid of the process that ssh action
> started is still there (by checking the returned value of command "{{ssh
> ps -p }}" ) to determine whether ssh action completes or not.
> However, we found cases where oozie fails to connect to host during action
> status check (e.g., the host is under heavy load, or network is bad etc.).
> In such cases, the return value of command "{{ssh ps -p }}"
> will be 255 (ssh command exits with the exit status of the remote command or
> with 255 if an error occurred.).
> According the current logic of method {{getActionStatus()}} in
> {{SshActionExecutor}}, the action status will be determined as OK which may
> not be correct.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (OOZIE-2791) ShareLib installation may fail on busy Hadoop clusters
[
https://issues.apache.org/jira/browse/OOZIE-2791?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Julia Kinga Marton updated OOZIE-2791:
--
Attachment: OOZIE-2791-005.patch
> ShareLib installation may fail on busy Hadoop clusters
> --
>
> Key: OOZIE-2791
> URL: https://issues.apache.org/jira/browse/OOZIE-2791
> Project: Oozie
> Issue Type: Bug
>Reporter: Attila Sasvari
>Assignee: Julia Kinga Marton
>Priority: Major
> Attachments: OOZIE-2791-003.patch, OOZIE-2791-004.patch,
> OOZIE-2791-005.patch, OOZIE-2791-01.patch, OOZIE-2791-02.patch
>
>
> On a busy Hadoop cluster it can happen that users cannot install properly
> Oozie ShareLib.
> Example on a Hadoop 2.4.0 pseudo cluster sharelib installion with a
> concurrency number set high (to simulate a busy cluster):
> {code}
> oozie-setup.sh sharelib create -fs hdfs://localhost:9000 -locallib
> oozie-sharelib-*.tar.gz -concurrency 150
> {code}
> You can see a lot of errors (failed copy tasks) on the output:
> {code}
> Running 464 copy tasks on 150 threads
> Error: Copy task failed with exception
> Stack trace for the error was (for debug purposes):
> --
> org.apache.hadoop.ipc.RemoteException(java.io.IOException): File
> /user/asasvari/share/lib/lib_20170207105926/distcp/hadoop-distcp-2.4.0.jar
> could only be replicated to 0 nodes instead of minReplication (=1). There
> are 1 datanode(s) running and no node(s) are excluded in this operation.
> at
> org.apache.hadoop.hdfs.server.blockmanagement.BlockManager.chooseTarget(BlockManager.java:1430)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:2684)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.addBlock(NameNodeRpcServer.java:584)
> at
> org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.addBlock(ClientNamenodeProtocolServerSideTranslatorPB.java:440)
> at
> org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:585)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:928)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2013)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2009)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1548)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2007)
> at org.apache.hadoop.ipc.Client.call(Client.java:1410)
> at org.apache.hadoop.ipc.Client.call(Client.java:1363)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:206)
> at com.sun.proxy.$Proxy9.addBlock(Unknown Source)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:190)
> at
> org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:103)
> at com.sun.proxy.$Proxy9.addBlock(Unknown Source)
> at
> org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.addBlock(ClientNamenodeProtocolTranslatorPB.java:361)
> at
> org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.locateFollowingBlock(DFSOutputStream.java:1439)
> at
> org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.nextBlockOutputStream(DFSOutputStream.java:1261)
> at
> org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.run(DFSOutputStream.java:525)
> --
> ...
> {code}
> You can see file is created but it's size is 0.
> {code}
> -rw-r--r-- 3 asasvari supergroup 0 2017-02-07 10:59
> share/lib/lib_20170207105926/distcp/hadoop-distcp-2.4.0.jar
> {code}
> This behaviour is clearly wrong.
> In case of such an exception, we should retry copying or rollback changes. We
> should also consider throttling HDFS requests.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
Re: Review Request 66930: OOZIE-2791 ShareLib installation may fail on busy Hadoop clusters
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/66930/ --- (Updated June 4, 2018, 1:55 p.m.) Review request for oozie, András Piros and Peter Cseh. Changes --- review fix + test classes for CopyTaskCallable and ConcurrentCopyFromLocal Repository: oozie-git Description --- On a busy Hadoop cluster it can happen that users cannot install properly Oozie ShareLib. Diffs (updated) - tools/src/main/java/org/apache/oozie/tools/OozieSharelibCLI.java dce1c559 tools/src/test/java/org/apache/oozie/tools/TestBlockSizeCalculator.java PRE-CREATION tools/src/test/java/org/apache/oozie/tools/TestConcurrentCopyFromLocal.java PRE-CREATION tools/src/test/java/org/apache/oozie/tools/TestCopyTaskCallable.java PRE-CREATION tools/src/test/java/org/apache/oozie/tools/TestOozieSharelibCLI.java ccad273b Diff: https://reviews.apache.org/r/66930/diff/3/ Changes: https://reviews.apache.org/r/66930/diff/2-3/ Testing --- Tested manually Thanks, Kinga Marton
[jira] [Commented] (OOZIE-2791) ShareLib installation may fail on busy Hadoop clusters
[
https://issues.apache.org/jira/browse/OOZIE-2791?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16500254#comment-16500254
]
Hadoop QA commented on OOZIE-2791:
--
PreCommit-OOZIE-Build started
> ShareLib installation may fail on busy Hadoop clusters
> --
>
> Key: OOZIE-2791
> URL: https://issues.apache.org/jira/browse/OOZIE-2791
> Project: Oozie
> Issue Type: Bug
>Reporter: Attila Sasvari
>Assignee: Julia Kinga Marton
>Priority: Major
> Attachments: OOZIE-2791-003.patch, OOZIE-2791-004.patch,
> OOZIE-2791-005.patch, OOZIE-2791-01.patch, OOZIE-2791-02.patch
>
>
> On a busy Hadoop cluster it can happen that users cannot install properly
> Oozie ShareLib.
> Example on a Hadoop 2.4.0 pseudo cluster sharelib installion with a
> concurrency number set high (to simulate a busy cluster):
> {code}
> oozie-setup.sh sharelib create -fs hdfs://localhost:9000 -locallib
> oozie-sharelib-*.tar.gz -concurrency 150
> {code}
> You can see a lot of errors (failed copy tasks) on the output:
> {code}
> Running 464 copy tasks on 150 threads
> Error: Copy task failed with exception
> Stack trace for the error was (for debug purposes):
> --
> org.apache.hadoop.ipc.RemoteException(java.io.IOException): File
> /user/asasvari/share/lib/lib_20170207105926/distcp/hadoop-distcp-2.4.0.jar
> could only be replicated to 0 nodes instead of minReplication (=1). There
> are 1 datanode(s) running and no node(s) are excluded in this operation.
> at
> org.apache.hadoop.hdfs.server.blockmanagement.BlockManager.chooseTarget(BlockManager.java:1430)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:2684)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.addBlock(NameNodeRpcServer.java:584)
> at
> org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.addBlock(ClientNamenodeProtocolServerSideTranslatorPB.java:440)
> at
> org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:585)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:928)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2013)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2009)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1548)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2007)
> at org.apache.hadoop.ipc.Client.call(Client.java:1410)
> at org.apache.hadoop.ipc.Client.call(Client.java:1363)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:206)
> at com.sun.proxy.$Proxy9.addBlock(Unknown Source)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:190)
> at
> org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:103)
> at com.sun.proxy.$Proxy9.addBlock(Unknown Source)
> at
> org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.addBlock(ClientNamenodeProtocolTranslatorPB.java:361)
> at
> org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.locateFollowingBlock(DFSOutputStream.java:1439)
> at
> org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.nextBlockOutputStream(DFSOutputStream.java:1261)
> at
> org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.run(DFSOutputStream.java:525)
> --
> ...
> {code}
> You can see file is created but it's size is 0.
> {code}
> -rw-r--r-- 3 asasvari supergroup 0 2017-02-07 10:59
> share/lib/lib_20170207105926/distcp/hadoop-distcp-2.4.0.jar
> {code}
> This behaviour is clearly wrong.
> In case of such an exception, we should retry copying or rollback changes. We
> should also consider throttling HDFS requests.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OOZIE-3156) SSH action status turns OK wrongly when failed to connect to host
[
https://issues.apache.org/jira/browse/OOZIE-3156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16500271#comment-16500271
]
Hadoop QA commented on OOZIE-3156:
--
Testing JIRA OOZIE-3156
Cleaning local git workspace
{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:green}+1 RAW_PATCH_ANALYSIS{color}
.{color:green}+1{color} the patch does not introduce any @author tags
.{color:green}+1{color} the patch does not introduce any tabs
.{color:green}+1{color} the patch does not introduce any trailing spaces
.{color:green}+1{color} the patch does not introduce any line longer than
132
.{color:green}+1{color} the patch adds/modifies 1 testcase(s)
{color:green}+1 RAT{color}
.{color:green}+1{color} the patch does not seem to introduce new RAT
warnings
{color:green}+1 JAVADOC{color}
{color:green}+1 JAVADOC{color}
.{color:green}+1{color} the patch does not seem to introduce new Javadoc
warning(s)
.{color:green}+1{color} the patch does not seem to introduce new Javadoc
error(s)
.{color:red}ERROR{color}: the current HEAD has 2 Javadoc error(s)
{color:green}+1 COMPILE{color}
.{color:green}+1{color} HEAD compiles
.{color:green}+1{color} patch compiles
.{color:green}+1{color} the patch does not seem to introduce new javac
warnings
{color:green}+1{color} There are no new bugs found in total.
. {color:green}+1{color} There are no new bugs found in [examples].
. {color:green}+1{color} There are no new bugs found in [webapp].
. {color:green}+1{color} There are no new bugs found in [core].
. {color:green}+1{color} There are no new bugs found in [tools].
. {color:green}+1{color} There are no new bugs found in [server].
. {color:green}+1{color} There are no new bugs found in [docs].
. {color:green}+1{color} There are no new bugs found in [sharelib/hive2].
. {color:green}+1{color} There are no new bugs found in [sharelib/pig].
. {color:green}+1{color} There are no new bugs found in [sharelib/streaming].
. {color:green}+1{color} There are no new bugs found in [sharelib/hive].
. {color:green}+1{color} There are no new bugs found in [sharelib/hcatalog].
. {color:green}+1{color} There are no new bugs found in [sharelib/sqoop].
. {color:green}+1{color} There are no new bugs found in [sharelib/oozie].
. {color:green}+1{color} There are no new bugs found in [sharelib/distcp].
. {color:green}+1{color} There are no new bugs found in [sharelib/spark].
. {color:green}+1{color} There are no new bugs found in [client].
{color:green}+1 BACKWARDS_COMPATIBILITY{color}
.{color:green}+1{color} the patch does not change any JPA
Entity/Colum/Basic/Lob/Transient annotations
.{color:green}+1{color} the patch does not modify JPA files
{color:green}+1 TESTS{color}
.Tests run: 2146
{color:green}+1 DISTRO{color}
.{color:green}+1{color} distro tarball builds with the patch
{color:green}*+1 Overall result, good!, no -1s*{color}
The full output of the test-patch run is available at
. https://builds.apache.org/job/PreCommit-OOZIE-Build/603/
> SSH action status turns OK wrongly when failed to connect to host
> -
>
> Key: OOZIE-3156
> URL: https://issues.apache.org/jira/browse/OOZIE-3156
> Project: Oozie
> Issue Type: Bug
> Components: action
>Affects Versions: 5.0.0
>Reporter: TIAN XING
>Assignee: TIAN XING
>Priority: Major
> Attachments: OOZIE-3156-v1.patch, OOZIE-3156-v2.patch,
> OOZIE-3156-v3.patch, OOZIE-3156-v4.patch, OOZIE-3156-v5.patch,
> OOZIE-3156-v6.patch, ssh-check-bug.patch
>
>
> When {{check()}} method of {{SshActionExecutor}} gets invoked, oozie will ssh
> connect to the host and check whether the pid of the process that ssh action
> started is still there (by checking the returned value of command "{{ssh
> ps -p }}" ) to determine whether ssh action completes or not.
> However, we found cases where oozie fails to connect to host during action
> status check (e.g., the host is under heavy load, or network is bad etc.).
> In such cases, the return value of command "{{ssh ps -p }}"
> will be 255 (ssh command exits with the exit status of the remote command or
> with 255 if an error occurred.).
> According the current logic of method {{getActionStatus()}} in
> {{SshActionExecutor}}, the action status will be determined as OK which may
> not be correct.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
Success: OOZIE-3156 PreCommit Build #603
Jira: https://issues.apache.org/jira/browse/OOZIE-3156
Build: https://builds.apache.org/job/PreCommit-OOZIE-Build/603/
###
## LAST 100 LINES OF THE CONSOLE
###
[...truncated 1.71 MB...]
[DEBUG] There are no new bugs found in [sharelib/pig].
[TRACE] New XMLLib present, calling 'xmllint --xpath' to get bug instance counts
[DEBUG] There are no new bugs found in [sharelib/streaming].
[TRACE] New XMLLib present, calling 'xmllint --xpath' to get bug instance counts
[DEBUG] There are no new bugs found in [sharelib/hive].
[TRACE] New XMLLib present, calling 'xmllint --xpath' to get bug instance counts
[DEBUG] There are no new bugs found in [sharelib/hcatalog].
[TRACE] New XMLLib present, calling 'xmllint --xpath' to get bug instance counts
[DEBUG] There are no new bugs found in [sharelib/sqoop].
[TRACE] New XMLLib present, calling 'xmllint --xpath' to get bug instance counts
[DEBUG] There are no new bugs found in [sharelib/oozie].
[TRACE] New XMLLib present, calling 'xmllint --xpath' to get bug instance counts
[DEBUG] There are no new bugs found in [sharelib/distcp].
[TRACE] New XMLLib present, calling 'xmllint --xpath' to get bug instance counts
[DEBUG] There are no new bugs found in [sharelib/spark].
[TRACE] New XMLLib present, calling 'xmllint --xpath' to get bug instance counts
[DEBUG] There are no new bugs found in [client].
[INFO] There are no new bugs found totally].
[TRACE] FindBugs diffs checked and reports created
[TRACE] Summary file size is 2365 bytes
[TRACE] Full summary file size is 1314 bytes
[TRACE] File
[/home/jenkins/jenkins-slave/workspace/PreCommit-OOZIE-Build/test-patch/tmp/FINDBUGS_DIFF/diff/findbugs-diff-0.1.0-all.jar]
removed
[TRACE] File
[/home/jenkins/jenkins-slave/workspace/PreCommit-OOZIE-Build/test-patch/tmp/FINDBUGS_DIFF/diff/findbugs-diff-0.1.0-all.jar.md5sum]
removed
Running test-patch task BACKWARDS_COMPATIBILITY
Running test-patch task TESTS
Running test-patch task DISTRO
Testing JIRA OOZIE-3156
Cleaning local git workspace
+1 PATCH_APPLIES
+1 CLEAN
+1 RAW_PATCH_ANALYSIS
+1 the patch does not introduce any @author tags
+1 the patch does not introduce any tabs
+1 the patch does not introduce any trailing spaces
+1 the patch does not introduce any line longer than 132
+1 the patch adds/modifies 1 testcase(s)
+1 RAT
+1 the patch does not seem to introduce new RAT warnings
+1 JAVADOC
+1 JAVADOC
+1 the patch does not seem to introduce new Javadoc warning(s)
+1 the patch does not seem to introduce new Javadoc error(s)
ERROR: the current HEAD has 2 Javadoc error(s)
+1 COMPILE
+1 HEAD compiles
+1 patch compiles
+1 the patch does not seem to introduce new javac warnings
+1 There are no new bugs found in total.
+1 There are no new bugs found in [examples].
+1 There are no new bugs found in [webapp].
+1 There are no new bugs found in [core].
+1 There are no new bugs found in [tools].
+1 There are no new bugs found in [server].
+1 There are no new bugs found in [docs].
+1 There are no new bugs found in [sharelib/hive2].
+1 There are no new bugs found in [sharelib/pig].
+1 There are no new bugs found in [sharelib/streaming].
+1 There are no new bugs found in [sharelib/hive].
+1 There are no new bugs found in [sharelib/hcatalog].
+1 There are no new bugs found in [sharelib/sqoop].
+1 There are no new bugs found in [sharelib/oozie].
+1 There are no new bugs found in [sharelib/distcp].
+1 There are no new bugs found in [sharelib/spark].
+1 There are no new bugs found in [client].
+1 BACKWARDS_COMPATIBILITY
+1 the patch does not change any JPA Entity/Colum/Basic/Lob/Transient
annotations
+1 the patch does not modify JPA files
+1 TESTS
Tests run: 2146
+1 DISTRO
+1 distro tarball builds with the patch
+1 Overall result, good!, no -1s
The full output of the test-patch run is available at
https://builds.apache.org/job/PreCommit-OOZIE-Build/603/
Adding comment to JIRA
% Total% Received % Xferd Average Speed TimeTime Time Current
Dload Upload Total SpentLeft Speed
0 00 00 0 0 0 --:--:-- --:--:-- --:--:--
0100 30020 0 100 3002 0 3484 --:--:-- --:--:-- --:--:--
3482{"self":"https://issues.apache.org/jira/rest/api/2/issue/13130096/comment/16500271","id":"16500271","author":{"self":"https://issues.apache.org/jira/rest/api/2/user?username=hadoopqa","name":"hadoopqa","key":"hadoopqa","emailAddress":"blackhole
at hadoop dot apache dot
org","avatarUrls":{"48x48":"https://issues.apache.org/jira/secure/useravatar?ownerId=hadoopqa&avatarId=10393","24x24":"https://issues.apache.org/jira/secure/useravatar?size=small&ownerId=hadoopqa&avatarId=10393","16x16":"https://issues.apache.org/jira/secure/useravatar?size=xs
[jira] [Updated] (OOZIE-3156) Retry SSH action check when there is a ConnectException
[
https://issues.apache.org/jira/browse/OOZIE-3156?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andras Piros updated OOZIE-3156:
Summary: Retry SSH action check when there is a ConnectException (was: SSH
action status turns OK wrongly when failed to connect to host)
> Retry SSH action check when there is a ConnectException
> ---
>
> Key: OOZIE-3156
> URL: https://issues.apache.org/jira/browse/OOZIE-3156
> Project: Oozie
> Issue Type: Bug
> Components: action
>Affects Versions: 5.0.0
>Reporter: TIAN XING
>Assignee: TIAN XING
>Priority: Major
> Attachments: OOZIE-3156-v1.patch, OOZIE-3156-v2.patch,
> OOZIE-3156-v3.patch, OOZIE-3156-v4.patch, OOZIE-3156-v5.patch,
> OOZIE-3156-v6.patch, ssh-check-bug.patch
>
>
> When {{check()}} method of {{SshActionExecutor}} gets invoked, oozie will ssh
> connect to the host and check whether the pid of the process that ssh action
> started is still there (by checking the returned value of command "{{ssh
> ps -p }}" ) to determine whether ssh action completes or not.
> However, we found cases where oozie fails to connect to host during action
> status check (e.g., the host is under heavy load, or network is bad etc.).
> In such cases, the return value of command "{{ssh ps -p }}"
> will be 255 (ssh command exits with the exit status of the remote command or
> with 255 if an error occurred.).
> According the current logic of method {{getActionStatus()}} in
> {{SshActionExecutor}}, the action status will be determined as OK which may
> not be correct.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (OOZIE-3156) Retry SSH action check when cannot connect to remote host
[
https://issues.apache.org/jira/browse/OOZIE-3156?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andras Piros updated OOZIE-3156:
Summary: Retry SSH action check when cannot connect to remote host (was:
Retry SSH action check when there is a ConnectException)
> Retry SSH action check when cannot connect to remote host
> -
>
> Key: OOZIE-3156
> URL: https://issues.apache.org/jira/browse/OOZIE-3156
> Project: Oozie
> Issue Type: Bug
> Components: action
>Affects Versions: 5.0.0
>Reporter: TIAN XING
>Assignee: TIAN XING
>Priority: Major
> Attachments: OOZIE-3156-v1.patch, OOZIE-3156-v2.patch,
> OOZIE-3156-v3.patch, OOZIE-3156-v4.patch, OOZIE-3156-v5.patch,
> OOZIE-3156-v6.patch, ssh-check-bug.patch
>
>
> When {{check()}} method of {{SshActionExecutor}} gets invoked, oozie will ssh
> connect to the host and check whether the pid of the process that ssh action
> started is still there (by checking the returned value of command "{{ssh
> ps -p }}" ) to determine whether ssh action completes or not.
> However, we found cases where oozie fails to connect to host during action
> status check (e.g., the host is under heavy load, or network is bad etc.).
> In such cases, the return value of command "{{ssh ps -p }}"
> will be 255 (ssh command exits with the exit status of the remote command or
> with 255 if an error occurred.).
> According the current logic of method {{getActionStatus()}} in
> {{SshActionExecutor}}, the action status will be determined as OK which may
> not be correct.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
Failed: OOZIE-2791 PreCommit Build #604
Jira: https://issues.apache.org/jira/browse/OOZIE-2791
Build: https://builds.apache.org/job/PreCommit-OOZIE-Build/604/
###
## LAST 100 LINES OF THE CONSOLE
###
[...truncated 1.71 MB...]
[TRACE] New XMLLib present, calling 'xmllint --xpath' to get bug instance counts
[DEBUG] There are no new bugs found in [sharelib/hcatalog].
[TRACE] New XMLLib present, calling 'xmllint --xpath' to get bug instance counts
[DEBUG] There are no new bugs found in [sharelib/sqoop].
[TRACE] New XMLLib present, calling 'xmllint --xpath' to get bug instance counts
[DEBUG] There are no new bugs found in [sharelib/oozie].
[TRACE] New XMLLib present, calling 'xmllint --xpath' to get bug instance counts
[DEBUG] There are no new bugs found in [sharelib/distcp].
[TRACE] New XMLLib present, calling 'xmllint --xpath' to get bug instance counts
[DEBUG] There are no new bugs found in [sharelib/spark].
[TRACE] New XMLLib present, calling 'xmllint --xpath' to get bug instance counts
[DEBUG] There are no new bugs found in [client].
[ERROR] There are [1] new bugs found below threshold in total that must be
fixed.
[TRACE] FindBugs diffs checked and reports created
[TRACE] Summary file size is 2874 bytes
[TRACE] Full summary file size is 1823 bytes
[TRACE] File
[/home/jenkins/jenkins-slave/workspace/PreCommit-OOZIE-Build@2/test-patch/tmp/FINDBUGS_DIFF/diff/findbugs-diff-0.1.0-all.jar]
removed
[TRACE] File
[/home/jenkins/jenkins-slave/workspace/PreCommit-OOZIE-Build@2/test-patch/tmp/FINDBUGS_DIFF/diff/findbugs-diff-0.1.0-all.jar.md5sum]
removed
Running test-patch task BACKWARDS_COMPATIBILITY
Running test-patch task TESTS
Running test-patch task DISTRO
Testing JIRA OOZIE-2791
Cleaning local git workspace
+1 PATCH_APPLIES
+1 CLEAN
+1 RAW_PATCH_ANALYSIS
+1 the patch does not introduce any @author tags
+1 the patch does not introduce any tabs
+1 the patch does not introduce any trailing spaces
+1 the patch does not introduce any line longer than 132
+1 the patch adds/modifies 4 testcase(s)
+1 RAT
+1 the patch does not seem to introduce new RAT warnings
+1 JAVADOC
+1 JAVADOC
+1 the patch does not seem to introduce new Javadoc warning(s)
+1 the patch does not seem to introduce new Javadoc error(s)
ERROR: the current HEAD has 2 Javadoc error(s)
+1 COMPILE
+1 HEAD compiles
+1 patch compiles
+1 the patch does not seem to introduce new javac warnings
-1 There are [1] new bugs found below threshold in total that must be fixed.
+1 There are no new bugs found in [examples].
+1 There are no new bugs found in [webapp].
+1 There are no new bugs found in [core].
-1 There are [1] new bugs found below threshold in [tools] that must be fixed.
You can find the FindBugs diff here (look for the red and orange ones):
tools/findbugs-new.html
The most important FindBugs errors are:
Dereferenced at OozieSharelibCLI.java:[line 388]: Possible null pointer
dereference in
org.apache.oozie.tools.OozieSharelibCLI$ConcurrentCopyFromLocal.copyFolderRecursively(OozieSharelibCLI$CopyTaskConfiguration)
due to return value of called method
Known null at OozieSharelibCLI.java:[line 388]
+1 There are no new bugs found in [server].
+1 There are no new bugs found in [docs].
+1 There are no new bugs found in [sharelib/hive2].
+1 There are no new bugs found in [sharelib/pig].
+1 There are no new bugs found in [sharelib/streaming].
+1 There are no new bugs found in [sharelib/hive].
+1 There are no new bugs found in [sharelib/hcatalog].
+1 There are no new bugs found in [sharelib/sqoop].
+1 There are no new bugs found in [sharelib/oozie].
+1 There are no new bugs found in [sharelib/distcp].
+1 There are no new bugs found in [sharelib/spark].
+1 There are no new bugs found in [client].
+1 BACKWARDS_COMPATIBILITY
+1 the patch does not change any JPA Entity/Colum/Basic/Lob/Transient
annotations
+1 the patch does not modify JPA files
+1 TESTS
Tests run: 2159
+1 DISTRO
+1 distro tarball builds with the patch
-1 Overall result, please check the reported -1(s)
The full output of the test-patch run is available at
https://builds.apache.org/job/PreCommit-OOZIE-Build/604/
Adding comment to JIRA
% Total% Received % Xferd Average Speed TimeTime Time Current
Dload Upload Total SpentLeft Speed
0 00 00 0 0 0 --:--:-- --:--:-- --:--:-- 0
0 00 00 0 0 0 --:--:-- --:--:-- --:--:--
0{"self":"https://issues.apache.org/jira/rest/api/2/issue/13040921/comment/16500373","id":"16500373","author":{"self":"https://issues.apache.org/jira/rest/api/2/user?username=hadoopqa","name":"hadoopqa","key":"hadoopqa","emailAddress":"blackhole
at hadoop dot apache dot
org","avatarUrls":{"48x48":"https://is
[jira] [Commented] (OOZIE-2791) ShareLib installation may fail on busy Hadoop clusters
[
https://issues.apache.org/jira/browse/OOZIE-2791?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16500373#comment-16500373
]
Hadoop QA commented on OOZIE-2791:
--
Testing JIRA OOZIE-2791
Cleaning local git workspace
{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:green}+1 RAW_PATCH_ANALYSIS{color}
.{color:green}+1{color} the patch does not introduce any @author tags
.{color:green}+1{color} the patch does not introduce any tabs
.{color:green}+1{color} the patch does not introduce any trailing spaces
.{color:green}+1{color} the patch does not introduce any line longer than
132
.{color:green}+1{color} the patch adds/modifies 4 testcase(s)
{color:green}+1 RAT{color}
.{color:green}+1{color} the patch does not seem to introduce new RAT
warnings
{color:green}+1 JAVADOC{color}
{color:green}+1 JAVADOC{color}
.{color:green}+1{color} the patch does not seem to introduce new Javadoc
warning(s)
.{color:green}+1{color} the patch does not seem to introduce new Javadoc
error(s)
.{color:red}ERROR{color}: the current HEAD has 2 Javadoc error(s)
{color:green}+1 COMPILE{color}
.{color:green}+1{color} HEAD compiles
.{color:green}+1{color} patch compiles
.{color:green}+1{color} the patch does not seem to introduce new javac
warnings
{color:red}-1{color} There are [1] new bugs found below threshold in total that
must be fixed.
. {color:green}+1{color} There are no new bugs found in [examples].
. {color:green}+1{color} There are no new bugs found in [webapp].
. {color:green}+1{color} There are no new bugs found in [core].
. {color:red}-1{color} There are [1] new bugs found below threshold in [tools]
that must be fixed.
. You can find the FindBugs diff here (look for the red and orange ones):
tools/findbugs-new.html
. The most important FindBugs errors are:
. Dereferenced at OozieSharelibCLI.java:[line 388]: Possible null pointer
dereference in
org.apache.oozie.tools.OozieSharelibCLI$ConcurrentCopyFromLocal.copyFolderRecursively(OozieSharelibCLI$CopyTaskConfiguration)
due to return value of called method
. Known null at OozieSharelibCLI.java:[line 388]
. {color:green}+1{color} There are no new bugs found in [server].
. {color:green}+1{color} There are no new bugs found in [docs].
. {color:green}+1{color} There are no new bugs found in [sharelib/hive2].
. {color:green}+1{color} There are no new bugs found in [sharelib/pig].
. {color:green}+1{color} There are no new bugs found in [sharelib/streaming].
. {color:green}+1{color} There are no new bugs found in [sharelib/hive].
. {color:green}+1{color} There are no new bugs found in [sharelib/hcatalog].
. {color:green}+1{color} There are no new bugs found in [sharelib/sqoop].
. {color:green}+1{color} There are no new bugs found in [sharelib/oozie].
. {color:green}+1{color} There are no new bugs found in [sharelib/distcp].
. {color:green}+1{color} There are no new bugs found in [sharelib/spark].
. {color:green}+1{color} There are no new bugs found in [client].
{color:green}+1 BACKWARDS_COMPATIBILITY{color}
.{color:green}+1{color} the patch does not change any JPA
Entity/Colum/Basic/Lob/Transient annotations
.{color:green}+1{color} the patch does not modify JPA files
{color:green}+1 TESTS{color}
.Tests run: 2159
{color:green}+1 DISTRO{color}
.{color:green}+1{color} distro tarball builds with the patch
{color:red}*-1 Overall result, please check the reported -1(s)*{color}
The full output of the test-patch run is available at
. https://builds.apache.org/job/PreCommit-OOZIE-Build/604/
> ShareLib installation may fail on busy Hadoop clusters
> --
>
> Key: OOZIE-2791
> URL: https://issues.apache.org/jira/browse/OOZIE-2791
> Project: Oozie
> Issue Type: Bug
>Reporter: Attila Sasvari
>Assignee: Julia Kinga Marton
>Priority: Major
> Attachments: OOZIE-2791-003.patch, OOZIE-2791-004.patch,
> OOZIE-2791-005.patch, OOZIE-2791-01.patch, OOZIE-2791-02.patch
>
>
> On a busy Hadoop cluster it can happen that users cannot install properly
> Oozie ShareLib.
> Example on a Hadoop 2.4.0 pseudo cluster sharelib installion with a
> concurrency number set high (to simulate a busy cluster):
> {code}
> oozie-setup.sh sharelib create -fs hdfs://localhost:9000 -locallib
> oozie-sharelib-*.tar.gz -concurrency 150
> {code}
> You can see a lot of errors (failed copy tasks) on the output:
> {code}
> Running 464 copy tasks on 150 threads
> Error: Copy task failed with exception
> Stack trace for the error was (for debug purposes):
> --
> org.apache.hadoop.ipc.RemoteException(java.io.IOException): File
> /user/asasvari/share/lib/lib_20170207105926/distcp/hadoop-distcp-2.4.0.jar
> c
[jira] [Commented] (OOZIE-2877) Oozie Git Action
[
https://issues.apache.org/jira/browse/OOZIE-2877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16500648#comment-16500648
]
Robert Kanter commented on OOZIE-2877:
--
I haven't been able to find time to look at the patch, but I saw mention of
"credential files" and just wanted to suggest something (I could be completely
off base here because I haven't looked at the code, so feel free to ignore me
if this doesn't make sense :)). Now that Oozie's directly on YARN, we have
access to the {{Credentials}} object in the {{ApplicationSubmissionContext}}.
You can add arbitrary secrets into a hashmap ({{}}) in the
{{Credentials}}. YARN will securely provide that to the Container (IIRC you
can retrieve it via {{Configuration}}). That might be a better alternative to
putting the credential file in HDFS.
> Oozie Git Action
>
>
> Key: OOZIE-2877
> URL: https://issues.apache.org/jira/browse/OOZIE-2877
> Project: Oozie
> Issue Type: Sub-task
> Components: action
>Reporter: Clay B.
>Assignee: Clay B.
>Priority: Major
> Labels: action
> Fix For: trunk
>
> Attachments: 0001-OOZIE-2877-Oozie-Git-Action.patch,
> 0002-OOZIE-2877-Oozie-Git-Action.patch,
> 0003-OOZIE-2877-Oozie-Git-Action.patch,
> 0004-OOZIE-2877-Oozie-Git-Action.patch,
> 0005-OOZIE-2877-Oozie-Git-Action.patch,
> 0006-OOZIE-2877-Oozie-Git-Action.patch,
> 0007-OOZIE-2877-Oozie-Git-Action.patch,
> 0008-OOZIE-2877-Oozie-Git-Action.patch,
> 0009-OOZIE-2877-Oozie-Git-Action.patch, OOZIE-2877.010.patch,
> OOZIE-2877.011.patch, OOZIE-2877.012.patch
>
>
> To aide in deploying ASCII artifacts to clusters, let's provide a tie-in for
> a source-code management system. Git would be my preferred choice.
> Ideally, this could handle a user's key material e.g. for an ssh key to pull
> down from a secured repository. This would free users from handling their own
> key staging and clean-up on YARN nodes and only require them to store the key
> secured in HDFS.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] Subscription: Oozie Patch Available
Issue Subscription Filter: Oozie Patch Available (99 issues) Subscriber: ooziedaily Key Summary OOZIE-3270 Upgrade Derby to 10.14.1.0 https://issues.apache.org/jira/browse/OOZIE-3270 OOZIE-3265 properties RERUN_FAIL_NODES and RERUN_SKIP_NODES should be able to appear together https://issues.apache.org/jira/browse/OOZIE-3265 OOZIE-3260 [sla] Remove stale item above max retries on JPA related errors from in-memory SLA map https://issues.apache.org/jira/browse/OOZIE-3260 OOZIE-3256 refactor OozieCLI class https://issues.apache.org/jira/browse/OOZIE-3256 OOZIE-3249 [tools] Instrumentation log parser https://issues.apache.org/jira/browse/OOZIE-3249 OOZIE-3218 Oozie Sqoop action with command splits the select clause into multiple parts due to delimiter being space https://issues.apache.org/jira/browse/OOZIE-3218 OOZIE-3199 Let system property restriction configurable https://issues.apache.org/jira/browse/OOZIE-3199 OOZIE-3196 Authorization: restrict world readability by user https://issues.apache.org/jira/browse/OOZIE-3196 OOZIE-3194 Oozie should set proper permissions to sharelib after upload https://issues.apache.org/jira/browse/OOZIE-3194 OOZIE-3193 Applications are not killed when submitted via subworkflow https://issues.apache.org/jira/browse/OOZIE-3193 OOZIE-3186 Oozie is unable to use configuration linked using jceks://file/... https://issues.apache.org/jira/browse/OOZIE-3186 OOZIE-3179 Adding a configurable config-default.xml location to a workflow https://issues.apache.org/jira/browse/OOZIE-3179 OOZIE-3170 Oozie Diagnostic Bundle tool fails with NPE due to missing service class https://issues.apache.org/jira/browse/OOZIE-3170 OOZIE-3160 PriorityDelayQueue put()/take() can cause significant CPU load due to busy waiting https://issues.apache.org/jira/browse/OOZIE-3160 OOZIE-3135 Configure log4j2 in SqoopMain https://issues.apache.org/jira/browse/OOZIE-3135 OOZIE-3109 Escape log-streaming's HTML-specific characters https://issues.apache.org/jira/browse/OOZIE-3109 OOZIE-3091 Oozie Sqoop Avro Import fails with "java.lang.NoClassDefFoundError: org/apache/avro/mapred/AvroWrapper" https://issues.apache.org/jira/browse/OOZIE-3091 OOZIE-3071 Oozie 4.3 Spark sharelib ueses a different version of commons-lang3 than Spark 2.2.0 https://issues.apache.org/jira/browse/OOZIE-3071 OOZIE-3063 Sanitizing variables that are part of openjpa.ConnectionProperties https://issues.apache.org/jira/browse/OOZIE-3063 OOZIE-3062 Set HADOOP_CONF_DIR for spark action https://issues.apache.org/jira/browse/OOZIE-3062 OOZIE-3061 Kill only those child jobs which are not already killed https://issues.apache.org/jira/browse/OOZIE-3061 OOZIE-2956 Fix Findbugs warnings related to reliance on default encoding in oozie-core https://issues.apache.org/jira/browse/OOZIE-2956 OOZIE-2955 Fix Findbugs warnings related to reliance on default encoding in oozie-client https://issues.apache.org/jira/browse/OOZIE-2955 OOZIE-2954 Fix Checkstyle issues in oozie-client https://issues.apache.org/jira/browse/OOZIE-2954 OOZIE-2953 Fix Checkstyle issues in oozie-tools https://issues.apache.org/jira/browse/OOZIE-2953 OOZIE-2952 Fix Findbugs warnings in oozie-sharelib-oozie https://issues.apache.org/jira/browse/OOZIE-2952 OOZIE-2949 Escape quotes whitespaces in Sqoop field https://issues.apache.org/jira/browse/OOZIE-2949 OOZIE-2942 [examples] Fix Findbugs warnings https://issues.apache.org/jira/browse/OOZIE-2942 OOZIE-2927 Append new line character for Hive2 query using query tag https://issues.apache.org/jira/browse/OOZIE-2927 OOZIE-2877 Oozie Git Action https://issues.apache.org/jira/browse/OOZIE-2877 OOZIE-2834 ParameterVerifier logging non-useful warning for workflow definition https://issues.apache.org/jira/browse/OOZIE-2834 OOZIE-2833 when using uber mode the regex pattern used in the extractHeapSizeMB method does not allow heap sizes specified in bytes. https://issues.apache.org/jira/browse/OOZIE-2833 OOZIE-2829 Improve sharelib upload to accept multiple source folders https://issues.apache.org/jira/browse/OOZIE-2829 OOZIE-2812 SparkConfigurationService should support loading configurations from multiple Spark versions https://issues.apache.org/jira/browse/OOZIE-2812 OOZIE-2795 Create lib directory or symlink for Oozie CLI during packaging https://issues.apache.org/jira/browse/OOZIE-2795 OOZIE-2791 ShareLib installation may fail on busy Hadoop clusters https://issues.apache.org/jira/browse/OOZIE-2791 OOZIE-2784 Include WEEK as a parameter in the Coordinator Expression Languag
