[jira] Subscription: Oozie Patch Available

[jira]

Issue Subscription
Filter: Oozie Patch Available (102 issues)

Subscriber: ooziedaily

Key Summary
OOZIE-3135  Configure log4j2 in SqoopMain
https://issues.apache.org/jira/browse/OOZIE-3135
OOZIE-3121  bump all maven plugins to latest versions
https://issues.apache.org/jira/browse/OOZIE-3121
OOZIE-3105  testJMXInstrumentation from the 
org.apache.oozie.util.TestMetricsInstrumentation class is flaky
https://issues.apache.org/jira/browse/OOZIE-3105
OOZIE-3094  fix for grammar mistake
https://issues.apache.org/jira/browse/OOZIE-3094
OOZIE-3091  Oozie Sqoop Avro Import fails with "java.lang.NoClassDefFoundError: 
org/apache/avro/mapred/AvroWrapper"
https://issues.apache.org/jira/browse/OOZIE-3091
OOZIE-3085  Improve logging in ActionExecutors
https://issues.apache.org/jira/browse/OOZIE-3085
OOZIE-3071  Oozie 4.3 Spark sharelib ueses a different version of commons-lang3 
than Spark 2.2.0
https://issues.apache.org/jira/browse/OOZIE-3071
OOZIE-3063  Sanitizing variables that are part of openjpa.ConnectionProperties
https://issues.apache.org/jira/browse/OOZIE-3063
OOZIE-3062  Set HADOOP_CONF_DIR for spark action
https://issues.apache.org/jira/browse/OOZIE-3062
OOZIE-3002  address findbugs errors in client lib
https://issues.apache.org/jira/browse/OOZIE-3002
OOZIE-2975  code clean up in pig sharelib, replace Exception with more 
explicit, add try with resources, StringBuilder instead of StringBuffer
https://issues.apache.org/jira/browse/OOZIE-2975
OOZIE-2973  Make sure Oozie works with Hadoop 3 
https://issues.apache.org/jira/browse/OOZIE-2973
OOZIE-2957  Documentation states that starting a coordinator is possible
https://issues.apache.org/jira/browse/OOZIE-2957
OOZIE-2956  Fix Findbugs warnings related to reliance on default encoding in 
oozie-core
https://issues.apache.org/jira/browse/OOZIE-2956
OOZIE-2955  Fix Findbugs warnings related to reliance on default encoding in 
oozie-client
https://issues.apache.org/jira/browse/OOZIE-2955
OOZIE-2954  Fix Checkstyle issues in oozie-client
https://issues.apache.org/jira/browse/OOZIE-2954
OOZIE-2953  Fix Checkstyle issues in oozie-tools
https://issues.apache.org/jira/browse/OOZIE-2953
OOZIE-2952  Fix Findbugs warnings in oozie-sharelib-oozie
https://issues.apache.org/jira/browse/OOZIE-2952
OOZIE-2949  Escape quotes whitespaces in Sqoop  field
https://issues.apache.org/jira/browse/OOZIE-2949
OOZIE-2942  Fix Findbugs warnings in oozie-examples
https://issues.apache.org/jira/browse/OOZIE-2942
OOZIE-2937  Remove redundant groupId from the child pom's
https://issues.apache.org/jira/browse/OOZIE-2937
OOZIE-2934  Fix "Exceptional return value of java.io.File.mkdirs() ignored" 
Findbugs error in oozie-sharelib-spark
https://issues.apache.org/jira/browse/OOZIE-2934
OOZIE-2927  Append new line character for Hive2 query using query tag
https://issues.apache.org/jira/browse/OOZIE-2927
OOZIE-2914  Consolidate Trim 
https://issues.apache.org/jira/browse/OOZIE-2914
OOZIE-2883  OOZIE throw the error "Missing 
[oozie.service.ProxyUserService.proxyuser.oozie.service.ProxyUserService.proxyuser.mr.groups]
 property"
https://issues.apache.org/jira/browse/OOZIE-2883
OOZIE-2877  Oozie Git Action
https://issues.apache.org/jira/browse/OOZIE-2877
OOZIE-2867  Timezone handling for Coordinators: emphasize "Continent/City" 
format
https://issues.apache.org/jira/browse/OOZIE-2867
OOZIE-2834  ParameterVerifier logging non-useful warning for workflow definition
https://issues.apache.org/jira/browse/OOZIE-2834
OOZIE-2833  when using uber mode the regex pattern used in the 
extractHeapSizeMB method does not allow heap sizes specified in bytes.
https://issues.apache.org/jira/browse/OOZIE-2833
OOZIE-2829  Improve sharelib upload to accept multiple source folders
https://issues.apache.org/jira/browse/OOZIE-2829
OOZIE-2826  Falcon feed fails to aws s3; Oozie joda time version does not meet 
required jar version 2.2 or later
https://issues.apache.org/jira/browse/OOZIE-2826
OOZIE-2812  SparkConfigurationService should support loading configurations 
from multiple Spark versions
https://issues.apache.org/jira/browse/OOZIE-2812
OOZIE-2795  Create lib directory or symlink for Oozie CLI during packaging
https://issues.apache.org/jira/browse/OOZIE-2795
OOZIE-2791  ShareLib installation may fail on busy Hadoop clusters
https://issues.apache.org/jira/browse/OOZIE-2791
OOZIE-2784  Include WEEK as a parameter in the Coordinator Expression Language 
Evaulator
https://issues.apache.org/jira/browse/OOZIE-2784
OOZIE-2779  Mask Hive2 action Beeline JDBC password
https://issues.apache.org/jira

[jira] Subscription: Oozie Patch Available

[jira]

Issue Subscription
Filter: Oozie Patch Available (111 issues)

Subscriber: ooziedaily

Key Summary
OOZIE-3129  Fix test TestConfigurationService.testOozieConfig
https://issues-test.apache.org/jira/browse/OOZIE-3129
OOZIE-3127  Remove redundant check for user
https://issues-test.apache.org/jira/browse/OOZIE-3127
OOZIE-3126  Add option to allow list of users to access system config
https://issues-test.apache.org/jira/browse/OOZIE-3126
OOZIE-3121  bump all maven plugins to latest versions
https://issues-test.apache.org/jira/browse/OOZIE-3121
OOZIE-3118  fix for error: self-closing element not allowed
https://issues-test.apache.org/jira/browse/OOZIE-3118
OOZIE-3117  fix for warning: no @throws for
https://issues-test.apache.org/jira/browse/OOZIE-3117
OOZIE-3116  fix for warning: no description for @throws
https://issues-test.apache.org/jira/browse/OOZIE-3116
OOZIE-3115  fix for javadoc warning: empty  tag
https://issues-test.apache.org/jira/browse/OOZIE-3115
OOZIE-3113  Retry for ZK lock release
https://issues-test.apache.org/jira/browse/OOZIE-3113
OOZIE-3112  SparkConfigrationService overwrites properties provided via 
--properties-file option in SparkAction
https://issues-test.apache.org/jira/browse/OOZIE-3112
OOZIE-3105  testJMXInstrumentation from the 
org.apache.oozie.util.TestMetricsInstrumentation class is flaky
https://issues-test.apache.org/jira/browse/OOZIE-3105
OOZIE-3094  fix for grammar mistake
https://issues-test.apache.org/jira/browse/OOZIE-3094
OOZIE-3091  Oozie Sqoop Avro Import fails with "java.lang.NoClassDefFoundError: 
org/apache/avro/mapred/AvroWrapper"
https://issues-test.apache.org/jira/browse/OOZIE-3091
OOZIE-3083  Make improved version Info backward compatible
https://issues-test.apache.org/jira/browse/OOZIE-3083
OOZIE-3071  Oozie 4.3 Spark sharelib ueses a different version of commons-lang3 
than Spark 2.2.0
https://issues-test.apache.org/jira/browse/OOZIE-3071
OOZIE-3063  Sanitizing variables that are part of openjpa.ConnectionProperties
https://issues-test.apache.org/jira/browse/OOZIE-3063
OOZIE-3062  Set HADOOP_CONF_DIR for spark action
https://issues-test.apache.org/jira/browse/OOZIE-3062
OOZIE-3002  address findbugs errors in client lib
https://issues-test.apache.org/jira/browse/OOZIE-3002
OOZIE-2975  code clean up in pig sharelib, replace Exception with more 
explicit, add try with resources, StringBuilder instead of StringBuffer
https://issues-test.apache.org/jira/browse/OOZIE-2975
OOZIE-2973  Make sure Oozie works with Hadoop 3 
https://issues-test.apache.org/jira/browse/OOZIE-2973
OOZIE-2969  Drop support for Java 1.7
https://issues-test.apache.org/jira/browse/OOZIE-2969
OOZIE-2957  Documentation states that starting a coordinator is possible
https://issues-test.apache.org/jira/browse/OOZIE-2957
OOZIE-2956  Fix Findbugs warnings related to reliance on default encoding in 
oozie-core
https://issues-test.apache.org/jira/browse/OOZIE-2956
OOZIE-2955  Fix Findbugs warnings related to reliance on default encoding in 
oozie-client
https://issues-test.apache.org/jira/browse/OOZIE-2955
OOZIE-2954  Fix Checkstyle issues in oozie-client
https://issues-test.apache.org/jira/browse/OOZIE-2954
OOZIE-2953  Fix Checkstyle issues in oozie-tools
https://issues-test.apache.org/jira/browse/OOZIE-2953
OOZIE-2952  Fix Findbugs warnings in oozie-sharelib-oozie
https://issues-test.apache.org/jira/browse/OOZIE-2952
OOZIE-2949  Escape quotes whitespaces in Sqoop  field
https://issues-test.apache.org/jira/browse/OOZIE-2949
OOZIE-2942  Fix Findbugs warnings in oozie-examples
https://issues-test.apache.org/jira/browse/OOZIE-2942
OOZIE-2937  Remove redundant groupId from the child pom's
https://issues-test.apache.org/jira/browse/OOZIE-2937
OOZIE-2934  Fix "Exceptional return value of java.io.File.mkdirs() ignored" 
Findbugs error in oozie-sharelib-spark
https://issues-test.apache.org/jira/browse/OOZIE-2934
OOZIE-2927  Append new line character for Hive2 query using query tag
https://issues-test.apache.org/jira/browse/OOZIE-2927
OOZIE-2914  Consolidate Trim 
https://issues-test.apache.org/jira/browse/OOZIE-2914
OOZIE-2883  OOZIE throw the error "Missing 
[oozie.service.ProxyUserService.proxyuser.oozie.service.ProxyUserService.proxyuser.mr.groups]
 property"
https://issues-test.apache.org/jira/browse/OOZIE-2883
OOZIE-2877  Oozie Git Action
https://issues-test.apache.org/jira/browse/OOZIE-2877
OOZIE-2867  Timezone handling for Coordinators: emphasize "Continent/City" 
format
https://issues-test.apache.org/jira/browse/OOZIE-2867
OOZIE-2834  ParameterVerifier logging non-useful

Re: [VOTE] Release Oozie 5.0.0-beta1 (candidate 1)

[Harsh J]

+1 (binding)

- Verified the md5 and sig (asc) against the tar
- Executed a build with tests

On Fri, Jan 12, 2018 at 12:09 AM Andras Piros 
wrote:

> Hi Attila,
>
> thanks for building the new RC1 and caring about the process.
>
> I've built Oozie against Hadoop 2.6.5 and tried some examples out, graph
> generation, log streaming etc.
>
> +1 (non-binding).
>
> Andras
>
> On Thu, Jan 11, 2018 at 5:01 PM, Peter Cseh  wrote:
>
> > Thanks Attila for building the RC1!
> >
> > 1) md5, gpg and rat report checks out.
> > 2) distro builds fine
> > 3) ran some examples, verified they are working as they should
> >
> > I'll try to verify the rc1 against a secure cluster soon, but +1
> > (non-binding) in the meantime
> >
> > gp
> >
> > On Tue, Jan 9, 2018 at 5:53 PM, Attila Sasvári 
> > wrote:
> >
> > > It is release candidate 1.
> > >
> > > On Tue, Jan 9, 2018 at 5:50 PM, Attila Sasvári 
> > > wrote:
> > >
> > > > Hi,
> > > >
> > > > I have created a build for Oozie 5.0.0-beta1, release candidate 0.
> > > >
> > > > This release introduces major changes:
> > > > - moving launcher from MapReduce AM to Oozie AM,
> > > > - dropped support for Hadoop 1.x,
> > > > - added new 1.0 workflow schemas,
> > > > - switching from Tomcat to embedded Jetty,
> > > > - completely rewritten graph generator,
> > > > - local sharelib,
> > > > - JDK 8 support.
> > > >
> > > > It also includes OOZIE-3083.
> > > >
> > > > Keys to verify the signature of the release artifact are available at
> > > >
> > > >  http://www.apache.org/dist/oozie/KEYS
> > > >
> > > > Please download, test, and try it out:
> > > >
> > > >   http://people.apache.org/~asasvari/oozie-5.0.0-beta1-rc1/
> > > >
> > > > The release, md5 signature, gpg signature, and rat report can all be
> > > found
> > > > at the above address.
> > > >
> > > > The vote is open until Friday, January 12, 2018 at 18:00 CEST.
> > > >
> > > > Thanks,
> > > > Attila Sasvari
> > > >
> > >
> >
> >
> >
> > --
> > Peter Cseh
> > Software Engineer
> > 
> >
>

Re: [VOTE] Release Oozie 5.0.0-beta1 (candidate 1)

[Andras Piros]

Hi Attila,

thanks for building the new RC1 and caring about the process.

I've built Oozie against Hadoop 2.6.5 and tried some examples out, graph
generation, log streaming etc.

+1 (non-binding).

Andras

On Thu, Jan 11, 2018 at 5:01 PM, Peter Cseh  wrote:

> Thanks Attila for building the RC1!
>
> 1) md5, gpg and rat report checks out.
> 2) distro builds fine
> 3) ran some examples, verified they are working as they should
>
> I'll try to verify the rc1 against a secure cluster soon, but +1
> (non-binding) in the meantime
>
> gp
>
> On Tue, Jan 9, 2018 at 5:53 PM, Attila Sasvári 
> wrote:
>
> > It is release candidate 1.
> >
> > On Tue, Jan 9, 2018 at 5:50 PM, Attila Sasvári 
> > wrote:
> >
> > > Hi,
> > >
> > > I have created a build for Oozie 5.0.0-beta1, release candidate 0.
> > >
> > > This release introduces major changes:
> > > - moving launcher from MapReduce AM to Oozie AM,
> > > - dropped support for Hadoop 1.x,
> > > - added new 1.0 workflow schemas,
> > > - switching from Tomcat to embedded Jetty,
> > > - completely rewritten graph generator,
> > > - local sharelib,
> > > - JDK 8 support.
> > >
> > > It also includes OOZIE-3083.
> > >
> > > Keys to verify the signature of the release artifact are available at
> > >
> > >  http://www.apache.org/dist/oozie/KEYS
> > >
> > > Please download, test, and try it out:
> > >
> > >   http://people.apache.org/~asasvari/oozie-5.0.0-beta1-rc1/
> > >
> > > The release, md5 signature, gpg signature, and rat report can all be
> > found
> > > at the above address.
> > >
> > > The vote is open until Friday, January 12, 2018 at 18:00 CEST.
> > >
> > > Thanks,
> > > Attila Sasvari
> > >
> >
>
>
>
> --
> Peter Cseh
> Software Engineer
> 
>

[jira] [Commented] (OOZIE-3085) Improve logging in ActionExecutors

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-3085?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16322629#comment-16322629
 ] 

Hadoop QA commented on OOZIE-3085:
--

Testing JIRA OOZIE-3085

Cleaning local git workspace



{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:red}-1 RAW_PATCH_ANALYSIS{color}
.{color:green}+1{color} the patch does not introduce any @author tags
.{color:green}+1{color} the patch does not introduce any tabs
.{color:green}+1{color} the patch does not introduce any trailing spaces
.{color:green}+1{color} the patch does not introduce any line longer than 
132
.{color:red}-1{color} the patch does not add/modify any testcase
{color:green}+1 RAT{color}
.{color:green}+1{color} the patch does not seem to introduce new RAT 
warnings
{color:green}+1 JAVADOC{color}
.{color:green}+1{color} the patch does not seem to introduce new Javadoc 
warnings
{color:green}+1 COMPILE{color}
.{color:green}+1{color} HEAD compiles
.{color:green}+1{color} patch compiles
.{color:green}+1{color} the patch does not seem to introduce new javac 
warnings
{color:green}+1{color} There are no new bugs found in total.
. {color:green}+1{color} There are no new bugs found in [docs].
. {color:green}+1{color} There are no new bugs found in [examples].
. {color:green}+1{color} There are no new bugs found in [tools].
. {color:green}+1{color} There are no new bugs found in [core].
. {color:green}+1{color} There are no new bugs found in [server].
. {color:green}+1{color} There are no new bugs found in [client].
. {color:green}+1{color} There are no new bugs found in [sharelib/distcp].
. {color:green}+1{color} There are no new bugs found in [sharelib/oozie].
. {color:green}+1{color} There are no new bugs found in [sharelib/hcatalog].
. {color:green}+1{color} There are no new bugs found in [sharelib/streaming].
. {color:green}+1{color} There are no new bugs found in [sharelib/hive].
. {color:green}+1{color} There are no new bugs found in [sharelib/sqoop].
. {color:green}+1{color} There are no new bugs found in [sharelib/spark].
. {color:green}+1{color} There are no new bugs found in [sharelib/hive2].
. {color:green}+1{color} There are no new bugs found in [sharelib/pig].
{color:green}+1 BACKWARDS_COMPATIBILITY{color}
.{color:green}+1{color} the patch does not change any JPA 
Entity/Colum/Basic/Lob/Transient annotations
.{color:green}+1{color} the patch does not modify JPA files
{color:green}+1 TESTS{color}
.Tests run: 2086
.{color:orange}Tests failed at first run:{color}
TestJavaActionExecutor#testCredentialsSkip
TestCoordActionInputCheckXCommandNonUTC>TestCoordActionInputCheckXCommand#testNone
TestCoordMaterializeTriggerService#testCoordMaterializeTriggerService3
TestStatusTransitService#testCoordStatusTransitWithLock
.For the complete list of flaky tests, see TEST-SUMMARY-FULL files.
{color:green}+1 DISTRO{color}
.{color:green}+1{color} distro tarball builds with the patch 


{color:red}*-1 Overall result, please check the reported -1(s)*{color}


The full output of the test-patch run is available at

. https://builds.apache.org/job/PreCommit-OOZIE-Build/319/

> Improve logging in ActionExecutors
> --
>
> Key: OOZIE-3085
> URL: https://issues.apache.org/jira/browse/OOZIE-3085
> Project: Oozie
>  Issue Type: Bug
>  Components: core
>Affects Versions: 4.3.0
>Reporter: Peter Cseh
>Assignee: Julia Kinga Marton
>  Labels: newbie
> Fix For: 5.0.0
>
> Attachments: OOZIE-3085-001.patch, OOZIE-3085-002.patch, 
> OOZIE-3085-003.patch, OOZIE-3085-004.patch
>
>
> Decision, ForkJoin, Email and FS ActionExecutors are producing no or very 
> little logs on info level. This makes it hard to gather usage information and 
> help with troubleshooting in the case of any issues. Please improve logging 
> in these classes.
> Several subtasks can be separated out from this one



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Failed: OOZIE-3085 PreCommit Build #319

[Apache Jenkins Server]

Jira: https://issues.apache.org/jira/browse/OOZIE-3085
Build: https://builds.apache.org/job/PreCommit-OOZIE-Build/319/

###
## LAST 100 LINES OF THE CONSOLE 
###
[...truncated 1.61 MB...]
[TRACE] New XMLLib present, calling 'xmllint --xpath' to get bug instance counts
[DEBUG] There are no new bugs found in [sharelib/streaming].
[TRACE] New XMLLib present, calling 'xmllint --xpath' to get bug instance counts
[DEBUG] There are no new bugs found in [sharelib/hive].
[TRACE] New XMLLib present, calling 'xmllint --xpath' to get bug instance counts
[DEBUG] There are no new bugs found in [sharelib/sqoop].
[TRACE] New XMLLib present, calling 'xmllint --xpath' to get bug instance counts
[DEBUG] There are no new bugs found in [sharelib/spark].
[TRACE] New XMLLib present, calling 'xmllint --xpath' to get bug instance counts
[DEBUG] There are no new bugs found in [sharelib/hive2].
[TRACE] New XMLLib present, calling 'xmllint --xpath' to get bug instance counts
[DEBUG] There are no new bugs found in [sharelib/pig].
[INFO] There are no new bugs found totally].
[TRACE] FindBugs diffs checked and reports created
[TRACE] Summary file size is 2111 bytes
[TRACE] Full summary file size is 1248 bytes
[TRACE] File 
[/home/jenkins/jenkins-slave/workspace/PreCommit-OOZIE-Build/test-patch/tmp/FINDBUGS_DIFF/diff/findbugs-diff-0.1.0-all.jar]
 removed
[TRACE] File 
[/home/jenkins/jenkins-slave/workspace/PreCommit-OOZIE-Build/test-patch/tmp/FINDBUGS_DIFF/diff/findbugs-diff-0.1.0-all.jar.md5sum]
 removed
  Running test-patch task BACKWARDS_COMPATIBILITY
  Running test-patch task TESTS
  Running test-patch task DISTRO


Testing JIRA OOZIE-3085

Cleaning local git workspace



+1 PATCH_APPLIES
+1 CLEAN
-1 RAW_PATCH_ANALYSIS
+1 the patch does not introduce any @author tags
+1 the patch does not introduce any tabs
+1 the patch does not introduce any trailing spaces
+1 the patch does not introduce any line longer than 132
-1 the patch does not add/modify any testcase
+1 RAT
+1 the patch does not seem to introduce new RAT warnings
+1 JAVADOC
+1 the patch does not seem to introduce new Javadoc warnings
+1 COMPILE
+1 HEAD compiles
+1 patch compiles
+1 the patch does not seem to introduce new javac warnings
+1 There are no new bugs found in total.
 +1 There are no new bugs found in [docs].
 +1 There are no new bugs found in [examples].
 +1 There are no new bugs found in [tools].
 +1 There are no new bugs found in [core].
 +1 There are no new bugs found in [server].
 +1 There are no new bugs found in [client].
 +1 There are no new bugs found in [sharelib/distcp].
 +1 There are no new bugs found in [sharelib/oozie].
 +1 There are no new bugs found in [sharelib/hcatalog].
 +1 There are no new bugs found in [sharelib/streaming].
 +1 There are no new bugs found in [sharelib/hive].
 +1 There are no new bugs found in [sharelib/sqoop].
 +1 There are no new bugs found in [sharelib/spark].
 +1 There are no new bugs found in [sharelib/hive2].
 +1 There are no new bugs found in [sharelib/pig].
+1 BACKWARDS_COMPATIBILITY
+1 the patch does not change any JPA Entity/Colum/Basic/Lob/Transient 
annotations
+1 the patch does not modify JPA files
+1 TESTS
Tests run: 2086
Tests failed at first run:
TestJavaActionExecutor#testCredentialsSkip
TestCoordActionInputCheckXCommandNonUTC>TestCoordActionInputCheckXCommand#testNone
TestCoordMaterializeTriggerService#testCoordMaterializeTriggerService3
TestStatusTransitService#testCoordStatusTransitWithLock
For the complete list of flaky tests, see TEST-SUMMARY-FULL files.
+1 DISTRO
+1 distro tarball builds with the patch 


-1 Overall result, please check the reported -1(s)


The full output of the test-patch run is available at

 https://builds.apache.org/job/PreCommit-OOZIE-Build/319/

  % Total% Received % Xferd  Average Speed   TimeTime Time  Current
 Dload  Upload   Total   SpentLeft  Speed
  0 00 00 0  0  0 --:--:-- --:--:-- --:--:-- 0  
0 00 00 0  0  0 --:--:-- --:--:-- --:--:-- 0  0 
00 00 0  0  0 --:--:-- --:--:-- --:--:-- 0
100 3706k  100 3706k0 0  4302k  0 --:--:-- --:--:-- --:--:-- 4302k
Adding comment to JIRA
Comment added.

test-patch exit code: 1

Build step 'Execute shell' marked build as failure
[description-setter] Description set: OOZIE-3085
Archiving artifacts
[Fast Archiver] Compressed 1.77 MB of artifacts by 47.7% relative to #318
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any



###
## FAILED TESTS (if any) 
##
All te

[jira] [Assigned] (OOZIE-3145) TestDistcpMain shall remove created files after test execution

[Julia Kinga Marton (JIRA)]

 [ 
https://issues.apache.org/jira/browse/OOZIE-3145?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Julia Kinga Marton reassigned OOZIE-3145:
-

Assignee: Julia Kinga Marton

> TestDistcpMain shall remove created files after test execution
> --
>
> Key: OOZIE-3145
> URL: https://issues.apache.org/jira/browse/OOZIE-3145
> Project: Oozie
>  Issue Type: Bug
>  Components: tests
>Affects Versions: 5.0.0b1
>Reporter: Attila Sasvari
>Assignee: Julia Kinga Marton
> Attachments: Oozie_test_garbage.png
>
>
> After running TestDistcpMain, local files created are not removed. Subsequent 
> test runs will reuse those files and fail:
> {code}
> $ ls -lrt /home/asasvari/workspace/oozie/core/distcp*
> -rw-rw-r-- 1 asasvari asasvari 1025 Dec 21 18:25 
> /home/asasvari/workspace/oozie/core/distcp-log4j.properties
> -rw-rw-r-- 1 asasvari asasvari 5756 Dec 21 18:26 
> /home/asasvari/workspace/oozie/core/distcp-oozie-1513876931701.log
> {code}
> Those files should be removed after test execution. It would be better to use 
> the oozie/core/target directory so that mvn could clean it.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (OOZIE-3157) Setup truststore so that it also works in HTTP only mode

[Attila Sasvari (JIRA)]

Attila Sasvari created OOZIE-3157:
-

 Summary: Setup truststore so that it also works in HTTP only mode
 Key: OOZIE-3157
 URL: https://issues.apache.org/jira/browse/OOZIE-3157
 Project: Oozie
  Issue Type: Bug
Affects Versions: trunk, 5.0.0b1
Reporter: Attila Sasvari
Priority: Blocker
 Fix For: 5.0.0


{{oozie.https.truststore.file}} is not read and used when 
{{oozie.https.enabled}} is false in {{oozie-site xml}}. As a result, the Oozie 
server will be unable to communicate with servers with unsigned certificate. It 
is a critical problem as authentication may involve external servers (for 
example KMS with self-signed certificate). Submitting a workflow in such an 
environment can result in an exception like:
{code}
2018-01-08 10:13:51,471 WARN 
org.apache.hadoop.crypto.key.kms.LoadBalancingKMSClientProvider: 
SERVER[myserver] KMS provider at [https://myserver:16000/kms/v1/] threw an 
IOException: 
javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: 
PKIX path building failed: 
sun.security.provider.certpath.SunCertPathBuilderException: unable to
 find valid certification path to requested target
at sun.security.ssl.Alerts.getSSLException(Alerts.java:192)
at sun.security.ssl.SSLSocketImpl.fatal(SSLSocketImpl.java:1959)
at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:302)
at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:296)
at 
sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1514)
at 
sun.security.ssl.ClientHandshaker.processMessage(ClientHandshaker.java:216)
at sun.security.ssl.Handshaker.processLoop(Handshaker.java:1026)
at sun.security.ssl.Handshaker.process_record(Handshaker.java:961)
at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:1072)
at 
sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1385)
at 
sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1413)
at 
sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1397)
at 
sun.net.www.protocol.https.HttpsClient.afterConnect(HttpsClient.java:559)
at 
sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(AbstractDelegateHttpsURLConnection.java:185)
at 
sun.net.www.protocol.https.HttpsURLConnectionImpl.connect(HttpsURLConnectionImpl.java:153)
at 
org.apache.hadoop.security.authentication.client.KerberosAuthenticator.authenticate(KerberosAuthenticator.java:186)
at 
org.apache.hadoop.security.token.delegation.web.DelegationTokenAuthenticator.authenticate(DelegationTokenAuthenticator.java:144)
at 
org.apache.hadoop.security.authentication.client.AuthenticatedURL.openConnection(AuthenticatedURL.java:348)
at 
org.apache.hadoop.security.token.delegation.web.DelegationTokenAuthenticatedURL.openConnection(DelegationTokenAuthenticatedURL.java:333)
at 
org.apache.hadoop.crypto.key.kms.KMSClientProvider$1.run(KMSClientProvider.java:477)
at 
org.apache.hadoop.crypto.key.kms.KMSClientProvider$1.run(KMSClientProvider.java:472)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at 
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1962)
at 
org.apache.hadoop.crypto.key.kms.KMSClientProvider.createConnection(KMSClientProvider.java:471)
at 
org.apache.hadoop.crypto.key.kms.KMSClientProvider.decryptEncryptedKey(KMSClientProvider.java:776)
at 
org.apache.hadoop.crypto.key.kms.LoadBalancingKMSClientProvider$5.call(LoadBalancingKMSClientProvider.java:287)
at 
org.apache.hadoop.crypto.key.kms.LoadBalancingKMSClientProvider$5.call(LoadBalancingKMSClientProvider.java:283)
at 
org.apache.hadoop.crypto.key.kms.LoadBalancingKMSClientProvider.doOp(LoadBalancingKMSClientProvider.java:123)
at 
org.apache.hadoop.crypto.key.kms.LoadBalancingKMSClientProvider.decryptEncryptedKey(LoadBalancingKMSClientProvider.java:283)
at 
org.apache.hadoop.crypto.key.KeyProviderCryptoExtension.decryptEncryptedKey(KeyProviderCryptoExtension.java:532)
at 
org.apache.hadoop.hdfs.DFSClient.decryptEncryptedDataEncryptionKey(DFSClient.java:926)
at 
org.apache.hadoop.hdfs.DFSClient.createWrappedInputStream(DFSClient.java:945)
at 
org.apache.hadoop.hdfs.DistributedFileSystem$4.doCall(DistributedFileSystem.java:315)
at 
org.apache.hadoop.hdfs.DistributedFileSystem$4.doCall(DistributedFileSystem.java:310)
at 
org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
at 
org.apache.hadoop.hdfs.DistributedFileSystem.open(DistributedFileSystem.java:322)
at org.apache.hadoop.fs.FileSystem.open(FileSystem.java:949)
 

[jira] [Commented] (OOZIE-3156) SSH action status turns OK wrongly when failed to connect to host

[Andras Piros (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-3156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16322500#comment-16322500
 ] 

Andras Piros commented on OOZIE-3156:
-

[~txsing] added you to the list of contributors, and assigned this JIRA to you. 
Thanks for the patch!

> SSH action status turns OK wrongly when failed to connect to host
> -
>
> Key: OOZIE-3156
> URL: https://issues.apache.org/jira/browse/OOZIE-3156
> Project: Oozie
>  Issue Type: Bug
>  Components: action
>Affects Versions: 4.0.0, 4.1.0, 4.2.0, 4.3.0
>Reporter: TIAN XING
>Assignee: TIAN XING
> Fix For: 4.3.0
>
> Attachments: ssh-check-bug.patch
>
>
> When {{check()}} method of {{SshActionExecutor}} gets invoked, oozie will ssh 
> connect to the host and check whether action shell pid is still there (by 
> checking the returned value of command {{ssh $hostIp ps -p $pid}} ) to 
> determine whether the action is running or not.
> However, there are cases where oozie fails to connect to the host during 
> action status check (e.g., the host is under heavy load, or network is bad 
> etc.).
> In such cases, the return value of the command {{ssh $hostIp ps -p $pid}} 
> will be 255 (ssh command exits with the exit status of the remote command or 
> with 255 if an error occurred.).
> According the current logic of method {{getActionStatus()}} in 
> {{SshActionExecutor}}, the action status will be determined as OK which may 
> not be correct. 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Assigned] (OOZIE-3156) SSH action status turns OK wrongly when failed to connect to host

[Andras Piros (JIRA)]

 [ 
https://issues.apache.org/jira/browse/OOZIE-3156?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andras Piros reassigned OOZIE-3156:
---

Assignee: TIAN XING

> SSH action status turns OK wrongly when failed to connect to host
> -
>
> Key: OOZIE-3156
> URL: https://issues.apache.org/jira/browse/OOZIE-3156
> Project: Oozie
>  Issue Type: Bug
>  Components: action
>Affects Versions: 4.0.0, 4.1.0, 4.2.0, 4.3.0
>Reporter: TIAN XING
>Assignee: TIAN XING
> Fix For: 4.3.0
>
> Attachments: ssh-check-bug.patch
>
>
> When {{check()}} method of {{SshActionExecutor}} gets invoked, oozie will ssh 
> connect to the host and check whether action shell pid is still there (by 
> checking the returned value of command {{ssh $hostIp ps -p $pid}} ) to 
> determine whether the action is running or not.
> However, there are cases where oozie fails to connect to the host during 
> action status check (e.g., the host is under heavy load, or network is bad 
> etc.).
> In such cases, the return value of the command {{ssh $hostIp ps -p $pid}} 
> will be 255 (ssh command exits with the exit status of the remote command or 
> with 255 if an error occurred.).
> According the current logic of method {{getActionStatus()}} in 
> {{SshActionExecutor}}, the action status will be determined as OK which may 
> not be correct. 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Assigned] (OOZIE-3155) Job DAG is not refreshed when a job is finished

[Andras Piros (JIRA)]

 [ 
https://issues.apache.org/jira/browse/OOZIE-3155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andras Piros reassigned OOZIE-3155:
---

Assignee: Andras Piros

> Job DAG is not refreshed when a job is finished
> ---
>
> Key: OOZIE-3155
> URL: https://issues.apache.org/jira/browse/OOZIE-3155
> Project: Oozie
>  Issue Type: Bug
>  Components: client
>Affects Versions: trunk, 5.0.0b1
>Reporter: Julia Kinga Marton
>Assignee: Andras Piros
>Priority: Critical
> Fix For: 5.0.0
>
> Attachments: Screen Shot 2018-01-10 at 16.23.28.png
>
>
> When opening a Job DAG when the job is still in progress, the DAG is OK, but 
> when the job will finish, this diagram is not refreshed.
> Steps to reproduce: 
> - submit a workflow (e.g. the subwf from the examples) - the wf should not 
> end very quickly, to have time to catch it when it is still running
> - check the Job DAG when the job is still running
> - wait for the job to finish
> - check again the Job DAG -> is the same with the running one
> Note: if check it in a new tab, is OK, and also a browser refresh will solve 
> it.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Re: Review Request 64808: OOZIE-3085 - Improve logging in ActionExecutors

[Kinga Marton via Review Board]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64808/
---

(Updated Jan. 11, 2018, 4:02 p.m.)


Review request for oozie and Attila Sasvari.


Repository: oozie-git


Description
---

Decision, ForkJoin, Email and FS ActionExecutors are producing no or very 
little logs on info level. This makes it hard to gather usage information and 
help with troubleshooting in the case of any issues. Please improve logging in 
these classes.
Several subtasks can be separated out from this one


Diffs (updated)
-

  
core/src/main/java/org/apache/oozie/action/control/ControlNodeActionExecutor.java
 cc1b6108 
  
core/src/main/java/org/apache/oozie/action/decision/DecisionActionExecutor.java 
8c235bda 
  core/src/main/java/org/apache/oozie/action/email/EmailActionExecutor.java 
d59f1d76 
  core/src/main/java/org/apache/oozie/action/hadoop/FsActionExecutor.java 
63f6104d 
  core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java 
5c8acae5 
  
core/src/main/java/org/apache/oozie/action/oozie/SubWorkflowActionExecutor.java 
d62cf686 
  core/src/main/java/org/apache/oozie/action/ssh/SshActionExecutor.java 
5890b8c1 


Diff: https://reviews.apache.org/r/64808/diff/3/

Changes: https://reviews.apache.org/r/64808/diff/2-3/


Testing
---

Manually tested the log entries for each affected ActionExecutor.
The log messages look like: 

`2018-01-11 16:24:06,967  INFO ForkActionExecutor:520 - 
SERVER[kmarton-MBP.local] USER[martonjuliakinga] GROUP[-] TOKEN[] 
APP[Fork_workflow] JOB[001-18055450212-oozie-mart-W] 
ACTION[001-18055450212-oozie-mart-W@fs-paralel] Starting action
`

`2018-01-11 14:40:22,770  INFO FsActionExecutor:520 - SERVER[kmarton-MBP.local] 
USER[martonjuliakinga] GROUP[-] TOKEN[] APP[one-op-wf] 
JOB[006-18034310829-oozie-mart-W] 
ACTION[006-18034310829-oozie-mart-W@action1] Action ended with external 
status [OK]
`


Thanks,

Kinga Marton

Re: [VOTE] Release Oozie 5.0.0-beta1 (candidate 1)

[Peter Cseh]

Thanks Attila for building the RC1!

1) md5, gpg and rat report checks out.
2) distro builds fine
3) ran some examples, verified they are working as they should

I'll try to verify the rc1 against a secure cluster soon, but +1
(non-binding) in the meantime

gp

On Tue, Jan 9, 2018 at 5:53 PM, Attila Sasvári  wrote:

> It is release candidate 1.
>
> On Tue, Jan 9, 2018 at 5:50 PM, Attila Sasvári 
> wrote:
>
> > Hi,
> >
> > I have created a build for Oozie 5.0.0-beta1, release candidate 0.
> >
> > This release introduces major changes:
> > - moving launcher from MapReduce AM to Oozie AM,
> > - dropped support for Hadoop 1.x,
> > - added new 1.0 workflow schemas,
> > - switching from Tomcat to embedded Jetty,
> > - completely rewritten graph generator,
> > - local sharelib,
> > - JDK 8 support.
> >
> > It also includes OOZIE-3083.
> >
> > Keys to verify the signature of the release artifact are available at
> >
> >  http://www.apache.org/dist/oozie/KEYS
> >
> > Please download, test, and try it out:
> >
> >   http://people.apache.org/~asasvari/oozie-5.0.0-beta1-rc1/
> >
> > The release, md5 signature, gpg signature, and rat report can all be
> found
> > at the above address.
> >
> > The vote is open until Friday, January 12, 2018 at 18:00 CEST.
> >
> > Thanks,
> > Attila Sasvari
> >
>



-- 
Peter Cseh
Software Engineer

Re: Review Request 64808: OOZIE-3085 - Improve logging in ActionExecutors

[Kinga Marton via Review Board]

> On Dec. 28, 2017, 11:44 a.m., Attila Sasvari wrote:
> > core/src/main/java/org/apache/oozie/action/control/ControlNodeActionExecutor.java
> > Lines 52 (patched)
> > 
> >
> > What does ``getId()`` return for a control node action?

the getId() for the ActionExecutors as I saw are following the pattern: 
JobId@ActionName. In case of the ControlNodeActionExecutors looks like: 
000-171228181538280-oozie-mart-W@end, 
000-171228181538280-oozie-mart-W@:start:, 
004-171228181538280-oozie-mart-W@fs-paralel (this was my fork action), 
004-171228181538280-oozie-mart-W@join-fs (this was my join action). But as 
we agreed I will remove it from the log message because is duplicated.


> On Dec. 28, 2017, 11:44 a.m., Attila Sasvari wrote:
> > core/src/main/java/org/apache/oozie/action/control/ControlNodeActionExecutor.java
> > Lines 59 (patched)
> > 
> >
> > What does ``getExternalStatus()`` return for a control node action? It 
> > would might sense to log ``getStatus()`` too.

In case of ControlNodeACtionExecutor the status is the same as the 
externalStatus (OK). It will be redundant to write both of them. There are 
cases when the two values are different, but the status is determined based on 
the value of the external status


> On Dec. 28, 2017, 11:44 a.m., Attila Sasvari wrote:
> > core/src/main/java/org/apache/oozie/action/decision/DecisionActionExecutor.java
> > Line 45 (original), 47 (patched)
> > 
> >
> > What does ``getId()`` return for a control node action? I've just 
> > checked TestDecisionActionExecutor (executed) and saw it was ``null`` that 
> > does not seem very useful. 
> > 
> > Can you create a workflow with all changed control nodes and try it out 
> > on a cluster (pseudo hadoop should be fine) and double check logs about 
> > starting and ending actions?

the getId() for the ActionExecutors as I saw are following the pattern: 
JobId@ActionName, where the action name comes from the workflow.xml, and here 
the name is mandatory. But as we discussed I will remove it from the log 
message because is duplicated.


> On Dec. 28, 2017, 11:44 a.m., Attila Sasvari wrote:
> > core/src/main/java/org/apache/oozie/action/decision/DecisionActionExecutor.java
> > Lines 83 (patched)
> > 
> >
> > There is a difference between status ``action.getStatus()`` (that can 
> > be "OK", "KILLED", "FAILED", etc.) and ``getExternalStatus()`` (in case of 
> > a decision action it returns where the execution will flow, transition to 
> > the next workflow action). 
> > 
> > For example:
> > ```
> > 
> >  false
> >  false
> >  false
> >  
> >  > ```
> > 
> > would return ``OK`` for ``action.getStatus()`` and ``d``for 
> > ``action.getExternalStatus()``. It might help to log both.

Yes, you are right. The external status will return the transition. As we 
discussed it will be unnecessary to log both ot them.


> On Dec. 28, 2017, 11:44 a.m., Attila Sasvari wrote:
> > core/src/main/java/org/apache/oozie/action/email/EmailActionExecutor.java
> > Lines 107 (patched)
> > 
> >
> > What does ``getId()`` return?

JobId@ActionName, where the action name comes from the workflow.xml. But as we 
discussed I will remove it from the log message because is duplicated.


> On Dec. 28, 2017, 11:44 a.m., Attila Sasvari wrote:
> > core/src/main/java/org/apache/oozie/action/email/EmailActionExecutor.java
> > Lines 313 (patched)
> > 
> >
> > What does ``getExternalStatus()`` return? It would might sense to log 
> > ``getStatus()`` too.

In this case the two values are the same


> On Dec. 28, 2017, 11:44 a.m., Attila Sasvari wrote:
> > core/src/main/java/org/apache/oozie/action/hadoop/FsActionExecutor.java
> > Lines 630 (patched)
> > 
> >
> > What does ``getId()`` return?

JobId@ActionName, where the action name comes from the workflow.xml. But as we 
discussed I will remove it from the log message because is duplicated.


> On Dec. 28, 2017, 11:44 a.m., Attila Sasvari wrote:
> > core/src/main/java/org/apache/oozie/action/oozie/SubWorkflowActionExecutor.java
> > Lines 167 (patched)
> > 
> >
> > What does ``getId()`` return?

JobId@ActionName, where the action name comes from the workflow.xml. But as we 
discussed I will remove it from the log message because is duplicated.


> On Dec. 28, 2017, 11:44 a.m., Attila Sasvari wrote:
> > core/src/main/

Re: Review Request 64808: OOZIE-3085 - Improve logging in ActionExecutors

[Kinga Marton via Review Board]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64808/
---

(Updated Jan. 11, 2018, 4:02 p.m.)


Review request for oozie and Attila Sasvari.


Repository: oozie-git


Description
---

Decision, ForkJoin, Email and FS ActionExecutors are producing no or very 
little logs on info level. This makes it hard to gather usage information and 
help with troubleshooting in the case of any issues. Please improve logging in 
these classes.
Several subtasks can be separated out from this one


Diffs
-

  
core/src/main/java/org/apache/oozie/action/control/ControlNodeActionExecutor.java
 cc1b6108 
  
core/src/main/java/org/apache/oozie/action/decision/DecisionActionExecutor.java 
8c235bda 
  core/src/main/java/org/apache/oozie/action/email/EmailActionExecutor.java 
d59f1d76 
  core/src/main/java/org/apache/oozie/action/hadoop/FsActionExecutor.java 
63f6104d 
  
core/src/main/java/org/apache/oozie/action/oozie/SubWorkflowActionExecutor.java 
d62cf686 
  core/src/main/java/org/apache/oozie/action/ssh/SshActionExecutor.java 
5890b8c1 


Diff: https://reviews.apache.org/r/64808/diff/2/


Testing (updated)
---

Manually tested the log entries for each affected ActionExecutor.
The log messages look like: 

`2018-01-11 16:24:06,967  INFO ForkActionExecutor:520 - 
SERVER[kmarton-MBP.local] USER[martonjuliakinga] GROUP[-] TOKEN[] 
APP[Fork_workflow] JOB[001-18055450212-oozie-mart-W] 
ACTION[001-18055450212-oozie-mart-W@fs-paralel] Starting action
`

`2018-01-11 14:40:22,770  INFO FsActionExecutor:520 - SERVER[kmarton-MBP.local] 
USER[martonjuliakinga] GROUP[-] TOKEN[] APP[one-op-wf] 
JOB[006-18034310829-oozie-mart-W] 
ACTION[006-18034310829-oozie-mart-W@action1] Action ended with external 
status [OK]
`


Thanks,

Kinga Marton

[jira] [Updated] (OOZIE-3085) Improve logging in ActionExecutors

[Julia Kinga Marton (JIRA)]

 [ 
https://issues.apache.org/jira/browse/OOZIE-3085?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Julia Kinga Marton updated OOZIE-3085:
--
Attachment: OOZIE-3085-004.patch

> Improve logging in ActionExecutors
> --
>
> Key: OOZIE-3085
> URL: https://issues.apache.org/jira/browse/OOZIE-3085
> Project: Oozie
>  Issue Type: Bug
>  Components: core
>Affects Versions: 4.3.0
>Reporter: Peter Cseh
>Assignee: Julia Kinga Marton
>  Labels: newbie
> Fix For: 5.0.0
>
> Attachments: OOZIE-3085-001.patch, OOZIE-3085-002.patch, 
> OOZIE-3085-003.patch, OOZIE-3085-004.patch
>
>
> Decision, ForkJoin, Email and FS ActionExecutors are producing no or very 
> little logs on info level. This makes it hard to gather usage information and 
> help with troubleshooting in the case of any issues. Please improve logging 
> in these classes.
> Several subtasks can be separated out from this one



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (OOZIE-3061) Kill only those child jobs which are not already killed

[Mate Juhasz (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-3061?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16322352#comment-16322352
 ] 

Mate Juhasz commented on OOZIE-3061:


Please add me to the contributors list, thank you!

> Kill only those child jobs which are not already killed
> ---
>
> Key: OOZIE-3061
> URL: https://issues.apache.org/jira/browse/OOZIE-3061
> Project: Oozie
>  Issue Type: Bug
>Reporter: Satish Subhashrao Saley
>Priority: Trivial
>  Labels: newbie, newbiee
>
> Here we kill all child jobs. 
> https://github.com/apache/oozie/blob/master/sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherMain.java#L265
> We should check before killing for already killed application.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (OOZIE-3156) SSH action status turns OK wrongly when failed to connect to host

[TIAN XING (JIRA)]

 [ 
https://issues.apache.org/jira/browse/OOZIE-3156?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

TIAN XING updated OOZIE-3156:
-
Remaining Estimate: (was: 1h)
 Original Estimate: (was: 1h)
   Description: 
When {{check()}} method of {{SshActionExecutor}} gets invoked, oozie will ssh 
connect to the host and check whether action shell pid is still there (by 
checking the returned value of command {{ssh $hostIp ps -p $pid}} ) to 
determine whether the action is running or not.

However, there are cases where oozie fails to connect to the host during action 
status check (e.g., the host is under heavy load, or network is bad etc.).

In such cases, the return value of the command {{ssh $hostIp ps -p $pid}} will 
be 255 (ssh command exits with the exit status of the remote command or with 
255 if an error occurred.).

According the current logic of method {{getActionStatus()}} in 
{{SshActionExecutor}}, the action status will be determined as OK which may not 
be correct. 

  was:
When {{check()}} method of {{SshActionExecutor}} gets invoked, oozie will ssh 
connect to the host and check whether action shell pid is still there (by 
checking the returned value of command `{{ssh $hostIp ps -p $pid}}` ) to 
determine whether the action is running or not.

However, there are cases where oozie fails to connect to the host during action 
status check (e.g., the host is under heavy load, or network is bad etc.).

In such cases, the return value of the command `{{ssh $hostIp ps -p $pid}}` 
will be 255 (ssh command exits with the exit status of the remote command or 
with 255 if an error occurred.).

According the current logic of method {{getActionStatus()}} in 
{{SshActionExecutor}}, the action status will be determined as OK which may not 
be correct. 


> SSH action status turns OK wrongly when failed to connect to host
> -
>
> Key: OOZIE-3156
> URL: https://issues.apache.org/jira/browse/OOZIE-3156
> Project: Oozie
>  Issue Type: Bug
>  Components: action
>Affects Versions: 4.0.0, 4.1.0, 4.2.0, 4.3.0
>Reporter: TIAN XING
> Fix For: 4.3.0
>
> Attachments: ssh-check-bug.patch
>
>
> When {{check()}} method of {{SshActionExecutor}} gets invoked, oozie will ssh 
> connect to the host and check whether action shell pid is still there (by 
> checking the returned value of command {{ssh $hostIp ps -p $pid}} ) to 
> determine whether the action is running or not.
> However, there are cases where oozie fails to connect to the host during 
> action status check (e.g., the host is under heavy load, or network is bad 
> etc.).
> In such cases, the return value of the command {{ssh $hostIp ps -p $pid}} 
> will be 255 (ssh command exits with the exit status of the remote command or 
> with 255 if an error occurred.).
> According the current logic of method {{getActionStatus()}} in 
> {{SshActionExecutor}}, the action status will be determined as OK which may 
> not be correct. 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)