Re: Review Request 71425: OOZIE-3405 - SSH action shows empty error Message and Error code

[Andras Salamon]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71425/#review217548
---




core/src/test/java/org/apache/oozie/action/ssh/TestSshActionExecutor.java
Lines 53 (patched)


Do we need this?



core/src/test/java/org/apache/oozie/action/ssh/TestSshActionExecutor.java
Line 593 (original), 596 (patched)


Why do we need this modification? This test is unrelated.



core/src/test/java/org/apache/oozie/action/ssh/TestSshActionExecutor.java
Lines 610-611 (patched)


This is not true for this method. Probably block-copy error.



core/src/test/java/org/apache/oozie/action/ssh/TestSshActionExecutor.java
Lines 613 (patched)


Were you able to run this test? The whole class is ignored right now: 
https://issues.apache.org/jira/browse/OOZIE-3391



core/src/test/java/org/apache/oozie/action/ssh/TestSshActionExecutor.java
Lines 629 (patched)


Could we use a less personal e-mail address here?



core/src/test/java/org/apache/oozie/action/ssh/TestSshActionExecutor.java
Lines 630 (patched)


script.sh is created at the beginning of the test method, but now used 
here. If we don't need it, then please delete all the related code.



core/src/test/java/org/apache/oozie/action/ssh/TestSshActionExecutor.java
Lines 631 (patched)


Do we need the semicolon?



core/src/test/java/org/apache/oozie/action/ssh/TestSshActionExecutor.java
Lines 647-649 (patched)


Can you please add assert messages.


- Andras Salamon


On Sept. 3, 2019, 1:10 p.m., Mate Juhasz wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/71425/
> ---
> 
> (Updated Sept. 3, 2019, 1:10 p.m.)
> 
> 
> Review request for oozie, Andras Salamon, Denes Bodo, and Kinga Marton.
> 
> 
> Bugs: OOZIE-3405
> https://issues.apache.org/jira/browse/OOZIE-3405
> 
> 
> Repository: oozie-git
> 
> 
> Description
> ---
> 
> Currently, when an SSH action fails the only message that is returned is the 
> Status. Neither the error Message nor Error code fields are filled. This 
> makes reporting on the causes of SSH Action failures via Oozie highly 
> impractical: the only meaningful bit of information there is on a failed SSH 
> Action is the Status.
> 
> The Status is filled based on the presence (or lack of) the .error file that 
> is produced in case the user submitted script returns with any other value 
> than 0.
> 
> SshActionExecutor#getActionStatus
>  ...
>  String outFile = getRemoteFileName(context, action, "error", false, true);
>  String checkErrorCmd = SSH_COMMAND_BASE + action.getTrackerUri() + " ls " + 
> outFile;
>  int retVal = getReturnValue(checkErrorCmd);
>  ...
>  
> User requirement is to provide some more detailed information on the 
> success/failure of the user-submitted script. That could be at a minimum the 
> return value, optionally the last ~1K of the stderr that is drained. This 
> information could then be communicated via errorMessage and ErrorCode
> 
> 
> Diffs
> -
> 
>   core/src/main/java/org/apache/oozie/ErrorCode.java 6b0ce4700 
>   core/src/main/java/org/apache/oozie/action/ssh/SshActionExecutor.java 
> 6956cba7b 
>   core/src/main/resources/ssh-wrapper.sh e2e6f7f9b 
>   core/src/test/java/org/apache/oozie/action/ssh/TestSshActionExecutor.java 
> d68aed094 
> 
> 
> Diff: https://reviews.apache.org/r/71425/diff/1/
> 
> 
> Testing
> ---
> 
> Unit testing is a bit awkward in thise case (I would like to ask for some 
> advice), although I have managed to test on a live cluster.
> 
> 
> Thanks,
> 
> Mate Juhasz
> 
>

Review Request 71425: OOZIE-3405 - SSH action shows empty error Message and Error code

[Mate Juhasz]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71425/
---

Review request for oozie, Andras Salamon, Denes Bodo, and Kinga Marton.


Bugs: OOZIE-3405
https://issues.apache.org/jira/browse/OOZIE-3405


Repository: oozie-git


Description
---

Currently, when an SSH action fails the only message that is returned is the 
Status. Neither the error Message nor Error code fields are filled. This makes 
reporting on the causes of SSH Action failures via Oozie highly impractical: 
the only meaningful bit of information there is on a failed SSH Action is the 
Status.

The Status is filled based on the presence (or lack of) the .error file that is 
produced in case the user submitted script returns with any other value than 0.

SshActionExecutor#getActionStatus
 ...
 String outFile = getRemoteFileName(context, action, "error", false, true);
 String checkErrorCmd = SSH_COMMAND_BASE + action.getTrackerUri() + " ls " + 
outFile;
 int retVal = getReturnValue(checkErrorCmd);
 ...
 
User requirement is to provide some more detailed information on the 
success/failure of the user-submitted script. That could be at a minimum the 
return value, optionally the last ~1K of the stderr that is drained. This 
information could then be communicated via errorMessage and ErrorCode


Diffs
-

  core/src/main/java/org/apache/oozie/ErrorCode.java 6b0ce4700 
  core/src/main/java/org/apache/oozie/action/ssh/SshActionExecutor.java 
6956cba7b 
  core/src/main/resources/ssh-wrapper.sh e2e6f7f9b 
  core/src/test/java/org/apache/oozie/action/ssh/TestSshActionExecutor.java 
d68aed094 


Diff: https://reviews.apache.org/r/71425/diff/1/


Testing
---

Unit testing is a bit awkward in thise case (I would like to ask for some 
advice), although I have managed to test on a live cluster.


Thanks,

Mate Juhasz

[jira] [Updated] (OOZIE-3405) SSH action shows empty error Message and Error code

[Mate Juhasz (Jira)]

 [ 
https://issues.apache.org/jira/browse/OOZIE-3405?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mate Juhasz updated OOZIE-3405:
---
Attachment: OOZIE-3405-V1.patch

> SSH action shows empty error Message and Error code
> ---
>
> Key: OOZIE-3405
> URL: https://issues.apache.org/jira/browse/OOZIE-3405
> Project: Oozie
>  Issue Type: Improvement
>Reporter: Peter Orova
>Assignee: Mate Juhasz
>Priority: Minor
> Attachments: OOZIE-3405-V1.patch
>
>
> Currently, when an SSH action fails the only message that is returned is the 
> Status. Neither the {{error Message}} nor {{Error code}} fields are filled. 
> This makes reporting on the causes of SSH Action failures via Oozie highly 
> impractical: the only meaningful bit of information there is on a failed SSH 
> Action is the Status.
> The Status is filled based on the presence (or lack of) the {{.error file}} 
> that is produced in case the user submitted script returns with any other 
> value than 0.
> {noformat}
> SshActionExecutor#getActionStatus
>  ...
>  String outFile = getRemoteFileName(context, action, "error", false, true);
>  String checkErrorCmd = SSH_COMMAND_BASE + action.getTrackerUri() + " ls " + 
> outFile;
>  int retVal = getReturnValue(checkErrorCmd);
>  ...
> {noformat}
>  
>  User requirement is to provide some more detailed information on the 
> success/failure of the user-submitted script. That could be at a minimum the 
> return value, optionally the last ~1K of the stderr that is drained. This 
> information could then be communicated via {{errorMessage}} and {{ErrorCode}}



--
This message was sent by Atlassian Jira
(v8.3.2#803003)

[jira] [Commented] (OOZIE-3405) SSH action shows empty error Message and Error code

[Mate Juhasz (Jira)]

[ 
https://issues.apache.org/jira/browse/OOZIE-3405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16921406#comment-16921406
 ] 

Mate Juhasz commented on OOZIE-3405:


Taking this over [~orova] if you dont mind :)

> SSH action shows empty error Message and Error code
> ---
>
> Key: OOZIE-3405
> URL: https://issues.apache.org/jira/browse/OOZIE-3405
> Project: Oozie
>  Issue Type: Improvement
>Reporter: Peter Orova
>Assignee: Mate Juhasz
>Priority: Minor
>
> Currently, when an SSH action fails the only message that is returned is the 
> Status. Neither the {{error Message}} nor {{Error code}} fields are filled. 
> This makes reporting on the causes of SSH Action failures via Oozie highly 
> impractical: the only meaningful bit of information there is on a failed SSH 
> Action is the Status.
> The Status is filled based on the presence (or lack of) the {{.error file}} 
> that is produced in case the user submitted script returns with any other 
> value than 0.
> {noformat}
> SshActionExecutor#getActionStatus
>  ...
>  String outFile = getRemoteFileName(context, action, "error", false, true);
>  String checkErrorCmd = SSH_COMMAND_BASE + action.getTrackerUri() + " ls " + 
> outFile;
>  int retVal = getReturnValue(checkErrorCmd);
>  ...
> {noformat}
>  
>  User requirement is to provide some more detailed information on the 
> success/failure of the user-submitted script. That could be at a minimum the 
> return value, optionally the last ~1K of the stderr that is drained. This 
> information could then be communicated via {{errorMessage}} and {{ErrorCode}}



--
This message was sent by Atlassian Jira
(v8.3.2#803003)

[jira] [Assigned] (OOZIE-3405) SSH action shows empty error Message and Error code

[Mate Juhasz (Jira)]

 [ 
https://issues.apache.org/jira/browse/OOZIE-3405?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mate Juhasz reassigned OOZIE-3405:
--

Assignee: Mate Juhasz  (was: Peter Orova)

> SSH action shows empty error Message and Error code
> ---
>
> Key: OOZIE-3405
> URL: https://issues.apache.org/jira/browse/OOZIE-3405
> Project: Oozie
>  Issue Type: Improvement
>Reporter: Peter Orova
>Assignee: Mate Juhasz
>Priority: Minor
>
> Currently, when an SSH action fails the only message that is returned is the 
> Status. Neither the {{error Message}} nor {{Error code}} fields are filled. 
> This makes reporting on the causes of SSH Action failures via Oozie highly 
> impractical: the only meaningful bit of information there is on a failed SSH 
> Action is the Status.
> The Status is filled based on the presence (or lack of) the {{.error file}} 
> that is produced in case the user submitted script returns with any other 
> value than 0.
> {noformat}
> SshActionExecutor#getActionStatus
>  ...
>  String outFile = getRemoteFileName(context, action, "error", false, true);
>  String checkErrorCmd = SSH_COMMAND_BASE + action.getTrackerUri() + " ls " + 
> outFile;
>  int retVal = getReturnValue(checkErrorCmd);
>  ...
> {noformat}
>  
>  User requirement is to provide some more detailed information on the 
> success/failure of the user-submitted script. That could be at a minimum the 
> return value, optionally the last ~1K of the stderr that is drained. This 
> information could then be communicated via {{errorMessage}} and {{ErrorCode}}



--
This message was sent by Atlassian Jira
(v8.3.2#803003)

[jira] [Commented] (OOZIE-3540) Use StringBuilder instead of StringBuffer if concurrent access is not required

[Andras Salamon (Jira)]

[ 
https://issues.apache.org/jira/browse/OOZIE-3540?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16921234#comment-16921234
 ] 

Andras Salamon commented on OOZIE-3540:
---

Thanks for the contribution [~zsombor], +1, committed to master.

> Use StringBuilder instead of StringBuffer if concurrent access is not required
> --
>
> Key: OOZIE-3540
> URL: https://issues.apache.org/jira/browse/OOZIE-3540
> Project: Oozie
>  Issue Type: Improvement
>  Components: core
>Affects Versions: trunk
>Reporter: Zsombor Gegesy
>Assignee: Zsombor Gegesy
>Priority: Minor
> Fix For: 5.2.0
>
> Attachments: OOZIE-3540.patch
>
>
> StringBuffer is an old relic from the distant past, Oozie shouldnt' need to 
> use it, unless concurrent access is expected.



--
This message was sent by Atlassian Jira
(v8.3.2#803003)

[jira] [Commented] (OOZIE-3540) Use StringBuilder instead of StringBuffer if concurrent access is not required

[ASF subversion and git services (Jira)]

[ 
https://issues.apache.org/jira/browse/OOZIE-3540?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16921233#comment-16921233
 ] 

ASF subversion and git services commented on OOZIE-3540:


Commit e39bd7ffef82f6f34ddbc6bbfc6f2b09ae424788 in oozie's branch 
refs/heads/master from Andras Salamon
[ https://gitbox.apache.org/repos/asf?p=oozie.git;h=e39bd7f ]

OOZIE-3540 Use StringBuilder instead of StringBuffer if concurrent access is 
not required (zsombor via asalamon74)


> Use StringBuilder instead of StringBuffer if concurrent access is not required
> --
>
> Key: OOZIE-3540
> URL: https://issues.apache.org/jira/browse/OOZIE-3540
> Project: Oozie
>  Issue Type: Improvement
>  Components: core
>Affects Versions: trunk
>Reporter: Zsombor Gegesy
>Assignee: Zsombor Gegesy
>Priority: Minor
> Fix For: 5.2.0
>
> Attachments: OOZIE-3540.patch
>
>
> StringBuffer is an old relic from the distant past, Oozie shouldnt' need to 
> use it, unless concurrent access is expected.



--
This message was sent by Atlassian Jira
(v8.3.2#803003)

Re: Review Request 71416: OOZIE-3540 - use StringBuilder

[Andras Salamon]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71416/#review217537
---


Ship it!




Ship It!

- Andras Salamon


On Aug. 30, 2019, 2:49 p.m., Zsombor Gegesy wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/71416/
> ---
> 
> (Updated Aug. 30, 2019, 2:49 p.m.)
> 
> 
> Review request for oozie.
> 
> 
> Bugs: OOZIE-3540
> https://issues.apache.org/jira/browse/OOZIE-3540
> 
> 
> Repository: oozie-git
> 
> 
> Description
> ---
> 
> Instead of StringBuffer, StringBuilder is the preferred way to construct 
> Strings, since Java 1.5
> 
> 
> Diffs
> -
> 
>   client/src/main/java/org/apache/oozie/client/OozieClient.java 
> 1073cb9241f7e27e72f7e37d2dd852e9bde9b52a 
>   core/src/main/java/org/apache/oozie/action/hadoop/OozieJobInfo.java 
> 1bf5b417bc81538ccab8dc8b68d33f3deaacd685 
>   
> core/src/main/java/org/apache/oozie/command/bundle/BundleJobChangeXCommand.java
>  8628ea42a76a0620e8e06a09187c3c5298c51c3e 
>   
> core/src/main/java/org/apache/oozie/command/bundle/BundleSLAAlertsXCommand.java
>  1e6f6aea90120f060b72862aa5199cf7775910c8 
>   core/src/main/java/org/apache/oozie/command/coord/CoordCommandUtils.java 
> 7632839f64c4ed516ec9616c6f99c1224bd9c727 
>   
> core/src/main/java/org/apache/oozie/command/coord/CoordSLAAlertsXCommand.java 
> ff6605ad50f99b7ccdd863952fb5e9515832c77a 
>   
> core/src/main/java/org/apache/oozie/coord/input/dependency/CoordOldInputDependency.java
>  296dd1aaadd3b4488dcddbfa23ecbfc0a08e1f75 
>   
> core/src/main/java/org/apache/oozie/coord/input/dependency/CoordPullInputDependency.java
>  9174a48baec511a553e366323e12cda826036fa9 
>   core/src/main/java/org/apache/oozie/coord/input/logic/InputLogicParser.java 
> 1571f317f9348552b65aabf2f38f7dad77225d7d 
>   core/src/main/java/org/apache/oozie/service/ShareLibService.java 
> 7d074a18ec8e7090ef6cdc263a0c80430515c514 
>   core/src/main/java/org/apache/oozie/servlet/JVMInfo.java 
> bf2b01db340a014d8831c5186188741f866e6559 
>   core/src/main/java/org/apache/oozie/sla/listener/SLAEmailEventListener.java 
> 96020538d8d8c23afdc328780bf1194226b939e6 
>   core/src/main/java/org/apache/oozie/util/IOUtils.java 
> c8ada2f3f4eca785961d0cfe1b88ac4cb7d6a759 
>   core/src/main/java/org/apache/oozie/util/db/SqlStatement.java 
> a35e2408e3f062a5116b0bfa5067706001ab97e3 
>   core/src/test/java/org/apache/oozie/service/TestZKXLogStreamingService.java 
> 5101b8d168c9c19d4e4123cd0290747f07e52683 
>   core/src/test/java/org/apache/oozie/util/DrainerTestCase.java 
> 2221e61f0a4e5c84b5940135f1a9a5e25e022dde 
>   
> sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherAMUtils.java
>  c577d252086e21562aea3fb339eaa95bac21c284 
>   
> sharelib/pig/src/main/java/org/apache/oozie/action/hadoop/OoziePigStats.java 
> b2fcfc444984571fa898fb9464162f938e15bcdc 
> 
> 
> Diff: https://reviews.apache.org/r/71416/diff/1/
> 
> 
> Testing
> ---
> 
> Manual testing.
> 
> 
> Thanks,
> 
> Zsombor Gegesy
> 
>

[jira] Subscription: Oozie Patch Available

[jira]

Issue Subscription
Filter: Oozie Patch Available (92 issues)

Subscriber: ooziedaily

Key Summary
OOZIE-3540  Use StringBuilder instead of StringBuffer if concurrent access is 
not required
https://issues.apache.org/jira/browse/OOZIE-3540
OOZIE-3536  oozie-main(pom.xml)  plugin maven-javadoc-plugin upgrade version 
caused configuration can't find the Tag 
https://issues.apache.org/jira/browse/OOZIE-3536
OOZIE-3529  Oozie not supported for s3 as filesystem
https://issues.apache.org/jira/browse/OOZIE-3529
OOZIE-3482  Fix bug in CoordSubmitXCommand#validateCoordinatorJob
https://issues.apache.org/jira/browse/OOZIE-3482
OOZIE-3480  Add windowactionstatus metrics in DBLiteWorkflowStoreService
https://issues.apache.org/jira/browse/OOZIE-3480
OOZIE-3461  CoordMaterializeTriggerService code cleanup
https://issues.apache.org/jira/browse/OOZIE-3461
OOZIE-3449  Make spark-2 as the default profile
https://issues.apache.org/jira/browse/OOZIE-3449
OOZIE-3447  Run test case in local : It shows oozie-hsqldb-orm.xml exception
https://issues.apache.org/jira/browse/OOZIE-3447
OOZIE-3418  Upgrade to Guava 27
https://issues.apache.org/jira/browse/OOZIE-3418
OOZIE-3404  The env variable of SPARK_HOME needs to be set when running pySpark
https://issues.apache.org/jira/browse/OOZIE-3404
OOZIE-3375  Can't use empty  in coordinator
https://issues.apache.org/jira/browse/OOZIE-3375
OOZIE-3367  Using && in EL expressions in oozie bundle.xml files generates 
parse errors
https://issues.apache.org/jira/browse/OOZIE-3367
OOZIE-3366  Update workflow status and subworkflow status on suspend command
https://issues.apache.org/jira/browse/OOZIE-3366
OOZIE-3364  Rerunning Oozie bundle jobs starts the coordinators in 
indeterminate order
https://issues.apache.org/jira/browse/OOZIE-3364
OOZIE-3362  When killed, SSH action should kill the spawned processes on target 
host
https://issues.apache.org/jira/browse/OOZIE-3362
OOZIE-3335  Cleanup parseFilter methods
https://issues.apache.org/jira/browse/OOZIE-3335
OOZIE-3328  Create Hive compatibility action executor to run hive actions using 
beeline
https://issues.apache.org/jira/browse/OOZIE-3328
OOZIE-3320  Oozie ShellAction should support absolute bash file path
https://issues.apache.org/jira/browse/OOZIE-3320
OOZIE-3319  Log SSH action callback error output
https://issues.apache.org/jira/browse/OOZIE-3319
OOZIE-3301  Update NOTICE file
https://issues.apache.org/jira/browse/OOZIE-3301
OOZIE-3274  Remove slf4j
https://issues.apache.org/jira/browse/OOZIE-3274
OOZIE-3266  Coord action rerun support RERUN_SKIP_NODES option
https://issues.apache.org/jira/browse/OOZIE-3266
OOZIE-3256  refactor OozieCLI class
https://issues.apache.org/jira/browse/OOZIE-3256
OOZIE-3254  [coordinator] LAST_ONLY and NONE execution modes: possible 
OutOfMemoryError when there are too many coordinator actions to materialize
https://issues.apache.org/jira/browse/OOZIE-3254
OOZIE-3199  Let system property restriction configurable
https://issues.apache.org/jira/browse/OOZIE-3199
OOZIE-3196  Authorization: restrict world readability by user
https://issues.apache.org/jira/browse/OOZIE-3196
OOZIE-3179  Adding a configurable config-default.xml location to a workflow
https://issues.apache.org/jira/browse/OOZIE-3179
OOZIE-3170  Oozie Diagnostic Bundle tool fails with NPE due to missing service 
class
https://issues.apache.org/jira/browse/OOZIE-3170
OOZIE-3137  Add support for log4j2 in HiveMain
https://issues.apache.org/jira/browse/OOZIE-3137
OOZIE-3135  Configure log4j2 in SqoopMain
https://issues.apache.org/jira/browse/OOZIE-3135
OOZIE-3091  Oozie Sqoop Avro Import fails with "java.lang.NoClassDefFoundError: 
org/apache/avro/mapred/AvroWrapper"
https://issues.apache.org/jira/browse/OOZIE-3091
OOZIE-3071  Oozie 4.3 Spark sharelib ueses a different version of commons-lang3 
than Spark 2.2.0
https://issues.apache.org/jira/browse/OOZIE-3071
OOZIE-3063  Sanitizing variables that are part of openjpa.ConnectionProperties
https://issues.apache.org/jira/browse/OOZIE-3063
OOZIE-3062  Set HADOOP_CONF_DIR for spark action
https://issues.apache.org/jira/browse/OOZIE-3062
OOZIE-2952  Fix Findbugs warnings in oozie-sharelib-oozie
https://issues.apache.org/jira/browse/OOZIE-2952
OOZIE-2834  ParameterVerifier logging non-useful warning for workflow definition
https://issues.apache.org/jira/browse/OOZIE-2834
OOZIE-2812  SparkConfigurationService should support loading configurations 
from multiple Spark versions
https://issues.apache.org/jira/browse/OOZIE-2812
OOZIE-2795  Create lib directory or symlink for