[jira] Subscription: Oozie Patch Available

[jira]

Issue Subscription
Filter: Oozie Patch Available (86 issues)

Subscriber: ooziedaily

Key Summary
OOZIE-2619  Make  Hive action defaults to match hive defaults when running from 
command line
https://issues-test.apache.org/jira/browse/OOZIE-2619
OOZIE-2618  Include hive-common in hive-beeline as compile component for 
hive-server2 (hive2 action)
https://issues-test.apache.org/jira/browse/OOZIE-2618
OOZIE-2617  Read properties files in action configurations
https://issues-test.apache.org/jira/browse/OOZIE-2617
OOZIE-2616  Add Tez profile for Hive and Pig sharelibs
https://issues-test.apache.org/jira/browse/OOZIE-2616
OOZIE-2606  Set spark.yarn.jars to fix Spark 2.0 with Oozie
https://issues-test.apache.org/jira/browse/OOZIE-2606
OOZIE-2589  CompletedActionXCommand priority param is of no effect
https://issues-test.apache.org/jira/browse/OOZIE-2589
OOZIE-2588  Support getting credentials for same cluster hcat when credentials 
config is empty
https://issues-test.apache.org/jira/browse/OOZIE-2588
OOZIE-2585  Disable or remove test case 
TestPartitionDependencyManagerEhcache.testMemoryUsageAndSpeedOverflowToDisk and 
testMemoryUsageAndSpeed
https://issues-test.apache.org/jira/browse/OOZIE-2585
OOZIE-2584  Eliminate Thread.sleep() calls in TestMemoryLocks
https://issues-test.apache.org/jira/browse/OOZIE-2584
OOZIE-2583  oozie throws EL Exception when reference variable name containing 
dot
https://issues-test.apache.org/jira/browse/OOZIE-2583
OOZIE-2582  Populating external child Ids for action failures
https://issues-test.apache.org/jira/browse/OOZIE-2582
OOZIE-2573  dataset url  which contains spaces can not be handled rightly
https://issues-test.apache.org/jira/browse/OOZIE-2573
OOZIE-2572  SLA DURATION miss not shown when job is running for longer than 
expected time
https://issues-test.apache.org/jira/browse/OOZIE-2572
OOZIE-2571  Add scala.binary.version Maven property so that Scala 2.11 can be 
used
https://issues-test.apache.org/jira/browse/OOZIE-2571
OOZIE-2569  Adding yarn-site, core-site, hdfs-site and mapred-site into spark 
launcher 
https://issues-test.apache.org/jira/browse/OOZIE-2569
OOZIE-2568  SSH  action pretends to retry automaticly when it failed
https://issues-test.apache.org/jira/browse/OOZIE-2568
OOZIE-2564  Create new log4j config for unit tests so that logged messages 
contain threads
https://issues-test.apache.org/jira/browse/OOZIE-2564
OOZIE-2552  Update ActiveMQ version for security and other fixes
https://issues-test.apache.org/jira/browse/OOZIE-2552
OOZIE-2538  Update HttpClient versions to close security vulnerabilities
https://issues-test.apache.org/jira/browse/OOZIE-2538
OOZIE-2534  Versioned action libs (similar to how SharedLibs works)
https://issues-test.apache.org/jira/browse/OOZIE-2534
OOZIE-2530  Remove jline from parent and sharelib pig pom
https://issues-test.apache.org/jira/browse/OOZIE-2530
OOZIE-2528  Print out environment variables in LauncherMapper
https://issues-test.apache.org/jira/browse/OOZIE-2528
OOZIE-2521  Filter options are case sensitive for jobtye=bundle
https://issues-test.apache.org/jira/browse/OOZIE-2521
OOZIE-2518  Oozie is creating many files and directories in /tmp per day
https://issues-test.apache.org/jira/browse/OOZIE-2518
OOZIE-2517  Add support for startCreatedTime and endCreatedTime filters for 
coord and bundles
https://issues-test.apache.org/jira/browse/OOZIE-2517
OOZIE-2513  log.scan.duration should not be used for error and audit logs
https://issues-test.apache.org/jira/browse/OOZIE-2513
OOZIE-2512  ShareLibservice returns incorrect path for jar
https://issues-test.apache.org/jira/browse/OOZIE-2512
OOZIE-2508  Documentation change for Coord action rerun [OOZIE-1735]
https://issues-test.apache.org/jira/browse/OOZIE-2508
OOZIE-2501  ZK reentrant lock doesn't work for few cases
https://issues-test.apache.org/jira/browse/OOZIE-2501
OOZIE-2500  -DtestJarSimple option mentioned in minioozie doc does not work
https://issues-test.apache.org/jira/browse/OOZIE-2500
OOZIE-2499  map-reduce launcher does not need distributed files, archives 
except  jar of input/outputformat class
https://issues-test.apache.org/jira/browse/OOZIE-2499
OOZIE-2498  Oozie CallerId configuration for downstream components 
https://issues-test.apache.org/jira/browse/OOZIE-2498
OOZIE-2495  change action status from  ErrorType.NON_TRANSIENT to TRANSIENT 
when SSH action occurs AUTH_FAILED occasionally
https://issues-test.apache.org/jira/browse/OOZIE-2495
OOZIE-2491  oozie acl cannot specify group,it does`t work
https://issues-test.apache.org/jira/browse/OOZIE-2491

[jira] [Commented] (OOZIE-1702) Reduce warnings thrown while building

[Robert Kanter (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-1702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15879689#comment-15879689
 ] 

Robert Kanter commented on OOZIE-1702:
--

Thanks for categorizing and listing the warnings [~eva].  

{quote}
a) where do the code triggering these warnings reside? I could not find them 
easily.
{quote}
I don't think these are anything code related.  "artifact" and "assembly" are 
all maven concepts, so this JIRA will likely involve solely tweaking pom or 
other XML files.  You can find the assembly files under 
{{src/main/assemblies/}} and pom files in each subproject directory.

{quote}
b) what would be the best practice for each of these warnings? 
1 - change the file description or format or the build process?
2 - change the character length accept level or change the path names/file 
names to fit the limit?
3 - what is the purpose of warning on non-triggered patterns?
{quote}
1 - I'm not really sure what's the best practice here.  Maybe there's some way 
to tell maven that these are directories instead of files and then the warning 
will go away?  Or maybe we need to use a regex or something?
2 - We can't really change the names (they're including the path as part of the 
name for some reason) to match the 100 character limit without restructuring 
everything, which I don't think we'll want to do.  Maybe there's a way to tell 
it to ignore this?
3 - I'm not really sure, but it sounds like we're excluding some files 
somewhere based on some patterns (i.e. 
"org.apache.oozie:oozie-hadoop-distcp:jar"), and those patterns don't end up 
actually excluding anything.  It's probably that those patterns are too old and 
should be removed?

It would be best to do some Googling to figure out what we should be doing in 
each of these cases.  Other than #2, I'm not really sure what the best thing to 
do is.

> Reduce warnings thrown while building
> -
>
> Key: OOZIE-1702
> URL: https://issues.apache.org/jira/browse/OOZIE-1702
> Project: Oozie
>  Issue Type: Wish
>  Components: build
>Affects Versions: trunk
>Reporter: Mona Chitnis
>Priority: Minor
>  Labels: documentation, newbie
>
> A lot of warnings are thrown during Oozie compilation, complaining about 
> javadoc mistakes, missing links etc among probably other severe ones. This 
> clutters the output. This JIRA is to fix these warnings



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Jenkins build is back to normal : oozie-trunk-find-patches-available #150667

[Apache Jenkins Server]

See 

Build failed in Jenkins: oozie-trunk-find-patches-available #150666

[Apache Jenkins Server]

See 


--
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on H1 (Hadoop) in workspace 

 > git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
 > git config remote.origin.url 
 > https://git-wip-us.apache.org/repos/asf/oozie.git # timeout=10
Fetching upstream changes from https://git-wip-us.apache.org/repos/asf/oozie.git
 > git --version # timeout=10
 > git fetch --tags --progress 
 > https://git-wip-us.apache.org/repos/asf/oozie.git 
 > +refs/heads/*:refs/remotes/origin/*
 > git rev-parse refs/remotes/origin/master^{commit} # timeout=10
 > git rev-parse refs/remotes/origin/origin/master^{commit} # timeout=10
Checking out Revision 77817e73588f665cbe43b0971cd93465d40711fb 
(refs/remotes/origin/master)
 > git config core.sparsecheckout # timeout=10
 > git checkout -f 77817e73588f665cbe43b0971cd93465d40711fb
 > git rev-list 77817e73588f665cbe43b0971cd93465d40711fb # timeout=10
[oozie-trunk-find-patches-available] $ /bin/bash 
/tmp/hudson4717833604610625423.sh
mkdir: cannot create directory 
':
 File exists
  % Total% Received % Xferd  Average Speed   TimeTime Time  Current
 Dload  Upload   Total   SpentLeft  Speed
  0 00 00 0  0  0 --:--:-- --:--:-- --:--:-- 0  
0 00 00 0  0  0 --:--:-- --:--:-- --:--:-- 0100 
 151k0  151k0 0   146k  0 --:--:--  0:00:01 --:--:--  147k
curl: (18) transfer closed with outstanding read data remaining
Could not retrieve available patches from JIRA
Build step 'Execute shell' marked build as failure

[jira] [Commented] (OOZIE-1702) Reduce warnings thrown while building

[Eva Andreasson (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-1702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15879064#comment-15879064
 ] 

Eva Andreasson commented on OOZIE-1702:
---

I guess I couldn't find a way to attach a file, so here is the current WARNINGs 
captured:

1. Non-regular files
[WARNING] Assembly file: 
/oozie/client/target/oozie-client-4.4.0-SNAPSHOT-client is not a regular file 
(it may be a directory). It cannot be attached to the project build for 
installation or deployment.
[WARNING] Assembly file: /oozie/sharelib/oozie/target/partial-sharelib is not a 
regular file (it may be a directory). It cannot be attached to the project 
build for installation or deployment.
[WARNING] Assembly file: /oozie/sharelib/hcatalog/target/partial-sharelib is 
not a regular file (it may be a directory). It cannot be attached to the 
project build for installation or deployment.
[WARNING] Assembly file: oozie/sharelib/distcp/target/partial-sharelib is not a 
regular file (it may be a directory). It cannot be attached to the project 
build for installation or deployment.
[WARNING] Assembly file: /oozie/sharelib/streaming/target/partial-sharelib is 
not a regular file (it may be a directory). It cannot be attached to the 
project build for installation or deployment.
[WARNING] Assembly file: /oozie/sharelib/pig/target/partial-sharelib is not a 
regular file (it may be a directory). It cannot be attached to the project 
build for installation or deployment.
[WARNING] Assembly file: /oozie/sharelib/hive/target/partial-sharelib is not a 
regular file (it may be a directory). It cannot be attached to the project 
build for installation or deployment.
[WARNING] Assembly file: /oozie/sharelib/hive2/target/partial-sharelib is not a 
regular file (it may be a directory). It cannot be attached to the project 
build for installation or deployment.
[WARNING] Assembly file: /oozie/sharelib/sqoop/target/partial-sharelib is not a 
regular file (it may be a directory). It cannot be attached to the project 
build for installation or deployment.
[WARNING] Assembly file: 
/oozie/examples/target/oozie-examples-4.4.0-SNAPSHOT-examples is not a regular 
file (it may be a directory). It cannot be attached to the project build for 
installation or deployment.
[WARNING] Assembly file: /oozie/sharelib/spark/target/partial-sharelib is not a 
regular file (it may be a directory). It cannot be attached to the project 
build for installation or deployment.
[WARNING] Assembly file: /oozie/sharelib/target/oozie-sharelib-4.4.0-SNAPSHOT 
is not a regular file (it may be a directory). It cannot be attached to the 
project build for installation or deployment.
[WARNING] Assembly file: /oozie/tools/target/oozie-tools-4.4.0-SNAPSHOT-tools 
is not a regular file (it may be a directory). It cannot be attached to the 
project build for installation or deployment.
[WARNING] Assembly file: /oozie/distro/target/oozie-4.4.0-SNAPSHOT-distro is 
not a regular file (it may be a directory). It cannot be attached to the 
project build for installation or deployment.

2. Character length issue
[WARNING] Entry: 
oozie-4.4.0-SNAPSHOT/embedded-oozie-server/webapp/console/sla/css/images/ui-bg_flat_100_DEECFD_40x100.png
 longer than 100 characters.
[WARNING] Resulting tar file can only be processed successfully by GNU 
compatible tar commands
[WARNING] Entry: 
oozie-4.4.0-SNAPSHOT/embedded-oozie-server/webapp/console/sla/css/images/ui-bg_glass_100_f6f6f6_1x400.png
 longer than 100 characters.
[WARNING] Entry: 
oozie-4.4.0-SNAPSHOT/embedded-oozie-server/webapp/console/sla/css/images/ui-bg_glass_65_ff_1x400.png
 longer than 100 characters.
[WARNING] Entry: 
oozie-4.4.0-SNAPSHOT/embedded-oozie-server/webapp/console/sla/css/images/ui-bg_highlight-soft_100_ee_1x100.png
 longer than 100 characters.
[WARNING] Entry: 
oozie-4.4.0-SNAPSHOT/embedded-oozie-server/webapp/console/sla/css/images/ui-icons_22_256x240.png
 longer than 100 characters.
[WARNING] Entry: 
oozie-4.4.0-SNAPSHOT/embedded-oozie-server/webapp/console/sla/js/table/jquery-ui-1.10.3.custom.min.js
 longer than 100 characters.
[WARNING] Entry: 
oozie-4.4.0-SNAPSHOT/embedded-oozie-server/webapp/console/sla/js/table/jquery-ui-timepicker-addon.js
 longer than 100 characters.
[WARNING] Entry: 
oozie-4.4.0-SNAPSHOT/embedded-oozie-server/webapp/WEB-INF/lib/geronimo-annotation_1.0_spec-1.1.1.jar
 longer than 100 characters.
[WARNING] Entry: 
oozie-4.4.0-SNAPSHOT/embedded-oozie-server/webapp/WEB-INF/lib/geronimo-j2ee-management_1.1_spec-1.0.1.jar
 longer than 100 characters.
[WARNING] Entry: 
oozie-4.4.0-SNAPSHOT/embedded-oozie-server/webapp/WEB-INF/lib/hadoop-yarn-server-applicationhistoryservice-2.6.0.jar
 longer than 100 characters.
[WARNING] Entry: 
oozie-4.4.0-SNAPSHOT/embedded-oozie-server/webapp/WEB-INF/lib/hadoop-yarn-server-resourcemanager-2.6.0.jar
 longer than 100 characters.
[WARNING] Entry: 

[jira] [Commented] (OOZIE-1702) Reduce warnings thrown while building

[Eva Andreasson (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-1702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15879058#comment-15879058
 ] 

Eva Andreasson commented on OOZIE-1702:
---

There seems to be (at least in my recently cleaned up build environment) three 
types of warnings:
1. The build system detecting a non-regular file
2. The build system detects an entry longer than 100 characters
3. Patterns not triggered in exclusion filters
 
I will attach a file with the full list of warnings, grouped into the three 
categories. 

However, since I don't know Oozie build system or code too well I need advise 
on two matters:
a) where do the code triggering these warnings reside? I could not find them 
easily.
b) what would be the best practice for each of these warnings? 
1 - change the file description or format or the build process?
2 - change the character length accept level or change the path names/file 
names to fit the limit?
3 - what is the purpose of warning on non-triggered patterns?

> Reduce warnings thrown while building
> -
>
> Key: OOZIE-1702
> URL: https://issues.apache.org/jira/browse/OOZIE-1702
> Project: Oozie
>  Issue Type: Wish
>  Components: build
>Affects Versions: trunk
>Reporter: Mona Chitnis
>Priority: Minor
>  Labels: documentation, newbie
>
> A lot of warnings are thrown during Oozie compilation, complaining about 
> javadoc mistakes, missing links etc among probably other severe ones. This 
> clutters the output. This JIRA is to fix these warnings



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (OOZIE-2802) Spark action failure on Spark 2.1.0 due to duplicate sharelibs

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-2802?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15878772#comment-15878772
 ] 

Hadoop QA commented on OOZIE-2802:
--

Testing JIRA OOZIE-2802

Cleaning local git workspace



{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:red}-1 RAW_PATCH_ANALYSIS{color}
.{color:green}+1{color} the patch does not introduce any @author tags
.{color:green}+1{color} the patch does not introduce any tabs
.{color:green}+1{color} the patch does not introduce any trailing spaces
.{color:green}+1{color} the patch does not introduce any line longer than 
132
.{color:red}-1{color} the patch does not add/modify any testcase
{color:green}+1 RAT{color}
.{color:green}+1{color} the patch does not seem to introduce new RAT 
warnings
{color:green}+1 JAVADOC{color}
.{color:green}+1{color} the patch does not seem to introduce new Javadoc 
warnings
{color:green}+1 COMPILE{color}
.{color:green}+1{color} HEAD compiles
.{color:green}+1{color} patch compiles
.{color:green}+1{color} the patch does not seem to introduce new javac 
warnings
{color:green}+1{color} There are no new bugs found in total.
.{color:green}+1{color} There are no new bugs found in [docs].
.{color:green}+1{color} There are no new bugs found in [server].
.{color:green}+1{color} There are no new bugs found in [client].
.{color:green}+1{color} There are no new bugs found in [tools].
.{color:green}+1{color} There are no new bugs found in [core].
.{color:green}+1{color} There are no new bugs found in [examples].
.{color:green}+1{color} There are no new bugs found in [sharelib/streaming].
.{color:green}+1{color} There are no new bugs found in [sharelib/hive2].
.{color:green}+1{color} There are no new bugs found in [sharelib/distcp].
.{color:green}+1{color} There are no new bugs found in [sharelib/hive].
.{color:green}+1{color} There are no new bugs found in [sharelib/pig].
.{color:green}+1{color} There are no new bugs found in [sharelib/oozie].
.{color:green}+1{color} There are no new bugs found in [sharelib/sqoop].
.{color:green}+1{color} There are no new bugs found in [sharelib/spark].
.{color:green}+1{color} There are no new bugs found in [sharelib/hcatalog].
.{color:green}+1{color} There are no new bugs found in 
[hadooplibs/hadoop-utils-2].
{color:green}+1 BACKWARDS_COMPATIBILITY{color}
.{color:green}+1{color} the patch does not change any JPA 
Entity/Colum/Basic/Lob/Transient annotations
.{color:green}+1{color} the patch does not modify JPA files
{color:red}-1 TESTS{color}
.Tests run: 1879
.Tests failed: 2
.Tests errors: 0

.The patch failed the following testcases:

.  
testMemoryUsageAndSpeed(org.apache.oozie.service.TestPartitionDependencyManagerEhcache)
.  
testMemoryUsageAndSpeed(org.apache.oozie.service.TestPartitionDependencyManagerService)

.Tests failing with errors:
.  

{color:green}+1 DISTRO{color}
.{color:green}+1{color} distro tarball builds with the patch 


{color:red}*-1 Overall result, please check the reported -1(s)*{color}


The full output of the test-patch run is available at

.   https://builds.apache.org/job/oozie-trunk-precommit-build/3656/

> Spark action failure on Spark 2.1.0 due to duplicate sharelibs
> --
>
> Key: OOZIE-2802
> URL: https://issues.apache.org/jira/browse/OOZIE-2802
> Project: Oozie
>  Issue Type: Bug
>Reporter: Satish Subhashrao Saley
>Assignee: Peter Cseh
> Attachments: OOZIE-2802-001.patch, spark-2.1-failure-launcherlib.txt
>
>
> There are some same jars in launcher lib and sharelib. These jars are causing 
> failures in spark action for spark 2.1.0. SPARK-18099 added an exception to 
> be thrown if same file gets added multiple times to distributed cache.
> {code}
> $ hdfs dfs -ls /user/saley/share/lib/launcher_20170216150752/oozie
> 17/02/16 15:14:22 WARN util.NativeCodeLoader: Unable to load native-hadoop 
> library for your platform... using builtin-java classes where applicable
> Found 3 items
> -rwxr-xr-x   1 saley supergroup  11871 2017-02-16 15:07 
> /user/saley/share/lib/launcher_20170216150752/oozie/oozie-hadoop-utils-hadoop-2-4.4.0-SNAPSHOT.jar
> -rwxr-xr-x   1 saley supergroup  17514 2017-02-16 15:07 
> /user/saley/share/lib/launcher_20170216150752/oozie/oozie-sharelib-hcatalog-4.4.0-SNAPSHOT.jar
> -rwxr-xr-x   1 saley supergroup  55092 2017-02-16 15:07 
> /user/saley/share/lib/launcher_20170216150752/oozie/oozie-sharelib-oozie-4.4.0-SNAPSHOT.jar
> $ hdfs dfs -ls /user/saley/share/lib/lib_20170216150706/oozie
> 17/02/16 15:14:39 WARN util.NativeCodeLoader: Unable to load native-hadoop 
> library for your platform... using 

Build failed in Jenkins: oozie-trunk-precommit-build #3656

[Apache Jenkins Server]

See 


--
[...truncated 1.53 MB...]
[WARNING] Entry: 
oozie-4.4.0-SNAPSHOT/embedded-oozie-server/webapp/docs/core/apidocs/src-html/org/apache/oozie/service/LiteWorkflowStoreService.LiteDecisionHandler.html
 longer than 100 characters.
[WARNING] Entry: 
oozie-4.4.0-SNAPSHOT/embedded-oozie-server/webapp/docs/core/apidocs/src-html/org/apache/oozie/service/ActionService.html
 longer than 100 characters.
[WARNING] Entry: 
oozie-4.4.0-SNAPSHOT/embedded-oozie-server/webapp/docs/core/apidocs/src-html/org/apache/oozie/service/JobsConcurrencyService.html
 longer than 100 characters.
[WARNING] Entry: 
oozie-4.4.0-SNAPSHOT/embedded-oozie-server/webapp/docs/core/apidocs/src-html/org/apache/oozie/service/HadoopAccessorService.html
 longer than 100 characters.
[WARNING] Entry: 
oozie-4.4.0-SNAPSHOT/embedded-oozie-server/webapp/docs/core/apidocs/src-html/org/apache/oozie/service/ProxyUserService.html
 longer than 100 characters.
[WARNING] Entry: 
oozie-4.4.0-SNAPSHOT/embedded-oozie-server/webapp/docs/core/apidocs/src-html/org/apache/oozie/service/StoreService.html
 longer than 100 characters.
[WARNING] Entry: 
oozie-4.4.0-SNAPSHOT/embedded-oozie-server/webapp/docs/core/apidocs/src-html/org/apache/oozie/service/GroupsService.html
 longer than 100 characters.
[WARNING] Entry: 
oozie-4.4.0-SNAPSHOT/embedded-oozie-server/webapp/docs/core/apidocs/src-html/org/apache/oozie/service/PartitionDependencyManagerService.html
 longer than 100 characters.
[WARNING] Entry: 
oozie-4.4.0-SNAPSHOT/embedded-oozie-server/webapp/docs/core/apidocs/src-html/org/apache/oozie/service/InstrumentationService.html
 longer than 100 characters.
[WARNING] Entry: 
oozie-4.4.0-SNAPSHOT/embedded-oozie-server/webapp/docs/core/apidocs/src-html/org/apache/oozie/service/CallableQueueService.html
 longer than 100 characters.
[WARNING] Entry: 
oozie-4.4.0-SNAPSHOT/embedded-oozie-server/webapp/docs/core/apidocs/src-html/org/apache/oozie/service/Service.html
 longer than 100 characters.
[WARNING] Entry: 
oozie-4.4.0-SNAPSHOT/embedded-oozie-server/webapp/docs/core/apidocs/src-html/org/apache/oozie/service/JMSAccessorService.JMSRetryRunnable.html
 longer than 100 characters.
[WARNING] Entry: 
oozie-4.4.0-SNAPSHOT/embedded-oozie-server/webapp/docs/core/apidocs/src-html/org/apache/oozie/service/SchemaCheckerService.html
 longer than 100 characters.
[WARNING] Entry: 
oozie-4.4.0-SNAPSHOT/embedded-oozie-server/webapp/docs/core/apidocs/src-html/org/apache/oozie/service/MemoryLocksService.Type.html
 longer than 100 characters.
[WARNING] Entry: 
oozie-4.4.0-SNAPSHOT/embedded-oozie-server/webapp/docs/core/apidocs/src-html/org/apache/oozie/service/CoordMaterializeTriggerService.html
 longer than 100 characters.
[WARNING] Entry: 
oozie-4.4.0-SNAPSHOT/embedded-oozie-server/webapp/docs/core/apidocs/src-html/org/apache/oozie/service/Services.html
 longer than 100 characters.
[WARNING] Entry: 
oozie-4.4.0-SNAPSHOT/embedded-oozie-server/webapp/docs/core/apidocs/src-html/org/apache/oozie/service/JPAService.QueryEntry.html
 longer than 100 characters.
[WARNING] Entry: 
oozie-4.4.0-SNAPSHOT/embedded-oozie-server/webapp/docs/core/apidocs/src-html/org/apache/oozie/service/AuthorizationService.html
 longer than 100 characters.
[WARNING] Entry: 
oozie-4.4.0-SNAPSHOT/embedded-oozie-server/webapp/docs/core/apidocs/src-html/org/apache/oozie/service/JMSTopicService.JobType.html
 longer than 100 characters.
[WARNING] Entry: 
oozie-4.4.0-SNAPSHOT/embedded-oozie-server/webapp/docs/core/apidocs/src-html/org/apache/oozie/service/SparkConfigurationService.html
 longer than 100 characters.
[WARNING] Entry: 
oozie-4.4.0-SNAPSHOT/embedded-oozie-server/webapp/docs/core/apidocs/src-html/org/apache/oozie/service/JMSTopicService.TopicType.html
 longer than 100 characters.
[WARNING] Entry: 
oozie-4.4.0-SNAPSHOT/embedded-oozie-server/webapp/docs/core/apidocs/src-html/org/apache/oozie/service/ConfigurationService.html
 longer than 100 characters.
[WARNING] Entry: 
oozie-4.4.0-SNAPSHOT/embedded-oozie-server/webapp/docs/core/apidocs/src-html/org/apache/oozie/service/URIHandlerService.html
 longer than 100 characters.
[WARNING] Entry: 
oozie-4.4.0-SNAPSHOT/embedded-oozie-server/webapp/docs/core/apidocs/src-html/org/apache/oozie/service/JMSAccessorService.html
 longer than 100 characters.
[WARNING] Entry: 
oozie-4.4.0-SNAPSHOT/embedded-oozie-server/webapp/docs/core/apidocs/src-html/org/apache/oozie/service/UUIDService.html
 longer than 100 characters.
[WARNING] Entry: 
oozie-4.4.0-SNAPSHOT/embedded-oozie-server/webapp/docs/core/apidocs/src-html/org/apache/oozie/service/UUIDService.ApplicationType.html
 longer than 100 characters.
[WARNING] Entry: 
oozie-4.4.0-SNAPSHOT/embedded-oozie-server/webapp/docs/core/apidocs/src-html/org/apache/oozie/service/SchemaService.html
 longer than 100 characters.
[WARNING] Entry: 

[jira] [Commented] (OOZIE-2806) Don't overwrite --files and --archives in Spark action

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-2806?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15878748#comment-15878748
 ] 

Hadoop QA commented on OOZIE-2806:
--

Testing JIRA OOZIE-2806

Cleaning local git workspace



{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:red}-1 RAW_PATCH_ANALYSIS{color}
.{color:green}+1{color} the patch does not introduce any @author tags
.{color:green}+1{color} the patch does not introduce any tabs
.{color:green}+1{color} the patch does not introduce any trailing spaces
.{color:green}+1{color} the patch does not introduce any line longer than 
132
.{color:red}-1{color} the patch does not add/modify any testcase
{color:green}+1 RAT{color}
.{color:green}+1{color} the patch does not seem to introduce new RAT 
warnings
{color:green}+1 JAVADOC{color}
.{color:green}+1{color} the patch does not seem to introduce new Javadoc 
warnings
{color:green}+1 COMPILE{color}
.{color:green}+1{color} HEAD compiles
.{color:green}+1{color} patch compiles
.{color:green}+1{color} the patch does not seem to introduce new javac 
warnings
{color:green}+1{color} There are no new bugs found in total.
.{color:green}+1{color} There are no new bugs found in [client].
.{color:green}+1{color} There are no new bugs found in [sharelib/hive].
.{color:green}+1{color} There are no new bugs found in [sharelib/sqoop].
.{color:green}+1{color} There are no new bugs found in [sharelib/spark].
.{color:green}+1{color} There are no new bugs found in [sharelib/streaming].
.{color:green}+1{color} There are no new bugs found in [sharelib/hive2].
.{color:green}+1{color} There are no new bugs found in [sharelib/oozie].
.{color:green}+1{color} There are no new bugs found in [sharelib/hcatalog].
.{color:green}+1{color} There are no new bugs found in [sharelib/pig].
.{color:green}+1{color} There are no new bugs found in [sharelib/distcp].
.{color:green}+1{color} There are no new bugs found in [docs].
.{color:green}+1{color} There are no new bugs found in [server].
.{color:green}+1{color} There are no new bugs found in [core].
.{color:green}+1{color} There are no new bugs found in 
[hadooplibs/hadoop-utils-2].
.{color:green}+1{color} There are no new bugs found in [examples].
.{color:green}+1{color} There are no new bugs found in [tools].
{color:green}+1 BACKWARDS_COMPATIBILITY{color}
.{color:green}+1{color} the patch does not change any JPA 
Entity/Colum/Basic/Lob/Transient annotations
.{color:green}+1{color} the patch does not modify JPA files
{color:red}-1 TESTS{color} - patch does not compile, cannot run testcases
{color:green}+1 DISTRO{color}
.{color:green}+1{color} distro tarball builds with the patch 


{color:red}*-1 Overall result, please check the reported -1(s)*{color}


The full output of the test-patch run is available at

.   https://builds.apache.org/job/oozie-trunk-precommit-build/3655/

> Don't overwrite --files and --archives in Spark action
> --
>
> Key: OOZIE-2806
> URL: https://issues.apache.org/jira/browse/OOZIE-2806
> Project: Oozie
>  Issue Type: Bug
>Reporter: Peter Cseh
>Assignee: Peter Cseh
> Attachments: OOZIE-2806-001.patch
>
>
> Since OOZIE-2547 we're using --files and --archives to pass jars to Spark.
> They should be merged with user-defined values if there are any.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Build failed in Jenkins: oozie-trunk-precommit-build #3655

[Apache Jenkins Server]

See 


--
[...truncated 1.50 MB...]
[WARNING] Entry: 
oozie-4.4.0-SNAPSHOT/embedded-oozie-server/webapp/docs/core/apidocs/org/apache/oozie/executor/jpa/CoordActionsRunningGetJPAExecutor.html
 longer than 100 characters.
[WARNING] Entry: 
oozie-4.4.0-SNAPSHOT/embedded-oozie-server/webapp/docs/core/apidocs/org/apache/oozie/executor/jpa/BundleJobQueryExecutor.BundleJobQuery.html
 longer than 100 characters.
[WARNING] Entry: 
oozie-4.4.0-SNAPSHOT/embedded-oozie-server/webapp/docs/core/apidocs/org/apache/oozie/executor/jpa/CoordinatorJobGetForUserJPAExecutor.html
 longer than 100 characters.
[WARNING] Entry: 
oozie-4.4.0-SNAPSHOT/embedded-oozie-server/webapp/docs/core/apidocs/org/apache/oozie/executor/jpa/package-summary.html
 longer than 100 characters.
[WARNING] Entry: 
oozie-4.4.0-SNAPSHOT/embedded-oozie-server/webapp/docs/core/apidocs/org/apache/oozie/executor/jpa/CoordJobInsertJPAExecutor.html
 longer than 100 characters.
[WARNING] Entry: 
oozie-4.4.0-SNAPSHOT/embedded-oozie-server/webapp/docs/core/apidocs/org/apache/oozie/executor/jpa/CoordActionGetForStartJPAExecutor.html
 longer than 100 characters.
[WARNING] Entry: 
oozie-4.4.0-SNAPSHOT/embedded-oozie-server/webapp/docs/core/apidocs/org/apache/oozie/executor/jpa/CoordJobGetActionsNotCompletedJPAExecutor.html
 longer than 100 characters.
[WARNING] Entry: 
oozie-4.4.0-SNAPSHOT/embedded-oozie-server/webapp/docs/core/apidocs/org/apache/oozie/executor/jpa/BundleJobInfoGetJPAExecutor.html
 longer than 100 characters.
[WARNING] Entry: 
oozie-4.4.0-SNAPSHOT/embedded-oozie-server/webapp/docs/core/apidocs/org/apache/oozie/executor/jpa/BundleJobInsertJPAExecutor.html
 longer than 100 characters.
[WARNING] Entry: 
oozie-4.4.0-SNAPSHOT/embedded-oozie-server/webapp/docs/core/apidocs/org/apache/oozie/executor/jpa/CoordJobGetReadyActionsJPAExecutor.html
 longer than 100 characters.
[WARNING] Entry: 
oozie-4.4.0-SNAPSHOT/embedded-oozie-server/webapp/docs/core/apidocs/org/apache/oozie/WorkflowsInfo.html
 longer than 100 characters.
[WARNING] Entry: 
oozie-4.4.0-SNAPSHOT/embedded-oozie-server/webapp/docs/core/apidocs/org/apache/oozie/jms/DefaultConnectionContext.html
 longer than 100 characters.
[WARNING] Entry: 
oozie-4.4.0-SNAPSHOT/embedded-oozie-server/webapp/docs/core/apidocs/org/apache/oozie/jms/JMSExceptionListener.html
 longer than 100 characters.
[WARNING] Entry: 
oozie-4.4.0-SNAPSHOT/embedded-oozie-server/webapp/docs/core/apidocs/org/apache/oozie/jms/class-use/DefaultConnectionContext.html
 longer than 100 characters.
[WARNING] Entry: 
oozie-4.4.0-SNAPSHOT/embedded-oozie-server/webapp/docs/core/apidocs/org/apache/oozie/jms/class-use/JMSExceptionListener.html
 longer than 100 characters.
[WARNING] Entry: 
oozie-4.4.0-SNAPSHOT/embedded-oozie-server/webapp/docs/core/apidocs/org/apache/oozie/jms/class-use/MessageReceiver.html
 longer than 100 characters.
[WARNING] Entry: 
oozie-4.4.0-SNAPSHOT/embedded-oozie-server/webapp/docs/core/apidocs/org/apache/oozie/jms/class-use/MessageHandler.html
 longer than 100 characters.
[WARNING] Entry: 
oozie-4.4.0-SNAPSHOT/embedded-oozie-server/webapp/docs/core/apidocs/org/apache/oozie/jms/class-use/JMSSLAEventListener.html
 longer than 100 characters.
[WARNING] Entry: 
oozie-4.4.0-SNAPSHOT/embedded-oozie-server/webapp/docs/core/apidocs/org/apache/oozie/jms/class-use/ConnectionContext.html
 longer than 100 characters.
[WARNING] Entry: 
oozie-4.4.0-SNAPSHOT/embedded-oozie-server/webapp/docs/core/apidocs/org/apache/oozie/jms/class-use/JMSConnectionInfo.html
 longer than 100 characters.
[WARNING] Entry: 
oozie-4.4.0-SNAPSHOT/embedded-oozie-server/webapp/docs/core/apidocs/org/apache/oozie/jms/class-use/JMSJobEventListener.html
 longer than 100 characters.
[WARNING] Entry: 
oozie-4.4.0-SNAPSHOT/embedded-oozie-server/webapp/docs/core/apidocs/org/apache/oozie/jms/MessageReceiver.html
 longer than 100 characters.
[WARNING] Entry: 
oozie-4.4.0-SNAPSHOT/embedded-oozie-server/webapp/docs/core/apidocs/org/apache/oozie/jms/package-use.html
 longer than 100 characters.
[WARNING] Entry: 
oozie-4.4.0-SNAPSHOT/embedded-oozie-server/webapp/docs/core/apidocs/org/apache/oozie/jms/MessageHandler.html
 longer than 100 characters.
[WARNING] Entry: 
oozie-4.4.0-SNAPSHOT/embedded-oozie-server/webapp/docs/core/apidocs/org/apache/oozie/jms/JMSSLAEventListener.html
 longer than 100 characters.
[WARNING] Entry: 
oozie-4.4.0-SNAPSHOT/embedded-oozie-server/webapp/docs/core/apidocs/org/apache/oozie/jms/package-tree.html
 longer than 100 characters.
[WARNING] Entry: 
oozie-4.4.0-SNAPSHOT/embedded-oozie-server/webapp/docs/core/apidocs/org/apache/oozie/jms/package-frame.html
 longer than 100 characters.
[WARNING] Entry: 
oozie-4.4.0-SNAPSHOT/embedded-oozie-server/webapp/docs/core/apidocs/org/apache/oozie/jms/ConnectionContext.html
 longer than 100 characters.
[WARNING] Entry: 

[jira] [Commented] (OOZIE-2803) Mask passwords when printing out configs/args in MapReduceMain and SparkMain

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-2803?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15878368#comment-15878368
 ] 

Hadoop QA commented on OOZIE-2803:
--

Testing JIRA OOZIE-2803

Cleaning local git workspace



{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:red}-1 RAW_PATCH_ANALYSIS{color}
.{color:green}+1{color} the patch does not introduce any @author tags
.{color:green}+1{color} the patch does not introduce any tabs
.{color:green}+1{color} the patch does not introduce any trailing spaces
.{color:red}-1{color} the patch contains 11 line(s) longer than 132 
characters
.{color:green}+1{color} the patch does adds/modifies 3 testcase(s)
{color:green}+1 RAT{color}
.{color:green}+1{color} the patch does not seem to introduce new RAT 
warnings
{color:green}+1 JAVADOC{color}
.{color:green}+1{color} the patch does not seem to introduce new Javadoc 
warnings
{color:green}+1 COMPILE{color}
.{color:green}+1{color} HEAD compiles
.{color:green}+1{color} patch compiles
.{color:green}+1{color} the patch does not seem to introduce new javac 
warnings
{color:orange}0{color} There are [1] new bugs found in total that would be nice 
to have fixed.
.{color:green}+1{color} There are no new bugs found in [server].
.{color:green}+1{color} There are no new bugs found in [client].
.{color:green}+1{color} There are no new bugs found in [core].
.{color:green}+1{color} There are no new bugs found in [docs].
.{color:green}+1{color} There are no new bugs found in 
[hadooplibs/hadoop-utils-2].
.{color:green}+1{color} There are no new bugs found in [tools].
.{color:green}+1{color} There are no new bugs found in [examples].
.{color:green}+1{color} There are no new bugs found in [sharelib/streaming].
.{color:green}+1{color} There are no new bugs found in [sharelib/sqoop].
.{color:green}+1{color} There are no new bugs found in [sharelib/distcp].
.{color:orange}0{color} There are [1] new bugs found in [sharelib/oozie] 
that would be nice to have fixed.
.You can find the FindBugs diff here: sharelib/oozie/findbugs-new.html
.{color:green}+1{color} There are no new bugs found in [sharelib/hcatalog].
.{color:green}+1{color} There are no new bugs found in [sharelib/hive].
.{color:green}+1{color} There are no new bugs found in [sharelib/hive2].
.{color:green}+1{color} There are no new bugs found in [sharelib/pig].
.{color:green}+1{color} There are no new bugs found in [sharelib/spark].
{color:green}+1 BACKWARDS_COMPATIBILITY{color}
.{color:green}+1{color} the patch does not change any JPA 
Entity/Colum/Basic/Lob/Transient annotations
.{color:green}+1{color} the patch does not modify JPA files
{color:green}+1 TESTS{color}
.Tests run: 1886
{color:green}+1 DISTRO{color}
.{color:green}+1{color} distro tarball builds with the patch 


{color:red}*-1 Overall result, please check the reported -1(s)*{color}


The full output of the test-patch run is available at

.   https://builds.apache.org/job/oozie-trunk-precommit-build/3654/

> Mask passwords when printing out configs/args in MapReduceMain and SparkMain
> 
>
> Key: OOZIE-2803
> URL: https://issues.apache.org/jira/browse/OOZIE-2803
> Project: Oozie
>  Issue Type: Bug
>  Components: action
>Reporter: Peter Bacsko
>Assignee: Peter Bacsko
>Priority: Critical
> Attachments: OOZIE-2803-001.patch, OOZIE-2803-002.patch, 
> OOZIE-2803-003.patch, OOZIE-2803-004.patch
>
>
> Sometimes passwords are displayed in both MapReduce and Spark action.
> *MapReduce*: when using {{HADOOP_CREDSTORE_PASSWORD}}, it must be passed to 
> some Hadoop-specific config values, like {{mapred.child.env}}. This is easy 
> to fix because we already have a method {{logMasking()}} where you can define 
> a {{maskSet}} which contains a list of property keys to be masked.
> Note that this is not necessarily the perfect solution, since you can pass 
> multiple env. vars separated by a colon, and only the password specific parts 
> should be masked. But we need a working solution relatively quickly - later 
> we can enhance this, eg. we can re-use {{PasswordMasker}} in some way (right 
> now it only works with {{Map}}).
> *Spark*:  for Spark, we have to pass passwords like this:
> {{--conf spark.executorEnv.HADOOP_CREDSTORE_PASSWORD= password>}}
> The Spark arguments are printed in {{SparkMain.run()}}. There is already a 
> code in {{LauncherMapper.printArgs()}} which deals with situations like this, 
> but it's not perfect because it only works if the args look something like 
> {{--password pwd123}}. So if a single arg contains a password, it doesn't 
> work, therefore 

Build failed in Jenkins: oozie-trunk-precommit-build #3654

[Apache Jenkins Server]

See 


--
[...truncated 1.53 MB...]
[WARNING] Entry: 
oozie-4.4.0-SNAPSHOT/embedded-oozie-server/webapp/docs/core/apidocs/src-html/org/apache/oozie/action/hadoop/HCatCredentialHelper.html
 longer than 100 characters.
[WARNING] Entry: 
oozie-4.4.0-SNAPSHOT/embedded-oozie-server/webapp/docs/core/apidocs/src-html/org/apache/oozie/action/hadoop/LauncherMapperHelper.html
 longer than 100 characters.
[WARNING] Entry: 
oozie-4.4.0-SNAPSHOT/embedded-oozie-server/webapp/docs/core/apidocs/src-html/org/apache/oozie/action/hadoop/Credentials.html
 longer than 100 characters.
[WARNING] Entry: 
oozie-4.4.0-SNAPSHOT/embedded-oozie-server/webapp/docs/core/apidocs/src-html/org/apache/oozie/action/hadoop/ShellActionExecutor.html
 longer than 100 characters.
[WARNING] Entry: 
oozie-4.4.0-SNAPSHOT/embedded-oozie-server/webapp/docs/core/apidocs/src-html/org/apache/oozie/action/hadoop/ScriptLanguageActionExecutor.html
 longer than 100 characters.
[WARNING] Entry: 
oozie-4.4.0-SNAPSHOT/embedded-oozie-server/webapp/docs/core/apidocs/src-html/org/apache/oozie/action/hadoop/PigActionExecutor.html
 longer than 100 characters.
[WARNING] Entry: 
oozie-4.4.0-SNAPSHOT/embedded-oozie-server/webapp/docs/core/apidocs/src-html/org/apache/oozie/action/hadoop/HCatCredentials.html
 longer than 100 characters.
[WARNING] Entry: 
oozie-4.4.0-SNAPSHOT/embedded-oozie-server/webapp/docs/core/apidocs/src-html/org/apache/oozie/action/hadoop/JavaActionExecutor.html
 longer than 100 characters.
[WARNING] Entry: 
oozie-4.4.0-SNAPSHOT/embedded-oozie-server/webapp/docs/core/apidocs/src-html/org/apache/oozie/action/hadoop/CredentialException.html
 longer than 100 characters.
[WARNING] Entry: 
oozie-4.4.0-SNAPSHOT/embedded-oozie-server/webapp/docs/core/apidocs/src-html/org/apache/oozie/action/hadoop/FsELFunctions.html
 longer than 100 characters.
[WARNING] Entry: 
oozie-4.4.0-SNAPSHOT/embedded-oozie-server/webapp/docs/core/apidocs/src-html/org/apache/oozie/action/hadoop/DistcpActionExecutor.html
 longer than 100 characters.
[WARNING] Entry: 
oozie-4.4.0-SNAPSHOT/embedded-oozie-server/webapp/docs/core/apidocs/src-html/org/apache/oozie/action/hadoop/HadoopELFunctions.html
 longer than 100 characters.
[WARNING] Entry: 
oozie-4.4.0-SNAPSHOT/embedded-oozie-server/webapp/docs/core/apidocs/src-html/org/apache/oozie/action/hadoop/CredentialsProperties.html
 longer than 100 characters.
[WARNING] Entry: 
oozie-4.4.0-SNAPSHOT/embedded-oozie-server/webapp/docs/core/apidocs/src-html/org/apache/oozie/action/hadoop/MapReduceActionExecutor.html
 longer than 100 characters.
[WARNING] Entry: 
oozie-4.4.0-SNAPSHOT/embedded-oozie-server/webapp/docs/core/apidocs/src-html/org/apache/oozie/action/hadoop/CredentialsProvider.html
 longer than 100 characters.
[WARNING] Entry: 
oozie-4.4.0-SNAPSHOT/embedded-oozie-server/webapp/docs/core/apidocs/src-html/org/apache/oozie/action/hadoop/HbaseCredentials.html
 longer than 100 characters.
[WARNING] Entry: 
oozie-4.4.0-SNAPSHOT/embedded-oozie-server/webapp/docs/core/apidocs/src-html/org/apache/oozie/action/hadoop/FsActionExecutor.html
 longer than 100 characters.
[WARNING] Entry: 
oozie-4.4.0-SNAPSHOT/embedded-oozie-server/webapp/docs/core/apidocs/src-html/org/apache/oozie/action/hadoop/Hive2ActionExecutor.html
 longer than 100 characters.
[WARNING] Entry: 
oozie-4.4.0-SNAPSHOT/embedded-oozie-server/webapp/docs/core/apidocs/src-html/org/apache/oozie/action/hadoop/SparkActionExecutor.html
 longer than 100 characters.
[WARNING] Entry: 
oozie-4.4.0-SNAPSHOT/embedded-oozie-server/webapp/docs/core/apidocs/src-html/org/apache/oozie/action/hadoop/HiveActionExecutor.html
 longer than 100 characters.
[WARNING] Entry: 
oozie-4.4.0-SNAPSHOT/embedded-oozie-server/webapp/docs/core/apidocs/src-html/org/apache/oozie/action/hadoop/OozieJobInfo.html
 longer than 100 characters.
[WARNING] Entry: 
oozie-4.4.0-SNAPSHOT/embedded-oozie-server/webapp/docs/core/apidocs/src-html/org/apache/oozie/action/ActionExecutor.Context.html
 longer than 100 characters.
[WARNING] Entry: 
oozie-4.4.0-SNAPSHOT/embedded-oozie-server/webapp/docs/core/apidocs/src-html/org/apache/oozie/CoordinatorEngine.html
 longer than 100 characters.
[WARNING] Entry: 
oozie-4.4.0-SNAPSHOT/embedded-oozie-server/webapp/docs/core/apidocs/src-html/org/apache/oozie/jms/MessageReceiver.html
 longer than 100 characters.
[WARNING] Entry: 
oozie-4.4.0-SNAPSHOT/embedded-oozie-server/webapp/docs/core/apidocs/src-html/org/apache/oozie/jms/DefaultConnectionContext.html
 longer than 100 characters.
[WARNING] Entry: 
oozie-4.4.0-SNAPSHOT/embedded-oozie-server/webapp/docs/core/apidocs/src-html/org/apache/oozie/jms/ConnectionContext.html
 longer than 100 characters.
[WARNING] Entry: 
oozie-4.4.0-SNAPSHOT/embedded-oozie-server/webapp/docs/core/apidocs/src-html/org/apache/oozie/jms/JMSExceptionListener.html
 longer than 100 characters.
[WARNING] Entry: 

[jira] [Updated] (OOZIE-2802) Spark action failure on Spark 2.1.0 due to duplicate sharelibs

[Peter Cseh (JIRA)]

 [ 
https://issues.apache.org/jira/browse/OOZIE-2802?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Peter Cseh updated OOZIE-2802:
--
Attachment: OOZIE-2802-001.patch

> Spark action failure on Spark 2.1.0 due to duplicate sharelibs
> --
>
> Key: OOZIE-2802
> URL: https://issues.apache.org/jira/browse/OOZIE-2802
> Project: Oozie
>  Issue Type: Bug
>Reporter: Satish Subhashrao Saley
>Assignee: Peter Cseh
> Attachments: OOZIE-2802-001.patch, spark-2.1-failure-launcherlib.txt
>
>
> There are some same jars in launcher lib and sharelib. These jars are causing 
> failures in spark action for spark 2.1.0. SPARK-18099 added an exception to 
> be thrown if same file gets added multiple times to distributed cache.
> {code}
> $ hdfs dfs -ls /user/saley/share/lib/launcher_20170216150752/oozie
> 17/02/16 15:14:22 WARN util.NativeCodeLoader: Unable to load native-hadoop 
> library for your platform... using builtin-java classes where applicable
> Found 3 items
> -rwxr-xr-x   1 saley supergroup  11871 2017-02-16 15:07 
> /user/saley/share/lib/launcher_20170216150752/oozie/oozie-hadoop-utils-hadoop-2-4.4.0-SNAPSHOT.jar
> -rwxr-xr-x   1 saley supergroup  17514 2017-02-16 15:07 
> /user/saley/share/lib/launcher_20170216150752/oozie/oozie-sharelib-hcatalog-4.4.0-SNAPSHOT.jar
> -rwxr-xr-x   1 saley supergroup  55092 2017-02-16 15:07 
> /user/saley/share/lib/launcher_20170216150752/oozie/oozie-sharelib-oozie-4.4.0-SNAPSHOT.jar
> $ hdfs dfs -ls /user/saley/share/lib/lib_20170216150706/oozie
> 17/02/16 15:14:39 WARN util.NativeCodeLoader: Unable to load native-hadoop 
> library for your platform... using builtin-java classes where applicable
> Found 3 items
> -rw-r--r--   1 saley supergroup  16046 2017-02-16 15:07 
> /user/saley/share/lib/lib_20170216150706/oozie/json-simple-1.1.jar
> -rw-r--r--   1 saley supergroup  11871 2017-02-16 15:07 
> /user/saley/share/lib/lib_20170216150706/oozie/oozie-hadoop-utils-hadoop-2-4.4.0-SNAPSHOT.jar
> -rw-r--r--   1 saley supergroup  55092 2017-02-16 15:07 
> /user/saley/share/lib/lib_20170216150706/oozie/oozie-sharelib-oozie-4.4.0-SNAPSHOT.jar
> {code}
> The file with same name can't be in twice in any of *--files , --archives, 
> --jars* (ie one file in both --files and --jars) and can't be in --files 
> twice and can't be in --archives twice. It is allowed to be in --jars twice.
> Possible solutions:
> 1. While specifying the --files  options, filter out the duplicate jars
> 2. Separate out jars and files. Use --jars options to distribute jars and 
> --files for files. Also remember to remove duplicates from --files, and make 
> sure that there is nothing common in --jars and --files.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Assigned] (OOZIE-2802) Spark action failure on Spark 2.1.0 due to duplicate sharelibs

[Peter Cseh (JIRA)]

 [ 
https://issues.apache.org/jira/browse/OOZIE-2802?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Peter Cseh reassigned OOZIE-2802:
-

Assignee: Peter Cseh

> Spark action failure on Spark 2.1.0 due to duplicate sharelibs
> --
>
> Key: OOZIE-2802
> URL: https://issues.apache.org/jira/browse/OOZIE-2802
> Project: Oozie
>  Issue Type: Bug
>Reporter: Satish Subhashrao Saley
>Assignee: Peter Cseh
> Attachments: spark-2.1-failure-launcherlib.txt
>
>
> There are some same jars in launcher lib and sharelib. These jars are causing 
> failures in spark action for spark 2.1.0. SPARK-18099 added an exception to 
> be thrown if same file gets added multiple times to distributed cache.
> {code}
> $ hdfs dfs -ls /user/saley/share/lib/launcher_20170216150752/oozie
> 17/02/16 15:14:22 WARN util.NativeCodeLoader: Unable to load native-hadoop 
> library for your platform... using builtin-java classes where applicable
> Found 3 items
> -rwxr-xr-x   1 saley supergroup  11871 2017-02-16 15:07 
> /user/saley/share/lib/launcher_20170216150752/oozie/oozie-hadoop-utils-hadoop-2-4.4.0-SNAPSHOT.jar
> -rwxr-xr-x   1 saley supergroup  17514 2017-02-16 15:07 
> /user/saley/share/lib/launcher_20170216150752/oozie/oozie-sharelib-hcatalog-4.4.0-SNAPSHOT.jar
> -rwxr-xr-x   1 saley supergroup  55092 2017-02-16 15:07 
> /user/saley/share/lib/launcher_20170216150752/oozie/oozie-sharelib-oozie-4.4.0-SNAPSHOT.jar
> $ hdfs dfs -ls /user/saley/share/lib/lib_20170216150706/oozie
> 17/02/16 15:14:39 WARN util.NativeCodeLoader: Unable to load native-hadoop 
> library for your platform... using builtin-java classes where applicable
> Found 3 items
> -rw-r--r--   1 saley supergroup  16046 2017-02-16 15:07 
> /user/saley/share/lib/lib_20170216150706/oozie/json-simple-1.1.jar
> -rw-r--r--   1 saley supergroup  11871 2017-02-16 15:07 
> /user/saley/share/lib/lib_20170216150706/oozie/oozie-hadoop-utils-hadoop-2-4.4.0-SNAPSHOT.jar
> -rw-r--r--   1 saley supergroup  55092 2017-02-16 15:07 
> /user/saley/share/lib/lib_20170216150706/oozie/oozie-sharelib-oozie-4.4.0-SNAPSHOT.jar
> {code}
> The file with same name can't be in twice in any of *--files , --archives, 
> --jars* (ie one file in both --files and --jars) and can't be in --files 
> twice and can't be in --archives twice. It is allowed to be in --jars twice.
> Possible solutions:
> 1. While specifying the --files  options, filter out the duplicate jars
> 2. Separate out jars and files. Use --jars options to distribute jars and 
> --files for files. Also remember to remove duplicates from --files, and make 
> sure that there is nothing common in --jars and --files.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (OOZIE-2806) Don't overwrite --files and --archives in Spark action

[Peter Cseh (JIRA)]

 [ 
https://issues.apache.org/jira/browse/OOZIE-2806?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Peter Cseh updated OOZIE-2806:
--
Attachment: OOZIE-2806-001.patch

Attaching first fix.


> Don't overwrite --files and --archives in Spark action
> --
>
> Key: OOZIE-2806
> URL: https://issues.apache.org/jira/browse/OOZIE-2806
> Project: Oozie
>  Issue Type: Bug
>Reporter: Peter Cseh
>Assignee: Peter Cseh
> Attachments: OOZIE-2806-001.patch
>
>
> Since OOZIE-2547 we're using --files and --archives to pass jars to Spark.
> They should be merged with user-defined values if there are any.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Assigned] (OOZIE-2806) Don't overwrite --files and --archives in Spark action

[Peter Cseh (JIRA)]

 [ 
https://issues.apache.org/jira/browse/OOZIE-2806?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Peter Cseh reassigned OOZIE-2806:
-

Assignee: Peter Cseh

> Don't overwrite --files and --archives in Spark action
> --
>
> Key: OOZIE-2806
> URL: https://issues.apache.org/jira/browse/OOZIE-2806
> Project: Oozie
>  Issue Type: Bug
>Reporter: Peter Cseh
>Assignee: Peter Cseh
>
> Since OOZIE-2547 we're using --files and --archives to pass jars to Spark.
> They should be merged with user-defined values if there are any.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (OOZIE-2802) Spark action failure on Spark 2.1.0 due to duplicate sharelibs

[Peter Cseh (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-2802?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15878297#comment-15878297
 ] 

Peter Cseh commented on OOZIE-2802:
---

Solution 1) seems cleaner.

> Spark action failure on Spark 2.1.0 due to duplicate sharelibs
> --
>
> Key: OOZIE-2802
> URL: https://issues.apache.org/jira/browse/OOZIE-2802
> Project: Oozie
>  Issue Type: Bug
>Reporter: Satish Subhashrao Saley
> Attachments: spark-2.1-failure-launcherlib.txt
>
>
> There are some same jars in launcher lib and sharelib. These jars are causing 
> failures in spark action for spark 2.1.0. SPARK-18099 added an exception to 
> be thrown if same file gets added multiple times to distributed cache.
> {code}
> $ hdfs dfs -ls /user/saley/share/lib/launcher_20170216150752/oozie
> 17/02/16 15:14:22 WARN util.NativeCodeLoader: Unable to load native-hadoop 
> library for your platform... using builtin-java classes where applicable
> Found 3 items
> -rwxr-xr-x   1 saley supergroup  11871 2017-02-16 15:07 
> /user/saley/share/lib/launcher_20170216150752/oozie/oozie-hadoop-utils-hadoop-2-4.4.0-SNAPSHOT.jar
> -rwxr-xr-x   1 saley supergroup  17514 2017-02-16 15:07 
> /user/saley/share/lib/launcher_20170216150752/oozie/oozie-sharelib-hcatalog-4.4.0-SNAPSHOT.jar
> -rwxr-xr-x   1 saley supergroup  55092 2017-02-16 15:07 
> /user/saley/share/lib/launcher_20170216150752/oozie/oozie-sharelib-oozie-4.4.0-SNAPSHOT.jar
> $ hdfs dfs -ls /user/saley/share/lib/lib_20170216150706/oozie
> 17/02/16 15:14:39 WARN util.NativeCodeLoader: Unable to load native-hadoop 
> library for your platform... using builtin-java classes where applicable
> Found 3 items
> -rw-r--r--   1 saley supergroup  16046 2017-02-16 15:07 
> /user/saley/share/lib/lib_20170216150706/oozie/json-simple-1.1.jar
> -rw-r--r--   1 saley supergroup  11871 2017-02-16 15:07 
> /user/saley/share/lib/lib_20170216150706/oozie/oozie-hadoop-utils-hadoop-2-4.4.0-SNAPSHOT.jar
> -rw-r--r--   1 saley supergroup  55092 2017-02-16 15:07 
> /user/saley/share/lib/lib_20170216150706/oozie/oozie-sharelib-oozie-4.4.0-SNAPSHOT.jar
> {code}
> The file with same name can't be in twice in any of *--files , --archives, 
> --jars* (ie one file in both --files and --jars) and can't be in --files 
> twice and can't be in --archives twice. It is allowed to be in --jars twice.
> Possible solutions:
> 1. While specifying the --files  options, filter out the duplicate jars
> 2. Separate out jars and files. Use --jars options to distribute jars and 
> --files for files. Also remember to remove duplicates from --files, and make 
> sure that there is nothing common in --jars and --files.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Created] (OOZIE-2806) Don't overwrite --files and --archives in Spark action

[Peter Cseh (JIRA)]

Peter Cseh created OOZIE-2806:
-

 Summary: Don't overwrite --files and --archives in Spark action
 Key: OOZIE-2806
 URL: https://issues.apache.org/jira/browse/OOZIE-2806
 Project: Oozie
  Issue Type: Bug
Reporter: Peter Cseh


Since OOZIE-2547 we're using --files and --archives to pass jars to Spark.
They should be merged with user-defined values if there are any.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (OOZIE-2803) Mask passwords when printing out configs/args in MapReduceMain and SparkMain

[Peter Bacsko (JIRA)]

 [ 
https://issues.apache.org/jira/browse/OOZIE-2803?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Peter Bacsko updated OOZIE-2803:

Attachment: OOZIE-2803-004.patch

> Mask passwords when printing out configs/args in MapReduceMain and SparkMain
> 
>
> Key: OOZIE-2803
> URL: https://issues.apache.org/jira/browse/OOZIE-2803
> Project: Oozie
>  Issue Type: Bug
>  Components: action
>Reporter: Peter Bacsko
>Assignee: Peter Bacsko
>Priority: Critical
> Attachments: OOZIE-2803-001.patch, OOZIE-2803-002.patch, 
> OOZIE-2803-003.patch, OOZIE-2803-004.patch
>
>
> Sometimes passwords are displayed in both MapReduce and Spark action.
> *MapReduce*: when using {{HADOOP_CREDSTORE_PASSWORD}}, it must be passed to 
> some Hadoop-specific config values, like {{mapred.child.env}}. This is easy 
> to fix because we already have a method {{logMasking()}} where you can define 
> a {{maskSet}} which contains a list of property keys to be masked.
> Note that this is not necessarily the perfect solution, since you can pass 
> multiple env. vars separated by a colon, and only the password specific parts 
> should be masked. But we need a working solution relatively quickly - later 
> we can enhance this, eg. we can re-use {{PasswordMasker}} in some way (right 
> now it only works with {{Map}}).
> *Spark*:  for Spark, we have to pass passwords like this:
> {{--conf spark.executorEnv.HADOOP_CREDSTORE_PASSWORD= password>}}
> The Spark arguments are printed in {{SparkMain.run()}}. There is already a 
> code in {{LauncherMapper.printArgs()}} which deals with situations like this, 
> but it's not perfect because it only works if the args look something like 
> {{--password pwd123}}. So if a single arg contains a password, it doesn't 
> work, therefore we need a different approach here.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (OOZIE-2803) Mask passwords when printing out configs/args in MapReduceMain and SparkMain

[Abhishek Bafna (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-2803?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15878103#comment-15878103
 ] 

Abhishek Bafna commented on OOZIE-2803:
---

[~pbacsko] Thanks for pointing to the previous discussion. I think we can keep 
it same.

For the JavaDoc: {{@param String}} and {{@param Input}} there should be 
parameter names, which is missing.


> Mask passwords when printing out configs/args in MapReduceMain and SparkMain
> 
>
> Key: OOZIE-2803
> URL: https://issues.apache.org/jira/browse/OOZIE-2803
> Project: Oozie
>  Issue Type: Bug
>  Components: action
>Reporter: Peter Bacsko
>Assignee: Peter Bacsko
>Priority: Critical
> Attachments: OOZIE-2803-001.patch, OOZIE-2803-002.patch, 
> OOZIE-2803-003.patch
>
>
> Sometimes passwords are displayed in both MapReduce and Spark action.
> *MapReduce*: when using {{HADOOP_CREDSTORE_PASSWORD}}, it must be passed to 
> some Hadoop-specific config values, like {{mapred.child.env}}. This is easy 
> to fix because we already have a method {{logMasking()}} where you can define 
> a {{maskSet}} which contains a list of property keys to be masked.
> Note that this is not necessarily the perfect solution, since you can pass 
> multiple env. vars separated by a colon, and only the password specific parts 
> should be masked. But we need a working solution relatively quickly - later 
> we can enhance this, eg. we can re-use {{PasswordMasker}} in some way (right 
> now it only works with {{Map}}).
> *Spark*:  for Spark, we have to pass passwords like this:
> {{--conf spark.executorEnv.HADOOP_CREDSTORE_PASSWORD= password>}}
> The Spark arguments are printed in {{SparkMain.run()}}. There is already a 
> code in {{LauncherMapper.printArgs()}} which deals with situations like this, 
> but it's not perfect because it only works if the args look something like 
> {{--password pwd123}}. So if a single arg contains a password, it doesn't 
> work, therefore we need a different approach here.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (OOZIE-2803) Mask passwords when printing out configs/args in MapReduceMain and SparkMain

[Peter Bacsko (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-2803?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15878078#comment-15878078
 ] 

Peter Bacsko commented on OOZIE-2803:
-

Thanks for the comment [~abhishekbafna].

1. What's wrong with the JavaDoc? I think it gives an accurate description of 
what the methods are doing. Are there spelling or grammatical errors?
2. As for PasswordMasker not being an utility class - I completely agree with 
you, originally, [~andras.piros] did not want to write static methods, his 
reasoning was the following: 
https://issues.apache.org/jira/browse/OOZIE-1814?focusedCommentId=15528763=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15528763

Nevertheless I can make it static, I don't think it's a big deal :)


> Mask passwords when printing out configs/args in MapReduceMain and SparkMain
> 
>
> Key: OOZIE-2803
> URL: https://issues.apache.org/jira/browse/OOZIE-2803
> Project: Oozie
>  Issue Type: Bug
>  Components: action
>Reporter: Peter Bacsko
>Assignee: Peter Bacsko
>Priority: Critical
> Attachments: OOZIE-2803-001.patch, OOZIE-2803-002.patch, 
> OOZIE-2803-003.patch
>
>
> Sometimes passwords are displayed in both MapReduce and Spark action.
> *MapReduce*: when using {{HADOOP_CREDSTORE_PASSWORD}}, it must be passed to 
> some Hadoop-specific config values, like {{mapred.child.env}}. This is easy 
> to fix because we already have a method {{logMasking()}} where you can define 
> a {{maskSet}} which contains a list of property keys to be masked.
> Note that this is not necessarily the perfect solution, since you can pass 
> multiple env. vars separated by a colon, and only the password specific parts 
> should be masked. But we need a working solution relatively quickly - later 
> we can enhance this, eg. we can re-use {{PasswordMasker}} in some way (right 
> now it only works with {{Map}}).
> *Spark*:  for Spark, we have to pass passwords like this:
> {{--conf spark.executorEnv.HADOOP_CREDSTORE_PASSWORD= password>}}
> The Spark arguments are printed in {{SparkMain.run()}}. There is already a 
> code in {{LauncherMapper.printArgs()}} which deals with situations like this, 
> but it's not perfect because it only works if the args look something like 
> {{--password pwd123}}. So if a single arg contains a password, it doesn't 
> work, therefore we need a different approach here.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Jenkins build is back to normal : oozie-trunk-find-patches-available #150585

[Apache Jenkins Server]

See 

Build failed in Jenkins: oozie-trunk-find-patches-available #150584

[Apache Jenkins Server]

See 


--
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on H1 (Hadoop) in workspace 

 > git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
 > git config remote.origin.url 
 > https://git-wip-us.apache.org/repos/asf/oozie.git # timeout=10
Fetching upstream changes from https://git-wip-us.apache.org/repos/asf/oozie.git
 > git --version # timeout=10
 > git fetch --tags --progress 
 > https://git-wip-us.apache.org/repos/asf/oozie.git 
 > +refs/heads/*:refs/remotes/origin/*
 > git rev-parse refs/remotes/origin/master^{commit} # timeout=10
 > git rev-parse refs/remotes/origin/origin/master^{commit} # timeout=10
Checking out Revision 77817e73588f665cbe43b0971cd93465d40711fb 
(refs/remotes/origin/master)
 > git config core.sparsecheckout # timeout=10
 > git checkout -f 77817e73588f665cbe43b0971cd93465d40711fb
 > git rev-list 77817e73588f665cbe43b0971cd93465d40711fb # timeout=10
[oozie-trunk-find-patches-available] $ /bin/bash 
/tmp/hudson2365912912457149786.sh
mkdir: cannot create directory 
':
 File exists
  % Total% Received % Xferd  Average Speed   TimeTime Time  Current
 Dload  Upload   Total   SpentLeft  Speed
  0 00 00 0  0  0 --:--:-- --:--:-- --:--:-- 
0100 982960 982960 0   121k  0 --:--:-- --:--:-- --:--:--  
121k100  751k0  751k0 0   428k  0 --:--:--  0:00:01 --:--:--  
428k100 1279k0 1279k0 0   466k  0 --:--:--  0:00:02 --:--:--  
466k100 1639k0 1639k0 0   475k  0 --:--:--  0:00:03 --:--:--  
476k
curl: (18) transfer closed with outstanding read data remaining
Could not retrieve available patches from JIRA
Build step 'Execute shell' marked build as failure