date:20160915

[jira] Subscription: Oozie Patch Available

2016-09-15 Thread jira

Issue Subscription
Filter: Oozie Patch Available (86 issues)

Subscriber: ooziedaily

Key Summary
OOZIE-2619  Make  Hive action defaults to match hive defaults when running from 
command line
https://issues-test.apache.org/jira/browse/OOZIE-2619
OOZIE-2618  Include hive-common in hive-beeline as compile component for 
hive-server2 (hive2 action)
https://issues-test.apache.org/jira/browse/OOZIE-2618
OOZIE-2617  Read properties files in action configurations
https://issues-test.apache.org/jira/browse/OOZIE-2617
OOZIE-2616  Add Tez profile for Hive and Pig sharelibs
https://issues-test.apache.org/jira/browse/OOZIE-2616
OOZIE-2606  Set spark.yarn.jars to fix Spark 2.0 with Oozie
https://issues-test.apache.org/jira/browse/OOZIE-2606
OOZIE-2589  CompletedActionXCommand priority param is of no effect
https://issues-test.apache.org/jira/browse/OOZIE-2589
OOZIE-2588  Support getting credentials for same cluster hcat when credentials 
config is empty
https://issues-test.apache.org/jira/browse/OOZIE-2588
OOZIE-2585  Disable or remove test case 
TestPartitionDependencyManagerEhcache.testMemoryUsageAndSpeedOverflowToDisk and 
testMemoryUsageAndSpeed
https://issues-test.apache.org/jira/browse/OOZIE-2585
OOZIE-2584  Eliminate Thread.sleep() calls in TestMemoryLocks
https://issues-test.apache.org/jira/browse/OOZIE-2584
OOZIE-2583  oozie throws EL Exception when reference variable name containing 
dot
https://issues-test.apache.org/jira/browse/OOZIE-2583
OOZIE-2582  Populating external child Ids for action failures
https://issues-test.apache.org/jira/browse/OOZIE-2582
OOZIE-2573  dataset url  which contains spaces can not be handled rightly
https://issues-test.apache.org/jira/browse/OOZIE-2573
OOZIE-2572  SLA DURATION miss not shown when job is running for longer than 
expected time
https://issues-test.apache.org/jira/browse/OOZIE-2572
OOZIE-2571  Add scala.binary.version Maven property so that Scala 2.11 can be 
used
https://issues-test.apache.org/jira/browse/OOZIE-2571
OOZIE-2569  Adding yarn-site, core-site, hdfs-site and mapred-site into spark 
launcher 
https://issues-test.apache.org/jira/browse/OOZIE-2569
OOZIE-2568  SSH  action pretends to retry automaticly when it failed
https://issues-test.apache.org/jira/browse/OOZIE-2568
OOZIE-2564  Create new log4j config for unit tests so that logged messages 
contain threads
https://issues-test.apache.org/jira/browse/OOZIE-2564
OOZIE-2552  Update ActiveMQ version for security and other fixes
https://issues-test.apache.org/jira/browse/OOZIE-2552
OOZIE-2538  Update HttpClient versions to close security vulnerabilities
https://issues-test.apache.org/jira/browse/OOZIE-2538
OOZIE-2534  Versioned action libs (similar to how SharedLibs works)
https://issues-test.apache.org/jira/browse/OOZIE-2534
OOZIE-2530  Remove jline from parent and sharelib pig pom
https://issues-test.apache.org/jira/browse/OOZIE-2530
OOZIE-2528  Print out environment variables in LauncherMapper
https://issues-test.apache.org/jira/browse/OOZIE-2528
OOZIE-2521  Filter options are case sensitive for jobtye=bundle
https://issues-test.apache.org/jira/browse/OOZIE-2521
OOZIE-2518  Oozie is creating many files and directories in /tmp per day
https://issues-test.apache.org/jira/browse/OOZIE-2518
OOZIE-2517  Add support for startCreatedTime and endCreatedTime filters for 
coord and bundles
https://issues-test.apache.org/jira/browse/OOZIE-2517
OOZIE-2513  log.scan.duration should not be used for error and audit logs
https://issues-test.apache.org/jira/browse/OOZIE-2513
OOZIE-2512  ShareLibservice returns incorrect path for jar
https://issues-test.apache.org/jira/browse/OOZIE-2512
OOZIE-2508  Documentation change for Coord action rerun [OOZIE-1735]
https://issues-test.apache.org/jira/browse/OOZIE-2508
OOZIE-2501  ZK reentrant lock doesn't work for few cases
https://issues-test.apache.org/jira/browse/OOZIE-2501
OOZIE-2500  -DtestJarSimple option mentioned in minioozie doc does not work
https://issues-test.apache.org/jira/browse/OOZIE-2500
OOZIE-2499  map-reduce launcher does not need distributed files, archives 
except  jar of input/outputformat class
https://issues-test.apache.org/jira/browse/OOZIE-2499
OOZIE-2498  Oozie CallerId configuration for downstream components 
https://issues-test.apache.org/jira/browse/OOZIE-2498
OOZIE-2495  change action status from  ErrorType.NON_TRANSIENT to TRANSIENT 
when SSH action occurs AUTH_FAILED occasionally
https://issues-test.apache.org/jira/browse/OOZIE-2495
OOZIE-2491  oozie acl cannot specify group,it does`t work
https://issues-test.apache.org/jira/browse/OOZIE-2491
OOZIE

[jira] Subscription: Oozie Patch Available

2016-09-15 Thread jira

Issue Subscription
Filter: Oozie Patch Available (83 issues)

Subscriber: ooziedaily

Key Summary
OOZIE-2670  Upgrade Hbase to 1.2
https://issues.apache.org/jira/browse/OOZIE-2670
OOZIE-2668  Status update and recovery problems when coord action and its 
children not in sync
https://issues.apache.org/jira/browse/OOZIE-2668
OOZIE-2662  DB migration fails is DB is too big
https://issues.apache.org/jira/browse/OOZIE-2662
OOZIE-2661  Coordinator Action status not updated when workflow job SUSPENDED
https://issues.apache.org/jira/browse/OOZIE-2661
OOZIE-2658  --driver-class-path can overwrite the classpath in SparkMain
https://issues.apache.org/jira/browse/OOZIE-2658
OOZIE-2652  Skip trash while deleting hive table partition
https://issues.apache.org/jira/browse/OOZIE-2652
OOZIE-2650  Retry coord start on database exceptions
https://issues.apache.org/jira/browse/OOZIE-2650
OOZIE-2641  rerunning an oozie coordinator job stucks on waiting
https://issues.apache.org/jira/browse/OOZIE-2641
OOZIE-2630  Oozie Coordinator EL Functions to get first day of the week/month
https://issues.apache.org/jira/browse/OOZIE-2630
OOZIE-2619  Make  Hive action defaults to match hive defaults when running from 
command line
https://issues.apache.org/jira/browse/OOZIE-2619
OOZIE-2618  Include hive-common in hive-beeline as compile component for 
hive-server2 (hive2 action)
https://issues.apache.org/jira/browse/OOZIE-2618
OOZIE-2616  Add Tez profile for Hive and Pig sharelibs
https://issues.apache.org/jira/browse/OOZIE-2616
OOZIE-2613  Upgrade hive version from 0.13.1 to 1.2.0
https://issues.apache.org/jira/browse/OOZIE-2613
OOZIE-2606  Set spark.yarn.jars to fix Spark 2.0 with Oozie
https://issues.apache.org/jira/browse/OOZIE-2606
OOZIE-2588  Support getting credentials for same cluster hcat when credentials 
config is empty
https://issues.apache.org/jira/browse/OOZIE-2588
OOZIE-2585  Disable or remove test case 
TestPartitionDependencyManagerEhcache.testMemoryUsageAndSpeedOverflowToDisk and 
testMemoryUsageAndSpeed
https://issues.apache.org/jira/browse/OOZIE-2585
OOZIE-2582  Populating external child Ids for action failures
https://issues.apache.org/jira/browse/OOZIE-2582
OOZIE-2573  dataset url  which contains spaces can not be handled rightly
https://issues.apache.org/jira/browse/OOZIE-2573
OOZIE-2572  SLA DURATION miss not shown when job is running for longer than 
expected time
https://issues.apache.org/jira/browse/OOZIE-2572
OOZIE-2569  Adding yarn-site, core-site, hdfs-site and mapred-site into spark 
launcher 
https://issues.apache.org/jira/browse/OOZIE-2569
OOZIE-2568  SSH  action pretends to retry automaticly when it failed
https://issues.apache.org/jira/browse/OOZIE-2568
OOZIE-2564  Create new log4j config for unit tests so that logged messages 
contain threads
https://issues.apache.org/jira/browse/OOZIE-2564
OOZIE-2538  Update HttpClient versions to close security vulnerabilities
https://issues.apache.org/jira/browse/OOZIE-2538
OOZIE-2534  Versioned action libs (similar to how SharedLibs works)
https://issues.apache.org/jira/browse/OOZIE-2534
OOZIE-2528  Print out environment variables in LauncherMapper
https://issues.apache.org/jira/browse/OOZIE-2528
OOZIE-2525  SchemaChecker fails with NPE
https://issues.apache.org/jira/browse/OOZIE-2525
OOZIE-2518  Oozie is creating many files and directories in /tmp per day
https://issues.apache.org/jira/browse/OOZIE-2518
OOZIE-2513  log.scan.duration should not be used for error and audit logs
https://issues.apache.org/jira/browse/OOZIE-2513
OOZIE-2501  ZK reentrant lock doesn't work for few cases
https://issues.apache.org/jira/browse/OOZIE-2501
OOZIE-2499  map-reduce launcher does not need distributed files, archives 
except  jar of input/outputformat class
https://issues.apache.org/jira/browse/OOZIE-2499
OOZIE-2498  Oozie CallerId configuration for downstream components 
https://issues.apache.org/jira/browse/OOZIE-2498
OOZIE-2495  change action status from  ErrorType.NON_TRANSIENT to TRANSIENT 
when SSH action occurs AUTH_FAILED occasionally
https://issues.apache.org/jira/browse/OOZIE-2495
OOZIE-2491  oozie acl cannot specify group,it does`t work
https://issues.apache.org/jira/browse/OOZIE-2491
OOZIE-2487  Unable to build distro on Java 8 (on Mac)
https://issues.apache.org/jira/browse/OOZIE-2487
OOZIE-2473  Connection pool for SMTP connection
https://issues.apache.org/jira/browse/OOZIE-2473
OOZIE-2461  Workflow, Coordinator and Bundle job querying should have last 
modified filter
https://issues.apache.org/jira/browse/OOZIE-2461
OOZIE-2458  'oozie-set

Jenkins build is back to normal : oozie-trunk-find-patches-available #133765

2016-09-15 Thread Apache Jenkins Server

See

Build failed in Jenkins: oozie-trunk-find-patches-available #133764

2016-09-15 Thread Apache Jenkins Server

See 

--
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on H1 (yahoo-not-h2 Hadoop Mapreduce Hdfs Pig) in workspace 

 > git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
 > git config remote.origin.url 
 > https://git-wip-us.apache.org/repos/asf/oozie.git # timeout=10
Fetching upstream changes from https://git-wip-us.apache.org/repos/asf/oozie.git
 > git --version # timeout=10
 > git -c core.askpass=true fetch --tags --progress 
 > https://git-wip-us.apache.org/repos/asf/oozie.git 
 > +refs/heads/*:refs/remotes/origin/*
 > git rev-parse refs/remotes/origin/master^{commit} # timeout=10
 > git rev-parse refs/remotes/origin/origin/master^{commit} # timeout=10
Checking out Revision 65f59ff0ecbc13e84b88d2debd45227f8296cd4a 
(refs/remotes/origin/master)
 > git config core.sparsecheckout # timeout=10
 > git checkout -f 65f59ff0ecbc13e84b88d2debd45227f8296cd4a
 > git rev-list 65f59ff0ecbc13e84b88d2debd45227f8296cd4a # timeout=10
[oozie-trunk-find-patches-available] $ /bin/bash 
/tmp/hudson3122957547221344594.sh
mkdir: cannot create directory 
':
 File exists
  % Total% Received % Xferd  Average Speed   TimeTime Time  Current
 Dload  Upload   Total   SpentLeft  Speed

  0 00 00 0  0  0 --:--:-- --:--:-- --:--:-- 0
  0 00 00 0  0  0 --:--:-- --:--:-- --:--:-- 0
curl: (52) Empty reply from server
Could not retrieve available patches from JIRA
Build step 'Execute shell' marked build as failure

[jira] [Commented] (OOZIE-2661) Coordinator Action status not updated when workflow job SUSPENDED

2016-09-15 Thread Hadoop QA (JIRA)


[ 
https://issues.apache.org/jira/browse/OOZIE-2661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15495232#comment-15495232
 ] 

Hadoop QA commented on OOZIE-2661:
--

Testing JIRA OOZIE-2661

Cleaning local git workspace



{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:green}+1 RAW_PATCH_ANALYSIS{color}
.{color:green}+1{color} the patch does not introduce any @author tags
.{color:green}+1{color} the patch does not introduce any tabs
.{color:green}+1{color} the patch does not introduce any trailing spaces
.{color:green}+1{color} the patch does not introduce any line longer than 
132
.{color:green}+1{color} the patch does adds/modifies 1 testcase(s)
{color:green}+1 RAT{color}
.{color:green}+1{color} the patch does not seem to introduce new RAT 
warnings
{color:green}+1 JAVADOC{color}
.{color:green}+1{color} the patch does not seem to introduce new Javadoc 
warnings
{color:green}+1 COMPILE{color}
.{color:green}+1{color} HEAD compiles
.{color:green}+1{color} patch compiles
.{color:green}+1{color} the patch does not seem to introduce new javac 
warnings
{color:green}+1 BACKWARDS_COMPATIBILITY{color}
.{color:green}+1{color} the patch does not change any JPA 
Entity/Colum/Basic/Lob/Transient annotations
.{color:green}+1{color} the patch does not modify JPA files
{color:green}+1 TESTS{color}
.Tests run: 1804
{color:green}+1 DISTRO{color}
.{color:green}+1{color} distro tarball builds with the patch 


{color:green}*+1 Overall result, good!, no -1s*{color}


The full output of the test-patch run is available at

.   https://builds.apache.org/job/oozie-trunk-precommit-build/3280/

> Coordinator Action status not updated when workflow job SUSPENDED
> -
>
> Key: OOZIE-2661
> URL: https://issues.apache.org/jira/browse/OOZIE-2661
> Project: Oozie
>  Issue Type: Bug
>Reporter: Satish Subhashrao Saley
>Assignee: Satish Subhashrao Saley
>Priority: Minor
> Attachments: OOZIE-2661-1.patch, OOZIE-2661-2.patch
>
>
> It's a bug in Oozie. CoordActionCheckXCommand doesn't take care of SUSPENDED 
> state. It only handles SUCCEEDED, FAILED and KILLED.
> {code}
> protected Void execute() throws CommandException {
> try {
> InstrumentUtils.incrJobCounter(getName(), 1, 
> getInstrumentation());
> Status slaStatus = null;
> CoordinatorAction.Status initialStatus = coordAction.getStatus();
> if (workflowJob.getStatus() == WorkflowJob.Status.SUCCEEDED) {
> coordAction.setStatus(CoordinatorAction.Status.SUCCEEDED);
> // set pending to false as the status is SUCCEEDED
> coordAction.setPending(0);
> slaStatus = Status.SUCCEEDED;
> }
> else {
> if (workflowJob.getStatus() == WorkflowJob.Status.FAILED) {
> coordAction.setStatus(CoordinatorAction.Status.FAILED);
> slaStatus = Status.FAILED;
> // set pending to false as the status is FAILED
> coordAction.setPending(0);
> }
> else {
> if (workflowJob.getStatus() == WorkflowJob.Status.KILLED) 
> {
> 
> coordAction.setStatus(CoordinatorAction.Status.KILLED);
> slaStatus = Status.KILLED;
> // set pending to false as the status is KILLED
> coordAction.setPending(0);
> }
> else {
> LOG.warn("Unexpected workflow " + workflowJob.getId() 
> + " STATUS " + workflowJob.getStatus());
> coordAction.setLastModifiedTime(new Date());
> CoordActionQueryExecutor.getInstance().executeUpdate(
> 
> CoordActionQueryExecutor.CoordActionQuery.UPDATE_COORD_ACTION_FOR_MODIFIED_DATE,
> coordAction);
> return null;
> }
> }
> }
> {code}
> Thank you [~puru] for pointing it out.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Jenkins build is back to normal : oozie-trunk-precommit-build #3280

2016-09-15 Thread Apache Jenkins Server

See

[jira] [Commented] (OOZIE-2572) SLA DURATION miss not shown when job is running for longer than expected time

2016-09-15 Thread Hadoop QA (JIRA)


[ 
https://issues.apache.org/jira/browse/OOZIE-2572?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15495102#comment-15495102
 ] 

Hadoop QA commented on OOZIE-2572:
--

Testing JIRA OOZIE-2572

Cleaning local git workspace



{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:green}+1 RAW_PATCH_ANALYSIS{color}
.{color:green}+1{color} the patch does not introduce any @author tags
.{color:green}+1{color} the patch does not introduce any tabs
.{color:green}+1{color} the patch does not introduce any trailing spaces
.{color:green}+1{color} the patch does not introduce any line longer than 
132
.{color:green}+1{color} the patch does adds/modifies 2 testcase(s)
{color:green}+1 RAT{color}
.{color:green}+1{color} the patch does not seem to introduce new RAT 
warnings
{color:green}+1 JAVADOC{color}
.{color:green}+1{color} the patch does not seem to introduce new Javadoc 
warnings
{color:green}+1 COMPILE{color}
.{color:green}+1{color} HEAD compiles
.{color:green}+1{color} patch compiles
.{color:green}+1{color} the patch does not seem to introduce new javac 
warnings
{color:green}+1 BACKWARDS_COMPATIBILITY{color}
.{color:green}+1{color} the patch does not change any JPA 
Entity/Colum/Basic/Lob/Transient annotations
.{color:green}+1{color} the patch does not modify JPA files
{color:red}-1 TESTS{color}
.Tests run: 1805
.Tests failed: 5
.Tests errors: 3

.The patch failed the following testcases:

.  testMain(org.apache.oozie.action.hadoop.TestHiveMain)
.  testPigScript(org.apache.oozie.action.hadoop.TestPigMain)
.  testPig_withNullExternalID(org.apache.oozie.action.hadoop.TestPigMain)
.  testEmbeddedPigWithinPython(org.apache.oozie.action.hadoop.TestPigMain)
.  testPigScript(org.apache.oozie.action.hadoop.TestPigMainWithOldAPI)

{color:green}+1 DISTRO{color}
.{color:green}+1{color} distro tarball builds with the patch 


{color:red}*-1 Overall result, please check the reported -1(s)*{color}


The full output of the test-patch run is available at

.   https://builds.apache.org/job/oozie-trunk-precommit-build/3279/

> SLA DURATION miss not shown when job is running for longer than expected time
> -
>
> Key: OOZIE-2572
> URL: https://issues.apache.org/jira/browse/OOZIE-2572
> Project: Oozie
>  Issue Type: Bug
>Reporter: Satish Subhashrao Saley
>Assignee: Satish Subhashrao Saley
> Fix For: 4.3.0
>
> Attachments: OOZIE-2572-1.patch, OOZIE-2572-10.patch, 
> OOZIE-2572-2.patch, OOZIE-2572-3.patch, OOZIE-2572-4.patch, 
> OOZIE-2572-5.patch, OOZIE-2572-6.patch, OOZIE-2572-7.patch, 
> OOZIE-2572-8.patch, OOZIE-2572-9.patch
>
>
> If job is running for beyond expected duration, Oozie UI is not slowing SLA 
> for duration miss.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Build failed in Jenkins: oozie-trunk-precommit-build #3279

2016-09-15 Thread Apache Jenkins Server

See 

--
[...truncated 6314 lines...]
[INFO] --- maven-surefire-plugin:2.12.2:test (default-test) @ oozie-tools ---
[INFO] Tests are skipped.
[INFO] 
[INFO] --- maven-jar-plugin:2.3.1:jar (default-jar) @ oozie-tools ---
[INFO] Building jar: 

[INFO] 
[INFO] --- maven-site-plugin:2.0-beta-6:attach-descriptor (attach-descriptor) @ 
oozie-tools ---
[INFO] 
[INFO] --- maven-assembly-plugin:2.2.1:single (default-cli) @ oozie-tools ---
[INFO] Reading assembly descriptor: ../src/main/assemblies/tools.xml
[WARNING] The following patterns were never triggered in this artifact 
exclusion filter:
o  '*:*:pom:*'

[INFO] Copying files to 

[WARNING] Assembly file: 

 is not a regular file (it may be a directory). It cannot be attached to the 
project build for installation or deployment.
[INFO] 
[INFO] 
[INFO] Building Apache Oozie MiniOozie 4.3.0-SNAPSHOT
[INFO] 
[INFO] 
[INFO] --- maven-remote-resources-plugin:1.5:process (default) @ oozie-mini ---
[INFO] 
[INFO] --- maven-resources-plugin:2.5:resources (default-resources) @ 
oozie-mini ---
[debug] execute contextualize
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory 

[INFO] Copying 3 resources
[INFO] 
[INFO] --- maven-compiler-plugin:2.3.2:compile (default-compile) @ oozie-mini 
---
[INFO] No sources to compile
[INFO] 
[INFO] --- maven-resources-plugin:2.5:testResources (default-testResources) @ 
oozie-mini ---
[debug] execute contextualize
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 3 resources
[INFO] Copying 3 resources
[INFO] 
[INFO] --- maven-compiler-plugin:2.3.2:testCompile (default-testCompile) @ 
oozie-mini ---
[INFO] Nothing to compile - all classes are up to date
[INFO] 
[INFO] --- maven-surefire-plugin:2.12.2:test (default-test) @ oozie-mini ---
[INFO] Tests are skipped.
[INFO] 
[INFO] --- maven-jar-plugin:2.3.1:jar (default-jar) @ oozie-mini ---
[INFO] Building jar: 

[INFO] 
[INFO] --- maven-site-plugin:2.0-beta-6:attach-descriptor (attach-descriptor) @ 
oozie-mini ---
[INFO] 
[INFO] --- maven-assembly-plugin:2.2.1:single (default-cli) @ oozie-mini ---
[INFO] Reading assembly descriptor: src/main/assemblies/empty.xml
[INFO] 
[INFO] 
[INFO] Building Apache Oozie Distro 4.3.0-SNAPSHOT
[INFO] 
[INFO] 
[INFO] --- maven-remote-resources-plugin:1.5:process (default) @ oozie-distro 
---
[INFO] 
[INFO] --- maven-resources-plugin:2.5:resources (default-resources) @ 
oozie-distro ---
[debug] execute contextualize
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory 

[INFO] Copying 3 resources
[INFO] 
[INFO] --- maven-compiler-plugin:2.3.2:compile (default-compile) @ oozie-distro 
---
[INFO] No sources to compile
[INFO] 
[INFO] --- maven-resources-plugin:2.5:testResources (default-testResources) @ 
oozie-distro ---
[debug] execute contextualize
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory 

[INFO] Copying 3 resources
[INFO] 
[INFO] --- maven-compiler-plugin:2.3.2:testCompile (default-testCompile) @ 
oozie-distro ---
[INFO] No sources to compile
[INFO] 
[INFO] --- maven-surefire-plugin:2.12.2:test (default-test) @ oozie-distro ---
[INFO] Tests are skipped.
[INFO] 
[INFO] --- maven-jar-plugin:2.3.1:jar (default-jar) @ oozie-distro ---
[INFO] Building jar: 

[INFO] 
[INFO] --- maven-site-plugin:2.0-beta-6:attach-descriptor (attach-descriptor) @ 
oozie-distro ---
[INFO] 
[INFO] --- maven-antrun-plugin:1.6:run (default) @ oozie-distro ---
[INFO] Executing tasks

main:
  [get] Getting: 
http://archive.apache.org/dist/tomcat/tomcat-6/v6.0.44/bin/apache-tomcat-6.0.44.t

[jira] [Commented] (OOZIE-1978) Forkjoin validation code is ridiculously slow in some cases

2016-09-15 Thread Robert Kanter (JIRA)


[ 
https://issues.apache.org/jira/browse/OOZIE-1978?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15495085#comment-15495085
 ] 

Robert Kanter commented on OOZIE-1978:
--

+1 LGTM

Though given the complexity of this patch, it would be good to have at least 
one other person review it (e.g. [~puru], [~rohini], [~satishsaley], 
[~jaydeepvishwakarma], [~abhishekbafna]).

> Forkjoin validation code is ridiculously slow in some cases
> ---
>
> Key: OOZIE-1978
> URL: https://issues.apache.org/jira/browse/OOZIE-1978
> Project: Oozie
>  Issue Type: Bug
>  Components: core
>Affects Versions: trunk, 4.0.1
>Reporter: Robert Kanter
>Assignee: Peter Bacsko
> Fix For: 4.3.0
>
> Attachments: OOZIE-1978-001.patch, OOZIE-1978-002.patch, 
> OOZIE-1978-002.patch, OOZIE-1978-003.patch, OOZIE-1978-004.patch, 
> OOZIE-1978-005.patch, OOZIE-1978-006.patch, OOZIE-1978_wip.001.patch, 
> workflow.xml
>
>
> We've had a few users who have run into problems where submitting a workflow 
> appears to hang (in the case of a subworkflow, it's similar but stuck in 
> PREP).  It turns out that if you wait long enough, it will actually go 
> through and the workflow will run normally.  The problem is that the forkjoin 
> validation code is taking a really long time.
> The attached example has a series of 20 forks where each fork has 6 actions 
> (it's based on an actual workflow, but all of the names were changed and the 
> actions were all replaced by simple shell actions).  One of our support guys 
> said it took 1-2 hours , but on my computer it was taking {color:red}*15+ 
> hours*{color} (I had to cancel it)
> While this example doesn't have any nested forks, those can also take a long 
> time too.
> It's easy to verify that it's the forkjoin validation code that's taking so 
> long by looking at a jstack of the Oozie server and seeing deep recursive 
> calls to 
> {{org.apache.oozie.workflow.lite.LiteWorkflowAppParser.validateForkJoin}}.  I 
> also noticed a lot of sitting around in calls LinkedList.contains.  
> I think we have 3 options:
> # See if we can make the existing code faster somehow.  Perhaps there's a way 
> to parallelize it?  Maybe there's some redundant checking that we can 
> identify and skip? Change some data structures? etc
> # See if we can write a new way to do this validation.  I had originally 
> completely rewritten this code a while ago, and we've since made a few fixes 
> to catch edge cases and things.  Perhaps it needs another rewrite?
> # Try to identify when it's taking a long time and at least let the user know 
> what's happening or something.  Right now, it just appears that the Oozie CLI 
> has hung and the job doesn't show up in the Oozie server.  Most users aren't 
> going to wait more than a minute or two.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Review Request 50202: [OOZIE-1978] Forkjoin validation code is ridiculously slow in some cases

2016-09-15 Thread Robert Kanter


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/50202/#review149163
---


Ship it!




Ship It!

- Robert Kanter


On Aug. 31, 2016, 1:02 p.m., Peter Bacsko wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/50202/
> ---
> 
> (Updated Aug. 31, 2016, 1:02 p.m.)
> 
> 
> Review request for oozie.
> 
> 
> Bugs: OOZIE-1978
> https://issues.apache.org/jira/browse/OOZIE-1978
> 
> 
> Repository: oozie-git
> 
> 
> Description
> ---
> 
> See OOZIE-1978 for details.
> 
> 
> Diffs
> -
> 
>   core/src/main/java/org/apache/oozie/ErrorCode.java 2907ca2 
>   core/src/main/java/org/apache/oozie/service/ActionService.java becc69b 
>   
> core/src/main/java/org/apache/oozie/workflow/lite/LiteWorkflowAppParser.java 
> 0541634 
>   
> core/src/main/java/org/apache/oozie/workflow/lite/LiteWorkflowValidator.java 
> PRE-CREATION 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitXCommand.java 
> 73464c8 
>   
> core/src/test/java/org/apache/oozie/workflow/lite/TestLiteWorkflowAppParser.java
>  9002b6c 
>   core/src/test/resources/wf-long.xml PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/50202/diff/
> 
> 
> Testing
> ---
> 
> Existing tests pass.
> 
> We might need to add some more, at least testing acyclic graph detection.
> 
> 
> Thanks,
> 
> Peter Bacsko
> 
>

[jira] [Updated] (OOZIE-2525) SchemaChecker fails with NPE

2016-09-15 Thread Robert Kanter (JIRA)


 [ 
https://issues.apache.org/jira/browse/OOZIE-2525?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Robert Kanter updated OOZIE-2525:
-
Priority: Blocker  (was: Critical)

> SchemaChecker fails with NPE
> 
>
> Key: OOZIE-2525
> URL: https://issues.apache.org/jira/browse/OOZIE-2525
> Project: Oozie
>  Issue Type: Bug
>Reporter: Purshotam Shah
>Assignee: Robert Kanter
>Priority: Blocker
> Fix For: 4.3.0
>
> Attachments: OOZIE-2525.001.patch
>
>
> {code}
> 2016-05-03 14:38:03,851  WARN SchedulerService:523 [pool-1-thread-2] - 
> SERVER[-] Error executing runnable [SchemaCheckerRunnable], null
> java.lang.NullPointerException
> at 
> org.apache.oozie.service.SchemaCheckerService.access$002(SchemaCheckerService.java:29)
> at 
> org.apache.oozie.service.SchemaCheckerService$SchemaCheckerRunnable.run(SchemaCheckerService.java:120)
> at 
> org.apache.oozie.service.SchedulerService$2.run(SchedulerService.java:175)
> at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
> at 
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
> at 
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> {code}
> It;s trying to use SchemaCheckerService before it is initialized.
> {code:title=SchemaCheckerService.java}
> Services.get().get(SchemaCheckerService.class).status = "DISABLED (not leader 
> in HA)";
> Services.get().get(SchemaCheckerService.class).lastCheck = 
> "N/A";
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (OOZIE-2613) Upgrade hive version from 0.13.1 to 1.2.0

2016-09-15 Thread Robert Kanter (JIRA)


[ 
https://issues.apache.org/jira/browse/OOZIE-2613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15495057#comment-15495057
 ] 

Robert Kanter commented on OOZIE-2613:
--

This seems good to me.  [~abhishekbafna], how much testing have you done?  
It's a little surprising that we don't need any code changes to go from Hive 
0.13.1 to 1.2.0.

Also, you'll need to do some minor refactoring because of OOZIE-2530.

> Upgrade hive version from 0.13.1 to 1.2.0
> -
>
> Key: OOZIE-2613
> URL: https://issues.apache.org/jira/browse/OOZIE-2613
> Project: Oozie
>  Issue Type: Bug
>  Components: core
>Reporter: Abhishek Bafna
>Assignee: Abhishek Bafna
> Fix For: 4.3.0
>
> Attachments: OOZIE-2613-00.patch, OOZIE-2613-01.patch, 
> OOZIE-2613-02.patch, OOZIE-2613-03.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (OOZIE-2037) Add TLSv1.1,TLSv1.2

2016-09-15 Thread Robert Kanter (JIRA)


[ 
https://issues.apache.org/jira/browse/OOZIE-2037?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15495047#comment-15495047
 ] 

Robert Kanter commented on OOZIE-2037:
--

[~rohini], can you take a look at this too?  It's a pretty trivial patch.  
We've been using it in CDH for probably a year now.

> Add TLSv1.1,TLSv1.2
> ---
>
> Key: OOZIE-2037
> URL: https://issues.apache.org/jira/browse/OOZIE-2037
> Project: Oozie
>  Issue Type: Sub-task
>  Components: security
>Affects Versions: trunk
>Reporter: Robert Kanter
>Assignee: Robert Kanter
>Priority: Blocker
>  Labels: newbie
> Fix For: 4.3.0
>
> Attachments: OOZIE-2037.001.patch
>
>
> OOZIE-2034 required us to specifically list the versions of TLS that Oozie 
> supports.  Java 7 supports TLSv1.1 and TLSv1.2, so we should add them to the 
> list.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (OOZIE-1793) Improve find bugs reporting for Oozie

2016-09-15 Thread Robert Kanter (JIRA)


[ 
https://issues.apache.org/jira/browse/OOZIE-1793?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15495038#comment-15495038
 ] 

Robert Kanter commented on OOZIE-1793:
--

[~rohini], can you review this?  It would be nice to get in.

> Improve find bugs reporting for Oozie
> -
>
> Key: OOZIE-1793
> URL: https://issues.apache.org/jira/browse/OOZIE-1793
> Project: Oozie
>  Issue Type: Sub-task
>  Components: build
>Affects Versions: trunk
>Reporter: Robert Kanter
>Assignee: Robert Kanter
> Fix For: 4.3.0
>
> Attachments: OOZIE-1793-01.patch, OOZIE-1793.patch, OOZIE-1793.patch
>
>
> I couldn't figure out how to run findbugs against Oozie.  And even if it does 
> work, it looks like it's just going to create an XML file.  
> Based on some minor refactoring to this 
> [tutorial|http://www.petrikainulainen.net/programming/maven/findbugs-maven-plugin-tutorial/]
>  and this [stack overflow 
> message|http://stackoverflow.com/questions/7035112/any-easy-way-to-generate-a-findbug-html-report-from-maven-without-sitesite/10365954#10365954],
>  I was able to improve our findbugs reporting.  Now, when you run {{mvn 
> verify}} (which we use to generate a checkstyle report), it will also run 
> findbugs and transform the resulting XML files into html files that are human 
> readable.
> The hard part here will be actually going through the reports and fixing 
> these bugs; but that's for other JIRAs :)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (OOZIE-2671) Update test-patch to run findbugs

2016-09-15 Thread Robert Kanter (JIRA)


 [ 
https://issues.apache.org/jira/browse/OOZIE-2671?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Robert Kanter resolved OOZIE-2671.
--
Resolution: Duplicate

> Update test-patch to run findbugs
> -
>
> Key: OOZIE-2671
> URL: https://issues.apache.org/jira/browse/OOZIE-2671
> Project: Oozie
>  Issue Type: Test
>  Components: build
>Reporter: Robert Kanter
>
> Now that we have findbugs working with OOZIE-1793, we should update 
> test-patch to run it and report it back on the JIRA when jenkins runs.  
> Given that there's a lot of bugs already that it finds, we should have the 
> report do some kind of diff to only show bugs introduced by the patch.
> Also keep in mind that each module gets its own report; there isn't an 
> overall report.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (OOZIE-2671) Update test-patch to run findbugs

2016-09-15 Thread Robert Kanter (JIRA)

Robert Kanter created OOZIE-2671:


 Summary: Update test-patch to run findbugs
 Key: OOZIE-2671
 URL: https://issues.apache.org/jira/browse/OOZIE-2671
 Project: Oozie
  Issue Type: Test
  Components: build
Reporter: Robert Kanter


Now that we have findbugs working with OOZIE-1793, we should update test-patch 
to run it and report it back on the JIRA when jenkins runs.  

Given that there's a lot of bugs already that it finds, we should have the 
report do some kind of diff to only show bugs introduced by the patch.

Also keep in mind that each module gets its own report; there isn't an overall 
report.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Jenkins build is back to normal : oozie-trunk-find-patches-available #133748

2016-09-15 Thread Apache Jenkins Server

See

[jira] [Commented] (OOZIE-2606) Set spark.yarn.jars to fix Spark 2.0 with Oozie

2016-09-15 Thread Hadoop QA (JIRA)


[ 
https://issues.apache.org/jira/browse/OOZIE-2606?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15495018#comment-15495018
 ] 

Hadoop QA commented on OOZIE-2606:
--

Testing JIRA OOZIE-2606

Cleaning local git workspace



{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:red}-1 RAW_PATCH_ANALYSIS{color}
.{color:green}+1{color} the patch does not introduce any @author tags
.{color:green}+1{color} the patch does not introduce any tabs
.{color:green}+1{color} the patch does not introduce any trailing spaces
.{color:green}+1{color} the patch does not introduce any line longer than 
132
.{color:red}-1{color} the patch does not add/modify any testcase
{color:green}+1 RAT{color}
.{color:green}+1{color} the patch does not seem to introduce new RAT 
warnings
{color:green}+1 JAVADOC{color}
.{color:green}+1{color} the patch does not seem to introduce new Javadoc 
warnings
{color:green}+1 COMPILE{color}
.{color:green}+1{color} HEAD compiles
.{color:green}+1{color} patch compiles
.{color:green}+1{color} the patch does not seem to introduce new javac 
warnings
{color:green}+1 BACKWARDS_COMPATIBILITY{color}
.{color:green}+1{color} the patch does not change any JPA 
Entity/Colum/Basic/Lob/Transient annotations
.{color:green}+1{color} the patch does not modify JPA files
{color:green}+1 TESTS{color}
.Tests run: 1804
{color:green}+1 DISTRO{color}
.{color:green}+1{color} distro tarball builds with the patch 


{color:red}*-1 Overall result, please check the reported -1(s)*{color}


The full output of the test-patch run is available at

.   https://builds.apache.org/job/oozie-trunk-precommit-build/3278/

> Set spark.yarn.jars to fix Spark 2.0 with Oozie
> ---
>
> Key: OOZIE-2606
> URL: https://issues.apache.org/jira/browse/OOZIE-2606
> Project: Oozie
>  Issue Type: Bug
>  Components: core
>Affects Versions: 4.2.0
>Reporter: Jonathan Kelly
>Assignee: Satish Subhashrao Saley
>  Labels: spark, spark2.0.0
> Fix For: 4.3.0
>
> Attachments: OOZIE-2606-2.patch, OOZIE-2606.patch
>
>
> Oozie adds all of the jars in the Oozie Spark sharelib to the 
> DistributedCache such that all jars will be present in the current working 
> directory of the YARN container (as well as in the container classpath). 
> However, this is not quite enough to make Spark 2.0 work, since Spark 2.0 by 
> default looks for the jars in assembly/target/scala-2.11/jars [1] (as if it 
> is a locally built distribution for development) and will not find them in 
> the current working directory.
> To fix this, we can set spark.yarn.jars to *.jar so that it finds the jars in 
> the current working directory rather than looking in the wrong place. [2]
> [1] 
> https://github.com/apache/spark/blob/v2.0.0-rc2/launcher/src/main/java/org/apache/spark/launcher/CommandBuilderUtils.java#L357
> [2] 
> https://github.com/apache/spark/blob/v2.0.0-rc2/yarn/src/main/scala/org/apache/spark/deploy/yarn/Client.scala#L476
> Note: This property will be ignored by Spark 1.x.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Build failed in Jenkins: oozie-trunk-find-patches-available #133747

2016-09-15 Thread Apache Jenkins Server

See 


Changes:

[rkanter] OOZIE-2530 Allow Hive to use a different jline version (poeppt via

[rkanter] OOZIE-2571 Add spark.scala.binary.version Maven property so that Scala

[rkanter] OOZIE-2552 Update ActiveMQ version for security and other fixes

[rkanter] OOZIE-2500 -DtestJarSimple option mentioned in minioozie doc does not

--
Started by timer
Started by timer
Started by timer
Started by timer
Started by timer
Started by timer
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on H1 (yahoo-not-h2 Hadoop Mapreduce Hdfs Pig) in workspace 

 > git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
 > git config remote.origin.url 
 > https://git-wip-us.apache.org/repos/asf/oozie.git # timeout=10
Fetching upstream changes from https://git-wip-us.apache.org/repos/asf/oozie.git
 > git --version # timeout=10
 > git -c core.askpass=true fetch --tags --progress 
 > https://git-wip-us.apache.org/repos/asf/oozie.git 
 > +refs/heads/*:refs/remotes/origin/*
 > git rev-parse refs/remotes/origin/master^{commit} # timeout=10
 > git rev-parse refs/remotes/origin/origin/master^{commit} # timeout=10
Checking out Revision 65f59ff0ecbc13e84b88d2debd45227f8296cd4a 
(refs/remotes/origin/master)
 > git config core.sparsecheckout # timeout=10
 > git checkout -f 65f59ff0ecbc13e84b88d2debd45227f8296cd4a
 > git rev-list 34c469dd2f7023c92bc0f5605274059c30beea0c # timeout=10
[oozie-trunk-find-patches-available] $ /bin/bash 
/tmp/hudson5182639126325149134.sh
mkdir: cannot create directory 
':
 File exists
  % Total% Received % Xferd  Average Speed   TimeTime Time  Current
 Dload  Upload   Total   SpentLeft  Speed

  0 00 00 0  0  0 --:--:-- --:--:-- --:--:-- 0
  0 00 00 0  0  0 --:--:-- --:--:-- --:--:-- 0
100  131k0  131k0 0   123k  0 --:--:--  0:00:01 --:--:--  123k
curl: (18) transfer closed with outstanding read data remaining
Could not retrieve available patches from JIRA
Build step 'Execute shell' marked build as failure

Build failed in Jenkins: oozie-trunk-precommit-build #3278

2016-09-15 Thread Apache Jenkins Server

See 

--
[...truncated 6272 lines...]
[INFO] Nothing to compile - all classes are up to date
[INFO] 
[INFO] --- maven-resources-plugin:2.5:testResources (default-testResources) @ 
oozie-tools ---
[debug] execute contextualize
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 11 resources
[INFO] Copying 3 resources
[INFO] 
[INFO] --- maven-compiler-plugin:2.3.2:testCompile (default-testCompile) @ 
oozie-tools ---
[INFO] Nothing to compile - all classes are up to date
[INFO] 
[INFO] --- maven-surefire-plugin:2.12.2:test (default-test) @ oozie-tools ---
[INFO] Tests are skipped.
[INFO] 
[INFO] --- maven-jar-plugin:2.3.1:jar (default-jar) @ oozie-tools ---
[INFO] Building jar: 

[INFO] 
[INFO] --- maven-site-plugin:2.0-beta-6:attach-descriptor (attach-descriptor) @ 
oozie-tools ---
[INFO] 
[INFO] --- maven-assembly-plugin:2.2.1:single (default-cli) @ oozie-tools ---
[INFO] Reading assembly descriptor: ../src/main/assemblies/tools.xml
[WARNING] The following patterns were never triggered in this artifact 
exclusion filter:
o  '*:*:pom:*'

[INFO] Copying files to 

[WARNING] Assembly file: 

 is not a regular file (it may be a directory). It cannot be attached to the 
project build for installation or deployment.
[INFO] 
[INFO] 
[INFO] Building Apache Oozie MiniOozie 4.3.0-SNAPSHOT
[INFO] 
[INFO] 
[INFO] --- maven-remote-resources-plugin:1.5:process (default) @ oozie-mini ---
[INFO] 
[INFO] --- maven-resources-plugin:2.5:resources (default-resources) @ 
oozie-mini ---
[debug] execute contextualize
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory 

[INFO] Copying 3 resources
[INFO] 
[INFO] --- maven-compiler-plugin:2.3.2:compile (default-compile) @ oozie-mini 
---
[INFO] No sources to compile
[INFO] 
[INFO] --- maven-resources-plugin:2.5:testResources (default-testResources) @ 
oozie-mini ---
[debug] execute contextualize
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 3 resources
[INFO] Copying 3 resources
[INFO] 
[INFO] --- maven-compiler-plugin:2.3.2:testCompile (default-testCompile) @ 
oozie-mini ---
[INFO] Nothing to compile - all classes are up to date
[INFO] 
[INFO] --- maven-surefire-plugin:2.12.2:test (default-test) @ oozie-mini ---
[INFO] Tests are skipped.
[INFO] 
[INFO] --- maven-jar-plugin:2.3.1:jar (default-jar) @ oozie-mini ---
[INFO] Building jar: 

[INFO] 
[INFO] --- maven-site-plugin:2.0-beta-6:attach-descriptor (attach-descriptor) @ 
oozie-mini ---
[INFO] 
[INFO] --- maven-assembly-plugin:2.2.1:single (default-cli) @ oozie-mini ---
[INFO] Reading assembly descriptor: src/main/assemblies/empty.xml
[INFO] 
[INFO] 
[INFO] Building Apache Oozie Distro 4.3.0-SNAPSHOT
[INFO] 
[INFO] 
[INFO] --- maven-remote-resources-plugin:1.5:process (default) @ oozie-distro 
---
[INFO] 
[INFO] --- maven-resources-plugin:2.5:resources (default-resources) @ 
oozie-distro ---
[debug] execute contextualize
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory 

[INFO] Copying 3 resources
[INFO] 
[INFO] --- maven-compiler-plugin:2.3.2:compile (default-compile) @ oozie-distro 
---
[INFO] No sources to compile
[INFO] 
[INFO] --- maven-resources-plugin:2.5:testResources (default-testResources) @ 
oozie-distro ---
[debug] execute contextualize
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory 

[INFO] Copying 3 resources
[INFO] 
[INFO] --- maven-compiler-plugin:2.3.2:testCompile (default-testCompile) @ 
oozie-distro ---
[INFO] No sources to compile
[INFO] 
[INFO] --- maven-surefire-plugin:2.12.2:test (default-test) @ oozie-distro ---
[INFO] Tests are skipped.
[INFO] 
[INFO] --- maven-jar-plugin:2.3.1:jar (default-jar) @

[jira] [Updated] (OOZIE-2037) Add TLSv1.1,TLSv1.2

2016-09-15 Thread Robert Kanter (JIRA)


 [ 
https://issues.apache.org/jira/browse/OOZIE-2037?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Robert Kanter updated OOZIE-2037:
-
Priority: Blocker  (was: Major)

Making this a blocker because it's a security thing.

> Add TLSv1.1,TLSv1.2
> ---
>
> Key: OOZIE-2037
> URL: https://issues.apache.org/jira/browse/OOZIE-2037
> Project: Oozie
>  Issue Type: Sub-task
>  Components: security
>Affects Versions: trunk
>Reporter: Robert Kanter
>Assignee: Robert Kanter
>Priority: Blocker
>  Labels: newbie
> Fix For: 4.3.0
>
> Attachments: OOZIE-2037.001.patch
>
>
> OOZIE-2034 required us to specifically list the versions of TLS that Oozie 
> supports.  Java 7 supports TLSv1.1 and TLSv1.2, so we should add them to the 
> list.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Issue Comment Deleted] (OOZIE-2296) Add an Oozie diagnostic bundle tool

2016-09-15 Thread Robert Kanter (JIRA)


 [ 
https://issues.apache.org/jira/browse/OOZIE-2296?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Robert Kanter updated OOZIE-2296:
-
Comment: was deleted

(was: RB: https://reviews.apache.org/r/36266/)

> Add an Oozie diagnostic bundle tool
> ---
>
> Key: OOZIE-2296
> URL: https://issues.apache.org/jira/browse/OOZIE-2296
> Project: Oozie
>  Issue Type: New Feature
>  Components: tools
>Affects Versions: trunk
>Reporter: Robert Kanter
>Assignee: Robert Kanter
>
> To help with our support cases, I've built a tool that collects a bunch of 
> job and other information from Oozie that throws it all in a tarball.  The 
> idea is that the user can just click a button in Cloudera Manager, and it 
> will run this tool.  
> This tool could be useful for others as an easy way to get information out of 
> Oozie, so I thought I'd contribute it here.  It's built as a "tool" (so it 
> sits next to the sharelib and database tools), and simply uses the Oozie 
> client for getting pretty much everything, so it doesn't require anything 
> special.
> Here's the information that it can get:
> # Sharelib: {{ooze admin -shareliblist}} and {{oozie admin -shareliblist 
> }}
> # Oozie Server's resolved loaded configuration (from admin endpoint)
> # Other admin commands output (queue dump, env vars, etc)
> # Thread dump (HOST:11000/oozie/admin/jvminfo.jsp)
> # Details from last n jobs and/or specific list of jobs
> #- job.properties contents
> #- XML definition
> #- verbose status for each job and each action etc
> #- Oozie logs
> #- Unfortunately, we can't get the launcher jobs' logs from Hadoop
> # Metrics/Instrumentation



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (OOZIE-2415) Build Error for oozie-docs

2016-09-15 Thread Robert Kanter (JIRA)


 [ 
https://issues.apache.org/jira/browse/OOZIE-2415?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Robert Kanter resolved OOZIE-2415.
--
   Resolution: Cannot Reproduce
Fix Version/s: (was: 4.3.0)

This doesn't seem to be a problem at this time.

> Build Error for oozie-docs
> --
>
> Key: OOZIE-2415
> URL: https://issues.apache.org/jira/browse/OOZIE-2415
> Project: Oozie
>  Issue Type: Improvement
>  Components: build
>Affects Versions: 4.2.0
> Environment: Darwin Kernel Version 15.0.0: Sat Sep 19 15:53:46 PDT 
> 2015; root:xnu-3247.10.11~1/RELEASE_X86_64 x86_64
> Apache Maven 3.2.1 (ea8b2b07643dbb1b84b6d16e1f08391b666bc1e9; 
> 2014-02-14T18:37:52+01:00)
> Java version: 1.8.0_25, vendor: Oracle Corporation
> Java home: /Library/Java/JavaVirtualMachines/jdk1.8.0_25.jdk/Contents/Home/jre
> Default locale: en_US, platform encoding: UTF-8
> OS name: "mac os x", version: "10.11.1", arch: "x86_64", family: "mac"
> MacBook Pro (13-inch, Early 2011)
>Reporter: Adebiy Abdurrahman
>Priority: Minor
>  Labels: maven
>
> Building oozie with the advised command.
> bin/mkdistro.sh -DskipTests
> [INFO] Apache Oozie Share Lib Hive 2 . SUCCESS [  8.926 s]
> [INFO] Apache Oozie Share Lib Sqoop .. SUCCESS [  2.571 s]
> [INFO] Apache Oozie Examples . SUCCESS [ 13.693 s]
> [INFO] Apache Oozie Share Lib Spark .. SUCCESS [ 20.181 s]
> [INFO] Apache Oozie Share Lib  SUCCESS [ 30.188 s]
> [INFO] Apache Oozie Docs . FAILURE [  9.458 s]
> [ERROR] Failed to execute goal 
> org.apache.maven.plugins:maven-assembly-plugin:2.2.1:single (default-cli) on 
> project oozie-distro: Failed to create assembly: Error adding file to 
> archive: 
> /Users/abd_adebiyi/sw/apache/oozie/oozie-4.2.0/distro/./../client/target/oozie-client-4.2.0-client.tar.gz
>  isn't a file. -> [Help 1]
> [ERROR] 
> [ERROR] To see the full stack trace of the errors, re-run Maven with the -e 
> switch.
> [ERROR] Re-run Maven using the -X switch to enable full debug logging.
> [ERROR] 
> [ERROR] For more information about the errors and possible solutions, please 
> read the following articles:
> [ERROR] [Help 1] 
> http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
> [ERROR] 
> [ERROR] After correcting the problems, you can resume the build with the 
> command
> [ERROR]   mvn  -rf :oozie-distro
> 
> Failure was due to the termination of Codehause services . See more details 
> at http://www.codehaus.org
> The maven pom.xml file needs mohave the repository removed.
> i.e 
>  
> Codehaus repository
> http://repository.codehaus.org/
> 
>   false
> 
>   



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (OOZIE-2487) Unable to build distro on Java 8 (on Mac)

2016-09-15 Thread Robert Kanter (JIRA)


 [ 
https://issues.apache.org/jira/browse/OOZIE-2487?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Robert Kanter updated OOZIE-2487:
-
Priority: Blocker  (was: Trivial)

> Unable to build distro on Java 8 (on Mac)
> -
>
> Key: OOZIE-2487
> URL: https://issues.apache.org/jira/browse/OOZIE-2487
> Project: Oozie
>  Issue Type: Bug
>  Components: core
>Affects Versions: 4.3.0
> Environment: Mac OS X (10.10.5), Java 1.8.0_66
>Reporter: Rajendra Patil
>Priority: Blocker
>  Labels: java8
> Fix For: 4.3.0
>
> Attachments: OOZIE-2487-1.patch, OOZIE-2487-2.patch
>
>
> While looking at another issue, I came across this. When I cloned latest 
> oozie and tried to build distro using `./bin/mkdistro.sh` I found that the 
> build failed with following error. 
> ---
> 
> .../oozie/core/src/main/java/org/apache/oozie/service/UUIDService.java:114: 
> error: reference not found
> [ERROR] * systemId is the value defined in the {@link #CONF_SYSTEM_ID} 
> configuration property.
> ...
> ---
> The error seems to be due to the Java 8 feature (DocLint) that checks for 
> errors in Java Docs strictly. The solution would be create a Java 8 specific 
> profile to pass '-Xdoclint:none' to deactivate it. Having this flag for older 
> java version fails to it has to be only for Java 8 using specific profile.
> I will create a patch for the same.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (OOZIE-2487) Unable to build distro on Java 8 (on Mac)

2016-09-15 Thread Robert Kanter (JIRA)


[ 
https://issues.apache.org/jira/browse/OOZIE-2487?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15494976#comment-15494976
 ] 

Robert Kanter commented on OOZIE-2487:
--

I think the original problem with {{UUIDService}} must have been fixed at some 
point because I'm able to build Oozie with Java 8.  However, if I run {{mvn 
clean javadoc:javadoc -DskipTests}} directly, it fails with a bunch of errors 
and doesn't get very far, so I bet there's more than just this:
{noformat}
[INFO] Apache Oozie Client ... SUCCESS [4.013s]
[INFO] Apache Oozie Share Lib Oozie .. FAILURE [1.480s]
[INFO] Apache Oozie Share Lib HCatalog ... SKIPPED
[INFO] Apache Oozie Share Lib Distcp . SKIPPED
[INFO] Apache Oozie Core . SKIPPED
[INFO] Apache Oozie Share Lib Streaming .. SKIPPED
[INFO] Apache Oozie Share Lib Pig  SKIPPED
[INFO] Apache Oozie Share Lib Hive ... SKIPPED
[INFO] Apache Oozie Share Lib Hive 2 . SKIPPED
[INFO] Apache Oozie Share Lib Sqoop .. SKIPPED
[INFO] Apache Oozie Examples . SKIPPED
[INFO] Apache Oozie Share Lib Spark .. SKIPPED
[INFO] Apache Oozie Share Lib  SKIPPED
[INFO] Apache Oozie Docs . SKIPPED
[INFO] Apache Oozie WebApp ... SKIPPED
[INFO] Apache Oozie Tools  SKIPPED
[INFO] Apache Oozie MiniOozie  SKIPPED
[INFO] Apache Oozie Distro ... SKIPPED
[INFO] Apache Oozie ZooKeeper Security Tests . SKIPPED
[INFO] 
[INFO] BUILD FAILURE
[INFO] 
[INFO] Total time: 7.667s
[INFO] Finished at: Thu Sep 15 17:36:18 PDT 2016
[INFO] Final Memory: 26M/437M
[INFO] 
[ERROR] Failed to execute goal 
org.apache.maven.plugins:maven-javadoc-plugin:2.10.3:javadoc (default-cli) on 
project oozie-sharelib-oozie: An error has occurred in JavaDocs report 
generation:
[ERROR] Exit code: 1 - Picked up _JAVA_OPTIONS: -Djava.awt.headless=true
[ERROR] 
/Users/rkanter/dev/oozie-git/sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherURIHandler.java:37:
 warning: no description for @throws
[ERROR] * @throws LauncherException
[ERROR] ^
[ERROR] 
/Users/rkanter/dev/oozie-git/sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherURIHandler.java:49:
 warning: no description for @throws
[ERROR] * @throws LauncherException
[ERROR] ^
[ERROR] 
/Users/rkanter/dev/oozie-git/sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/JavaMain.java:32:
 warning: no description for @throws
[ERROR] * @throws Exception
[ERROR] ^
[ERROR] 
/Users/rkanter/dev/oozie-git/sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherMain.java:142:
 warning: no @param for env
[ERROR] protected static String getFilePathFromEnv(String env) {
[ERROR] ^
[ERROR] 
/Users/rkanter/dev/oozie-git/sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherMain.java:142:
 warning: no @return
[ERROR] protected static String getFilePathFromEnv(String env) {
[ERROR] ^
[ERROR] 
/Users/rkanter/dev/oozie-git/sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherMain.java:161:
 warning: no description for @throws
[ERROR] * @throws OozieActionConfiguratorException
[ERROR] ^
[ERROR] 
/Users/rkanter/dev/oozie-git/sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherMain.java:185:
 warning: no description for @throws
[ERROR] * @throws IOException
[ERROR] ^
[ERROR] 
/Users/rkanter/dev/oozie-git/sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherMain.java:222:
 warning: no description for @throws
[ERROR] * @throws IOException
[ERROR] ^
[ERROR] 
/Users/rkanter/dev/oozie-git/sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherURIHandlerFactory.java:36:
 warning: no description for @param
[ERROR] * @param uri
[ERROR] ^
[ERROR] 
/Users/rkanter/dev/oozie-git/sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherURIHandlerFactory.java:38:
 warning: no description for @throws
[ERROR] * @throws LauncherException
[ERROR] ^
[ERROR] 
/Users/rkanter/dev/oozie-git/sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/OozieActionConfigurator.java:34:
 warning: no description for @throws
[ERROR] * @throws OozieActionConfiguratorException
[ERROR] ^
[ERROR] 
/Users/rkanter/dev/oozie-git/sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/OozieLauncherOutputCommitter.java:40:
 error: @param name not found
[ERROR] * @param context the task

[jira] [Commented] (OOZIE-2500) -DtestJarSimple option mentioned in minioozie doc does not work

2016-09-15 Thread Robert Kanter (JIRA)


[ 
https://issues.apache.org/jira/browse/OOZIE-2500?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15494953#comment-15494953
 ] 

Robert Kanter commented on OOZIE-2500:
--

+1

> -DtestJarSimple option mentioned in minioozie doc does not work
> ---
>
> Key: OOZIE-2500
> URL: https://issues.apache.org/jira/browse/OOZIE-2500
> Project: Oozie
>  Issue Type: Bug
>Reporter: Satish Subhashrao Saley
>Assignee: Abhishek Bafna
> Fix For: 4.3.0
>
> Attachments: OOZIE-2500-01.patch, out.txt
>
>
> [Minioozie doc|https://oozie.apache.org/docs/4.2.0/ENG_MiniOozie.html] says 
> to run {{$ mvn clean install -DskipTests -DtestJarSimple}} to populate local 
> maven repo with required jars. But the command fails with following message:
> {code}
> [INFO] 100 errors 
> [INFO] -
> [INFO] 
> 
> [INFO] Reactor Summary:
> [INFO] 
> [INFO] Apache Oozie Main .. SUCCESS [  1.339 
> s]
> [INFO] Apache Oozie Hadoop Utils .. SUCCESS [  0.999 
> s]
> [INFO] Apache Oozie Hadoop Distcp hadoop-1-4.3.0-SNAPSHOT . SUCCESS [  0.138 
> s]
> [INFO] Apache Oozie Hadoop Auth hadoop-1-4.3.0-SNAPSHOT ... SUCCESS [  0.244 
> s]
> [INFO] Apache Oozie Hadoop Libs ... SUCCESS [  0.052 
> s]
> [INFO] Apache Oozie Client  SUCCESS [  3.282 
> s]
> [INFO] Apache Oozie Share Lib Oozie ... SUCCESS [  2.257 
> s]
> [INFO] Apache Oozie Share Lib HCatalog  SUCCESS [  2.088 
> s]
> [INFO] Apache Oozie Share Lib Distcp .. SUCCESS [  0.693 
> s]
> [INFO] Apache Oozie Core .. SUCCESS [ 20.951 
> s]
> [INFO] Apache Oozie Share Lib Streaming ... FAILURE [  2.688 
> s]
> [INFO] Apache Oozie Share Lib Pig . SKIPPED
> [INFO] Apache Oozie Share Lib Hive  SKIPPED
> [INFO] Apache Oozie Share Lib Hive 2 .. SKIPPED
> [INFO] Apache Oozie Share Lib Sqoop ... SKIPPED
> [INFO] Apache Oozie Examples .. SKIPPED
> [INFO] Apache Oozie Share Lib Spark ... SKIPPED
> [INFO] Apache Oozie Share Lib . SKIPPED
> [INFO] Apache Oozie Docs .. SKIPPED
> [INFO] Apache Oozie WebApp  SKIPPED
> [INFO] Apache Oozie Tools . SKIPPED
> [INFO] Apache Oozie MiniOozie . SKIPPED
> [INFO] Apache Oozie Distro  SKIPPED
> [INFO] Apache Oozie ZooKeeper Security Tests .. SKIPPED
> [INFO] 
> 
> [INFO] BUILD FAILURE
> [INFO] 
> 
> [INFO] Total time: 35.474 s
> [INFO] Finished at: 2016-04-04T16:04:41-07:00
> [INFO] Final Memory: 161M/1262M
> [INFO] 
> 
> [ERROR] Failed to execute goal 
> org.apache.maven.plugins:maven-compiler-plugin:2.3.2:testCompile 
> (default-testCompile) on project oozie-sharelib-streaming: Compilation 
> failure: Compilation failure:
> [ERROR] 
> /Users/saley/src/oozie/sharelib/streaming/src/test/java/org/apache/oozie/action/hadoop/TestStreamingMain.java:[31,39]
>  error: cannot find symbol
> [ERROR] class MainTestCase
> [ERROR] 
> /Users/saley/src/oozie/sharelib/streaming/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionExecutor.java:[76,49]
>  error: cannot find symbol
> [ERROR] class ActionExecutorTestCase
> [ERROR] 
> /Users/saley/src/oozie/sharelib/streaming/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionExecutor.java:[342,14]
>  error: cannot find symbol
> [ERROR] class TestMapReduceActionExecutor
> [ERROR] 
> /Users/saley/src/oozie/sharelib/streaming/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionExecutor.java:[365,14]
>  error: cannot find symbol
> [ERROR] class TestMapReduceActionExecutor
> [ERROR] 
> /Users/saley/src/oozie/sharelib/streaming/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionExecutor.java:[389,38]
>  error: cannot find symbol
> [ERROR] class TestMapReduceActionExecutor
> [ERROR] 
> /Users/saley/src/oozie/sharelib/streaming/src/test/java/org/apache/oozie/action/hadoop/TestStreamingMain.java:[34,24]
>  error: cannot find symbol
> [ERROR] class TestStreamingMain
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (OOZIE-2525) SchemaChecker fails with NPE

2016-09-15 Thread Robert Kanter (JIRA)


 [ 
https://issues.apache.org/jira/browse/OOZIE-2525?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Robert Kanter updated OOZIE-2525:
-
Priority: Critical  (was: Major)

> SchemaChecker fails with NPE
> 
>
> Key: OOZIE-2525
> URL: https://issues.apache.org/jira/browse/OOZIE-2525
> Project: Oozie
>  Issue Type: Bug
>Reporter: Purshotam Shah
>Assignee: Robert Kanter
>Priority: Critical
> Fix For: 4.3.0
>
> Attachments: OOZIE-2525.001.patch
>
>
> {code}
> 2016-05-03 14:38:03,851  WARN SchedulerService:523 [pool-1-thread-2] - 
> SERVER[-] Error executing runnable [SchemaCheckerRunnable], null
> java.lang.NullPointerException
> at 
> org.apache.oozie.service.SchemaCheckerService.access$002(SchemaCheckerService.java:29)
> at 
> org.apache.oozie.service.SchemaCheckerService$SchemaCheckerRunnable.run(SchemaCheckerService.java:120)
> at 
> org.apache.oozie.service.SchedulerService$2.run(SchedulerService.java:175)
> at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
> at 
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
> at 
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> {code}
> It;s trying to use SchemaCheckerService before it is initialized.
> {code:title=SchemaCheckerService.java}
> Services.get().get(SchemaCheckerService.class).status = "DISABLED (not leader 
> in HA)";
> Services.get().get(SchemaCheckerService.class).lastCheck = 
> "N/A";
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (OOZIE-2661) Coordinator Action status not updated when workflow job SUSPENDED

2016-09-15 Thread Satish Subhashrao Saley (JIRA)


 [ 
https://issues.apache.org/jira/browse/OOZIE-2661?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Satish Subhashrao Saley updated OOZIE-2661:
---
Attachment: OOZIE-2661-2.patch

> Coordinator Action status not updated when workflow job SUSPENDED
> -
>
> Key: OOZIE-2661
> URL: https://issues.apache.org/jira/browse/OOZIE-2661
> Project: Oozie
>  Issue Type: Bug
>Reporter: Satish Subhashrao Saley
>Assignee: Satish Subhashrao Saley
>Priority: Minor
> Attachments: OOZIE-2661-1.patch, OOZIE-2661-2.patch
>
>
> It's a bug in Oozie. CoordActionCheckXCommand doesn't take care of SUSPENDED 
> state. It only handles SUCCEEDED, FAILED and KILLED.
> {code}
> protected Void execute() throws CommandException {
> try {
> InstrumentUtils.incrJobCounter(getName(), 1, 
> getInstrumentation());
> Status slaStatus = null;
> CoordinatorAction.Status initialStatus = coordAction.getStatus();
> if (workflowJob.getStatus() == WorkflowJob.Status.SUCCEEDED) {
> coordAction.setStatus(CoordinatorAction.Status.SUCCEEDED);
> // set pending to false as the status is SUCCEEDED
> coordAction.setPending(0);
> slaStatus = Status.SUCCEEDED;
> }
> else {
> if (workflowJob.getStatus() == WorkflowJob.Status.FAILED) {
> coordAction.setStatus(CoordinatorAction.Status.FAILED);
> slaStatus = Status.FAILED;
> // set pending to false as the status is FAILED
> coordAction.setPending(0);
> }
> else {
> if (workflowJob.getStatus() == WorkflowJob.Status.KILLED) 
> {
> 
> coordAction.setStatus(CoordinatorAction.Status.KILLED);
> slaStatus = Status.KILLED;
> // set pending to false as the status is KILLED
> coordAction.setPending(0);
> }
> else {
> LOG.warn("Unexpected workflow " + workflowJob.getId() 
> + " STATUS " + workflowJob.getStatus());
> coordAction.setLastModifiedTime(new Date());
> CoordActionQueryExecutor.getInstance().executeUpdate(
> 
> CoordActionQueryExecutor.CoordActionQuery.UPDATE_COORD_ACTION_FOR_MODIFIED_DATE,
> coordAction);
> return null;
> }
> }
> }
> {code}
> Thank you [~puru] for pointing it out.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (OOZIE-2661) Coordinator Action status not updated when workflow job SUSPENDED

2016-09-15 Thread Satish Subhashrao Saley (JIRA)


 [ 
https://issues.apache.org/jira/browse/OOZIE-2661?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Satish Subhashrao Saley updated OOZIE-2661:
---
Attachment: (was: OOZIE-2661-2.patch)

> Coordinator Action status not updated when workflow job SUSPENDED
> -
>
> Key: OOZIE-2661
> URL: https://issues.apache.org/jira/browse/OOZIE-2661
> Project: Oozie
>  Issue Type: Bug
>Reporter: Satish Subhashrao Saley
>Assignee: Satish Subhashrao Saley
>Priority: Minor
> Attachments: OOZIE-2661-1.patch
>
>
> It's a bug in Oozie. CoordActionCheckXCommand doesn't take care of SUSPENDED 
> state. It only handles SUCCEEDED, FAILED and KILLED.
> {code}
> protected Void execute() throws CommandException {
> try {
> InstrumentUtils.incrJobCounter(getName(), 1, 
> getInstrumentation());
> Status slaStatus = null;
> CoordinatorAction.Status initialStatus = coordAction.getStatus();
> if (workflowJob.getStatus() == WorkflowJob.Status.SUCCEEDED) {
> coordAction.setStatus(CoordinatorAction.Status.SUCCEEDED);
> // set pending to false as the status is SUCCEEDED
> coordAction.setPending(0);
> slaStatus = Status.SUCCEEDED;
> }
> else {
> if (workflowJob.getStatus() == WorkflowJob.Status.FAILED) {
> coordAction.setStatus(CoordinatorAction.Status.FAILED);
> slaStatus = Status.FAILED;
> // set pending to false as the status is FAILED
> coordAction.setPending(0);
> }
> else {
> if (workflowJob.getStatus() == WorkflowJob.Status.KILLED) 
> {
> 
> coordAction.setStatus(CoordinatorAction.Status.KILLED);
> slaStatus = Status.KILLED;
> // set pending to false as the status is KILLED
> coordAction.setPending(0);
> }
> else {
> LOG.warn("Unexpected workflow " + workflowJob.getId() 
> + " STATUS " + workflowJob.getStatus());
> coordAction.setLastModifiedTime(new Date());
> CoordActionQueryExecutor.getInstance().executeUpdate(
> 
> CoordActionQueryExecutor.CoordActionQuery.UPDATE_COORD_ACTION_FOR_MODIFIED_DATE,
> coordAction);
> return null;
> }
> }
> }
> {code}
> Thank you [~puru] for pointing it out.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (OOZIE-2538) Update HttpClient versions to close security vulnerabilities

2016-09-15 Thread Robert Kanter (JIRA)


[ 
https://issues.apache.org/jira/browse/OOZIE-2538?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15494931#comment-15494931
 ] 

Robert Kanter commented on OOZIE-2538:
--

Why do we specify the versions in the webapp pom?  Shouldn't they go in the 
root pom?

> Update HttpClient versions to close security vulnerabilities
> 
>
> Key: OOZIE-2538
> URL: https://issues.apache.org/jira/browse/OOZIE-2538
> Project: Oozie
>  Issue Type: Bug
>  Components: core
>Reporter: Abhishek Bafna
>Assignee: Abhishek Bafna
> Fix For: 4.3.0
>
> Attachments: OOZIE-2538-01.patch, OOZIE-2538.patch
>
>
> We learned that
> https://web.nvd.nist.gov/view/vuln/detail?vulnId=CVE-2015-5262 : 
> http/conn/ssl/SSLConnectionSocketFactory.java in Apache HttpComponents 
> HttpClient before 4.3.6 ignores the http.socket.timeout configuration setting 
> during an SSL handshake, which allows remote attackers to cause a denial of 
> service (HTTPS call hang) via unspecified vectors.
> Also, Commons HttpClient project is now end of life, and is no longer being 
> developed. It has been replaced by the Apache HttpComponents project in its 
> HttpClient and HttpCore modules, which offer better performance and more 
> flexibility.  http://hc.apache.org/httpclient-3.x/
> Hence, HttpClient version should be updated.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (OOZIE-2571) Add spark.scala.binary.version Maven property so that Scala 2.11 can be used

2016-09-15 Thread Robert Kanter (JIRA)


 [ 
https://issues.apache.org/jira/browse/OOZIE-2571?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Robert Kanter updated OOZIE-2571:
-
Summary: Add spark.scala.binary.version Maven property so that Scala 2.11 
can be used  (was: Add scala.binary.version Maven property so that Scala 2.11 
can be used)

> Add spark.scala.binary.version Maven property so that Scala 2.11 can be used
> 
>
> Key: OOZIE-2571
> URL: https://issues.apache.org/jira/browse/OOZIE-2571
> Project: Oozie
>  Issue Type: New Feature
>  Components: build
>Affects Versions: 4.2.0
>Reporter: Jonathan Kelly
>Assignee: Jonathan Kelly
> Fix For: 4.3.0
>
> Attachments: OOZIE-2571.patch
>
>
> The pom.xml files hardcode a Scala binary version of Scala 2.10 for its Spark 
> dependencies, making it impossible to use a version of Spark built against 
> Scala 2.11.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (OOZIE-2571) Add scala.binary.version Maven property so that Scala 2.11 can be used

2016-09-15 Thread Robert Kanter (JIRA)


 [ 
https://issues.apache.org/jira/browse/OOZIE-2571?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Robert Kanter updated OOZIE-2571:
-
Assignee: Jonathan Kelly

> Add scala.binary.version Maven property so that Scala 2.11 can be used
> --
>
> Key: OOZIE-2571
> URL: https://issues.apache.org/jira/browse/OOZIE-2571
> Project: Oozie
>  Issue Type: New Feature
>  Components: build
>Affects Versions: 4.2.0
>Reporter: Jonathan Kelly
>Assignee: Jonathan Kelly
> Fix For: 4.3.0
>
> Attachments: OOZIE-2571.patch
>
>
> The pom.xml files hardcode a Scala binary version of Scala 2.10 for its Spark 
> dependencies, making it impossible to use a version of Spark built against 
> Scala 2.11.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (OOZIE-2571) Add scala.binary.version Maven property so that Scala 2.11 can be used

2016-09-15 Thread Robert Kanter (JIRA)


[ 
https://issues.apache.org/jira/browse/OOZIE-2571?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15494897#comment-15494897
 ] 

Robert Kanter commented on OOZIE-2571:
--

+1, though I'm going to rename the property from {{scala.binary.version}} to 
{{spark.scala.binary.version}}.

> Add scala.binary.version Maven property so that Scala 2.11 can be used
> --
>
> Key: OOZIE-2571
> URL: https://issues.apache.org/jira/browse/OOZIE-2571
> Project: Oozie
>  Issue Type: New Feature
>  Components: build
>Affects Versions: 4.2.0
>Reporter: Jonathan Kelly
> Fix For: 4.3.0
>
> Attachments: OOZIE-2571.patch
>
>
> The pom.xml files hardcode a Scala binary version of Scala 2.10 for its Spark 
> dependencies, making it impossible to use a version of Spark built against 
> Scala 2.11.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (OOZIE-2613) Upgrade hive version from 0.13.1 to 1.2.0

2016-09-15 Thread Robert Kanter (JIRA)


[ 
https://issues.apache.org/jira/browse/OOZIE-2613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15494890#comment-15494890
 ] 

Robert Kanter commented on OOZIE-2613:
--

I think it's fine to drop it in 4.3 (I assume that's what you meant, not 4.2).

> Upgrade hive version from 0.13.1 to 1.2.0
> -
>
> Key: OOZIE-2613
> URL: https://issues.apache.org/jira/browse/OOZIE-2613
> Project: Oozie
>  Issue Type: Bug
>  Components: core
>Reporter: Abhishek Bafna
>Assignee: Abhishek Bafna
> Fix For: 4.3.0
>
> Attachments: OOZIE-2613-00.patch, OOZIE-2613-01.patch, 
> OOZIE-2613-02.patch, OOZIE-2613-03.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (OOZIE-2646) Spark Action Improvement

2016-09-15 Thread Robert Kanter (JIRA)


 [ 
https://issues.apache.org/jira/browse/OOZIE-2646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Robert Kanter resolved OOZIE-2646.
--
   Resolution: Invalid
Fix Version/s: (was: 4.4.0)
   (was: 4.3.0)

> Spark Action Improvement
> 
>
> Key: OOZIE-2646
> URL: https://issues.apache.org/jira/browse/OOZIE-2646
> Project: Oozie
>  Issue Type: Improvement
>  Components: action
>Affects Versions: 4.2.0
>Reporter: Rui Xue
>  Labels: easyfix
>
> In the current mapr release (4.2.0), Oozie's Spark action still cannot accept 
> the value of the "--conf" option with double quotation. For instance, in 
> Spark-submit, the value of --conf "xxx" is allowed, whereas in Oozie it is 
> not allowed, and this is required sometimes anyway in practice



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (OOZIE-2646) Spark Action Improvement

2016-09-15 Thread Robert Kanter (JIRA)


[ 
https://issues.apache.org/jira/browse/OOZIE-2646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15494882#comment-15494882
 ] 

Robert Kanter commented on OOZIE-2646:
--

[~rui.xue], thanks for your interest in contributing to Apache Oozie!  We don't 
actually do pull requests; we attach a diff patch file to the JIRA.  However, 
your pull request looks like it's meant to be against MapR's version of Oozie, 
and contains a lot of other changes and doesn't match Apache Oozie's current 
code.  On top of that, as [~gezapeti] mentioned, OOZIE-2391 already fixes this 
issue.  I recommend you contact MapR about including OOZIE-2391.

> Spark Action Improvement
> 
>
> Key: OOZIE-2646
> URL: https://issues.apache.org/jira/browse/OOZIE-2646
> Project: Oozie
>  Issue Type: Improvement
>  Components: action
>Affects Versions: 4.2.0
>Reporter: Rui Xue
>  Labels: easyfix
>
> In the current mapr release (4.2.0), Oozie's Spark action still cannot accept 
> the value of the "--conf" option with double quotation. For instance, in 
> Spark-submit, the value of --conf "xxx" is allowed, whereas in Oozie it is 
> not allowed, and this is required sometimes anyway in practice



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (OOZIE-2651) Set javax.xml.parsers.DocumentBuilderFactory sys prop to make XML handling faster

2016-09-15 Thread Robert Kanter (JIRA)


 [ 
https://issues.apache.org/jira/browse/OOZIE-2651?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Robert Kanter updated OOZIE-2651:
-
Fix Version/s: (was: 4.3.0)

> Set javax.xml.parsers.DocumentBuilderFactory sys prop to make XML handling 
> faster
> -
>
> Key: OOZIE-2651
> URL: https://issues.apache.org/jira/browse/OOZIE-2651
> Project: Oozie
>  Issue Type: Improvement
>  Components: core
>Affects Versions: trunk
>Reporter: Robert Kanter
>Assignee: Robert Kanter
> Attachments: OOZIE-2651.001.patch
>
>
> We noticed from some jstacks that Oozie was spending a lot of time looking 
> for XML related jars (we do a lot of XML parsing in Oozie and Hadoop code).  
> For instance, we'd see hundreds of threads blocked here
> {noformat}
> "pool-5-thread-57" prio=10 tid=0x7f9579923000 nid=0x6b7f waiting for 
> monitor entry [0x7f94225e3000]
>java.lang.Thread.State: BLOCKED (on object monitor)
> at 
> org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1559)
> - waiting to lock <0x000500026148> (a 
> org.apache.catalina.loader.WebappClassLoader)
> at 
> org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1526)
> at 
> javax.xml.parsers.FactoryFinder.getProviderClass(FactoryFinder.java:117)
> at javax.xml.parsers.FactoryFinder.newInstance(FactoryFinder.java:178)
> at 
> javax.xml.parsers.FactoryFinder.findJarServiceProvider(FactoryFinder.java:333)
> at javax.xml.parsers.FactoryFinder.find(FactoryFinder.java:255)
> at 
> javax.xml.parsers.DocumentBuilderFactory.newInstance(DocumentBuilderFactory.java:121)
> at 
> org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:2526)
> at 
> org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:2503)
> at 
> org.apache.hadoop.conf.Configuration.getProps(Configuration.java:2409)
> - locked <0x0007eac5ea28> (a org.apache.hadoop.conf.Configuration)
> at org.apache.hadoop.conf.Configuration.set(Configuration.java:1144)
> at org.apache.hadoop.conf.Configuration.set(Configuration.java:1116)
> at org.apache.oozie.util.XConfiguration.copy(XConfiguration.java:206)
> ...
> {noformat}
> There's nothing we can do about the lock at the {{WebappClassLoader}}, but by 
> setting the {{javax.xml.parsers.DocumentBuilderFactory}} system property to 
> {{org.apache.xerces.jaxp.DocumentBuilderFactoryImpl}}, we can speed this up a 
> lot by getting rid of all of the searching that it's doing to find the 
> implementation.  
> See 
> https://docs.oracle.com/javase/7/docs/api/javax/xml/parsers/DocumentBuilderFactory.html#newInstance()
>  
> and 
> https://www.databasesandlife.com/java-always-explicitly-specify-which-xml-parser-to-use/
> In our testing, we had a metric around creating a new {{JobClient}}, which 
> also calls {{DocumentBuilderFactory.newInstance}}, and it dropped from ~3min 
> max to ~2sec max (not an exact test of the savings here).  And we also didn't 
> see so much blocking on this in the jstacks anymore.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (OOZIE-2530) Allow Hive to use a different jline version

2016-09-15 Thread Robert Kanter (JIRA)


 [ 
https://issues.apache.org/jira/browse/OOZIE-2530?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Robert Kanter updated OOZIE-2530:
-
Assignee: Thomas Poepping

> Allow Hive to use a different jline version
> ---
>
> Key: OOZIE-2530
> URL: https://issues.apache.org/jira/browse/OOZIE-2530
> Project: Oozie
>  Issue Type: Improvement
>Affects Versions: trunk
>Reporter: Thomas Poepping
>Assignee: Thomas Poepping
> Attachments: OOZIE-2530.01.patch, OOZIE-2530.02.patch, 
> OOZIE-2530.03.patch, OOZIE-2530.patch
>
>
> This patch removes an unnecessary hardcoded dependency on jline in the oozie 
> parent pom and pig sharelib pom.
> The jline dependencies were required before OOZIE-704, which removed the 
> direct dependencies on jline in oozie code. Since that patch, the direct 
> dependency on jline in oozie has been unnecessary. This will also fix 
> OOZIE-2409.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (OOZIE-2530) Allow Hive to use a different jline version

2016-09-15 Thread Robert Kanter (JIRA)


[ 
https://issues.apache.org/jira/browse/OOZIE-2530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15494859#comment-15494859
 ] 

Robert Kanter commented on OOZIE-2530:
--

+1

> Allow Hive to use a different jline version
> ---
>
> Key: OOZIE-2530
> URL: https://issues.apache.org/jira/browse/OOZIE-2530
> Project: Oozie
>  Issue Type: Improvement
>Affects Versions: trunk
>Reporter: Thomas Poepping
> Attachments: OOZIE-2530.01.patch, OOZIE-2530.02.patch, 
> OOZIE-2530.03.patch, OOZIE-2530.patch
>
>
> This patch removes an unnecessary hardcoded dependency on jline in the oozie 
> parent pom and pig sharelib pom.
> The jline dependencies were required before OOZIE-704, which removed the 
> direct dependencies on jline in oozie code. Since that patch, the direct 
> dependency on jline in oozie has been unnecessary. This will also fix 
> OOZIE-2409.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (OOZIE-2572) SLA DURATION miss not shown when job is running for longer than expected time

2016-09-15 Thread Satish Subhashrao Saley (JIRA)


 [ 
https://issues.apache.org/jira/browse/OOZIE-2572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Satish Subhashrao Saley updated OOZIE-2572:
---
Attachment: OOZIE-2572-10.patch

> SLA DURATION miss not shown when job is running for longer than expected time
> -
>
> Key: OOZIE-2572
> URL: https://issues.apache.org/jira/browse/OOZIE-2572
> Project: Oozie
>  Issue Type: Bug
>Reporter: Satish Subhashrao Saley
>Assignee: Satish Subhashrao Saley
> Fix For: 4.3.0
>
> Attachments: OOZIE-2572-1.patch, OOZIE-2572-10.patch, 
> OOZIE-2572-2.patch, OOZIE-2572-3.patch, OOZIE-2572-4.patch, 
> OOZIE-2572-5.patch, OOZIE-2572-6.patch, OOZIE-2572-7.patch, 
> OOZIE-2572-8.patch, OOZIE-2572-9.patch
>
>
> If job is running for beyond expected duration, Oozie UI is not slowing SLA 
> for duration miss.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (OOZIE-2530) Allow Hive to use a different jline version

2016-09-15 Thread Robert Kanter (JIRA)


 [ 
https://issues.apache.org/jira/browse/OOZIE-2530?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Robert Kanter updated OOZIE-2530:
-
Summary: Allow Hive to use a different jline version  (was: Remove jline 
from parent and sharelib pig pom)

> Allow Hive to use a different jline version
> ---
>
> Key: OOZIE-2530
> URL: https://issues.apache.org/jira/browse/OOZIE-2530
> Project: Oozie
>  Issue Type: Improvement
>Affects Versions: trunk
>Reporter: Thomas Poepping
> Attachments: OOZIE-2530.01.patch, OOZIE-2530.02.patch, 
> OOZIE-2530.03.patch, OOZIE-2530.patch
>
>
> This patch removes an unnecessary hardcoded dependency on jline in the oozie 
> parent pom and pig sharelib pom.
> The jline dependencies were required before OOZIE-704, which removed the 
> direct dependencies on jline in oozie code. Since that patch, the direct 
> dependency on jline in oozie has been unnecessary. This will also fix 
> OOZIE-2409.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (OOZIE-2606) Set spark.yarn.jars to fix Spark 2.0 with Oozie

2016-09-15 Thread Satish Subhashrao Saley (JIRA)


 [ 
https://issues.apache.org/jira/browse/OOZIE-2606?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Satish Subhashrao Saley updated OOZIE-2606:
---
Attachment: (was: OOZIE-2606-2.patch)

> Set spark.yarn.jars to fix Spark 2.0 with Oozie
> ---
>
> Key: OOZIE-2606
> URL: https://issues.apache.org/jira/browse/OOZIE-2606
> Project: Oozie
>  Issue Type: Bug
>  Components: core
>Affects Versions: 4.2.0
>Reporter: Jonathan Kelly
>Assignee: Satish Subhashrao Saley
>  Labels: spark, spark2.0.0
> Fix For: 4.3.0
>
> Attachments: OOZIE-2606-2.patch, OOZIE-2606.patch
>
>
> Oozie adds all of the jars in the Oozie Spark sharelib to the 
> DistributedCache such that all jars will be present in the current working 
> directory of the YARN container (as well as in the container classpath). 
> However, this is not quite enough to make Spark 2.0 work, since Spark 2.0 by 
> default looks for the jars in assembly/target/scala-2.11/jars [1] (as if it 
> is a locally built distribution for development) and will not find them in 
> the current working directory.
> To fix this, we can set spark.yarn.jars to *.jar so that it finds the jars in 
> the current working directory rather than looking in the wrong place. [2]
> [1] 
> https://github.com/apache/spark/blob/v2.0.0-rc2/launcher/src/main/java/org/apache/spark/launcher/CommandBuilderUtils.java#L357
> [2] 
> https://github.com/apache/spark/blob/v2.0.0-rc2/yarn/src/main/scala/org/apache/spark/deploy/yarn/Client.scala#L476
> Note: This property will be ignored by Spark 1.x.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (OOZIE-2606) Set spark.yarn.jars to fix Spark 2.0 with Oozie

2016-09-15 Thread Satish Subhashrao Saley (JIRA)


 [ 
https://issues.apache.org/jira/browse/OOZIE-2606?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Satish Subhashrao Saley updated OOZIE-2606:
---
Attachment: OOZIE-2606-2.patch

> Set spark.yarn.jars to fix Spark 2.0 with Oozie
> ---
>
> Key: OOZIE-2606
> URL: https://issues.apache.org/jira/browse/OOZIE-2606
> Project: Oozie
>  Issue Type: Bug
>  Components: core
>Affects Versions: 4.2.0
>Reporter: Jonathan Kelly
>Assignee: Satish Subhashrao Saley
>  Labels: spark, spark2.0.0
> Fix For: 4.3.0
>
> Attachments: OOZIE-2606-2.patch, OOZIE-2606.patch
>
>
> Oozie adds all of the jars in the Oozie Spark sharelib to the 
> DistributedCache such that all jars will be present in the current working 
> directory of the YARN container (as well as in the container classpath). 
> However, this is not quite enough to make Spark 2.0 work, since Spark 2.0 by 
> default looks for the jars in assembly/target/scala-2.11/jars [1] (as if it 
> is a locally built distribution for development) and will not find them in 
> the current working directory.
> To fix this, we can set spark.yarn.jars to *.jar so that it finds the jars in 
> the current working directory rather than looking in the wrong place. [2]
> [1] 
> https://github.com/apache/spark/blob/v2.0.0-rc2/launcher/src/main/java/org/apache/spark/launcher/CommandBuilderUtils.java#L357
> [2] 
> https://github.com/apache/spark/blob/v2.0.0-rc2/yarn/src/main/scala/org/apache/spark/deploy/yarn/Client.scala#L476
> Note: This property will be ignored by Spark 1.x.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (OOZIE-2606) Set spark.yarn.jars to fix Spark 2.0 with Oozie

2016-09-15 Thread Hadoop QA (JIRA)


[ 
https://issues.apache.org/jira/browse/OOZIE-2606?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15494710#comment-15494710
 ] 

Hadoop QA commented on OOZIE-2606:
--

Testing JIRA OOZIE-2606

Cleaning local git workspace



{color:red}-1{color} Patch failed to apply to head of branch



> Set spark.yarn.jars to fix Spark 2.0 with Oozie
> ---
>
> Key: OOZIE-2606
> URL: https://issues.apache.org/jira/browse/OOZIE-2606
> Project: Oozie
>  Issue Type: Bug
>  Components: core
>Affects Versions: 4.2.0
>Reporter: Jonathan Kelly
>Assignee: Satish Subhashrao Saley
>  Labels: spark, spark2.0.0
> Fix For: 4.3.0
>
> Attachments: OOZIE-2606-2.patch, OOZIE-2606.patch
>
>
> Oozie adds all of the jars in the Oozie Spark sharelib to the 
> DistributedCache such that all jars will be present in the current working 
> directory of the YARN container (as well as in the container classpath). 
> However, this is not quite enough to make Spark 2.0 work, since Spark 2.0 by 
> default looks for the jars in assembly/target/scala-2.11/jars [1] (as if it 
> is a locally built distribution for development) and will not find them in 
> the current working directory.
> To fix this, we can set spark.yarn.jars to *.jar so that it finds the jars in 
> the current working directory rather than looking in the wrong place. [2]
> [1] 
> https://github.com/apache/spark/blob/v2.0.0-rc2/launcher/src/main/java/org/apache/spark/launcher/CommandBuilderUtils.java#L357
> [2] 
> https://github.com/apache/spark/blob/v2.0.0-rc2/yarn/src/main/scala/org/apache/spark/deploy/yarn/Client.scala#L476
> Note: This property will be ignored by Spark 1.x.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Build failed in Jenkins: oozie-trunk-precommit-build #3277

2016-09-15 Thread Apache Jenkins Server

See 

--
[...truncated 1430 lines...]
[INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ oozie-docs ---
[INFO] Deleting 

[INFO] 
[INFO] --- maven-remote-resources-plugin:1.5:process (default) @ oozie-docs ---
[INFO] 
[INFO] --- maven-resources-plugin:2.5:resources (default-resources) @ 
oozie-docs ---
[debug] execute contextualize
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory 

[INFO] Copying 3 resources
[INFO] 
[INFO] --- maven-compiler-plugin:2.3.2:compile (default-compile) @ oozie-docs 
---
[INFO] Compiling 1 source file to 

[INFO] 
[INFO] --- maven-resources-plugin:2.5:testResources (default-testResources) @ 
oozie-docs ---
[debug] execute contextualize
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory 

[INFO] Copying 3 resources
[INFO] 
[INFO] --- maven-compiler-plugin:2.3.2:testCompile (default-testCompile) @ 
oozie-docs ---
[INFO] No sources to compile
[INFO] 
[INFO] --- maven-surefire-plugin:2.12.2:test (default-test) @ oozie-docs ---
[INFO] Tests are skipped.
[INFO] 
[INFO] 
[INFO] Building Apache Oozie WebApp 4.3.0-SNAPSHOT
[INFO] 
[INFO] 
[INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ oozie-webapp ---
[INFO] Deleting 

[INFO] 
[INFO] --- maven-remote-resources-plugin:1.5:process (default) @ oozie-webapp 
---
[INFO] 
[INFO] --- maven-resources-plugin:2.5:resources (default-resources) @ 
oozie-webapp ---
[debug] execute contextualize
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 1 resource
[INFO] Copying 3 resources
[INFO] 
[INFO] --- maven-compiler-plugin:2.3.2:compile (default-compile) @ oozie-webapp 
---
[INFO] No sources to compile
[INFO] 
[INFO] --- maven-resources-plugin:2.5:testResources (default-testResources) @ 
oozie-webapp ---
[debug] execute contextualize
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory 

[INFO] Copying 3 resources
[INFO] 
[INFO] --- maven-compiler-plugin:2.3.2:testCompile (default-testCompile) @ 
oozie-webapp ---
[INFO] No sources to compile
[INFO] 
[INFO] --- maven-surefire-plugin:2.12.2:test (default-test) @ oozie-webapp ---
[INFO] Tests are skipped.
[INFO] 
[INFO] 
[INFO] Building Apache Oozie Tools 4.3.0-SNAPSHOT
[INFO] 
[INFO] 
[INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ oozie-tools ---
[INFO] Deleting 

[INFO] 
[INFO] --- maven-remote-resources-plugin:1.5:process (default) @ oozie-tools ---
[INFO] 
[INFO] --- maven-resources-plugin:2.5:resources (default-resources) @ 
oozie-tools ---
[debug] execute contextualize
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory 

[INFO] Copying 3 resources
[INFO] 
[INFO] --- maven-compiler-plugin:2.3.2:compile (default-compile) @ oozie-tools 
---
[INFO] Compiling 4 source files to 

[INFO] 
[INFO] --- maven-resources-plugin:2.5:testResources (default-testResources) @ 
oozie-tools ---
[debug] execute contextualize
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 11 resources
[INFO] Copying 3 resources
[INFO] 
[INFO] --- maven-compiler-plugin:2.3.2:testCompile (default-testCompile) @ 
oozie-tools ---
[INFO] Compiling 7 source files to 

[INFO] 
[INFO] --- maven-surefire-plugin:2.12.2:test (default-test) @ oozie-tools ---
[INFO] Tests are skipped.
[INFO] 
[INFO] 
[INFO] Building Apache Oozie MiniOozie 4.3.0-SNAPSHOT
[INFO] --

[jira] [Comment Edited] (OOZIE-2606) Set spark.yarn.jars to fix Spark 2.0 with Oozie

2016-09-15 Thread Satish Subhashrao Saley (JIRA)


[ 
https://issues.apache.org/jira/browse/OOZIE-2606?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15494685#comment-15494685
 ] 

Satish Subhashrao Saley edited comment on OOZIE-2606 at 9/15/16 10:20 PM:
--

- In case of spark 1.X, we *can* have both {{spark.yarn.jar}} and 
{{spark.yarn.jars}}. {{spark.yarn.jars}} will get ignored anyway.
- By setting {{spark.yarn.jar}} configuration in case of spark-1, we will avoid 
multiple distribution of spark-yarn/spark-assembly jar.
- In case of spark 2.X, we *cannot* have both {{spark.yarn.jar}} and 
{{spark.yarn.jars}}. It causes problems.

- The approach in the patch is to look at the spark version and populate the 
configuration accordingly. For checking the spark version, I am checking the 
"Specification-Version" field in the jar manifest. (Any cleaner alternatives?)
- We are still keeping the {{-files}} option as it is required in case of 
spark-1 and not causing any issues with spark-2 even if some of the uris 
present in {{spark.yarn.jars}} and {{-files}} are same. Files will get 
distributed only once. 
- Also, for spark-2.X, we need to bump up the versions of some libraries. I 
created a profile for spark-2 and spark-1 (spark-1 being the default). For me, 
spark-1.X did not work with newer version of those libraries.


was (Author: satishsaley):
- In case of spark 1.X, we *can* have both {{spark.yarn.jar}} and 
{{spark.yarn.jars}}. {{spark.yarn.jars}} will get ignored anyway.
- By setting {{spark.yarn.jar}} configuration in case of spark-1, we will avoid 
multiple distribution of spark-yarn/spark-assembly jar.
- In case of spark 2.X, we *cannot* have both {{spark.yarn.jar}} and 
{{spark.yarn.jars}}. It causes problems.

- The approach in the patch is to look at the spark version and populate the 
configuration accordingly. For checking the spark version, I am checking the 
"Specification-Version" field in the jar manifest. (Any cleaner alternatives?)
- We are still keeping the {{--files}} option as it is required in case of 
spark-1 and not causing any issues with spark-2 even if some of the uris 
present in {{spark.yarn.jars}} and {{--files}} are same. Files will get 
distributed only once. 
- Also, for spark-2.X, we need to bump up the versions of some libraries. I 
created a profile for spark-2 and spark-1 (spark-1 being the default). For me, 
spark-1.X did not work with newer version of those libraries.

> Set spark.yarn.jars to fix Spark 2.0 with Oozie
> ---
>
> Key: OOZIE-2606
> URL: https://issues.apache.org/jira/browse/OOZIE-2606
> Project: Oozie
>  Issue Type: Bug
>  Components: core
>Affects Versions: 4.2.0
>Reporter: Jonathan Kelly
>Assignee: Satish Subhashrao Saley
>  Labels: spark, spark2.0.0
> Fix For: 4.3.0
>
> Attachments: OOZIE-2606-2.patch, OOZIE-2606.patch
>
>
> Oozie adds all of the jars in the Oozie Spark sharelib to the 
> DistributedCache such that all jars will be present in the current working 
> directory of the YARN container (as well as in the container classpath). 
> However, this is not quite enough to make Spark 2.0 work, since Spark 2.0 by 
> default looks for the jars in assembly/target/scala-2.11/jars [1] (as if it 
> is a locally built distribution for development) and will not find them in 
> the current working directory.
> To fix this, we can set spark.yarn.jars to *.jar so that it finds the jars in 
> the current working directory rather than looking in the wrong place. [2]
> [1] 
> https://github.com/apache/spark/blob/v2.0.0-rc2/launcher/src/main/java/org/apache/spark/launcher/CommandBuilderUtils.java#L357
> [2] 
> https://github.com/apache/spark/blob/v2.0.0-rc2/yarn/src/main/scala/org/apache/spark/deploy/yarn/Client.scala#L476
> Note: This property will be ignored by Spark 1.x.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Comment Edited] (OOZIE-2606) Set spark.yarn.jars to fix Spark 2.0 with Oozie

2016-09-15 Thread Satish Subhashrao Saley (JIRA)


[ 
https://issues.apache.org/jira/browse/OOZIE-2606?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15494685#comment-15494685
 ] 

Satish Subhashrao Saley edited comment on OOZIE-2606 at 9/15/16 10:18 PM:
--

- In case of spark 1.X, we *can* have both {{spark.yarn.jar}} and 
{{spark.yarn.jars}}. {{spark.yarn.jars}} will get ignored anyway.
- By setting {{spark.yarn.jar}} configuration in case of spark-1, we will avoid 
multiple distribution of spark-yarn/spark-assembly jar.
- In case of spark 2.X, we *cannot* have both {{spark.yarn.jar}} and 
{{spark.yarn.jars}}. It causes problems.

- The approach in the patch is to look at the spark version and populate the 
configuration accordingly. For checking the spark version, I am checking the 
"Specification-Version" field in the jar manifest. (Any cleaner alternatives?)
- We are still keeping the {{--files}} option as it is required in case of 
spark-1 and not causing any issues with spark-2 even if some of the uris 
present in {{spark.yarn.jars}} and {{--files}} are same. Files will get 
distributed only once. 
- Also, for spark-2.X, we need to bump up the versions of some libraries. I 
created a profile for spark-2 and spark-1 (spark-1 being the default). For me, 
spark-1.X did not work with newer version of those libraries.


was (Author: satishsaley):
- In case of spark 1.X, we *can* have both {{spark.yarn.jar}} and 
{{spark.yarn.jars}}. {{spark.yarn.jars}} will get ignored anyway.
- By setting {{spark.yarn.jar}} configuration in case of spark-1, we will avoid 
multiple distribution of spark-yarn/spark-assembly jar.
- In case of spark 2.X, we *cannot* have both {{spark.yarn.jar}} and 
{{spark.yarn.jars}}. It causes problems.

-The approach in the patch is to look at the spark version and populate the 
configuration accordingly. For checking the spark version, I am checking the 
"Specification-Version" field in the jar manifest. (Any cleaner alternatives?)
- We are still keeping the {{--files}} option as it is required in case of 
spark-1 and not causing any issues with spark-2 even if some of the uris 
present in {{spark.yarn.jars}} and {{--files}} are same. Files will get 
distributed only once. 
- Also, for spark-2.X, we need to bump up the versions of some libraries. I 
created a profile for spark-2 and spark-1 (spark-1 being the default). For me, 
spark-1.X did not work with newer version of those libraries.

> Set spark.yarn.jars to fix Spark 2.0 with Oozie
> ---
>
> Key: OOZIE-2606
> URL: https://issues.apache.org/jira/browse/OOZIE-2606
> Project: Oozie
>  Issue Type: Bug
>  Components: core
>Affects Versions: 4.2.0
>Reporter: Jonathan Kelly
>Assignee: Satish Subhashrao Saley
>  Labels: spark, spark2.0.0
> Fix For: 4.3.0
>
> Attachments: OOZIE-2606-2.patch, OOZIE-2606.patch
>
>
> Oozie adds all of the jars in the Oozie Spark sharelib to the 
> DistributedCache such that all jars will be present in the current working 
> directory of the YARN container (as well as in the container classpath). 
> However, this is not quite enough to make Spark 2.0 work, since Spark 2.0 by 
> default looks for the jars in assembly/target/scala-2.11/jars [1] (as if it 
> is a locally built distribution for development) and will not find them in 
> the current working directory.
> To fix this, we can set spark.yarn.jars to *.jar so that it finds the jars in 
> the current working directory rather than looking in the wrong place. [2]
> [1] 
> https://github.com/apache/spark/blob/v2.0.0-rc2/launcher/src/main/java/org/apache/spark/launcher/CommandBuilderUtils.java#L357
> [2] 
> https://github.com/apache/spark/blob/v2.0.0-rc2/yarn/src/main/scala/org/apache/spark/deploy/yarn/Client.scala#L476
> Note: This property will be ignored by Spark 1.x.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (OOZIE-2606) Set spark.yarn.jars to fix Spark 2.0 with Oozie

2016-09-15 Thread Satish Subhashrao Saley (JIRA)


 [ 
https://issues.apache.org/jira/browse/OOZIE-2606?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Satish Subhashrao Saley updated OOZIE-2606:
---
Attachment: OOZIE-2606-2.patch

> Set spark.yarn.jars to fix Spark 2.0 with Oozie
> ---
>
> Key: OOZIE-2606
> URL: https://issues.apache.org/jira/browse/OOZIE-2606
> Project: Oozie
>  Issue Type: Bug
>  Components: core
>Affects Versions: 4.2.0
>Reporter: Jonathan Kelly
>Assignee: Satish Subhashrao Saley
>  Labels: spark, spark2.0.0
> Fix For: 4.3.0
>
> Attachments: OOZIE-2606-2.patch, OOZIE-2606.patch
>
>
> Oozie adds all of the jars in the Oozie Spark sharelib to the 
> DistributedCache such that all jars will be present in the current working 
> directory of the YARN container (as well as in the container classpath). 
> However, this is not quite enough to make Spark 2.0 work, since Spark 2.0 by 
> default looks for the jars in assembly/target/scala-2.11/jars [1] (as if it 
> is a locally built distribution for development) and will not find them in 
> the current working directory.
> To fix this, we can set spark.yarn.jars to *.jar so that it finds the jars in 
> the current working directory rather than looking in the wrong place. [2]
> [1] 
> https://github.com/apache/spark/blob/v2.0.0-rc2/launcher/src/main/java/org/apache/spark/launcher/CommandBuilderUtils.java#L357
> [2] 
> https://github.com/apache/spark/blob/v2.0.0-rc2/yarn/src/main/scala/org/apache/spark/deploy/yarn/Client.scala#L476
> Note: This property will be ignored by Spark 1.x.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (OOZIE-2606) Set spark.yarn.jars to fix Spark 2.0 with Oozie

2016-09-15 Thread Satish Subhashrao Saley (JIRA)


[ 
https://issues.apache.org/jira/browse/OOZIE-2606?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15494685#comment-15494685
 ] 

Satish Subhashrao Saley commented on OOZIE-2606:


- In case of spark 1.X, we *can* have both {{spark.yarn.jar}} and 
{{spark.yarn.jars}}. {{spark.yarn.jars}} will get ignored anyway.
- By setting {{spark.yarn.jar}} configuration in case of spark-1, we will avoid 
multiple distribution of spark-yarn/spark-assembly jar.
- In case of spark 2.X, we *cannot* have both {{spark.yarn.jar}} and 
{{spark.yarn.jars}}. It causes problems.

-The approach in the patch is to look at the spark version and populate the 
configuration accordingly. For checking the spark version, I am checking the 
"Specification-Version" field in the jar manifest. (Any cleaner alternatives?)
- We are still keeping the {{--files}} option as it is required in case of 
spark-1 and not causing any issues with spark-2 even if some of the uris 
present in {{spark.yarn.jars}} and {{--files}} are same. Files will get 
distributed only once. 
- Also, for spark-2.X, we need to bump up the versions of some libraries. I 
created a profile for spark-2 and spark-1 (spark-1 being the default). For me, 
spark-1.X did not work with newer version of those libraries.

> Set spark.yarn.jars to fix Spark 2.0 with Oozie
> ---
>
> Key: OOZIE-2606
> URL: https://issues.apache.org/jira/browse/OOZIE-2606
> Project: Oozie
>  Issue Type: Bug
>  Components: core
>Affects Versions: 4.2.0
>Reporter: Jonathan Kelly
>Assignee: Satish Subhashrao Saley
>  Labels: spark, spark2.0.0
> Fix For: 4.3.0
>
> Attachments: OOZIE-2606-2.patch, OOZIE-2606.patch
>
>
> Oozie adds all of the jars in the Oozie Spark sharelib to the 
> DistributedCache such that all jars will be present in the current working 
> directory of the YARN container (as well as in the container classpath). 
> However, this is not quite enough to make Spark 2.0 work, since Spark 2.0 by 
> default looks for the jars in assembly/target/scala-2.11/jars [1] (as if it 
> is a locally built distribution for development) and will not find them in 
> the current working directory.
> To fix this, we can set spark.yarn.jars to *.jar so that it finds the jars in 
> the current working directory rather than looking in the wrong place. [2]
> [1] 
> https://github.com/apache/spark/blob/v2.0.0-rc2/launcher/src/main/java/org/apache/spark/launcher/CommandBuilderUtils.java#L357
> [2] 
> https://github.com/apache/spark/blob/v2.0.0-rc2/yarn/src/main/scala/org/apache/spark/deploy/yarn/Client.scala#L476
> Note: This property will be ignored by Spark 1.x.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Jenkins build is back to normal : oozie-trunk-find-patches-available #133731

2016-09-15 Thread Apache Jenkins Server

See

Build failed in Jenkins: oozie-trunk-find-patches-available #133730

2016-09-15 Thread Apache Jenkins Server

See 

--
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on H1 (yahoo-not-h2 Hadoop Mapreduce Hdfs Pig) in workspace 

 > git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
 > git config remote.origin.url 
 > https://git-wip-us.apache.org/repos/asf/oozie.git # timeout=10
Fetching upstream changes from https://git-wip-us.apache.org/repos/asf/oozie.git
 > git --version # timeout=10
 > git -c core.askpass=true fetch --tags --progress 
 > https://git-wip-us.apache.org/repos/asf/oozie.git 
 > +refs/heads/*:refs/remotes/origin/*
 > git rev-parse refs/remotes/origin/master^{commit} # timeout=10
 > git rev-parse refs/remotes/origin/origin/master^{commit} # timeout=10
Checking out Revision 34c469dd2f7023c92bc0f5605274059c30beea0c 
(refs/remotes/origin/master)
 > git config core.sparsecheckout # timeout=10
 > git checkout -f 34c469dd2f7023c92bc0f5605274059c30beea0c
 > git rev-list 34c469dd2f7023c92bc0f5605274059c30beea0c # timeout=10
[oozie-trunk-find-patches-available] $ /bin/bash 
/tmp/hudson3705260978457566430.sh
mkdir: cannot create directory 
':
 File exists
  % Total% Received % Xferd  Average Speed   TimeTime Time  Current
 Dload  Upload   Total   SpentLeft  Speed

  0 00 00 0  0  0 --:--:-- --:--:-- --:--:-- 0
100  184k0  184k0 0   238k  0 --:--:-- --:--:-- --:--:--  238k
100  482k0  482k0 0   275k  0 --:--:--  0:00:01 --:--:--  275k
100  675k0  675k0 0   266k  0 --:--:--  0:00:02 --:--:--  266k
curl: (18) transfer closed with outstanding read data remaining
Could not retrieve available patches from JIRA
Build step 'Execute shell' marked build as failure

[jira] [Commented] (OOZIE-2569) Adding yarn-site, core-site, hdfs-site and mapred-site into spark launcher

2016-09-15 Thread Abhishek Bafna (JIRA)


[ 
https://issues.apache.org/jira/browse/OOZIE-2569?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15494155#comment-15494155
 ] 

Abhishek Bafna commented on OOZIE-2569:
---

Ping: [~rkanter], [~jaydeepvishwakarma]

> Adding yarn-site, core-site, hdfs-site and mapred-site into spark launcher 
> ---
>
> Key: OOZIE-2569
> URL: https://issues.apache.org/jira/browse/OOZIE-2569
> Project: Oozie
>  Issue Type: Bug
>  Components: core
>Reporter: Abhishek Bafna
>Assignee: Abhishek Bafna
> Fix For: 4.3.0
>
> Attachments: OOZIE-2569-00.patch, OOZIE-2569-02.patch, 
> OOZIE-2569-03.patch
>
>
> Oozie Spark action requires yarn-site, core-site, hdfs-site and mapred-site 
> to be present in the classpath for launching the spark applications. 
> Currently we manually copy these files into oozie spark sharelib.
> Until it is fixed through the spark, we can copy the 
> {{oozie.action.conf.xml}} file as yarn-site, core-site, hdfs-site and 
> mapred-site, which will avoid manual steps of copying those files.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (OOZIE-2582) Populating external child Ids for action failures

2016-09-15 Thread Abhishek Bafna (JIRA)


[ 
https://issues.apache.org/jira/browse/OOZIE-2582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15494154#comment-15494154
 ] 

Abhishek Bafna commented on OOZIE-2582:
---

Ping: [~rkanter], [~jaydeepvishwakarma]

> Populating external child Ids for action failures
> -
>
> Key: OOZIE-2582
> URL: https://issues.apache.org/jira/browse/OOZIE-2582
> Project: Oozie
>  Issue Type: Bug
>  Components: core
>Reporter: Abhishek Bafna
>Assignee: Abhishek Bafna
> Fix For: 4.3.0
>
> Attachments: OOZIE-2582-00.patch, OOZIE-2582-01.patch, 
> OOZIE-2582-02.patch, OOZIE-2582-03.patch, OOZIE-2582-04.patch, 
> OOZIE-2582-05.patch
>
>
> Currently Oozie external child ids are populated into workflow bean, when the 
> job/action completes successfully. It should populate external child ids in 
> case of job failures as well.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (OOZIE-2670) Upgrade Hbase to 1.2

2016-09-15 Thread Robert Kanter (JIRA)


[ 
https://issues.apache.org/jira/browse/OOZIE-2670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15493864#comment-15493864
 ] 

Robert Kanter commented on OOZIE-2670:
--

Thanks for taking care of this.  HBase 0.94 is pretty old now.
Two questions:
# Should {{ConnectionFactory.createConnection(jobConf);}} go into the {{try}} 
block?  i.e. can it throw a connection?
# Have you verified that this properly impersonates the user?  We've had 
problems with that in the past (see OOZIE-2419).

> Upgrade Hbase to 1.2
> 
>
> Key: OOZIE-2670
> URL: https://issues.apache.org/jira/browse/OOZIE-2670
> Project: Oozie
>  Issue Type: Bug
>Reporter: Peter Cseh
>Assignee: Peter Cseh
> Attachments: OOZIE-2670.001.patch
>
>
> We should upgrade Hbase to a newer version, say 1.2



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (OOZIE-2670) Upgrade Hbase to 1.2

2016-09-15 Thread Hadoop QA (JIRA)


[ 
https://issues.apache.org/jira/browse/OOZIE-2670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15493263#comment-15493263
 ] 

Hadoop QA commented on OOZIE-2670:
--

Testing JIRA OOZIE-2670

Cleaning local git workspace



{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:red}-1 RAW_PATCH_ANALYSIS{color}
.{color:green}+1{color} the patch does not introduce any @author tags
.{color:green}+1{color} the patch does not introduce any tabs
.{color:green}+1{color} the patch does not introduce any trailing spaces
.{color:green}+1{color} the patch does not introduce any line longer than 
132
.{color:red}-1{color} the patch does not add/modify any testcase
{color:green}+1 RAT{color}
.{color:green}+1{color} the patch does not seem to introduce new RAT 
warnings
{color:green}+1 JAVADOC{color}
.{color:green}+1{color} the patch does not seem to introduce new Javadoc 
warnings
{color:red}-1 COMPILE{color}
.{color:green}+1{color} HEAD compiles
.{color:red}-1{color} patch does not compile
.{color:red}-1{color} the patch seems to introduce 1 new javac warning(s)
{color:green}+1 BACKWARDS_COMPATIBILITY{color}
.{color:green}+1{color} the patch does not change any JPA 
Entity/Colum/Basic/Lob/Transient annotations
.{color:green}+1{color} the patch does not modify JPA files
{color:red}-1 TESTS{color} - patch does not compile, cannot run testcases
{color:red}-1 DISTRO{color}
.{color:red}-1{color} distro tarball fails with the patch


{color:red}*-1 Overall result, please check the reported -1(s)*{color}


The full output of the test-patch run is available at

.   https://builds.apache.org/job/oozie-trunk-precommit-build/3276/

> Upgrade Hbase to 1.2
> 
>
> Key: OOZIE-2670
> URL: https://issues.apache.org/jira/browse/OOZIE-2670
> Project: Oozie
>  Issue Type: Bug
>Reporter: Peter Cseh
>Assignee: Peter Cseh
> Attachments: OOZIE-2670.001.patch
>
>
> We should upgrade Hbase to a newer version, say 1.2



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Build failed in Jenkins: oozie-trunk-precommit-build #3276

2016-09-15 Thread Apache Jenkins Server

See 

--
[...truncated 5238 lines...]
[INFO] Copying 3 resources
[INFO] 
[INFO] --- maven-compiler-plugin:2.3.2:compile (default-compile) @ 
oozie-sharelib-oozie ---
[INFO] Nothing to compile - all classes are up to date
[INFO] 
[INFO] --- maven-dependency-plugin:2.4:build-classpath (gen-classpath) @ 
oozie-sharelib-oozie ---
[INFO] Wrote classpath file 
'
[INFO] 
[INFO] --- maven-dependency-plugin:2.4:build-classpath 
(create-mrapp-generated-classpath) @ oozie-sharelib-oozie ---
[INFO] Wrote classpath file 
'
[INFO] 
[INFO] --- maven-antrun-plugin:1.6:run (default) @ oozie-sharelib-oozie ---
[INFO] Executing tasks

main:
[INFO] Executed tasks
[INFO] 
[INFO] --- maven-resources-plugin:2.5:testResources (default-testResources) @ 
oozie-sharelib-oozie ---
[debug] execute contextualize
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory 

[INFO] Copying 3 resources
[INFO] 
[INFO] --- maven-compiler-plugin:2.3.2:testCompile (default-testCompile) @ 
oozie-sharelib-oozie ---
[INFO] No sources to compile
[INFO] 
[INFO] --- maven-surefire-plugin:2.12.2:test (default-test) @ 
oozie-sharelib-oozie ---
[INFO] Tests are skipped.
[INFO] 
[INFO] --- maven-jar-plugin:2.3.1:jar (default-jar) @ oozie-sharelib-oozie ---
[INFO] Building jar: 

[INFO] 
[INFO] --- maven-site-plugin:2.0-beta-6:attach-descriptor (attach-descriptor) @ 
oozie-sharelib-oozie ---
[INFO] 
[INFO] --- maven-assembly-plugin:2.2.1:single (default-cli) @ 
oozie-sharelib-oozie ---
[INFO] Reading assembly descriptor: 
../../src/main/assemblies/partial-sharelib.xml
[WARNING] The following patterns were never triggered in this artifact 
exclusion filter:
o  'org.apache.oozie:oozie-hadoop-distcp:jar'

[INFO] Copying files to 

[WARNING] Assembly file: 

 is not a regular file (it may be a directory). It cannot be attached to the 
project build for installation or deployment.
[INFO] 
[INFO] 
[INFO] Building Apache Oozie Share Lib HCatalog 4.3.0-SNAPSHOT
[INFO] 
[INFO] 
[INFO] --- maven-remote-resources-plugin:1.5:process (default) @ 
oozie-sharelib-hcatalog ---
[INFO] 
[INFO] --- maven-resources-plugin:2.5:resources (default-resources) @ 
oozie-sharelib-hcatalog ---
[debug] execute contextualize
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory 

[INFO] Copying 3 resources
[INFO] 
[INFO] --- maven-compiler-plugin:2.3.2:compile (default-compile) @ 
oozie-sharelib-hcatalog ---
[INFO] Nothing to compile - all classes are up to date
[INFO] 
[INFO] --- maven-dependency-plugin:2.4:build-classpath (gen-classpath) @ 
oozie-sharelib-hcatalog ---
[INFO] Wrote classpath file 
'
[INFO] 
[INFO] --- maven-dependency-plugin:2.4:build-classpath 
(create-mrapp-generated-classpath) @ oozie-sharelib-hcatalog ---
[INFO] Wrote classpath file 
'
[INFO] 
[INFO] --- maven-antrun-plugin:1.6:run (default) @ oozie-sharelib-hcatalog ---
[INFO] Executing tasks

main:
[INFO] Executed tasks
[INFO] 
[INFO] --- maven-resources-plugin:2.5:testResources (default-testResources) @ 
oozie-sharelib-hcatalog ---
[debug] execute contextualize
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory 

[INFO] Copying 3 resources
[INFO] 
[INFO] --- maven-compiler-plugin:2.3.2:testCompile (default-testCompile) @ 
oozie-sharelib-hcatalog ---
[INFO] Nothing to compile - all classes are up to date
[INFO] 
[INFO] --- maven-surefire-plugin:2.12.2:test (default-test) @ 
oozie-sharelib-hcatalog ---
[INFO] Tests are skipped.
[INFO] 
[INFO] --- maven-jar-plugin:2.3.1:jar (default-jar) @ oozie-sharelib-hcatalog

[jira] [Updated] (OOZIE-2670) Upgrade Hbase to 1.2

2016-09-15 Thread Peter Cseh (JIRA)


 [ 
https://issues.apache.org/jira/browse/OOZIE-2670?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Peter Cseh updated OOZIE-2670:
--
Attachment: OOZIE-2670.001.patch

> Upgrade Hbase to 1.2
> 
>
> Key: OOZIE-2670
> URL: https://issues.apache.org/jira/browse/OOZIE-2670
> Project: Oozie
>  Issue Type: Bug
>Reporter: Peter Cseh
>Assignee: Peter Cseh
> Attachments: OOZIE-2670.001.patch
>
>
> We should upgrade Hbase to a newer version, say 1.2



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (OOZIE-2670) Upgrade Hbase to 1.2

2016-09-15 Thread Peter Cseh (JIRA)


 [ 
https://issues.apache.org/jira/browse/OOZIE-2670?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Peter Cseh updated OOZIE-2670:
--
Attachment: (was: OOZIE-2670.001.patch)

> Upgrade Hbase to 1.2
> 
>
> Key: OOZIE-2670
> URL: https://issues.apache.org/jira/browse/OOZIE-2670
> Project: Oozie
>  Issue Type: Bug
>Reporter: Peter Cseh
>Assignee: Peter Cseh
> Attachments: OOZIE-2670.001.patch
>
>
> We should upgrade Hbase to a newer version, say 1.2



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (OOZIE-2670) Upgrade Hbase to 1.2

2016-09-15 Thread Peter Cseh (JIRA)


 [ 
https://issues.apache.org/jira/browse/OOZIE-2670?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Peter Cseh updated OOZIE-2670:
--
Attachment: OOZIE-2670.001.patch

hbase.jar is not available after 0.95, Oozie needs hbase-client after that.

{{TokenUtil.obtainToken(jobConf);}} was deprecated, so I've replaced it.

I've found only two references to hbase in the docs, those are updated.

> Upgrade Hbase to 1.2
> 
>
> Key: OOZIE-2670
> URL: https://issues.apache.org/jira/browse/OOZIE-2670
> Project: Oozie
>  Issue Type: Bug
>Reporter: Peter Cseh
>Assignee: Peter Cseh
> Attachments: OOZIE-2670.001.patch
>
>
> We should upgrade Hbase to a newer version, say 1.2



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Jenkins build is back to normal : oozie-trunk-find-patches-available #133657

2016-09-15 Thread Apache Jenkins Server

See

Build failed in Jenkins: oozie-trunk-find-patches-available #133656

2016-09-15 Thread Apache Jenkins Server

See 

--
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on H1 (yahoo-not-h2 Hadoop Mapreduce Hdfs Pig) in workspace 

 > git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
 > git config remote.origin.url 
 > https://git-wip-us.apache.org/repos/asf/oozie.git # timeout=10
Fetching upstream changes from https://git-wip-us.apache.org/repos/asf/oozie.git
 > git --version # timeout=10
 > git -c core.askpass=true fetch --tags --progress 
 > https://git-wip-us.apache.org/repos/asf/oozie.git 
 > +refs/heads/*:refs/remotes/origin/*
 > git rev-parse refs/remotes/origin/master^{commit} # timeout=10
 > git rev-parse refs/remotes/origin/origin/master^{commit} # timeout=10
Checking out Revision 34c469dd2f7023c92bc0f5605274059c30beea0c 
(refs/remotes/origin/master)
 > git config core.sparsecheckout # timeout=10
 > git checkout -f 34c469dd2f7023c92bc0f5605274059c30beea0c
 > git rev-list 34c469dd2f7023c92bc0f5605274059c30beea0c # timeout=10
[oozie-trunk-find-patches-available] $ /bin/bash 
/tmp/hudson8603856683039070467.sh
mkdir: cannot create directory 
':
 File exists
  % Total% Received % Xferd  Average Speed   TimeTime Time  Current
 Dload  Upload   Total   SpentLeft  Speed

  0 00 00 0  0  0 --:--:-- --:--:-- --:--:-- 0
100 983220 983220 0   125k  0 --:--:-- --:--:-- --:--:--  125k
100  403k0  403k0 0   227k  0 --:--:--  0:00:01 --:--:--  227k
100  720k0  720k0 0   254k  0 --:--:--  0:00:02 --:--:--  254k
100 1026k0 1026k0 0   269k  0 --:--:--  0:00:03 --:--:--  269k
100 1298k0 1298k0 0   268k  0 --:--:--  0:00:04 --:--:--  268k
100 1528k0 1528k0 0   262k  0 --:--:--  0:00:05 --:--:--  283k
100 1843k0 1843k0 0   272k  0 --:--:--  0:00:06 --:--:--  288k
100 1878k0 1878k0 0   270k  0 --:--:--  0:00:06 --:--:--  280k
curl: (18) transfer closed with outstanding read data remaining
Could not retrieve available patches from JIRA
Build step 'Execute shell' marked build as failure

[jira] [Commented] (OOZIE-2572) SLA DURATION miss not shown when job is running for longer than expected time

2016-09-15 Thread Hadoop QA (JIRA)


[ 
https://issues.apache.org/jira/browse/OOZIE-2572?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15492645#comment-15492645
 ] 

Hadoop QA commented on OOZIE-2572:
--

Testing JIRA OOZIE-2572

Cleaning local git workspace



{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:green}+1 RAW_PATCH_ANALYSIS{color}
.{color:green}+1{color} the patch does not introduce any @author tags
.{color:green}+1{color} the patch does not introduce any tabs
.{color:green}+1{color} the patch does not introduce any trailing spaces
.{color:green}+1{color} the patch does not introduce any line longer than 
132
.{color:green}+1{color} the patch does adds/modifies 2 testcase(s)
{color:green}+1 RAT{color}
.{color:green}+1{color} the patch does not seem to introduce new RAT 
warnings
{color:green}+1 JAVADOC{color}
.{color:green}+1{color} the patch does not seem to introduce new Javadoc 
warnings
{color:green}+1 COMPILE{color}
.{color:green}+1{color} HEAD compiles
.{color:green}+1{color} patch compiles
.{color:green}+1{color} the patch does not seem to introduce new javac 
warnings
{color:green}+1 BACKWARDS_COMPATIBILITY{color}
.{color:green}+1{color} the patch does not change any JPA 
Entity/Colum/Basic/Lob/Transient annotations
.{color:green}+1{color} the patch does not modify JPA files
{color:green}+1 TESTS{color}
.Tests run: 1805
{color:green}+1 DISTRO{color}
.{color:green}+1{color} distro tarball builds with the patch 


{color:green}*+1 Overall result, good!, no -1s*{color}


The full output of the test-patch run is available at

.   https://builds.apache.org/job/oozie-trunk-precommit-build/3275/

> SLA DURATION miss not shown when job is running for longer than expected time
> -
>
> Key: OOZIE-2572
> URL: https://issues.apache.org/jira/browse/OOZIE-2572
> Project: Oozie
>  Issue Type: Bug
>Reporter: Satish Subhashrao Saley
>Assignee: Satish Subhashrao Saley
> Fix For: 4.3.0
>
> Attachments: OOZIE-2572-1.patch, OOZIE-2572-2.patch, 
> OOZIE-2572-3.patch, OOZIE-2572-4.patch, OOZIE-2572-5.patch, 
> OOZIE-2572-6.patch, OOZIE-2572-7.patch, OOZIE-2572-8.patch, OOZIE-2572-9.patch
>
>
> If job is running for beyond expected duration, Oozie UI is not slowing SLA 
> for duration miss.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Jenkins build is back to normal : oozie-trunk-precommit-build #3275

2016-09-15 Thread Apache Jenkins Server

See

63 matches

Mail list logo