[jira] [Commented] (OOZIE-2632) Provide database dump/load functionality to make database migration easier

[Jaydeep Vishwakarma (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-2632?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15451251#comment-15451251
 ] 

Jaydeep Vishwakarma commented on OOZIE-2632:


ok, Lets go ahead with this. 

> Provide database dump/load functionality to make database migration easier
> --
>
> Key: OOZIE-2632
> URL: https://issues.apache.org/jira/browse/OOZIE-2632
> Project: Oozie
>  Issue Type: New Feature
>Reporter: Peter Cseh
>Assignee: Peter Cseh
> Attachments: OOZIE-2632.001.patch, OOZIE-2632.002.patch, 
> OOZIE-2632.003.patch
>
>
> When there is a need to change the database under Oozie one has to submit all 
> the workflows again or migrate the data manually which is not easy because of 
> the binary blobs stored in the DB. 
> oozie-setup.sh could have a db export/import command that would save the 
> contents of the Db to a file and read it back from it.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (OOZIE-2615) Flaky tests TestCoordActionsKillXCommand.testActionKillCommandActionNumbers and testActionKillCommandDate

[Abhishek Bafna (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-2615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15451156#comment-15451156
 ] 

Abhishek Bafna commented on OOZIE-2615:
---

 Just a thought, Instead of destroying the services, can we use 
{{XDataTestCase#setClassesToBeExcluded}} method for excluding the service in 
first. An example can be found in the {{TestStatusTransitService}} class. 
Thanks.

> Flaky tests TestCoordActionsKillXCommand.testActionKillCommandActionNumbers 
> and testActionKillCommandDate 
> --
>
> Key: OOZIE-2615
> URL: https://issues.apache.org/jira/browse/OOZIE-2615
> Project: Oozie
>  Issue Type: Bug
>  Components: tests
>Reporter: Peter Bacsko
>Assignee: Peter Bacsko
>Priority: Minor
> Attachments: OOZIE-2615-001.patch
>
>
> Two test cases are unstable because StatusTransitService might change the 
> coord job status.
> {code:title=testActionKillCommandDate|borderStyle=solid}
> 08:15:16,122 [pool-3063-thread-2] TRACE CoordStatusTransitXCommand:529 - 
> USER[test] GROUP[testg] TOKEN[] APP[COORD-TEST] 
> JOB[000-160708081515125-oozie-root-C] ACTION[-] Precondition check for 
> command [coord_status_transit] key [000-160708081515125-oozie-root-C]
> 08:15:16,122 [pool-3063-thread-2] DEBUG CoordStatusTransitXCommand:526 - 
> USER[test] GROUP[testg] TOKEN[] APP[COORD-TEST] 
> JOB[000-160708081515125-oozie-root-C] ACTION[-] Execute command 
> [coord_status_transit] key [000-160708081515125-oozie-root-C]
> 08:15:16,122 [pool-3063-thread-2]  INFO CoordStatusTransitXCommand:520 - 
> USER[test] GROUP[testg] TOKEN[] APP[COORD-TEST] 
> JOB[000-160708081515125-oozie-root-C] ACTION[-] Set coordinator job 
> [000-160708081515125-oozie-root-C] status to 'KILLED' from 'RUNNING'
> {code}
> {code:title=testActionKillCommandActionNumbers|borderStyle=solid}
> 08:15:13,886 [pool-3059-thread-2] TRACE CoordStatusTransitXCommand:529 - 
> USER[test] GROUP[testg] TOKEN[] APP[COORD-TEST] 
> JOB[000-160708081512561-oozie-root-C] ACTION[-] Precondition check for 
> command [coord_status_transit] key [000-160708081512561-oozie-root-C]
> 08:15:13,886 [pool-3059-thread-2] DEBUG CoordStatusTransitXCommand:526 - 
> USER[test] GROUP[testg] TOKEN[] APP[COORD-TEST] 
> JOB[000-160708081512561-oozie-root-C] ACTION[-] Execute command 
> [coord_status_transit] key [000-160708081512561-oozie-root-C]
> 08:15:13,886 [pool-3059-thread-2]  INFO CoordStatusTransitXCommand:520 - 
> USER[test] GROUP[testg] TOKEN[] APP[COORD-TEST] 
> JOB[000-160708081512561-oozie-root-C] ACTION[-] Set coordinator job 
> [000-160708081512561-oozie-root-C] status to 'RUNNINGWITHERROR' from 
> 'RUNNING'
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (OOZIE-2632) Provide database dump/load functionality to make database migration easier

[Robert Kanter (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-2632?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15450762#comment-15450762
 ] 

Robert Kanter commented on OOZIE-2632:
--

[~jaydeepvishwakarma], given the info in the last few comments, do you agree 
that this is okay?  We're hoping to get this in by EoD PDT Wednesday for a 
deadline.

> Provide database dump/load functionality to make database migration easier
> --
>
> Key: OOZIE-2632
> URL: https://issues.apache.org/jira/browse/OOZIE-2632
> Project: Oozie
>  Issue Type: New Feature
>Reporter: Peter Cseh
>Assignee: Peter Cseh
> Attachments: OOZIE-2632.001.patch, OOZIE-2632.002.patch, 
> OOZIE-2632.003.patch
>
>
> When there is a need to change the database under Oozie one has to submit all 
> the workflows again or migrate the data manually which is not easy because of 
> the binary blobs stored in the DB. 
> oozie-setup.sh could have a db export/import command that would save the 
> contents of the Db to a file and read it back from it.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (OOZIE-2615) Flaky tests TestCoordActionsKillXCommand.testActionKillCommandActionNumbers and testActionKillCommandDate

[Robert Kanter (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-2615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15450752#comment-15450752
 ] 

Robert Kanter commented on OOZIE-2615:
--

Two minor things:
- Instead of destroying the {{SchedulerService}}, shouldn't we only need to 
destroy the {{StatusTransitService}}?
- Can we move the destroy before the init?  That way we don't even initialize 
the Service (just in case it has a chance to do something before it's 
destroyed).

> Flaky tests TestCoordActionsKillXCommand.testActionKillCommandActionNumbers 
> and testActionKillCommandDate 
> --
>
> Key: OOZIE-2615
> URL: https://issues.apache.org/jira/browse/OOZIE-2615
> Project: Oozie
>  Issue Type: Bug
>  Components: tests
>Reporter: Peter Bacsko
>Assignee: Peter Bacsko
>Priority: Minor
> Attachments: OOZIE-2615-001.patch
>
>
> Two test cases are unstable because StatusTransitService might change the 
> coord job status.
> {code:title=testActionKillCommandDate|borderStyle=solid}
> 08:15:16,122 [pool-3063-thread-2] TRACE CoordStatusTransitXCommand:529 - 
> USER[test] GROUP[testg] TOKEN[] APP[COORD-TEST] 
> JOB[000-160708081515125-oozie-root-C] ACTION[-] Precondition check for 
> command [coord_status_transit] key [000-160708081515125-oozie-root-C]
> 08:15:16,122 [pool-3063-thread-2] DEBUG CoordStatusTransitXCommand:526 - 
> USER[test] GROUP[testg] TOKEN[] APP[COORD-TEST] 
> JOB[000-160708081515125-oozie-root-C] ACTION[-] Execute command 
> [coord_status_transit] key [000-160708081515125-oozie-root-C]
> 08:15:16,122 [pool-3063-thread-2]  INFO CoordStatusTransitXCommand:520 - 
> USER[test] GROUP[testg] TOKEN[] APP[COORD-TEST] 
> JOB[000-160708081515125-oozie-root-C] ACTION[-] Set coordinator job 
> [000-160708081515125-oozie-root-C] status to 'KILLED' from 'RUNNING'
> {code}
> {code:title=testActionKillCommandActionNumbers|borderStyle=solid}
> 08:15:13,886 [pool-3059-thread-2] TRACE CoordStatusTransitXCommand:529 - 
> USER[test] GROUP[testg] TOKEN[] APP[COORD-TEST] 
> JOB[000-160708081512561-oozie-root-C] ACTION[-] Precondition check for 
> command [coord_status_transit] key [000-160708081512561-oozie-root-C]
> 08:15:13,886 [pool-3059-thread-2] DEBUG CoordStatusTransitXCommand:526 - 
> USER[test] GROUP[testg] TOKEN[] APP[COORD-TEST] 
> JOB[000-160708081512561-oozie-root-C] ACTION[-] Execute command 
> [coord_status_transit] key [000-160708081512561-oozie-root-C]
> 08:15:13,886 [pool-3059-thread-2]  INFO CoordStatusTransitXCommand:520 - 
> USER[test] GROUP[testg] TOKEN[] APP[COORD-TEST] 
> JOB[000-160708081512561-oozie-root-C] ACTION[-] Set coordinator job 
> [000-160708081512561-oozie-root-C] status to 'RUNNINGWITHERROR' from 
> 'RUNNING'
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (OOZIE-2462) When calling ParamChecker.notNull() in CoordActionsIgnoreXCommand.java, "Action" should be passed instead of "Action cannot be null".

[Robert Kanter (JIRA)]

 [ 
https://issues.apache.org/jira/browse/OOZIE-2462?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Robert Kanter updated OOZIE-2462:
-
Assignee: Manjunath Ballur

> When calling ParamChecker.notNull() in CoordActionsIgnoreXCommand.java, 
> "Action" should be passed instead of "Action cannot be null".
> -
>
> Key: OOZIE-2462
> URL: https://issues.apache.org/jira/browse/OOZIE-2462
> Project: Oozie
>  Issue Type: Bug
>Reporter: Manjunath Ballur
>Assignee: Manjunath Ballur
>Priority: Trivial
>  Labels: newbie
> Fix For: 4.3.0
>
> Attachments: OOZIE-2462-001.patch, OOZIE-2462-002.patch, 
> OOZIE-2462-002.patch
>
>
> In ParamChecker::notNull() method, if the passed object is null, then 
> following exception is thrown:
> if (obj == null) {
> throw new IllegalArgumentException(name + " cannot be null");
> }
> Here, "name" is the name of the object being checked.
> In "CoordActionsIgnoreXCommand::checkAllActionsStatus()" method, following 
> piece of code calls "ParamChecker::notNull()" method:
> ParamChecker.notNull(action, "Action cannot be null");
> The above statement should be changed to:
> ParamChecker.notNull(action, "Action");
> With the current code, when "action" is null, then the message printed will 
> be:
> "Action cannot be null cannot be null"
> "cannot be null" is printed twice.
> Hence, we need to remove "cannot be null" while calling 
> "ParamChecker::notNull()" method.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Jenkins build is back to normal : oozie-trunk-find-patches-available #131721

[Apache Jenkins Server]

See 

Build failed in Jenkins: oozie-trunk-find-patches-available #131720

[Apache Jenkins Server]

See 

--
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on H1 (yahoo-not-h2 Hadoop Mapreduce Hdfs Pig) in workspace 

 > git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
 > git config remote.origin.url 
 > https://git-wip-us.apache.org/repos/asf/oozie.git # timeout=10
Fetching upstream changes from https://git-wip-us.apache.org/repos/asf/oozie.git
 > git --version # timeout=10
 > git -c core.askpass=true fetch --tags --progress 
 > https://git-wip-us.apache.org/repos/asf/oozie.git 
 > +refs/heads/*:refs/remotes/origin/*
 > git rev-parse refs/remotes/origin/master^{commit} # timeout=10
 > git rev-parse refs/remotes/origin/origin/master^{commit} # timeout=10
Checking out Revision 57152acd5c9aa081d9c8357009f8741ee026e352 
(refs/remotes/origin/master)
 > git config core.sparsecheckout # timeout=10
 > git checkout -f 57152acd5c9aa081d9c8357009f8741ee026e352
 > git rev-list 57152acd5c9aa081d9c8357009f8741ee026e352 # timeout=10
[oozie-trunk-find-patches-available] $ /bin/bash 
/tmp/hudson8331063670379402730.sh
mkdir: cannot create directory 
':
 File exists
  % Total% Received % Xferd  Average Speed   TimeTime Time  Current
 Dload  Upload   Total   SpentLeft  Speed

  0 00 00 0  0  0 --:--:-- --:--:-- --:--:-- 0
100 533700 533700 0  84068  0 --:--:-- --:--:-- --:--:-- 84047
100  385k0  385k0 0   239k  0 --:--:--  0:00:01 --:--:--  239k
100  816k0  816k0 0   305k  0 --:--:--  0:00:02 --:--:--  305k
100 1080k0 1080k0 0   270k  0 --:--:--  0:00:03 --:--:--  270k
100 1281k0 1281k0 0   273k  0 --:--:--  0:00:04 --:--:--  273k
100 1588k0 1588k0 0   282k  0 --:--:--  0:00:05 --:--:--  308k
100 1799k0 1799k0 0   288k  0 --:--:--  0:00:06 --:--:--  305k
curl: (18) transfer closed with outstanding read data remaining
Could not retrieve available patches from JIRA
Build step 'Execute shell' marked build as failure

[jira] [Commented] (OOZIE-2632) Provide database dump/load functionality to make database migration easier

[Peter Bacsko (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-2632?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15450279#comment-15450279
 ] 

Peter Bacsko commented on OOZIE-2632:
-

An example of a migration process right now:
http://blog.shmuma.ru/2014/04/07/how-to-migrate-oozie-from-derby-to-mysql-an-easy-way/

We tried this out a couple of months ago and it didn't work - as [~gezapeti] 
mentioned, BLOB fields were screwed up and coordinator jobs failed to start up 
properly. I think we also had to modify the {{sed}} command to have proper SQL 
syntax (and even without modification, having to run a command like this it's 
simply risky).

The proposed code is very easy to read and maintain, all DB details are hidden. 
Even if it's slower than using more direct methods/tools, it's very reliable. 



> Provide database dump/load functionality to make database migration easier
> --
>
> Key: OOZIE-2632
> URL: https://issues.apache.org/jira/browse/OOZIE-2632
> Project: Oozie
>  Issue Type: New Feature
>Reporter: Peter Cseh
>Assignee: Peter Cseh
> Attachments: OOZIE-2632.001.patch, OOZIE-2632.002.patch, 
> OOZIE-2632.003.patch
>
>
> When there is a need to change the database under Oozie one has to submit all 
> the workflows again or migrate the data manually which is not easy because of 
> the binary blobs stored in the DB. 
> oozie-setup.sh could have a db export/import command that would save the 
> contents of the Db to a file and read it back from it.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (OOZIE-2632) Provide database dump/load functionality to make database migration easier

[Robert Kanter (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-2632?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15450219#comment-15450219
 ] 

Robert Kanter commented on OOZIE-2632:
--

I would also add that even if it takes a long time to run, there's a few things 
to consider:
- [~gezapeti]'s point that the blobs making using DB tools 
problematic/complicated
- I'm sure users would rather have an automated simple tool that takes a while 
rather than an unsupported complicated manual process that might be faster
- I think the main usecase of this tool would be for migrating off of Derby, 
which
-- will start running into scaling problems before it gets big enough where 
this tool will take _that_ long
-- doesn't have great tools.  AFAIK, there's been a few suggested ways of 
migrating Oozie data out of Derby, but they've all been very hacky and don't 
always work 100%
- Migrating the Oozie DB isn't going to be a common operation.  It's something 
that most users won't even ever do.  I'd say it's better to have an easy 
process for this, even if it's slow, than no process at all.  As it stands now, 
users typically have to loose all of their Oozie history and running 
Coordinators if they need to migrate; something they're not happy about.

If we need to improve the speed, we can look into making this multithreaded 
(though probably not for Derby).  But that's something to save for a future 
JIRA.

> Provide database dump/load functionality to make database migration easier
> --
>
> Key: OOZIE-2632
> URL: https://issues.apache.org/jira/browse/OOZIE-2632
> Project: Oozie
>  Issue Type: New Feature
>Reporter: Peter Cseh
>Assignee: Peter Cseh
> Attachments: OOZIE-2632.001.patch, OOZIE-2632.002.patch, 
> OOZIE-2632.003.patch
>
>
> When there is a need to change the database under Oozie one has to submit all 
> the workflows again or migrate the data manually which is not easy because of 
> the binary blobs stored in the DB. 
> oozie-setup.sh could have a db export/import command that would save the 
> contents of the Db to a file and read it back from it.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Jenkins build is back to normal : oozie-trunk-find-patches-available #131713

[Apache Jenkins Server]

See 

Build failed in Jenkins: oozie-trunk-find-patches-available #131712

[Apache Jenkins Server]

See 

--
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on H1 (yahoo-not-h2 Hadoop Mapreduce Hdfs Pig) in workspace 

 > git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
 > git config remote.origin.url 
 > https://git-wip-us.apache.org/repos/asf/oozie.git # timeout=10
Fetching upstream changes from https://git-wip-us.apache.org/repos/asf/oozie.git
 > git --version # timeout=10
 > git -c core.askpass=true fetch --tags --progress 
 > https://git-wip-us.apache.org/repos/asf/oozie.git 
 > +refs/heads/*:refs/remotes/origin/*
 > git rev-parse refs/remotes/origin/master^{commit} # timeout=10
 > git rev-parse refs/remotes/origin/origin/master^{commit} # timeout=10
Checking out Revision 57152acd5c9aa081d9c8357009f8741ee026e352 
(refs/remotes/origin/master)
 > git config core.sparsecheckout # timeout=10
 > git checkout -f 57152acd5c9aa081d9c8357009f8741ee026e352
 > git rev-list 57152acd5c9aa081d9c8357009f8741ee026e352 # timeout=10
[oozie-trunk-find-patches-available] $ /bin/bash 
/tmp/hudson7957637030883978288.sh
mkdir: cannot create directory 
':
 File exists
  % Total% Received % Xferd  Average Speed   TimeTime Time  Current
 Dload  Upload   Total   SpentLeft  Speed

  0 00 00 0  0  0 --:--:-- --:--:-- --:--:-- 0
100  148k0  148k0 0   154k  0 --:--:-- --:--:-- --:--:--  154k
100  464k0  464k0 0   244k  0 --:--:--  0:00:01 --:--:--  244k
100  464k0  464k0 0   244k  0 --:--:--  0:00:01 --:--:--  244k
curl: (18) transfer closed with outstanding read data remaining
Could not retrieve available patches from JIRA
Build step 'Execute shell' marked build as failure

[jira] [Commented] (OOZIE-2632) Provide database dump/load functionality to make database migration easier

[Jaydeep Vishwakarma (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-2632?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15449922#comment-15449922
 ] 

Jaydeep Vishwakarma commented on OOZIE-2632:


I think all DBs have inbuilt tool or command by help of them anyone can do the 
migration and all those tools are very efficient. If we still wants to help 
user, we should integrate those tool in oozie.

> Provide database dump/load functionality to make database migration easier
> --
>
> Key: OOZIE-2632
> URL: https://issues.apache.org/jira/browse/OOZIE-2632
> Project: Oozie
>  Issue Type: New Feature
>Reporter: Peter Cseh
>Assignee: Peter Cseh
> Attachments: OOZIE-2632.001.patch, OOZIE-2632.002.patch, 
> OOZIE-2632.003.patch
>
>
> When there is a need to change the database under Oozie one has to submit all 
> the workflows again or migrate the data manually which is not easy because of 
> the binary blobs stored in the DB. 
> oozie-setup.sh could have a db export/import command that would save the 
> contents of the Db to a file and read it back from it.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Assigned] (OOZIE-2658) --driver-class-path can overwrite the classpath in SparkMain

[Peter Cseh (JIRA)]

 [ 
https://issues.apache.org/jira/browse/OOZIE-2658?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Peter Cseh reassigned OOZIE-2658:
-

Assignee: Peter Cseh

> --driver-class-path can overwrite the classpath in SparkMain
> 
>
> Key: OOZIE-2658
> URL: https://issues.apache.org/jira/browse/OOZIE-2658
> Project: Oozie
>  Issue Type: Bug
>Reporter: Peter Cseh
>Assignee: Peter Cseh
>
> In SparkMain we use the option {{--conf spark.driver.extraClassPath=}} to put 
> together the jars needed for Spark. If the user specifies this option, Oozie 
> appends to the value.
> If {{--driver-class-path}} is defined in {{}} then it's value 
> overwrites the other ones, causing the Spark action to fail.
> We should handle both ways to define classpaths the same.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (OOZIE-2658) --driver-class-path can overwrite the classpath in SparkMain

[Peter Cseh (JIRA)]

Peter Cseh created OOZIE-2658:
-

 Summary: --driver-class-path can overwrite the classpath in 
SparkMain
 Key: OOZIE-2658
 URL: https://issues.apache.org/jira/browse/OOZIE-2658
 Project: Oozie
  Issue Type: Bug
Reporter: Peter Cseh


In SparkMain we use the option {{--conf spark.driver.extraClassPath=}} to put 
together the jars needed for Spark. If the user specifies this option, Oozie 
appends to the value.
If {{--driver-class-path}} is defined in {{}} then it's value 
overwrites the other ones, causing the Spark action to fail.
We should handle both ways to define classpaths the same.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (OOZIE-2649) Can't override sub-workflow configuration property if defined in parent workflow XML

[Attila Sasvari (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-2649?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15449765#comment-15449765
 ] 

Attila Sasvari commented on OOZIE-2649:
---

Many thanks for the review!

> Can't override sub-workflow configuration property if defined in parent 
> workflow XML
> 
>
> Key: OOZIE-2649
> URL: https://issues.apache.org/jira/browse/OOZIE-2649
> Project: Oozie
>  Issue Type: Bug
>Reporter: Attila Sasvari
>Assignee: Attila Sasvari
> Fix For: 4.3.0
>
> Attachments: OOZIE-2649.1.patch, OOZIE-2649.2.patch, 
> OOZIE-2649.3.patch, OOZIE-2649.4.patch, OOZIE-2649.5.patch
>
>
> After OOZIE-2511, configuration property cannot overriden in sub-workflow.
> Unfortunately some scenarios concerning subworkflows are not covered with 
> tests.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (OOZIE-2649) Can't override sub-workflow configuration property if defined in parent workflow XML

[Robert Kanter (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-2649?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15449686#comment-15449686
 ] 

Robert Kanter commented on OOZIE-2649:
--

+1

> Can't override sub-workflow configuration property if defined in parent 
> workflow XML
> 
>
> Key: OOZIE-2649
> URL: https://issues.apache.org/jira/browse/OOZIE-2649
> Project: Oozie
>  Issue Type: Bug
>Reporter: Attila Sasvari
>Assignee: Attila Sasvari
> Attachments: OOZIE-2649.1.patch, OOZIE-2649.2.patch, 
> OOZIE-2649.3.patch, OOZIE-2649.4.patch, OOZIE-2649.5.patch
>
>
> After OOZIE-2511, configuration property cannot overriden in sub-workflow.
> Unfortunately some scenarios concerning subworkflows are not covered with 
> tests.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Jenkins build is back to normal : oozie-trunk-find-patches-available #131690

[Apache Jenkins Server]

See 

Build failed in Jenkins: oozie-trunk-find-patches-available #131689

[Apache Jenkins Server]

See 

--
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on H1 (yahoo-not-h2 Hadoop Mapreduce Hdfs Pig) in workspace 

 > git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
 > git config remote.origin.url 
 > https://git-wip-us.apache.org/repos/asf/oozie.git # timeout=10
Fetching upstream changes from https://git-wip-us.apache.org/repos/asf/oozie.git
 > git --version # timeout=10
 > git -c core.askpass=true fetch --tags --progress 
 > https://git-wip-us.apache.org/repos/asf/oozie.git 
 > +refs/heads/*:refs/remotes/origin/*
 > git rev-parse refs/remotes/origin/master^{commit} # timeout=10
 > git rev-parse refs/remotes/origin/origin/master^{commit} # timeout=10
Checking out Revision 43b73d836575c6fe8ac4b39c120eef7c2cb1b8f4 
(refs/remotes/origin/master)
 > git config core.sparsecheckout # timeout=10
 > git checkout -f 43b73d836575c6fe8ac4b39c120eef7c2cb1b8f4
 > git rev-list 43b73d836575c6fe8ac4b39c120eef7c2cb1b8f4 # timeout=10
[oozie-trunk-find-patches-available] $ /bin/bash 
/tmp/hudson8492829181454255378.sh
mkdir: cannot create directory 
':
 File exists
  % Total% Received % Xferd  Average Speed   TimeTime Time  Current
 Dload  Upload   Total   SpentLeft  Speed

  0 00 00 0  0  0 --:--:-- --:--:-- --:--:-- 0
  0 00 00 0  0  0 --:--:-- --:--:-- --:--:-- 0
100  272k0  272k0 0   218k  0 --:--:--  0:00:01 --:--:--  218k
100  632k0  632k0 0   269k  0 --:--:--  0:00:02 --:--:--  269k
100  992k0  992k0 0   302k  0 --:--:--  0:00:03 --:--:--  302k
100 1158k0 1158k0 0   309k  0 --:--:--  0:00:03 --:--:--  309k
curl: (18) transfer closed with outstanding read data remaining
Could not retrieve available patches from JIRA
Build step 'Execute shell' marked build as failure

ApacheCon Seville CFP closes September 9th

[Rich Bowen]

It's traditional. We wait for the last minute to get our talk proposals
in for conferences.

Well, the last minute has arrived. The CFP for ApacheCon Seville closes
on September 9th, which is less than 2 weeks away. It's time to get your
talks in, so that we can make this the best ApacheCon yet.

It's also time to discuss with your developer and user community whether
there's a track of talks that you might want to propose, so that you
have more complete coverage of your project than a talk or two.

For Apache Big Data, the relevant URLs are:
Event details:
http://events.linuxfoundation.org/events/apache-big-data-europe
CFP:
http://events.linuxfoundation.org/events/apache-big-data-europe/program/cfp

For ApacheCon Europe, the relevant URLs are:
Event details: http://events.linuxfoundation.org/events/apachecon-europe
CFP: http://events.linuxfoundation.org/events/apachecon-europe/program/cfp

This year, we'll be reviewing papers "blind" - that is, looking at the
abstracts without knowing who the speaker is. This has been shown to
eliminate the "me and my buddies" nature of many tech conferences,
producing more diversity, and more new speakers. So make sure your
abstracts clearly explain what you'll be talking about.

For further updated about ApacheCon, follow us on Twitter, @ApacheCon,
or drop by our IRC channel, #apachecon on the Freenode IRC network.

-- 
Rich Bowen
WWW: http://apachecon.com/
Twitter: @ApacheCon

[jira] [Resolved] (OOZIE-2642) Extend TestSubWorkflowActionExecutor to the test if subworkflow action's parameter is properly set

[Attila Sasvari (JIRA)]

 [ 
https://issues.apache.org/jira/browse/OOZIE-2642?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Attila Sasvari resolved OOZIE-2642.
---
Resolution: Won't Fix

handled in OOZIE-2649

> Extend TestSubWorkflowActionExecutor to the test if subworkflow action's 
> parameter is properly set
> --
>
> Key: OOZIE-2642
> URL: https://issues.apache.org/jira/browse/OOZIE-2642
> Project: Oozie
>  Issue Type: Test
>Affects Versions: trunk
>Reporter: Attila Sasvari
>Assignee: Attila Sasvari
>Priority: Trivial
> Attachments: OOZIE-2642-01.patch, OOZIE-2642-02.patch
>
>
> Currently not test exercises whether a configuration property that was set in 
> a parent workflow is properly overriden by the configuration property in the 
> subworkflow's action. Now, what is checked is that global config properties 
> are handled as expected (i.e. subworkflow can override). 
> We need to check if config property defined in a subworkflow's action is 
> overrided.
> However, if  is included in parent workflow, then 
> the properties defined in job.properties (of the parent) shall not be 
> overriden.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (OOZIE-2649) Can't override sub-workflow configuration property if defined in parent workflow XML

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-2649?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15449132#comment-15449132
 ] 

Hadoop QA commented on OOZIE-2649:
--

Testing JIRA OOZIE-2649

Cleaning local git workspace



{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:green}+1 RAW_PATCH_ANALYSIS{color}
.{color:green}+1{color} the patch does not introduce any @author tags
.{color:green}+1{color} the patch does not introduce any tabs
.{color:green}+1{color} the patch does not introduce any trailing spaces
.{color:green}+1{color} the patch does not introduce any line longer than 
132
.{color:green}+1{color} the patch does adds/modifies 1 testcase(s)
{color:green}+1 RAT{color}
.{color:green}+1{color} the patch does not seem to introduce new RAT 
warnings
{color:green}+1 JAVADOC{color}
.{color:green}+1{color} the patch does not seem to introduce new Javadoc 
warnings
{color:green}+1 COMPILE{color}
.{color:green}+1{color} HEAD compiles
.{color:green}+1{color} patch compiles
.{color:green}+1{color} the patch does not seem to introduce new javac 
warnings
{color:green}+1 BACKWARDS_COMPATIBILITY{color}
.{color:green}+1{color} the patch does not change any JPA 
Entity/Colum/Basic/Lob/Transient annotations
.{color:green}+1{color} the patch does not modify JPA files
{color:green}+1 TESTS{color}
.Tests run: 1798
{color:green}+1 DISTRO{color}
.{color:green}+1{color} distro tarball builds with the patch 


{color:green}*+1 Overall result, good!, no -1s*{color}


The full output of the test-patch run is available at

.   https://builds.apache.org/job/oozie-trunk-precommit-build/3254/

> Can't override sub-workflow configuration property if defined in parent 
> workflow XML
> 
>
> Key: OOZIE-2649
> URL: https://issues.apache.org/jira/browse/OOZIE-2649
> Project: Oozie
>  Issue Type: Bug
>Reporter: Attila Sasvari
>Assignee: Attila Sasvari
> Attachments: OOZIE-2649.1.patch, OOZIE-2649.2.patch, 
> OOZIE-2649.3.patch, OOZIE-2649.4.patch, OOZIE-2649.5.patch
>
>
> After OOZIE-2511, configuration property cannot overriden in sub-workflow.
> Unfortunately some scenarios concerning subworkflows are not covered with 
> tests.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Jenkins build is back to normal : oozie-trunk-precommit-build #3254

[Apache Jenkins Server]

See 

[jira] [Updated] (OOZIE-2649) Can't override sub-workflow configuration property if defined in parent workflow XML

[Attila Sasvari (JIRA)]

 [ 
https://issues.apache.org/jira/browse/OOZIE-2649?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Attila Sasvari updated OOZIE-2649:
--
Attachment: OOZIE-2649.5.patch

same as previous patch, to retrigger test execution

> Can't override sub-workflow configuration property if defined in parent 
> workflow XML
> 
>
> Key: OOZIE-2649
> URL: https://issues.apache.org/jira/browse/OOZIE-2649
> Project: Oozie
>  Issue Type: Bug
>Reporter: Attila Sasvari
>Assignee: Attila Sasvari
> Attachments: OOZIE-2649.1.patch, OOZIE-2649.2.patch, 
> OOZIE-2649.3.patch, OOZIE-2649.4.patch, OOZIE-2649.5.patch
>
>
> After OOZIE-2511, configuration property cannot overriden in sub-workflow.
> Unfortunately some scenarios concerning subworkflows are not covered with 
> tests.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Jenkins build is back to normal : oozie-trunk-find-patches-available #131643

[Apache Jenkins Server]

See 

Build failed in Jenkins: oozie-trunk-find-patches-available #131642

[Apache Jenkins Server]

See 

--
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on H1 (yahoo-not-h2 Hadoop Mapreduce Hdfs Pig) in workspace 

 > git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
 > git config remote.origin.url 
 > https://git-wip-us.apache.org/repos/asf/oozie.git # timeout=10
Fetching upstream changes from https://git-wip-us.apache.org/repos/asf/oozie.git
 > git --version # timeout=10
 > git -c core.askpass=true fetch --tags --progress 
 > https://git-wip-us.apache.org/repos/asf/oozie.git 
 > +refs/heads/*:refs/remotes/origin/*
 > git rev-parse refs/remotes/origin/master^{commit} # timeout=10
 > git rev-parse refs/remotes/origin/origin/master^{commit} # timeout=10
Checking out Revision 43b73d836575c6fe8ac4b39c120eef7c2cb1b8f4 
(refs/remotes/origin/master)
 > git config core.sparsecheckout # timeout=10
 > git checkout -f 43b73d836575c6fe8ac4b39c120eef7c2cb1b8f4
 > git rev-list 43b73d836575c6fe8ac4b39c120eef7c2cb1b8f4 # timeout=10
[oozie-trunk-find-patches-available] $ /bin/bash 
/tmp/hudson1036657580676146723.sh
mkdir: cannot create directory 
':
 File exists
  % Total% Received % Xferd  Average Speed   TimeTime Time  Current
 Dload  Upload   Total   SpentLeft  Speed

  0 00 00 0  0  0 --:--:-- --:--:-- --:--:-- 0
  0 00 00 0  0  0 --:--:-- --:--:-- --:--:-- 0
100  192k0  192k0 0   166k  0 --:--:--  0:00:01 --:--:--  166k
100  499k0  499k0 0   221k  0 --:--:--  0:00:02 --:--:--  221k
100  614k0  614k0 0   226k  0 --:--:--  0:00:02 --:--:--  226k
curl: (18) transfer closed with outstanding read data remaining
Could not retrieve available patches from JIRA
Build step 'Execute shell' marked build as failure

Jenkins build is back to normal : oozie-trunk-find-patches-available #131637

[Apache Jenkins Server]

See 

Build failed in Jenkins: oozie-trunk-find-patches-available #131636

[Apache Jenkins Server]

See 

--
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on H1 (yahoo-not-h2 Hadoop Mapreduce Hdfs Pig) in workspace 

 > git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
 > git config remote.origin.url 
 > https://git-wip-us.apache.org/repos/asf/oozie.git # timeout=10
Fetching upstream changes from https://git-wip-us.apache.org/repos/asf/oozie.git
 > git --version # timeout=10
 > git -c core.askpass=true fetch --tags --progress 
 > https://git-wip-us.apache.org/repos/asf/oozie.git 
 > +refs/heads/*:refs/remotes/origin/*
 > git rev-parse refs/remotes/origin/master^{commit} # timeout=10
 > git rev-parse refs/remotes/origin/origin/master^{commit} # timeout=10
Checking out Revision 43b73d836575c6fe8ac4b39c120eef7c2cb1b8f4 
(refs/remotes/origin/master)
 > git config core.sparsecheckout # timeout=10
 > git checkout -f 43b73d836575c6fe8ac4b39c120eef7c2cb1b8f4
 > git rev-list 43b73d836575c6fe8ac4b39c120eef7c2cb1b8f4 # timeout=10
[oozie-trunk-find-patches-available] $ /bin/bash /tmp/hudson56941949981653807.sh
mkdir: cannot create directory 
':
 File exists
  % Total% Received % Xferd  Average Speed   TimeTime Time  Current
 Dload  Upload   Total   SpentLeft  Speed

  0 00 00 0  0  0 --:--:-- --:--:-- --:--:-- 0
100  113k0  113k0 0   151k  0 --:--:-- --:--:-- --:--:--  151k
100  420k0  420k0 0   169k  0 --:--:--  0:00:02 --:--:--  169k
100  544k0  544k0 0   199k  0 --:--:--  0:00:02 --:--:--  199k
100  719k0  719k0 0   211k  0 --:--:--  0:00:03 --:--:--  211k
curl: (18) transfer closed with outstanding read data remaining
Could not retrieve available patches from JIRA
Build step 'Execute shell' marked build as failure

[jira] Subscription: Oozie Patch Available

[jira]

Issue Subscription
Filter: Oozie Patch Available (92 issues)

Subscriber: ooziedaily

Key Summary
OOZIE-2653  Close active connection to hcat server in fs action
https://issues.apache.org/jira/browse/OOZIE-2653
OOZIE-2650  Retry coord start on database exceptions
https://issues.apache.org/jira/browse/OOZIE-2650
OOZIE-2649  Can't override sub-workflow configuration property if defined in 
parent workflow XML
https://issues.apache.org/jira/browse/OOZIE-2649
OOZIE-2641  rerunning an oozie coordinator job stucks on waiting
https://issues.apache.org/jira/browse/OOZIE-2641
OOZIE-2635  TimeZone.getTimeZone has performance issue.
https://issues.apache.org/jira/browse/OOZIE-2635
OOZIE-2632  Provide database dump/load functionality to make database migration 
easier
https://issues.apache.org/jira/browse/OOZIE-2632
OOZIE-2630  Oozie Coordinator EL Functions to get first day of the week/month
https://issues.apache.org/jira/browse/OOZIE-2630
OOZIE-2619  Make  Hive action defaults to match hive defaults when running from 
command line
https://issues.apache.org/jira/browse/OOZIE-2619
OOZIE-2618  Include hive-common in hive-beeline as compile component for 
hive-server2 (hive2 action)
https://issues.apache.org/jira/browse/OOZIE-2618
OOZIE-2617  Read properties files in action configurations
https://issues.apache.org/jira/browse/OOZIE-2617
OOZIE-2616  Add Tez profile for Hive and Pig sharelibs
https://issues.apache.org/jira/browse/OOZIE-2616
OOZIE-2615  Flaky tests 
TestCoordActionsKillXCommand.testActionKillCommandActionNumbers and 
testActionKillCommandDate 
https://issues.apache.org/jira/browse/OOZIE-2615
OOZIE-2613  Upgrade hive version from 0.13.1 to 1.2.0
https://issues.apache.org/jira/browse/OOZIE-2613
OOZIE-2606  Set spark.yarn.jars to fix Spark 2.0 with Oozie
https://issues.apache.org/jira/browse/OOZIE-2606
OOZIE-2588  Support getting credentials for same cluster hcat when credentials 
config is empty
https://issues.apache.org/jira/browse/OOZIE-2588
OOZIE-2585  Disable or remove test case 
TestPartitionDependencyManagerEhcache.testMemoryUsageAndSpeedOverflowToDisk and 
testMemoryUsageAndSpeed
https://issues.apache.org/jira/browse/OOZIE-2585
OOZIE-2584  Eliminate Thread.sleep() calls in TestMemoryLocks
https://issues.apache.org/jira/browse/OOZIE-2584
OOZIE-2583  oozie throws EL Exception when reference variable name containing 
dot
https://issues.apache.org/jira/browse/OOZIE-2583
OOZIE-2582  Populating external child Ids for action failures
https://issues.apache.org/jira/browse/OOZIE-2582
OOZIE-2573  dataset url  which contains spaces can not be handled rightly
https://issues.apache.org/jira/browse/OOZIE-2573
OOZIE-2572  SLA DURATION miss not shown when job is running for longer than 
expected time
https://issues.apache.org/jira/browse/OOZIE-2572
OOZIE-2571  Add scala.binary.version Maven property so that Scala 2.11 can be 
used
https://issues.apache.org/jira/browse/OOZIE-2571
OOZIE-2569  Adding yarn-site, core-site, hdfs-site and mapred-site into spark 
launcher 
https://issues.apache.org/jira/browse/OOZIE-2569
OOZIE-2568  SSH  action pretends to retry automaticly when it failed
https://issues.apache.org/jira/browse/OOZIE-2568
OOZIE-2564  Create new log4j config for unit tests so that logged messages 
contain threads
https://issues.apache.org/jira/browse/OOZIE-2564
OOZIE-2552  Update ActiveMQ version for security and other fixes
https://issues.apache.org/jira/browse/OOZIE-2552
OOZIE-2538  Update HttpClient versions to close security vulnerabilities
https://issues.apache.org/jira/browse/OOZIE-2538
OOZIE-2534  Versioned action libs (similar to how SharedLibs works)
https://issues.apache.org/jira/browse/OOZIE-2534
OOZIE-2530  Remove jline from parent and sharelib pig pom
https://issues.apache.org/jira/browse/OOZIE-2530
OOZIE-2528  Print out environment variables in LauncherMapper
https://issues.apache.org/jira/browse/OOZIE-2528
OOZIE-2525  SchemaChecker fails with NPE
https://issues.apache.org/jira/browse/OOZIE-2525
OOZIE-2518  Oozie is creating many files and directories in /tmp per day
https://issues.apache.org/jira/browse/OOZIE-2518
OOZIE-2513  log.scan.duration should not be used for error and audit logs
https://issues.apache.org/jira/browse/OOZIE-2513
OOZIE-2501  ZK reentrant lock doesn't work for few cases
https://issues.apache.org/jira/browse/OOZIE-2501
OOZIE-2500  -DtestJarSimple option mentioned in minioozie doc does not work
https://issues.apache.org/jira/browse/OOZIE-2500
OOZIE-2499  map-reduce launcher does not need distributed files, archives 
except  jar of input/outputformat class
 

[jira] Subscription: Oozie Patch Available

[jira]

Issue Subscription
Filter: Oozie Patch Available (0 issues)

Subscriber: ooziedaily

You may edit this subscription at:
https://issues.apache.org/jira/secure/FilterSubscription!default.jspa?subId=16327=12325939