[jira] [Created] (OOZIE-2704) Oozie fails to start if default_realm does not match Oozie prinicpal realm

2016-10-07 Thread Prabhu Joseph (JIRA) 
Prabhu Joseph created OOZIE-2704:


 Summary: Oozie fails to start if default_realm does not match 
Oozie prinicpal realm
 Key: OOZIE-2704
 URL: https://issues.apache.org/jira/browse/OOZIE-2704
 Project: Oozie
  Issue Type: Bug
  Components: core, security
Affects Versions: 4.2.0
 Environment: CentOS-6.6
Reporter: Prabhu Joseph
Priority: Critical


Problem:

Oozie fails to start with below exception when default_realm in /etc/krb5.conf 
does not match with oozie principal realm. (krb5.conf managed by DC Centrify)

{code}

2016-10-06 04:10:15,991 FATAL Services:514 - SERVER[] E0100: Could not 
initialize service [org.apache.oozie.service.HadoopAccessorService], Login 
failure for oozie/kerberos-2.openstacklo...@example.com from keytab 
/etc/security/keytabs/oozie.service.keytab: 
javax.security.auth.login.LoginException: java.lang.IllegalArgumentException: 
Illegal principal name oozie/kerberos-2.openstacklo...@example.com: 
org.apache.hadoop.security.authentication.util.KerberosName$NoMatchingRule: No 
rules applied to oozie/kerberos-2.openstacklo...@example.com
org.apache.oozie.service.ServiceException: E0100: Could not initialize service 
[org.apache.oozie.service.HadoopAccessorService], Login failure for 
oozie/kerberos-2.openstacklo...@example.com from keytab 
/etc/security/keytabs/oozie.service.keytab: 
javax.security.auth.login.LoginException: java.lang.IllegalArgumentException: 
Illegal principal name oozie/kerberos-2.openstacklo...@example.com: 
org.apache.hadoop.security.authentication.util.KerberosName$NoMatchingRule: No 
rules applied to oozie/kerberos-2.openstacklo...@example.com
at 
org.apache.oozie.service.HadoopAccessorService.kerberosInit(HadoopAccessorService.java:209)
at 
org.apache.oozie.service.HadoopAccessorService.init(HadoopAccessorService.java:136)
at 
org.apache.oozie.service.HadoopAccessorService.init(HadoopAccessorService.java:107)
at 
org.apache.oozie.service.Services.setServiceInternal(Services.java:386)
at org.apache.oozie.service.Services.setService(Services.java:372)
at org.apache.oozie.service.Services.loadServices(Services.java:305)
at org.apache.oozie.service.Services.init(Services.java:213)
at org.apache.oozie.tools.OozieDBCLI.getJdbcConf(OozieDBCLI.java:177)
at 
org.apache.oozie.tools.OozieDBCLI.createConnection(OozieDBCLI.java:943)
at 
org.apache.oozie.tools.OozieDBCLI.validateConnection(OozieDBCLI.java:951)
at org.apache.oozie.tools.OozieDBCLI.createDB(OozieDBCLI.java:190)
at org.apache.oozie.tools.OozieDBCLI.run(OozieDBCLI.java:128)
at org.apache.oozie.tools.OozieDBCLI.main(OozieDBCLI.java:79)
Caused by: java.io.IOException: Login failure for 
oozie/kerberos-2.openstacklo...@example.com from keytab 
/etc/security/keytabs/oozie.service.keytab: 
javax.security.auth.login.LoginException: java.lang.IllegalArgumentException: 
Illegal principal name oozie/kerberos-2.openstacklo...@example.com: 
org.apache.hadoop.security.authentication.util.KerberosName$NoMatchingRule: No 
rules applied to oozie/kerberos-2.openstacklo...@example.com
at 
org.apache.hadoop.security.UserGroupInformation.loginUserFromKeytab(UserGroupInformation.java:976)
at 
org.apache.oozie.service.HadoopAccessorService.kerberosInit(HadoopAccessorService.java:201)
... 12 more
Caused by: javax.security.auth.login.LoginException: 
java.lang.IllegalArgumentException: Illegal principal name 
oozie/kerberos-2.openstacklo...@example.com: 
org.apache.hadoop.security.authentication.util.KerberosName$NoMatchingRule: No 
rules applied to oozie/kerberos-2.openstacklo...@example.com
at 
org.apache.hadoop.security.UserGroupInformation$HadoopLoginModule.commit(UserGroupInformation.java:202)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at javax.security.auth.login.LoginContext.invoke(LoginContext.java:755)
at 
javax.security.auth.login.LoginContext.access$000(LoginContext.java:195)
at javax.security.auth.login.LoginContext$4.run(LoginContext.java:682)
at javax.security.auth.login.LoginContext$4.run(LoginContext.java:680)
at java.security.AccessController.doPrivileged(Native Method)
at 
javax.security.auth.login.LoginContext.invokePriv(LoginContext.java:680)
at javax.security.auth.login.LoginContext.login(LoginContext.java:588)
at 
org.apache.hadoop.security.UserGroupInformation.loginUserFromKeytab(UserGroupInformation.java:967)
... 13 more
Caused by: java.lang.IllegalArgumentException: Illegal principal name

[jira] [Commented] (OOZIE-2619) Make Hive action defaults to match hive defaults when running from command line

2016-10-07 Thread Venkat Ranganathan (JIRA) 

[ 
https://issues.apache.org/jira/browse/OOZIE-2619?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15556996#comment-15556996
 ] 

Venkat Ranganathan commented on OOZIE-2619:
---

Test failures unrelated to patch

> Make  Hive action defaults to match hive defaults when running from command 
> line
> 
>
> Key: OOZIE-2619
> URL: https://issues.apache.org/jira/browse/OOZIE-2619
> Project: Oozie
>  Issue Type: Bug
>  Components: core
>Affects Versions: 3.3.0, 4.2.0
>Reporter: Venkat Ranganathan
>Assignee: Venkat Ranganathan
> Attachments: OOZIE-2619.patch
>
>
> Over a few patches, we have done a few fixes to make Oozie Hive actions 
> easier for users.
> One of them was OOZIE-2051 which allows default hive and tez site xml configs 
> to be added to hive actions automatically by introducing action specific 
> configuration directory under oozie conf/action-conf directory and as a bonus 
> in an Ambari managed cluster the hive site changes done as part of the Hive 
> components are automatically reflected into the oozie hive action defaults.
> But there is one issue pending for Oozie hive actions.
> Oozie Hive jobs launched via hive action  are historically restricted to one 
> reducer by default (and also there are few other in terms of split sizes 
> etc).   Thisvis because of the way Oozie action config management is done and 
> how Hive was determining the reducers.   Hive uses mapreduce.job.reduces to 
> determine if the reducers have to be dynamically determined (when this 
> parameter is initialized to an invalid value -1) or explicitly determined by 
> the users.   In HiveConf, this is internally set to -1 if not in 
> hive-site.xml or in one of the set statements.
> Oozie, when it prepares the action configuration, has the 
> mapreduce.job.reduces set to 1 (from mapred-default).   As part of the hive 
> action, Oozie writes the action configuration prepared (the action.xml) also 
> as hive-site.xml with the value for mapreduce.job.reduces set to 1.
> There are a few ways to overcome this issue, true to Oozie being very
> flexible with lots of options :).  I may be missing a few other
> options here!
> # Explicitly set the mapreduce.job.reduces parameter in the configuration 
> element of the action
> Every hive workflow configuration has be changed
> #  Add the parameter to a job-xml for the action
> Once again affects all actions
> #  Set the parameter to disable loading of the default *-site.xml
> files as provided by OOZIE-2205
>We need to make sure that the  *-site.xml are otherwise available to the 
> containers - either have hadoop conf directory (typically /etc/hadoop/conf) 
> in the mapred framework classpath or explicitly make the files using other 
> mechanisms available (as files, archives, in sharelib ec).   The big issue is 
> that this affects rolling upgrades once you add explicit config dependency
> Unfortunately we can't use the default action config addition introduced in 
> OOZIE-2051 for adding one more configuration file to the oozie hive action 
> conf directory with hive MR defaults.
> The config files under the action-conf/hive/*.xml or action-conf/hive.xml are 
> all merged using the method injectDefaults which only updates the target only 
> if it does not exist in the target configuration map.   In our case, 
> mapreduce.job.reduces already exsits in the action default configuration 
> (coming from mapred-default.xml) and hence does not get overwritten from the 
> action-conf/hive configuration files.
> The fix (essentially one line of code change) is to use the copy method of 
> XConfiguration  to copy the action-default config instead of using the 
> injectDefaults method and then provide the action-default/hive.xm with the 
> required mapred hive parameters with hive expected initial values.
> This patch introduces a change that has potential backward compatibility 
> issues.
> * If the action-conf/.xml currently has entries that were no-ops so 
> far, they can be added to the action configuration.
> * Hive will work as expected when run as an Oozie action without users 
> needing to resort to changes!



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (OOZIE-2613) Upgrade hive version from 0.13.1 to 1.2.0

2016-10-07 Thread Abhishek Bafna (JIRA) 

[ 
https://issues.apache.org/jira/browse/OOZIE-2613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15556946#comment-15556946
 ] 

Abhishek Bafna commented on OOZIE-2613:
---

[~rkanter] For three testcase patch uses {{XTestCase#setSystemProperty}} to set 
the current user, because with the {{test}} user they are failing with 
{{Permission denied}}. Otherwise it is ready. Thanks.

> Upgrade hive version from 0.13.1 to 1.2.0
> -
>
> Key: OOZIE-2613
> URL: https://issues.apache.org/jira/browse/OOZIE-2613
> Project: Oozie
>  Issue Type: Bug
>  Components: core
>Reporter: Abhishek Bafna
>Assignee: Abhishek Bafna
>Priority: Blocker
> Fix For: 4.3.0
>
> Attachments: OOZIE-2613-00.patch, OOZIE-2613-01.patch, 
> OOZIE-2613-02.patch, OOZIE-2613-03.patch, OOZIE-2613-05.patch, 
> OOZIE-2613-06.patch, OOZIE-2613-07.patch, OOZIE-2613-08.patch, 
> OOZIE-2613-09.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: [SCOPE] Oozie 4.3.0 Release

2016-10-07 Thread Robert Kanter 
- I took care of OOZIE-1814 yesterday.
- OOZIE-2658 should be good after it gets rebased (doesn't apply cleanly);
I'm guessing that will have to wait until Monday PDT due to timezones.
- I'm not sure what's the current status of OOZIE-2613.  test-patch looks
like it's ready, but from subsequent comments, it sounds like it's not?

- Robert

On Fri, Oct 7, 2016 at 5:18 AM, Abhishek Bafna  wrote:

> Hi All,
>
> Today EOD (UTC), we will freeze the scope and further commits into
> branch-4.3 expect any blocker issues.
>
> By early next week, we will have our first release candidate for review.
>
> Regards,
> Abhishek
> > On Sep 23, 2016, at 11:11 PM, Abhishek Bafna 
> wrote:
> >
> > Hi All,
> >
> > The new branch creation and update of the version in the current branch
> will be done on Monday (IST).
> >
> > Thanks,
> > Abhishek
> >> On Sep 21, 2016, at 10:56 PM, Robert Kanter 
> wrote:
> >>
> >> --94eb2c04982ae57b7b053d07d72b
> >> Content-Type: text/plain; charset=UTF-8
> >>
> >> Oh, right.  I forgot that Shwetha was going to help you with that.
> That's
> >> fine with me.
> >>
> >> On Wed, Sep 21, 2016 at 3:22 AM, Abhishek Bafna 
> >> wrote:
> >>
> >>> Hello Robert,
> >>>
> >>> Yes. I do not have permissions to do it. I was planning to take
> Shwetha's
> >>> help. I am fine with you doing it.
> >>> Please let me know if there is any change in the plan.
> >>>
> >>> Thanks,
> >>> Abhishek
>  On Sep 21, 2016, at 2:11 AM, Robert Kanter 
> wrote:
> 
>  Abhishek,
> 
>  You don't have git permissions to create the release branch.  I can
> take
>  care of that on Thursday if you want.
> 
> 
>  - Robert
> 
>  On Tue, Sep 20, 2016 at 1:30 PM, Abhishek Bafna  >
>  wrote:
> 
> > Hello All,
> >
> > I think we can create the new branch by Thursday (IST) and update
> >>> version
> > for new branch and master. If we have any important pending Jira's by
> >>> then
> > we will commit them to both new branch and master.
> >
> > I will start working on the next steps for release from release doc.
> > https://cwiki.apache.org/confluence/display/OOZIE/How+To+Release <
> > https://cwiki.apache.org/confluence/display/OOZIE/How+To+Release>
> >
> > Please do provide if you have any feedback or suggestion.
> >
> > Thanks,
> > Abhishek
> >> On Sep 20, 2016, at 12:52 AM, Rohini Palaniswamy <
> > rohini.adi...@gmail.com> wrote:
> >>
> >> Sorry Abhishek for the long delay. Was on vacation and then busy
> >>> catching
> >> up with stuff. I did review a bunch of patches today which were
> marked
> >> 4.3.0 and Patch Available apart from those which Robert or others
> have
> >> already not commented on. Skipping those as they have better
> context.
> >>> Tag
> >> my name if there is something else that needs to be looked at.
> >>
> >> Regards,
> >> Rohini
> >>
> >> On Mon, Sep 19, 2016 at 6:33 AM, Abhishek Bafna <
> bafna.i...@gmail.com>
> >> wrote:
> >>
> >>> Hello Robert, Rohini, Purshotam, Jaydeep
> >>>
> >>> As we discussed earlier, will be bumping the priority of some
> Jira's
> >>> to
> >>> 'blocker' for to be included into Oozie 4.3.0 release. We will try
> to
> >>> complete as much possible out of that. (It would to good complete
> >>> all.)
> >>>
> >>> @All, If I miss something, Please do priorities or send a message
> >>> here.
> >>>
> >>> Thanks a lot for all the efforts and time we have put in last few
> >>> weeks.
> >>>
> >>> Thanks,
> >>> Abhishek
>  On Sep 8, 2016, at 10:14 AM, Abhishek Bafna  >
> >>> wrote:
> 
>  Hi All,
> 
>  It has been a long time since we started the scope discussion for
> the
> >>> Oozie-4.3.0 release.
>  Since then a lot of patches got reviewed and committed and I
> would to
> >>> thank each one you for contributing your efforts for that.
> 
>  I think we should move to the next stage and go into blocker only
> >>> mode
> >>> from next week (one week). After that 7-10 days for preparing and
> >>> completing release condidate.
> 
>  We did plan to update some of the old tech and some of them are
> still
> >>> pending for review. If can spend time into reviewing them, it
> would be
> >>> good, else we might have to move without them.
> 
>  There are also some good to have patches available for review and
> >>> commit, I will bump up the priority for them. If I miss something,
> > please
> >>> feel free to do the same.
> 
>  Regards,
>  Abhishek
> 
> > On Aug 31, 2016, at 2:30 PM, Abhishek Bafna <
> bafna.i...@gmail.com>
> >>> wrote:
> >
> > Hi All,

[jira] [Commented] (OOZIE-2613) Upgrade hive version from 0.13.1 to 1.2.0

2016-10-07 Thread Robert Kanter (JIRA) 

[ 
https://issues.apache.org/jira/browse/OOZIE-2613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15556597#comment-15556597
 ] 

Robert Kanter commented on OOZIE-2613:
--

The latest test-patch run shows all tests green.  Is this not ready?

> Upgrade hive version from 0.13.1 to 1.2.0
> -
>
> Key: OOZIE-2613
> URL: https://issues.apache.org/jira/browse/OOZIE-2613
> Project: Oozie
>  Issue Type: Bug
>  Components: core
>Reporter: Abhishek Bafna
>Assignee: Abhishek Bafna
>Priority: Blocker
> Fix For: 4.3.0
>
> Attachments: OOZIE-2613-00.patch, OOZIE-2613-01.patch, 
> OOZIE-2613-02.patch, OOZIE-2613-03.patch, OOZIE-2613-05.patch, 
> OOZIE-2613-06.patch, OOZIE-2613-07.patch, OOZIE-2613-08.patch, 
> OOZIE-2613-09.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Comment Edited] (OOZIE-2658) --driver-class-path can overwrite the classpath in SparkMain

2016-10-07 Thread Robert Kanter (JIRA) 

[ 
https://issues.apache.org/jira/browse/OOZIE-2658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15556573#comment-15556573
 ] 

Robert Kanter edited comment on OOZIE-2658 at 10/7/16 11:26 PM:


Can you rebase the patch?  It no longer applies cleanly.

+1 after that (and pending Jenkins again).


was (Author: rkanter):
+1

> --driver-class-path can overwrite the classpath in SparkMain
> 
>
> Key: OOZIE-2658
> URL: https://issues.apache.org/jira/browse/OOZIE-2658
> Project: Oozie
>  Issue Type: Bug
>Reporter: Peter Cseh
>Assignee: Peter Cseh
> Fix For: 4.3.0
>
> Attachments: OOZIE-2658.001.patch, OOZIE-2658.002.patch
>
>
> In SparkMain we use the option {{--conf spark.driver.extraClassPath=}} to put 
> together the jars needed for Spark. If the user specifies this option, Oozie 
> appends to the value.
> If {{--driver-class-path}} is defined in {{}} then it's value 
> overwrites the other ones, causing the Spark action to fail.
> We should handle both ways to define classpaths the same.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (OOZIE-2658) --driver-class-path can overwrite the classpath in SparkMain

2016-10-07 Thread Robert Kanter (JIRA) 

 [ 
https://issues.apache.org/jira/browse/OOZIE-2658?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Robert Kanter updated OOZIE-2658:
-
 Priority: Blocker  (was: Major)
Fix Version/s: 4.3.0

> --driver-class-path can overwrite the classpath in SparkMain
> 
>
> Key: OOZIE-2658
> URL: https://issues.apache.org/jira/browse/OOZIE-2658
> Project: Oozie
>  Issue Type: Bug
>Reporter: Peter Cseh
>Assignee: Peter Cseh
>Priority: Blocker
> Fix For: 4.3.0
>
> Attachments: OOZIE-2658.001.patch, OOZIE-2658.002.patch
>
>
> In SparkMain we use the option {{--conf spark.driver.extraClassPath=}} to put 
> together the jars needed for Spark. If the user specifies this option, Oozie 
> appends to the value.
> If {{--driver-class-path}} is defined in {{}} then it's value 
> overwrites the other ones, causing the Spark action to fail.
> We should handle both ways to define classpaths the same.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (OOZIE-2658) --driver-class-path can overwrite the classpath in SparkMain

2016-10-07 Thread Robert Kanter (JIRA) 

[ 
https://issues.apache.org/jira/browse/OOZIE-2658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15556573#comment-15556573
 ] 

Robert Kanter commented on OOZIE-2658:
--

+1

> --driver-class-path can overwrite the classpath in SparkMain
> 
>
> Key: OOZIE-2658
> URL: https://issues.apache.org/jira/browse/OOZIE-2658
> Project: Oozie
>  Issue Type: Bug
>Reporter: Peter Cseh
>Assignee: Peter Cseh
> Attachments: OOZIE-2658.001.patch, OOZIE-2658.002.patch
>
>
> In SparkMain we use the option {{--conf spark.driver.extraClassPath=}} to put 
> together the jars needed for Spark. If the user specifies this option, Oozie 
> appends to the value.
> If {{--driver-class-path}} is defined in {{}} then it's value 
> overwrites the other ones, causing the Spark action to fail.
> We should handle both ways to define classpaths the same.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (OOZIE-2406) Completely rewrite GraphGenerator code

2016-10-07 Thread Robert Kanter (JIRA) 

[ 
https://issues.apache.org/jira/browse/OOZIE-2406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15556563#comment-15556563
 ] 

Robert Kanter commented on OOZIE-2406:
--

Another option (which might be the best) is to have the Oozie Server send out 
the DAG as JSON and use a client-side library for rendering the graph (there 
must be many Javascript libraries for this).

> Completely rewrite GraphGenerator code
> --
>
> Key: OOZIE-2406
> URL: https://issues.apache.org/jira/browse/OOZIE-2406
> Project: Oozie
>  Issue Type: New Feature
>Affects Versions: 4.2.0
>Reporter: Robert Kanter
>
> The Web UI currently generates a graph of the workflow DAG as a png image 
> that it can show to the user ({{GraphGenerator}} class).  Unfortunately, 
> there are a number of downsides to the current implementation:
> # The image is generated server-side, which doesn't scale well and eats up 
> lots of memory
> #- To help combat this issue, we only generate graphs for workflows with less 
> than 25 nodes and we disabled the refresh button in the UI
> # It's slow
> # It uses a library ({{net.sf.jung}} from http://jung.sourceforge.net), which 
> hasn't been updated since 2010.  
> #- This library also has a dependency on a fork of Commons-Collections 
> ({{net.sourceforge.collections}} from 
> http://sourceforge.net/projects/collections/), which similarly hasn't been 
> updated since 2010.  This is a problem because we can't update 
> Commons-Collections when there are security concerns such as COLLECTIONS-580 
> (though Oozie is *not* susceptible to this attack).
> It would be good to do a complete rewrite, using a different library and 
> remove Jung and the Commons-Collections fork.  Whatever we choose, it should 
> an svg which will draw the image on the browser.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (OOZIE-2644) Skip queuing Notification Commands when there's nothing to notify

2016-10-07 Thread Robert Kanter (JIRA) 

[ 
https://issues.apache.org/jira/browse/OOZIE-2644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15556539#comment-15556539
 ] 

Robert Kanter commented on OOZIE-2644:
--

It might make sense to generalize this.  For instance, we have the 
{{eagerVerifyPrecondition}} method in {{XCommand}}, which checks if the 
preconditions are met before acquiring the lock to be more efficient.  We could 
add a submission time check (say {{submissionVerifyPrecondition}} or something) 
that gets called when an {{XCommand}} is added to the callablequeue.  If it 
fails, then we wouldn't even add the {{XCommand}} to the queue.  So for these 
Notification XCommands, we'd have them check if there's a url set or not.

> Skip queuing Notification Commands when there's nothing to notify
> -
>
> Key: OOZIE-2644
> URL: https://issues.apache.org/jira/browse/OOZIE-2644
> Project: Oozie
>  Issue Type: Improvement
>Affects Versions: 3.1.3
>Reporter: Robert Kanter
>Assignee: Azrael Seoeun
> Fix For: 5.0.0
>
>
> When you use the 
> [Workflow|https://oozie.apache.org/docs/4.2.0/WorkflowFunctionalSpec.html#a5_Workflow_Notifications]
>  or 
> [Coordinator|https://oozie.apache.org/docs/4.2.0/CoordinatorFunctionalSpec.html#a15._Coordinator_Notifications]
>  Notification features, Oozie can end up queuing up a lot of 
> {{WorkflowNotificationXCommand}} and {{CoordActionNotificationXCommand}}.  
> This happens even if there's no notification configured on the job (which I 
> imagine is most of the time); in this case, the {{execute}} method simply 
> does nothing.  This is wasteful and clogs the queue up.  
> We should change the code so that it doesn't queue up one of these Commands 
> unless there's actually a URL to notify.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (OOZIE-1986) Add FindBugs report to pre-commit build

2016-10-07 Thread Robert Kanter (JIRA) 

[ 
https://issues.apache.org/jira/browse/OOZIE-1986?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15556006#comment-15556006
 ] 

Robert Kanter commented on OOZIE-1986:
--

That's fine.  I think we can ignore the line length check on this one.

Can we make the results of the findbugs diff show up in the comment made by 
Hadoop QA?  Otherwise, you have to go digging around in the output text.

> Add FindBugs report to pre-commit build
> ---
>
> Key: OOZIE-1986
> URL: https://issues.apache.org/jira/browse/OOZIE-1986
> Project: Oozie
>  Issue Type: Bug
>  Components: tests
>Reporter: Robert Kanter
>Assignee: Andras Piros
> Fix For: 5.0.0
>
> Attachments: OOZIE-1986.001.patch, OOZIE-1986.002.patch, 
> OOZIE-1986.003.patch, OOZIE-1986.004.patch
>
>
> Now that we have OOZIE-1793, it will be really useful to have the pre-commit 
> build generate a findbugs report.  It should probably before and after the 
> patch and only complain it there are more after.  And it should link to the 
> report.  Ideally, this would do some kind of diff (like Hadoop does) so it's 
> easy to identify the new findbugs items.
> You can generate the findbugs reports by running:
> {noformat}
> mvn verify -DskipTests
> {noformat}
> This also runs checkstyle



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Assigned] (OOZIE-2616) Add Tez profile for Hive and Pig sharelibs

2016-10-07 Thread Thomas Poepping (JIRA) 

 [ 
https://issues.apache.org/jira/browse/OOZIE-2616?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Thomas Poepping reassigned OOZIE-2616:
--

Assignee: Thomas Poepping

> Add Tez profile for Hive and Pig sharelibs
> --
>
> Key: OOZIE-2616
> URL: https://issues.apache.org/jira/browse/OOZIE-2616
> Project: Oozie
>  Issue Type: Improvement
>Affects Versions: trunk
>Reporter: Thomas Poepping
>Assignee: Thomas Poepping
> Attachments: OOZIE-2616-1.patch, OOZIE-2616-2.patch
>
>
> More recent versions of Hive and Pig depend on Tez as their default execution 
> engine. As the sharelib for those applications stands today, those 
> applications are unable to run Tez jobs. This patch adds an optional profile 
> to the root pom that will include the proper Tez dependencies into those 
> sharelibs, while excluding problematic jars to keep the sharelibs from having 
> classpath duplication issues. *Note*: this patch will only be required for 
> those clients who specify their own, newer versions of Hive or Pig. This 
> change is not required for the current supported versions of those 
> applications (hence why the profile is off by default).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (OOZIE-2616) Add Tez profile for Hive and Pig sharelibs

2016-10-07 Thread Thomas Poepping (JIRA) 

[ 
https://issues.apache.org/jira/browse/OOZIE-2616?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1843#comment-1843
 ] 

Thomas Poepping commented on OOZIE-2616:


double ping? If we're freezing Oozie-4.3.0 at EOD today, it'd be great to get 
this in beforehand.

> Add Tez profile for Hive and Pig sharelibs
> --
>
> Key: OOZIE-2616
> URL: https://issues.apache.org/jira/browse/OOZIE-2616
> Project: Oozie
>  Issue Type: Improvement
>Affects Versions: trunk
>Reporter: Thomas Poepping
> Attachments: OOZIE-2616-1.patch, OOZIE-2616-2.patch
>
>
> More recent versions of Hive and Pig depend on Tez as their default execution 
> engine. As the sharelib for those applications stands today, those 
> applications are unable to run Tez jobs. This patch adds an optional profile 
> to the root pom that will include the proper Tez dependencies into those 
> sharelibs, while excluding problematic jars to keep the sharelibs from having 
> classpath duplication issues. *Note*: this patch will only be required for 
> those clients who specify their own, newer versions of Hive or Pig. This 
> change is not required for the current supported versions of those 
> applications (hence why the profile is off by default).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (OOZIE-2666) Support embedding Jetty into Oozie

2016-10-07 Thread Hadoop QA (JIRA) 

[ 
https://issues.apache.org/jira/browse/OOZIE-2666?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1628#comment-1628
 ] 

Hadoop QA commented on OOZIE-2666:
--

Testing JIRA OOZIE-2666

Cleaning local git workspace



{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:red}-1 RAW_PATCH_ANALYSIS{color}
.{color:green}+1{color} the patch does not introduce any @author tags
.{color:red}-1{color} the patch contains 13 line(s) with tabs
.{color:green}+1{color} the patch does not introduce any trailing spaces
.{color:red}-1{color} the patch contains 9 line(s) longer than 132 
characters
.{color:green}+1{color} the patch does adds/modifies 2 testcase(s)
{color:red}-1 RAT{color}
.{color:red}-1{color} the patch seems to introduce 3 new RAT warning(s)
{color:green}+1 JAVADOC{color}
.{color:green}+1{color} the patch does not seem to introduce new Javadoc 
warnings
{color:red}-1 COMPILE{color}
.{color:green}+1{color} HEAD compiles
.{color:green}+1{color} patch compiles
.{color:red}-1{color} the patch seems to introduce 3 new javac warning(s)
{color:green}+1 BACKWARDS_COMPATIBILITY{color}
.{color:green}+1{color} the patch does not change any JPA 
Entity/Colum/Basic/Lob/Transient annotations
.{color:green}+1{color} the patch does not modify JPA files
{color:green}+1 TESTS{color}
.Tests run: 1821
{color:green}+1 DISTRO{color}
.{color:green}+1{color} distro tarball builds with the patch 


{color:red}*-1 Overall result, please check the reported -1(s)*{color}


The full output of the test-patch run is available at

.   https://builds.apache.org/job/oozie-trunk-precommit-build/3363/

> Support embedding Jetty into Oozie
> --
>
> Key: OOZIE-2666
> URL: https://issues.apache.org/jira/browse/OOZIE-2666
> Project: Oozie
>  Issue Type: Sub-task
>  Components: core
>Reporter: Attila Sasvari
>Assignee: Attila Sasvari
> Attachments: OOZIE-2666.PATCH-01.patch, OOZIE-2666.PATCH-02.patch, 
> OOZIE-2666.PATCH-03.patch, OOZIE-2666.PATCH-05.patch, 
> OOZIE-2666.PATCH-06.patch, OOZIE-2666.PATCH-07.patch
>
>
> The purpose of this task is to allow Oozie to run as a standalone application 
> rather than being deployed to a container inside a WAR file.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Build failed in Jenkins: oozie-trunk-precommit-build #3363

2016-10-07 Thread Apache Jenkins Server 
See 

--
[...truncated 5816 lines...]
[INFO] Copying jdo-api-3.0.1.jar to 

[INFO] Copying hive-shims-0.20S-0.13.1.jar to 

[INFO] Copying oozie-hadoop-utils-hadoop-2-4.4.0-SNAPSHOT.jar to 

[INFO] Copying log4j-1.2.17.jar to 

[INFO] Copying tools.jar to 

[INFO] Copying netty-3.6.2.Final.jar to 

[INFO] Copying apache-log4j-extras-1.2.17.jar to 

[INFO] Copying commons-digester-1.8.jar to 

[INFO] Copying activation-1.1.jar to 

[INFO] Copying collections-generic-4.01.jar to 

[INFO] Copying hive-shims-0.23-0.13.1.jar to 

[INFO] Copying xmlenc-0.52.jar to 

[INFO] Copying httpcore-4.3.3.jar to 

[INFO] Copying openjpa-persistence-2.4.1.jar to 

[INFO] Copying jetty-io-9.2.19.v20160908.jar to 

[INFO] Copying hive-shims-0.20-0.13.1.jar to 

[INFO] Copying jetty-http-9.2.19.v20160908.jar to 

[INFO] Copying jetty-plus-9.2.19.v20160908.jar to 

[INFO] Copying hive-serde-0.13.1.jar to 

[INFO] Copying antlr-2.7.7.jar to 

[INFO] Copying jackson-databind-2.2.2.jar to 

[INFO] Copying oozie-sharelib-oozie-4.4.0-SNAPSHOT.jar to 

[INFO] Copying commons-beanutils-core-1.8.0.jar to 

[INFO] Copying hadoop-yarn-server-common-2.4.0.jar to 

[INFO] Copying geronimo-j2ee-management_1.1_spec-1.0.1.jar to 

[INFO] Copying datanucleus-core-3.2.10.jar to 

[INFO] Copying slf4j-log4j12-1.6.6.jar to 

[INFO] Copying aopalliance-1.0.jar to 

[INFO] Copying hadoop-mapreduce-client-core-2.4.0.jar to 

[INFO] Copying datanucleus-api-jdo-3.2.6.jar to

[jira] [Updated] (OOZIE-2666) Support embedding Jetty into Oozie

2016-10-07 Thread Attila Sasvari (JIRA) 

 [ 
https://issues.apache.org/jira/browse/OOZIE-2666?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Attila Sasvari updated OOZIE-2666:
--
Attachment: OOZIE-2666.PATCH-07.patch

> Support embedding Jetty into Oozie
> --
>
> Key: OOZIE-2666
> URL: https://issues.apache.org/jira/browse/OOZIE-2666
> Project: Oozie
>  Issue Type: Sub-task
>  Components: core
>Reporter: Attila Sasvari
>Assignee: Attila Sasvari
> Attachments: OOZIE-2666.PATCH-01.patch, OOZIE-2666.PATCH-02.patch, 
> OOZIE-2666.PATCH-03.patch, OOZIE-2666.PATCH-05.patch, 
> OOZIE-2666.PATCH-06.patch, OOZIE-2666.PATCH-07.patch
>
>
> The purpose of this task is to allow Oozie to run as a standalone application 
> rather than being deployed to a container inside a WAR file.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Review Request 52399: OOZIE-2666 Support embedding Jetty into Oozie

2016-10-07 Thread Attila Sasvari 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52399/
---

(Updated Oct. 7, 2016, 2:59 p.m.)


Review request for oozie, András Piros, Peter Cseh, Peter Bacsko, and Robert 
Kanter.


Changes
---

- Make configurable
- HTTP request / response header size 
- thread pool size
- Remove jetty opts that are defined in oozie-default.xml (i.e. not required to 
be set via system properties)
- Add HttpConfigurationWrapper


Repository: oozie-git


Description
---

Embedding jetty into Oozie so that it can run as a standalone application. The 
changes also try to address OOZIE-2317 (i.e. Tomcat 6 is EOL).

New functionality
- New module (server) is added that sets up an embedded Jetty server and start 
Oozie services. Servlet mapping is done by reading web.xml of webapp at 
runtime. JSP is handled with custom code. Server version is not revealed in 
server repsonses.
- SSL protocols and ciphers can be configured via system properties and 
environment variables. Precedence: system properties, environment variables, 
default values
   
Changes in existing code
- Excluded jetty 6 dependencies from core and updated tests accordingly  

Packaging
- oozie.sh is modified so that it starts Oozie with embedded jetty by default. 
If someone would like to use tomcat for any reason, they can set an environment 
variable (e.g. OOZIE_USE_TOMCAT=1).

TODO:
- Add more tests
- Add more documentation
- Code cleanup + refactoring in packaging and core parts
- Maven clean up
- Allow to tune more Jetty settings (for example threadpool)
- More security measures (e.g. protect against clickjacking, CSRF, etc.)
- Update Oozie Documentation


Diffs (updated)
-

  core/src/main/java/org/apache/oozie/util/Instrumentation.java 
fa1e92a0f1fadfb66c5e66fea5f26f57579080e7 
  core/src/main/resources/oozie-default.xml 
e71ebe3b7a85e6b23176ef30713af63847144498 
  distro/pom.xml c50572c57a376b28963d4e7da8ac7df777fe0480 
  distro/src/main/bin/oozie-jetty-server.sh PRE-CREATION 
  distro/src/main/bin/oozie-setup.sh 79b049bccceb2690f8a673a885a615c8d4d9578c 
  distro/src/main/bin/oozied.sh a869c3da177c863a068f2af45c7bca9d5cb771ac 
  pom.xml 704a29f4e4805e3e08c2a547b2a375f6b1b2 
  server/pom.xml PRE-CREATION 
  server/src/main/assemblies/empty.xml PRE-CREATION 
  server/src/main/java/org/apache/oozie/server/EmbeddedOozieServer.java 
PRE-CREATION 
  server/src/main/java/org/apache/oozie/server/HttpConfigurationWrapper.java 
PRE-CREATION 
  server/src/main/java/org/apache/oozie/server/JspHandler.java PRE-CREATION 
  server/src/main/java/org/apache/oozie/server/SSLServerConnectorFactory.java 
PRE-CREATION 
  server/src/main/resources/checkstyle-header.txt PRE-CREATION 
  server/src/main/resources/checkstyle.xml PRE-CREATION 
  server/src/test/java/org/apache/oozie/server/TestEmbeddedOozieServer.java 
PRE-CREATION 
  
server/src/test/java/org/apache/oozie/server/TestSSLServerConnectorFactory.java 
PRE-CREATION 
  src/main/assemblies/distro.xml 1ffbfd6d2ba33b390999e9094cbb336fbce45c21 

Diff: https://reviews.apache.org/r/52399/diff/


Testing
---

- Tested basic functionality by executing a workflow that uses the sample 
JavaAction
- without SSL - on a 2.4.0 pseudo Hadoop cluster
- SSL with Kerberos is using a test CDH cluster 
- Added new unit tests that check
- If oozie.ssl.enabled is not specified, server starts without SSL settings 
- If oozie.ssl.enabled is specified, server starts with SSL settings
- SSL protocols and ciphers can be configured via system properties and 
environment variables 
- Ran subset of tests using Hadoop-2 profile
- mvn clean package assembly:single   -DjavaVersion=1.8 -DtargetVersion=1.7 
 -Dtest=TestJavaActionExecutor  -Phadoop-2 -Dhadoop.version=2.4.0


Thanks,

Attila Sasvari

Re: Review Request 52429: OOZIE-1986 Add FindBugs report to pre-commit build

2016-10-07 Thread Attila Sasvari 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52429/#review151797
---




bin/test-patch-11-findbugs-diff (line 121)


Should not we clean up / delete this directory at the end of the execution?

If not and we decide to keep the jar / directory for some reason, then it 
is not necessary to download findbugs diff jar each time. For example it can 
happen that jenkins jobs run on the same build server. In this case we could 
just reuse the already downloaded jar (if its md5 checksum is OK). What do you 
think?


- Attila Sasvari


On Oct. 7, 2016, 10:39 a.m., András Piros wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/52429/
> ---
> 
> (Updated Oct. 7, 2016, 10:39 a.m.)
> 
> 
> Review request for oozie and Robert Kanter.
> 
> 
> Repository: oozie-git
> 
> 
> Description
> ---
> 
> We go for that solution:
> 
> 1. create a new script as `bin/test-patch-11-findbugs-diff` so that it comes 
> right after compile
> 
> 2. `mvn clean verify -DskipTests` before and after applying the patch, and 
> saving each and every module's `target/findbugs/findbugsXml.xml` to a secure 
> folder
> 
> 3. call the `findbugs-diff` app for all the before / after FindBugs xml pairs
> 
> 4. analyze all the `findbugs-new.xml` entries for nodes with XPath 
> `/BugCollection/BugInstance` entries. Stop on any BugInstance w/ `priority` 
> and / or `rank` lower than threshold
> 
> 5. have the reports merged and be downloadable in any case in following 
> scenarios:
> 
> * -1 for bugs that are too serious to apply the patch with them
> * 0 for other new bugs
> * +1 if there is no new bug
> 
> 
> Diffs
> -
> 
>   bin/test-patch-11-findbugs-diff PRE-CREATION 
>   bin/test-patch-11-findbugs-diff-0.1.0-all.jar.md5 PRE-CREATION 
>   pom.xml a33d2c5 
> 
> Diff: https://reviews.apache.org/r/52429/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> András Piros
> 
>

[jira] [Resolved] (OOZIE-2703) OYA: Streaming action

2016-10-07 Thread Peter Bacsko (JIRA) 

 [ 
https://issues.apache.org/jira/browse/OOZIE-2703?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Peter Bacsko resolved OOZIE-2703.
-
Resolution: Fixed

> OYA: Streaming action
> -
>
> Key: OOZIE-2703
> URL: https://issues.apache.org/jira/browse/OOZIE-2703
> Project: Oozie
>  Issue Type: Sub-task
>Reporter: Peter Bacsko
>Assignee: Peter Bacsko
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (OOZIE-1986) Add FindBugs report to pre-commit build

2016-10-07 Thread Andras Piros (JIRA) 

[ 
https://issues.apache.org/jira/browse/OOZIE-1986?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1047#comment-1047
 ] 

Andras Piros commented on OOZIE-1986:
-

As said before, bash files are quite cumbersome to cut line length...

> Add FindBugs report to pre-commit build
> ---
>
> Key: OOZIE-1986
> URL: https://issues.apache.org/jira/browse/OOZIE-1986
> Project: Oozie
>  Issue Type: Bug
>  Components: tests
>Reporter: Robert Kanter
>Assignee: Andras Piros
> Fix For: 5.0.0
>
> Attachments: OOZIE-1986.001.patch, OOZIE-1986.002.patch, 
> OOZIE-1986.003.patch, OOZIE-1986.004.patch
>
>
> Now that we have OOZIE-1793, it will be really useful to have the pre-commit 
> build generate a findbugs report.  It should probably before and after the 
> patch and only complain it there are more after.  And it should link to the 
> report.  Ideally, this would do some kind of diff (like Hadoop does) so it's 
> easy to identify the new findbugs items.
> You can generate the findbugs reports by running:
> {noformat}
> mvn verify -DskipTests
> {noformat}
> This also runs checkstyle



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (OOZIE-1986) Add FindBugs report to pre-commit build

2016-10-07 Thread Hadoop QA (JIRA) 

[ 
https://issues.apache.org/jira/browse/OOZIE-1986?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1041#comment-1041
 ] 

Hadoop QA commented on OOZIE-1986:
--

Testing JIRA OOZIE-1986

Cleaning local git workspace



{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:red}-1 RAW_PATCH_ANALYSIS{color}
.{color:green}+1{color} the patch does not introduce any @author tags
.{color:green}+1{color} the patch does not introduce any tabs
.{color:green}+1{color} the patch does not introduce any trailing spaces
.{color:red}-1{color} the patch contains 7 line(s) longer than 132 
characters
.{color:green}+1{color} the patch does adds/modifies 2 testcase(s)
{color:green}+1 RAT{color}
.{color:green}+1{color} the patch does not seem to introduce new RAT 
warnings
{color:green}+1 JAVADOC{color}
.{color:green}+1{color} the patch does not seem to introduce new Javadoc 
warnings
{color:green}+1 COMPILE{color}
.{color:green}+1{color} HEAD compiles
.{color:green}+1{color} patch compiles
.{color:green}+1{color} the patch does not seem to introduce new javac 
warnings
{color:green}+1 BACKWARDS_COMPATIBILITY{color}
.{color:green}+1{color} the patch does not change any JPA 
Entity/Colum/Basic/Lob/Transient annotations
.{color:green}+1{color} the patch does not modify JPA files
{color:green}+1 TESTS{color}
.Tests run: 1815
{color:green}+1 DISTRO{color}
.{color:green}+1{color} distro tarball builds with the patch 


{color:red}*-1 Overall result, please check the reported -1(s)*{color}


The full output of the test-patch run is available at

.   https://builds.apache.org/job/oozie-trunk-precommit-build/3362/

> Add FindBugs report to pre-commit build
> ---
>
> Key: OOZIE-1986
> URL: https://issues.apache.org/jira/browse/OOZIE-1986
> Project: Oozie
>  Issue Type: Bug
>  Components: tests
>Reporter: Robert Kanter
>Assignee: Andras Piros
> Fix For: 5.0.0
>
> Attachments: OOZIE-1986.001.patch, OOZIE-1986.002.patch, 
> OOZIE-1986.003.patch, OOZIE-1986.004.patch
>
>
> Now that we have OOZIE-1793, it will be really useful to have the pre-commit 
> build generate a findbugs report.  It should probably before and after the 
> patch and only complain it there are more after.  And it should link to the 
> report.  Ideally, this would do some kind of diff (like Hadoop does) so it's 
> easy to identify the new findbugs items.
> You can generate the findbugs reports by running:
> {noformat}
> mvn verify -DskipTests
> {noformat}
> This also runs checkstyle



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (OOZIE-2393) Allow table drop in hcat prepare

2016-10-07 Thread Abhishek Bafna (JIRA) 

[ 
https://issues.apache.org/jira/browse/OOZIE-2393?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15554949#comment-15554949
 ] 

Abhishek Bafna commented on OOZIE-2393:
---

Thank you [~gezapeti] for the feedback and review.

> Allow table drop in hcat prepare
> 
>
> Key: OOZIE-2393
> URL: https://issues.apache.org/jira/browse/OOZIE-2393
> Project: Oozie
>  Issue Type: Improvement
>  Components: core
>Affects Versions: 4.1.0, 4.2.0
>Reporter: Muhammad Ehsan ul Haque
>Assignee: Abhishek Bafna
>Priority: Minor
>  Labels: hcatalog, prepare
> Fix For: 5.0.0
>
> Attachments: OOZIE-2393-00.patch, OOZIE-2393-01.patch
>
>
> The hcat prepare only allows to drop partitions. It would be nice to also 
> allow dropping of table, depending on the URL. The current format of the URL 
> is
> {code}hcat://[metastore server]:[port]/[database name]/[table 
> name]/[partkey1]=[value];[partkey2]=[value]{code}
> where at least one partition must be provided, otherwise the prepare step 
> fails with the fololwing exception.
> {code}
> Starting the execution of prepare actions
> Creating HCatClient for user=ehsan.haq (auth:SIMPLE) and 
> server=thrift://datavault-prod-app2.internal.machines:9083
> Prepare execution in the Launcher Mapper has failed
> Failing Oozie Launcher, Main class 
> [org.apache.oozie.action.hadoop.SqoopMain], exception invoking main(), Error 
> trying to drop 
> hcat://datavault-prod-app2.internal.machines:9083/test_rdbms_import_2015110600/test
> org.apache.oozie.action.hadoop.LauncherException: Error trying to drop 
> hcat://datavault-prod-app2.internal.machines:9083/test_rdbms_import_2015110600/test
> at org.apache.oozie.action.hadoop.LauncherMapper.map(LauncherMapper.java:178)
> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
> at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:450)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
> at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at 
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
> Caused by: org.apache.oozie.action.hadoop.LauncherException: Error trying to 
> drop 
> hcat://datavault-prod-app2.internal.machines:9083/test_rdbms_import_2015110600/test
> at 
> org.apache.oozie.action.hadoop.HCatLauncherURIHandler.delete(HCatLauncherURIHandler.java:64)
> at 
> org.apache.oozie.action.hadoop.PrepareActionsDriver.execute(PrepareActionsDriver.java:89)
> at 
> org.apache.oozie.action.hadoop.PrepareActionsDriver.doOperations(PrepareActionsDriver.java:67)
> at 
> org.apache.oozie.action.hadoop.LauncherMapper.executePrepare(LauncherMapper.java:446)
> at org.apache.oozie.action.hadoop.LauncherMapper.map(LauncherMapper.java:174)
> ... 8 more
> Caused by: java.net.URISyntaxException: URI path is not in expected format: 
> hcat://datavault-prod-app2.internal.machines:9083/test_rdbms_import_2015110600/test
> at org.apache.oozie.util.HCatURI.parse(HCatURI.java:66)
> at org.apache.oozie.util.HCatURI.(HCatURI.java:52)
> at org.apache.oozie.util.HCatURI.(HCatURI.java:48)
> at 
> org.apache.oozie.action.hadoop.HCatLauncherURIHandler.delete(HCatLauncherURIHandler.java:52)
> ... 12 more
> Oozie Launcher failed, finishing Hadoop job gracefully
> {code}
> h3. Suggestion
> In the URL if the partition is not provided then it should delete the entire 
> table.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: [SCOPE] Oozie 4.3.0 Release

2016-10-07 Thread Abhishek Bafna 
Hi All,

Today EOD (UTC), we will freeze the scope and further commits into branch-4.3 
expect any blocker issues.

By early next week, we will have our first release candidate for review.

Regards,
Abhishek
> On Sep 23, 2016, at 11:11 PM, Abhishek Bafna  wrote:
> 
> Hi All,
> 
> The new branch creation and update of the version in the current branch will 
> be done on Monday (IST).
> 
> Thanks,
> Abhishek
>> On Sep 21, 2016, at 10:56 PM, Robert Kanter  wrote:
>> 
>> --94eb2c04982ae57b7b053d07d72b
>> Content-Type: text/plain; charset=UTF-8
>> 
>> Oh, right.  I forgot that Shwetha was going to help you with that.  That's
>> fine with me.
>> 
>> On Wed, Sep 21, 2016 at 3:22 AM, Abhishek Bafna 
>> wrote:
>> 
>>> Hello Robert,
>>> 
>>> Yes. I do not have permissions to do it. I was planning to take Shwetha's
>>> help. I am fine with you doing it.
>>> Please let me know if there is any change in the plan.
>>> 
>>> Thanks,
>>> Abhishek
 On Sep 21, 2016, at 2:11 AM, Robert Kanter  wrote:
 
 Abhishek,
 
 You don't have git permissions to create the release branch.  I can take
 care of that on Thursday if you want.
 
 
 - Robert
 
 On Tue, Sep 20, 2016 at 1:30 PM, Abhishek Bafna 
 wrote:
 
> Hello All,
> 
> I think we can create the new branch by Thursday (IST) and update
>>> version
> for new branch and master. If we have any important pending Jira's by
>>> then
> we will commit them to both new branch and master.
> 
> I will start working on the next steps for release from release doc.
> https://cwiki.apache.org/confluence/display/OOZIE/How+To+Release <
> https://cwiki.apache.org/confluence/display/OOZIE/How+To+Release>
> 
> Please do provide if you have any feedback or suggestion.
> 
> Thanks,
> Abhishek
>> On Sep 20, 2016, at 12:52 AM, Rohini Palaniswamy <
> rohini.adi...@gmail.com> wrote:
>> 
>> Sorry Abhishek for the long delay. Was on vacation and then busy
>>> catching
>> up with stuff. I did review a bunch of patches today which were marked
>> 4.3.0 and Patch Available apart from those which Robert or others have
>> already not commented on. Skipping those as they have better context.
>>> Tag
>> my name if there is something else that needs to be looked at.
>> 
>> Regards,
>> Rohini
>> 
>> On Mon, Sep 19, 2016 at 6:33 AM, Abhishek Bafna 
>> wrote:
>> 
>>> Hello Robert, Rohini, Purshotam, Jaydeep
>>> 
>>> As we discussed earlier, will be bumping the priority of some Jira's
>>> to
>>> 'blocker' for to be included into Oozie 4.3.0 release. We will try to
>>> complete as much possible out of that. (It would to good complete
>>> all.)
>>> 
>>> @All, If I miss something, Please do priorities or send a message
>>> here.
>>> 
>>> Thanks a lot for all the efforts and time we have put in last few
>>> weeks.
>>> 
>>> Thanks,
>>> Abhishek
 On Sep 8, 2016, at 10:14 AM, Abhishek Bafna 
>>> wrote:
 
 Hi All,
 
 It has been a long time since we started the scope discussion for the
>>> Oozie-4.3.0 release.
 Since then a lot of patches got reviewed and committed and I would to
>>> thank each one you for contributing your efforts for that.
 
 I think we should move to the next stage and go into blocker only
>>> mode
>>> from next week (one week). After that 7-10 days for preparing and
>>> completing release condidate.
 
 We did plan to update some of the old tech and some of them are still
>>> pending for review. If can spend time into reviewing them, it would be
>>> good, else we might have to move without them.
 
 There are also some good to have patches available for review and
>>> commit, I will bump up the priority for them. If I miss something,
> please
>>> feel free to do the same.
 
 Regards,
 Abhishek
 
> On Aug 31, 2016, at 2:30 PM, Abhishek Bafna 
>>> wrote:
> 
> Hi All,
> 
> Can we please look into these upgrade related bugs?
> 
> ActiveMQ https://issues.apache.org/jira/browse/OOZIE-2552
> HttpClient   https://issues.apache.org/jira/browse/OOZIE-2538
> OpenJPA  https://issues.apache.org/
>>> jira/browse/OOZIE-2488
> Curator  https://issues.apache.org/
>>> jira/browse/OOZIE-2231
> Hive https://issues.apache.org/
>>> jira/browse/OOZIE-2613
> 
> Are the current patches good and feedback if there is any.
> 
> Also, If we can do some reviews collectively.
> 
> Thanks,
> Abhishek

[jira] [Assigned] (OOZIE-2703) OYA: Streaming action

2016-10-07 Thread Peter Bacsko (JIRA) 

 [ 
https://issues.apache.org/jira/browse/OOZIE-2703?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Peter Bacsko reassigned OOZIE-2703:
---

Assignee: Peter Bacsko

> OYA: Streaming action
> -
>
> Key: OOZIE-2703
> URL: https://issues.apache.org/jira/browse/OOZIE-2703
> Project: Oozie
>  Issue Type: Sub-task
>Reporter: Peter Bacsko
>Assignee: Peter Bacsko
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (OOZIE-2703) OYA: Streaming action

2016-10-07 Thread Peter Bacsko (JIRA) 
Peter Bacsko created OOZIE-2703:
---

 Summary: OYA: Streaming action
 Key: OOZIE-2703
 URL: https://issues.apache.org/jira/browse/OOZIE-2703
 Project: Oozie
  Issue Type: Sub-task
Reporter: Peter Bacsko






--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Review Request 52429: OOZIE-1986 Add FindBugs report to pre-commit build

2016-10-07 Thread András Piros 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52429/
---

(Updated Oct. 7, 2016, 10:39 a.m.)


Review request for oozie and Robert Kanter.


Changes
---

Some small refactor in order to have method names more bash-ish.


Repository: oozie-git


Description
---

We go for that solution:

1. create a new script as `bin/test-patch-11-findbugs-diff` so that it comes 
right after compile

2. `mvn clean verify -DskipTests` before and after applying the patch, and 
saving each and every module's `target/findbugs/findbugsXml.xml` to a secure 
folder

3. call the `findbugs-diff` app for all the before / after FindBugs xml pairs

4. analyze all the `findbugs-new.xml` entries for nodes with XPath 
`/BugCollection/BugInstance` entries. Stop on any BugInstance w/ `priority` and 
/ or `rank` lower than threshold

5. have the reports merged and be downloadable in any case in following 
scenarios:

* -1 for bugs that are too serious to apply the patch with them
* 0 for other new bugs
* +1 if there is no new bug


Diffs (updated)
-

  bin/test-patch-11-findbugs-diff PRE-CREATION 
  bin/test-patch-11-findbugs-diff-0.1.0-all.jar.md5 PRE-CREATION 
  pom.xml a33d2c5 

Diff: https://reviews.apache.org/r/52429/diff/


Testing
---


Thanks,

András Piros

[jira] [Comment Edited] (OOZIE-1986) Add FindBugs report to pre-commit build

2016-10-07 Thread Andras Piros (JIRA) 

[ 
https://issues.apache.org/jira/browse/OOZIE-1986?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15554744#comment-15554744
 ] 

Andras Piros edited comment on OOZIE-1986 at 10/7/16 10:24 AM:
---

Some small refactors to make function names more {{bash}}-like.


was (Author: andras.piros):
Some small refactors to make function names more {{bash]]-like.

> Add FindBugs report to pre-commit build
> ---
>
> Key: OOZIE-1986
> URL: https://issues.apache.org/jira/browse/OOZIE-1986
> Project: Oozie
>  Issue Type: Bug
>  Components: tests
>Reporter: Robert Kanter
>Assignee: Andras Piros
> Fix For: 5.0.0
>
> Attachments: OOZIE-1986.001.patch, OOZIE-1986.002.patch, 
> OOZIE-1986.003.patch, OOZIE-1986.004.patch
>
>
> Now that we have OOZIE-1793, it will be really useful to have the pre-commit 
> build generate a findbugs report.  It should probably before and after the 
> patch and only complain it there are more after.  And it should link to the 
> report.  Ideally, this would do some kind of diff (like Hadoop does) so it's 
> easy to identify the new findbugs items.
> You can generate the findbugs reports by running:
> {noformat}
> mvn verify -DskipTests
> {noformat}
> This also runs checkstyle



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (OOZIE-1986) Add FindBugs report to pre-commit build

2016-10-07 Thread Andras Piros (JIRA) 

 [ 
https://issues.apache.org/jira/browse/OOZIE-1986?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andras Piros updated OOZIE-1986:

Attachment: OOZIE-1986.004.patch

Some small refactors to make function names more {{bash]]-like.

> Add FindBugs report to pre-commit build
> ---
>
> Key: OOZIE-1986
> URL: https://issues.apache.org/jira/browse/OOZIE-1986
> Project: Oozie
>  Issue Type: Bug
>  Components: tests
>Reporter: Robert Kanter
>Assignee: Andras Piros
> Fix For: 5.0.0
>
> Attachments: OOZIE-1986.001.patch, OOZIE-1986.002.patch, 
> OOZIE-1986.003.patch, OOZIE-1986.004.patch
>
>
> Now that we have OOZIE-1793, it will be really useful to have the pre-commit 
> build generate a findbugs report.  It should probably before and after the 
> patch and only complain it there are more after.  And it should link to the 
> report.  Ideally, this would do some kind of diff (like Hadoop does) so it's 
> easy to identify the new findbugs items.
> You can generate the findbugs reports by running:
> {noformat}
> mvn verify -DskipTests
> {noformat}
> This also runs checkstyle



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (OOZIE-2666) Support embedding Jetty into Oozie

2016-10-07 Thread Hadoop QA (JIRA) 

[ 
https://issues.apache.org/jira/browse/OOZIE-2666?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15554615#comment-15554615
 ] 

Hadoop QA commented on OOZIE-2666:
--

Testing JIRA OOZIE-2666

Cleaning local git workspace



{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:red}-1 RAW_PATCH_ANALYSIS{color}
.{color:green}+1{color} the patch does not introduce any @author tags
.{color:red}-1{color} the patch contains 12 line(s) with tabs
.{color:green}+1{color} the patch does not introduce any trailing spaces
.{color:red}-1{color} the patch contains 9 line(s) longer than 132 
characters
.{color:green}+1{color} the patch does adds/modifies 2 testcase(s)
{color:red}-1 RAT{color}
.{color:red}-1{color} the patch seems to introduce 3 new RAT warning(s)
{color:green}+1 JAVADOC{color}
.{color:green}+1{color} the patch does not seem to introduce new Javadoc 
warnings
{color:red}-1 COMPILE{color}
.{color:green}+1{color} HEAD compiles
.{color:green}+1{color} patch compiles
.{color:red}-1{color} the patch seems to introduce 3 new javac warning(s)
{color:green}+1 BACKWARDS_COMPATIBILITY{color}
.{color:green}+1{color} the patch does not change any JPA 
Entity/Colum/Basic/Lob/Transient annotations
.{color:green}+1{color} the patch does not modify JPA files
{color:green}+1 TESTS{color}
.Tests run: 1821
{color:green}+1 DISTRO{color}
.{color:green}+1{color} distro tarball builds with the patch 


{color:red}*-1 Overall result, please check the reported -1(s)*{color}


The full output of the test-patch run is available at

.   https://builds.apache.org/job/oozie-trunk-precommit-build/3361/

> Support embedding Jetty into Oozie
> --
>
> Key: OOZIE-2666
> URL: https://issues.apache.org/jira/browse/OOZIE-2666
> Project: Oozie
>  Issue Type: Sub-task
>  Components: core
>Reporter: Attila Sasvari
>Assignee: Attila Sasvari
> Attachments: OOZIE-2666.PATCH-01.patch, OOZIE-2666.PATCH-02.patch, 
> OOZIE-2666.PATCH-03.patch, OOZIE-2666.PATCH-05.patch, 
> OOZIE-2666.PATCH-06.patch
>
>
> The purpose of this task is to allow Oozie to run as a standalone application 
> rather than being deployed to a container inside a WAR file.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Build failed in Jenkins: oozie-trunk-precommit-build #3361

2016-10-07 Thread Apache Jenkins Server 
See 

--
[...truncated 5815 lines...]
[INFO] Copying jdo-api-3.0.1.jar to 

[INFO] Copying hive-shims-0.20S-0.13.1.jar to 

[INFO] Copying oozie-hadoop-utils-hadoop-2-4.4.0-SNAPSHOT.jar to 

[INFO] Copying log4j-1.2.17.jar to 

[INFO] Copying tools.jar to 

[INFO] Copying netty-3.6.2.Final.jar to 

[INFO] Copying apache-log4j-extras-1.2.17.jar to 

[INFO] Copying commons-digester-1.8.jar to 

[INFO] Copying activation-1.1.jar to 

[INFO] Copying collections-generic-4.01.jar to 

[INFO] Copying hive-shims-0.23-0.13.1.jar to 

[INFO] Copying xmlenc-0.52.jar to 

[INFO] Copying httpcore-4.3.3.jar to 

[INFO] Copying openjpa-persistence-2.4.1.jar to 

[INFO] Copying jetty-io-9.2.19.v20160908.jar to 

[INFO] Copying hive-shims-0.20-0.13.1.jar to 

[INFO] Copying jetty-http-9.2.19.v20160908.jar to 

[INFO] Copying jetty-plus-9.2.19.v20160908.jar to 

[INFO] Copying hive-serde-0.13.1.jar to 

[INFO] Copying antlr-2.7.7.jar to 

[INFO] Copying jackson-databind-2.2.2.jar to 

[INFO] Copying oozie-sharelib-oozie-4.4.0-SNAPSHOT.jar to 

[INFO] Copying commons-beanutils-core-1.8.0.jar to 

[INFO] Copying hadoop-yarn-server-common-2.4.0.jar to 

[INFO] Copying geronimo-j2ee-management_1.1_spec-1.0.1.jar to 

[INFO] Copying datanucleus-core-3.2.10.jar to 

[INFO] Copying slf4j-log4j12-1.6.6.jar to 

[INFO] Copying aopalliance-1.0.jar to 

[INFO] Copying hadoop-mapreduce-client-core-2.4.0.jar to 

[INFO] Copying datanucleus-api-jdo-3.2.6.jar to

[jira] [Updated] (OOZIE-2666) Support embedding Jetty into Oozie

2016-10-07 Thread Attila Sasvari (JIRA) 

 [ 
https://issues.apache.org/jira/browse/OOZIE-2666?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Attila Sasvari updated OOZIE-2666:
--
Attachment: OOZIE-2666.PATCH-06.patch

> Support embedding Jetty into Oozie
> --
>
> Key: OOZIE-2666
> URL: https://issues.apache.org/jira/browse/OOZIE-2666
> Project: Oozie
>  Issue Type: Sub-task
>  Components: core
>Reporter: Attila Sasvari
>Assignee: Attila Sasvari
> Attachments: OOZIE-2666.PATCH-01.patch, OOZIE-2666.PATCH-02.patch, 
> OOZIE-2666.PATCH-03.patch, OOZIE-2666.PATCH-05.patch, 
> OOZIE-2666.PATCH-06.patch
>
>
> The purpose of this task is to allow Oozie to run as a standalone application 
> rather than being deployed to a container inside a WAR file.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Review Request 52399: OOZIE-2666 Support embedding Jetty into Oozie

2016-10-07 Thread Attila Sasvari 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52399/
---

(Updated Oct. 7, 2016, 7:18 a.m.)


Review request for oozie, András Piros, Peter Cseh, Peter Bacsko, and Robert 
Kanter.


Changes
---

Fixes according to review comments:
- rename oozie.ssl to oozie.https, 
- added ShutdownHook to stop oozie services.


Repository: oozie-git


Description
---

Embedding jetty into Oozie so that it can run as a standalone application. The 
changes also try to address OOZIE-2317 (i.e. Tomcat 6 is EOL).

New functionality
- New module (server) is added that sets up an embedded Jetty server and start 
Oozie services. Servlet mapping is done by reading web.xml of webapp at 
runtime. JSP is handled with custom code. Server version is not revealed in 
server repsonses.
- SSL protocols and ciphers can be configured via system properties and 
environment variables. Precedence: system properties, environment variables, 
default values
   
Changes in existing code
- Excluded jetty 6 dependencies from core and updated tests accordingly  

Packaging
- oozie.sh is modified so that it starts Oozie with embedded jetty by default. 
If someone would like to use tomcat for any reason, they can set an environment 
variable (e.g. OOZIE_USE_TOMCAT=1).

TODO:
- Add more tests
- Add more documentation
- Code cleanup + refactoring in packaging and core parts
- Maven clean up
- Allow to tune more Jetty settings (for example threadpool)
- More security measures (e.g. protect against clickjacking, CSRF, etc.)
- Update Oozie Documentation


Diffs (updated)
-

  core/src/main/java/org/apache/oozie/util/Instrumentation.java 
fa1e92a0f1fadfb66c5e66fea5f26f57579080e7 
  core/src/main/resources/oozie-default.xml 
e71ebe3b7a85e6b23176ef30713af63847144498 
  distro/pom.xml c50572c57a376b28963d4e7da8ac7df777fe0480 
  distro/src/main/bin/oozie-jetty-server.sh PRE-CREATION 
  distro/src/main/bin/oozie-setup.sh 79b049bccceb2690f8a673a885a615c8d4d9578c 
  distro/src/main/bin/oozie-sys.sh 97d55a2b69c34ede007d4f65cdfc66f1ac2cfd13 
  distro/src/main/bin/oozied.sh a869c3da177c863a068f2af45c7bca9d5cb771ac 
  pom.xml 704a29f4e4805e3e08c2a547b2a375f6b1b2 
  server/pom.xml PRE-CREATION 
  server/src/main/assemblies/empty.xml PRE-CREATION 
  server/src/main/java/org/apache/oozie/server/EmbeddedOozieServer.java 
PRE-CREATION 
  server/src/main/java/org/apache/oozie/server/JspHandler.java PRE-CREATION 
  server/src/main/java/org/apache/oozie/server/SSLServerConnectorFactory.java 
PRE-CREATION 
  server/src/main/resources/checkstyle-header.txt PRE-CREATION 
  server/src/main/resources/checkstyle.xml PRE-CREATION 
  server/src/test/java/org/apache/oozie/server/TestEmbeddedOozieServer.java 
PRE-CREATION 
  
server/src/test/java/org/apache/oozie/server/TestSSLServerConnectorFactory.java 
PRE-CREATION 
  src/main/assemblies/distro.xml 1ffbfd6d2ba33b390999e9094cbb336fbce45c21 

Diff: https://reviews.apache.org/r/52399/diff/


Testing
---

- Tested basic functionality by executing a workflow that uses the sample 
JavaAction
- without SSL - on a 2.4.0 pseudo Hadoop cluster
- SSL with Kerberos is using a test CDH cluster 
- Added new unit tests that check
- If oozie.ssl.enabled is not specified, server starts without SSL settings 
- If oozie.ssl.enabled is specified, server starts with SSL settings
- SSL protocols and ciphers can be configured via system properties and 
environment variables 
- Ran subset of tests using Hadoop-2 profile
- mvn clean package assembly:single   -DjavaVersion=1.8 -DtargetVersion=1.7 
 -Dtest=TestJavaActionExecutor  -Phadoop-2 -Dhadoop.version=2.4.0


Thanks,

Attila Sasvari

Jenkins build is back to normal : oozie-trunk-find-patches-available #135523

2016-10-07 Thread Apache Jenkins Server 
See

Build failed in Jenkins: oozie-trunk-find-patches-available #135522

2016-10-07 Thread Apache Jenkins Server 
See 

--
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on H1 (yahoo-not-h2 Hadoop Mapreduce Hdfs Pig) in workspace 

 > git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
 > git config remote.origin.url 
 > https://git-wip-us.apache.org/repos/asf/oozie.git # timeout=10
Fetching upstream changes from https://git-wip-us.apache.org/repos/asf/oozie.git
 > git --version # timeout=10
 > git -c core.askpass=true fetch --tags --progress 
 > https://git-wip-us.apache.org/repos/asf/oozie.git 
 > +refs/heads/*:refs/remotes/origin/*
 > git rev-parse refs/remotes/origin/master^{commit} # timeout=10
 > git rev-parse refs/remotes/origin/origin/master^{commit} # timeout=10
Checking out Revision 962d650bdc5522d39d49ba8ee2b3fc1634c625ec 
(refs/remotes/origin/master)
 > git config core.sparsecheckout # timeout=10
 > git checkout -f 962d650bdc5522d39d49ba8ee2b3fc1634c625ec
 > git rev-list 962d650bdc5522d39d49ba8ee2b3fc1634c625ec # timeout=10
[oozie-trunk-find-patches-available] $ /bin/bash 
/tmp/hudson7732711959075163322.sh
mkdir: cannot create directory 
':
 File exists
  % Total% Received % Xferd  Average Speed   TimeTime Time  Current
 Dload  Upload   Total   SpentLeft  Speed

  0 00 00 0  0  0 --:--:-- --:--:-- --:--:-- 0
100 533690 533690 0  74106  0 --:--:-- --:--:-- --:--:-- 74020
100  192k0  192k0 0   168k  0 --:--:--  0:00:01 --:--:--  168k
curl: (18) transfer closed with outstanding read data remaining
Could not retrieve available patches from JIRA
Build step 'Execute shell' marked build as failure

[jira] Subscription: Oozie Patch Available

2016-10-07 Thread jira 
Issue Subscription
Filter: Oozie Patch Available (86 issues)

Subscriber: ooziedaily

Key Summary
OOZIE-2619  Make  Hive action defaults to match hive defaults when running from 
command line
https://issues-test.apache.org/jira/browse/OOZIE-2619
OOZIE-2618  Include hive-common in hive-beeline as compile component for 
hive-server2 (hive2 action)
https://issues-test.apache.org/jira/browse/OOZIE-2618
OOZIE-2617  Read properties files in action configurations
https://issues-test.apache.org/jira/browse/OOZIE-2617
OOZIE-2616  Add Tez profile for Hive and Pig sharelibs
https://issues-test.apache.org/jira/browse/OOZIE-2616
OOZIE-2606  Set spark.yarn.jars to fix Spark 2.0 with Oozie
https://issues-test.apache.org/jira/browse/OOZIE-2606
OOZIE-2589  CompletedActionXCommand priority param is of no effect
https://issues-test.apache.org/jira/browse/OOZIE-2589
OOZIE-2588  Support getting credentials for same cluster hcat when credentials 
config is empty
https://issues-test.apache.org/jira/browse/OOZIE-2588
OOZIE-2585  Disable or remove test case 
TestPartitionDependencyManagerEhcache.testMemoryUsageAndSpeedOverflowToDisk and 
testMemoryUsageAndSpeed
https://issues-test.apache.org/jira/browse/OOZIE-2585
OOZIE-2584  Eliminate Thread.sleep() calls in TestMemoryLocks
https://issues-test.apache.org/jira/browse/OOZIE-2584
OOZIE-2583  oozie throws EL Exception when reference variable name containing 
dot
https://issues-test.apache.org/jira/browse/OOZIE-2583
OOZIE-2582  Populating external child Ids for action failures
https://issues-test.apache.org/jira/browse/OOZIE-2582
OOZIE-2573  dataset url  which contains spaces can not be handled rightly
https://issues-test.apache.org/jira/browse/OOZIE-2573
OOZIE-2572  SLA DURATION miss not shown when job is running for longer than 
expected time
https://issues-test.apache.org/jira/browse/OOZIE-2572
OOZIE-2571  Add scala.binary.version Maven property so that Scala 2.11 can be 
used
https://issues-test.apache.org/jira/browse/OOZIE-2571
OOZIE-2569  Adding yarn-site, core-site, hdfs-site and mapred-site into spark 
launcher 
https://issues-test.apache.org/jira/browse/OOZIE-2569
OOZIE-2568  SSH  action pretends to retry automaticly when it failed
https://issues-test.apache.org/jira/browse/OOZIE-2568
OOZIE-2564  Create new log4j config for unit tests so that logged messages 
contain threads
https://issues-test.apache.org/jira/browse/OOZIE-2564
OOZIE-2552  Update ActiveMQ version for security and other fixes
https://issues-test.apache.org/jira/browse/OOZIE-2552
OOZIE-2538  Update HttpClient versions to close security vulnerabilities
https://issues-test.apache.org/jira/browse/OOZIE-2538
OOZIE-2534  Versioned action libs (similar to how SharedLibs works)
https://issues-test.apache.org/jira/browse/OOZIE-2534
OOZIE-2530  Remove jline from parent and sharelib pig pom
https://issues-test.apache.org/jira/browse/OOZIE-2530
OOZIE-2528  Print out environment variables in LauncherMapper
https://issues-test.apache.org/jira/browse/OOZIE-2528
OOZIE-2521  Filter options are case sensitive for jobtye=bundle
https://issues-test.apache.org/jira/browse/OOZIE-2521
OOZIE-2518  Oozie is creating many files and directories in /tmp per day
https://issues-test.apache.org/jira/browse/OOZIE-2518
OOZIE-2517  Add support for startCreatedTime and endCreatedTime filters for 
coord and bundles
https://issues-test.apache.org/jira/browse/OOZIE-2517
OOZIE-2513  log.scan.duration should not be used for error and audit logs
https://issues-test.apache.org/jira/browse/OOZIE-2513
OOZIE-2512  ShareLibservice returns incorrect path for jar
https://issues-test.apache.org/jira/browse/OOZIE-2512
OOZIE-2508  Documentation change for Coord action rerun [OOZIE-1735]
https://issues-test.apache.org/jira/browse/OOZIE-2508
OOZIE-2501  ZK reentrant lock doesn't work for few cases
https://issues-test.apache.org/jira/browse/OOZIE-2501
OOZIE-2500  -DtestJarSimple option mentioned in minioozie doc does not work
https://issues-test.apache.org/jira/browse/OOZIE-2500
OOZIE-2499  map-reduce launcher does not need distributed files, archives 
except  jar of input/outputformat class
https://issues-test.apache.org/jira/browse/OOZIE-2499
OOZIE-2498  Oozie CallerId configuration for downstream components 
https://issues-test.apache.org/jira/browse/OOZIE-2498
OOZIE-2495  change action status from  ErrorType.NON_TRANSIENT to TRANSIENT 
when SSH action occurs AUTH_FAILED occasionally
https://issues-test.apache.org/jira/browse/OOZIE-2495
OOZIE-2491  oozie acl cannot specify group,it does`t work
https://issues-test.apache.org/jira/browse/OOZIE-2491

[jira] Subscription: Oozie Patch Available

2016-10-07 Thread jira 
Issue Subscription
Filter: Oozie Patch Available (78 issues)

Subscriber: ooziedaily

Key Summary
OOZIE-2694  Add logging for FsActionExecutor 
https://issues.apache.org/jira/browse/OOZIE-2694
OOZIE-2693  SimpleHCatDependencyCache.removeMissingDependency can throw NPE
https://issues.apache.org/jira/browse/OOZIE-2693
OOZIE-2692  Oozie job submit doesn't report error message to user if there is 
any issue with job conf
https://issues.apache.org/jira/browse/OOZIE-2692
OOZIE-2691  Show workflow action retry count to UI
https://issues.apache.org/jira/browse/OOZIE-2691
OOZIE-2690  OOZIE NPE while executing kill()
https://issues.apache.org/jira/browse/OOZIE-2690
OOZIE-2684  Bad database schema error for WF_ACTIONS table
https://issues.apache.org/jira/browse/OOZIE-2684
OOZIE-2674  Improve oozie commads documentation
https://issues.apache.org/jira/browse/OOZIE-2674
OOZIE-2670  Upgrade Hbase to 1.2
https://issues.apache.org/jira/browse/OOZIE-2670
OOZIE-2668  Status update and recovery problems when coord action and its 
children not in sync
https://issues.apache.org/jira/browse/OOZIE-2668
OOZIE-2666  Support embedding Jetty into Oozie
https://issues.apache.org/jira/browse/OOZIE-2666
OOZIE-2662  DB migration fails if DB is too big
https://issues.apache.org/jira/browse/OOZIE-2662
OOZIE-2658  --driver-class-path can overwrite the classpath in SparkMain
https://issues.apache.org/jira/browse/OOZIE-2658
OOZIE-2654  Zookeeper dependent services should not depend on Connectionstate 
to be valid before cleaning up
https://issues.apache.org/jira/browse/OOZIE-2654
OOZIE-2652  Skip trash while deleting hive table partition
https://issues.apache.org/jira/browse/OOZIE-2652
OOZIE-2650  Retry coord start on database exceptions
https://issues.apache.org/jira/browse/OOZIE-2650
OOZIE-2641  rerunning an oozie coordinator job stucks on waiting
https://issues.apache.org/jira/browse/OOZIE-2641
OOZIE-2630  Oozie Coordinator EL Functions to get first day of the week/month
https://issues.apache.org/jira/browse/OOZIE-2630
OOZIE-2619  Make  Hive action defaults to match hive defaults when running from 
command line
https://issues.apache.org/jira/browse/OOZIE-2619
OOZIE-2618  Include hive-common in hive-beeline as compile component for 
hive-server2 (hive2 action)
https://issues.apache.org/jira/browse/OOZIE-2618
OOZIE-2616  Add Tez profile for Hive and Pig sharelibs
https://issues.apache.org/jira/browse/OOZIE-2616
OOZIE-2613  Upgrade hive version from 0.13.1 to 1.2.0
https://issues.apache.org/jira/browse/OOZIE-2613
OOZIE-2585  Disable or remove test case 
TestPartitionDependencyManagerEhcache.testMemoryUsageAndSpeedOverflowToDisk and 
testMemoryUsageAndSpeed
https://issues.apache.org/jira/browse/OOZIE-2585
OOZIE-2573  dataset url  which contains spaces can not be handled rightly
https://issues.apache.org/jira/browse/OOZIE-2573
OOZIE-2568  SSH  action pretends to retry automaticly when it failed
https://issues.apache.org/jira/browse/OOZIE-2568
OOZIE-2564  Create new log4j config for unit tests so that logged messages 
contain threads
https://issues.apache.org/jira/browse/OOZIE-2564
OOZIE-2534  Versioned action libs (similar to how SharedLibs works)
https://issues.apache.org/jira/browse/OOZIE-2534
OOZIE-2528  Print out environment variables in LauncherMapper
https://issues.apache.org/jira/browse/OOZIE-2528
OOZIE-2518  Oozie is creating many files and directories in /tmp per day
https://issues.apache.org/jira/browse/OOZIE-2518
OOZIE-2513  log.scan.duration should not be used for error and audit logs
https://issues.apache.org/jira/browse/OOZIE-2513
OOZIE-2499  map-reduce launcher does not need distributed files, archives 
except  jar of input/outputformat class
https://issues.apache.org/jira/browse/OOZIE-2499
OOZIE-2495  change action status from  ErrorType.NON_TRANSIENT to TRANSIENT 
when SSH action occurs AUTH_FAILED occasionally
https://issues.apache.org/jira/browse/OOZIE-2495
OOZIE-2473  Connection pool for SMTP connection
https://issues.apache.org/jira/browse/OOZIE-2473
OOZIE-2461  Workflow, Coordinator and Bundle job querying should have last 
modified filter
https://issues.apache.org/jira/browse/OOZIE-2461
OOZIE-2458  'oozie-setup.sh sharelib create' should ensure uploaded jars are 
world readable
https://issues.apache.org/jira/browse/OOZIE-2458
OOZIE-2457  Oozie log parsing regex consume more than 90% cpu
https://issues.apache.org/jira/browse/OOZIE-2457
OOZIE-2427  With Kerberos enabled, the authentication failure for the 
javascript resources under /ext-2.2 
https://issues.apache.org/jira/browse/OOZIE-2427