date:20181122

[jira] Subscription: Oozie Patch Available

2018-11-22 Thread jira

Issue Subscription
Filter: Oozie Patch Available (96 issues)

Subscriber: ooziedaily

Key Summary
OOZIE-3382  Optimize SshActionExecutor's drainBuffers method
https://issues.apache.org/jira/browse/OOZIE-3382
OOZIE-3379  Auth token cache file name should include Oozie URL
https://issues.apache.org/jira/browse/OOZIE-3379
OOZIE-3375  Can't use empty  in coordinator
https://issues.apache.org/jira/browse/OOZIE-3375
OOZIE-3367  Using && in EL expressions in oozie bundle.xml files generates 
parse errors
https://issues.apache.org/jira/browse/OOZIE-3367
OOZIE-3366  Update workflow status and subworkflow status on suspend command
https://issues.apache.org/jira/browse/OOZIE-3366
OOZIE-3364  Rerunning Oozie bundle jobs starts the coordinators in 
indeterminate order
https://issues.apache.org/jira/browse/OOZIE-3364
OOZIE-3362  When killed, SSH action should kill the spawned processes on target 
host
https://issues.apache.org/jira/browse/OOZIE-3362
OOZIE-3326  Sqoop Action should support tez delegation tokens for hive-import
https://issues.apache.org/jira/browse/OOZIE-3326
OOZIE-3320  Oozie ShellAction should support absolute bash file path
https://issues.apache.org/jira/browse/OOZIE-3320
OOZIE-3319  Log SSH action callback error output
https://issues.apache.org/jira/browse/OOZIE-3319
OOZIE-3301  Update NOTICE file
https://issues.apache.org/jira/browse/OOZIE-3301
OOZIE-3274  Remove slf4j
https://issues.apache.org/jira/browse/OOZIE-3274
OOZIE-3266  Coord action rerun support RERUN_SKIP_NODES option
https://issues.apache.org/jira/browse/OOZIE-3266
OOZIE-3265  properties RERUN_FAIL_NODES and RERUN_SKIP_NODES should be able to 
appear together
https://issues.apache.org/jira/browse/OOZIE-3265
OOZIE-3256  refactor OozieCLI class
https://issues.apache.org/jira/browse/OOZIE-3256
OOZIE-3249  [tools] Instrumentation log parser
https://issues.apache.org/jira/browse/OOZIE-3249
OOZIE-3218  Oozie Sqoop action with command splits the select clause into 
multiple parts due to delimiter being space
https://issues.apache.org/jira/browse/OOZIE-3218
OOZIE-3199  Let system property restriction configurable
https://issues.apache.org/jira/browse/OOZIE-3199
OOZIE-3196  Authorization: restrict world readability by user
https://issues.apache.org/jira/browse/OOZIE-3196
OOZIE-3194  Oozie should set proper permissions to sharelib after upload
https://issues.apache.org/jira/browse/OOZIE-3194
OOZIE-3186  Oozie is unable to use configuration linked using jceks://file/...
https://issues.apache.org/jira/browse/OOZIE-3186
OOZIE-3179  Adding a configurable config-default.xml location to a workflow
https://issues.apache.org/jira/browse/OOZIE-3179
OOZIE-3170  Oozie Diagnostic Bundle tool fails with NPE due to missing service 
class
https://issues.apache.org/jira/browse/OOZIE-3170
OOZIE-3137  Add support for log4j2 in HiveMain
https://issues.apache.org/jira/browse/OOZIE-3137
OOZIE-3135  Configure log4j2 in SqoopMain
https://issues.apache.org/jira/browse/OOZIE-3135
OOZIE-3120  maven-assembly-plugin fails when bumped from 2.2.1
https://issues.apache.org/jira/browse/OOZIE-3120
OOZIE-3091  Oozie Sqoop Avro Import fails with "java.lang.NoClassDefFoundError: 
org/apache/avro/mapred/AvroWrapper"
https://issues.apache.org/jira/browse/OOZIE-3091
OOZIE-3071  Oozie 4.3 Spark sharelib ueses a different version of commons-lang3 
than Spark 2.2.0
https://issues.apache.org/jira/browse/OOZIE-3071
OOZIE-3063  Sanitizing variables that are part of openjpa.ConnectionProperties
https://issues.apache.org/jira/browse/OOZIE-3063
OOZIE-3062  Set HADOOP_CONF_DIR for spark action
https://issues.apache.org/jira/browse/OOZIE-3062
OOZIE-2952  Fix Findbugs warnings in oozie-sharelib-oozie
https://issues.apache.org/jira/browse/OOZIE-2952
OOZIE-2949  Escape quotes whitespaces in Sqoop  field
https://issues.apache.org/jira/browse/OOZIE-2949
OOZIE-2927  Append new line character for Hive2 query using query tag
https://issues.apache.org/jira/browse/OOZIE-2927
OOZIE-2834  ParameterVerifier logging non-useful warning for workflow definition
https://issues.apache.org/jira/browse/OOZIE-2834
OOZIE-2833  when using uber mode the regex pattern used in the 
extractHeapSizeMB method does not allow heap sizes specified in bytes.
https://issues.apache.org/jira/browse/OOZIE-2833
OOZIE-2812  SparkConfigurationService should support loading configurations 
from multiple Spark versions
https://issues.apache.org/jira/browse/OOZIE-2812
OOZIE-2795  Create lib directory or symlink for Oozie CLI during packaging
https://issues.apache.org/jira/browse/OOZIE-2795

Re: [VOTE] Release Oozie 5.1.0 (candidate 1)

2018-11-22 Thread Andras Piros

Thanks Attila,

fixing the JIRAs you've revealed, and soon creating RC2 on top of them.
Thanks for taking the time and the effort to thoroughly test the RC1.

Regards,

Andras

On Thu, Nov 22, 2018 at 4:24 PM Attila Sasvari
 wrote:

>  Hi,
>
> Thanks for driving the release Andras.
>
> -1
>
> Here is what I did:
> - Tested with Hadoop 2.7.5, Java 8
> - signature, checksum is fine
> - rat report is ok
> - distro builds
> - I submitted example workflows and coordinators and examined jobs via the
> Oozie Web UI, I have found and filed the following issues:
> https://issues.apache.org/jira/browse/OOZIE-3386
> https://issues.apache.org/jira/browse/OOZIE-3389
> https://issues.apache.org/jira/browse/OOZIE-3390
>
> - I ran mkdistro.sh, then executed "mvn site" that failed with the
> following
> [ERROR] Failed to execute goal on project oozie-client: Could not resolve
> dependencies for project org.apache.oozie:oozie-client:jar:5.1.0: Could not
> find artifact org.apache.oozie:oozie-fluent-job-api:jar:5.1.0 in central (
> http://repo1.maven.org/maven2) -> [Help 1]
> ( Installing dependencies locally solves this problem )
>
> Please fix OOZIE-3386, OOZIE-3389, OOZIE-3390 and create a new RC.
>
> Best regards,
> - Attila
>
> On Wed, Nov 14, 2018 at 2:29 PM Andras Piros
>  wrote:
>
> > Thanks Gp,
> >
> > extending the vote by two weeks (Fri Nov 23, 12pm PDT) to see what others
> > find.
> >
> > Andras
> >
> > On Tue, Nov 13, 2018 at 2:01 PM Peter Cseh  >
> > wrote:
> >
> > > Hi!
> > >
> > > Tested with Hadoop2.7.2
> > > - signature, checksum is fine
> > > - rat report is ok
> > > - distro builds
> > > - tried some of the examples (MR, Hive, Spark), they are running fine.
> > >
> > > +1
> > >
> > >
> > > On Tue, Nov 13, 2018 at 12:59 PM Andras Piros
> > >  wrote:
> > >
> > > > Thanks Kinga for checking out RC1, and good to hear it passed from
> your
> > > > perspective.
> > > >
> > > > Regards,
> > > >
> > > > Andras
> > > >
> > > > On Tue, Nov 13, 2018 at 12:36 PM Julia Kinga Marton
> > > >  wrote:
> > > >
> > > > > Hi,
> > > > >
> > > > > I made some testing on the RC, and it seems OK.
> > > > >
> > > > > Here is what I checked:
> > > > > - sha512
> > > > > - gpg signature
> > > > > - rat report
> > > > > - built Oozie against Hadoop-2.6.0
> > > > > - ran a few examples (wf and coordinator), clicked the Web UI, ran
> > some
> > > > CLI
> > > > > commands
> > > > > - checked Fluent Job API by running Java and Shell examples
> > > > > - looked briefly at the documentation
> > > > >
> > > > > +1 from my side (non-binding)
> > > > >
> > > > > Best regards,
> > > > > Kinga
> > > > >
> > > > > On Mon, Nov 5, 2018 at 8:44 PM Andras Piros 
> > wrote:
> > > > >
> > > > > > Hi there,
> > > > > >
> > > > > > I have created a build for Oozie 5.1.0, release candidate 1.
> > > > > >
> > > > > > This release introduces following major changes and fixes:
> > > > > > - Fluent Job API
> > > > > > - Git action
> > > > > > - reduced heap waste
> > > > > > - switched documentation from TWiki to Markdown
> > > > > > - callable queue improvements
> > > > > > - Spark action fixes
> > > > > > - SLA improvements
> > > > > > - third party dependency revamp
> > > > > > - fixed CVE-2018-11799
> > > > > >
> > > > > > Keys to verify the signature of the release artifact are
> available
> > at
> > > > > >
> > > > > >  http://www.apache.org/dist/oozie/KEYS
> > > > > >
> > > > > > Please download, test, and try it out:
> > > > > >
> > > > > > http://people.apache.org/~andras/oozie-5.1.0-rc1/
> > > > > >
> > > > > > The release candidate, sha512 signature, gpg signature, and rat
> > > report
> > > > > can
> > > > > > all be found
> > > > > > at the above address.
> > > > > >
> > > > > > The vote is open until Friday, November 9, 2018 at 12:00am PDT.
> > > > > >
> > > > > > Thanks,
> > > > > >
> > > > > > Andras Piros
> > > > > >
> > > > >
> > > >
> > >
> > >
> > > --
> > > *Peter Cseh *| Software Engineer
> > > cloudera.com 
> > >
> > > [image: Cloudera] 
> > >
> > > [image: Cloudera on Twitter]  [image:
> > > Cloudera on Facebook]  [image:
> > Cloudera
> > > on LinkedIn] 
> > > --
> > >
> >
>
>
> --
> --
> Attila Sasvari
> Software Engineer
> 
>

[jira] [Assigned] (OOZIE-3390) shell action's stderr contains a bogus error message (since 5.0.0)

2018-11-22 Thread Andras Piros (JIRA)



 [ 
https://issues.apache.org/jira/browse/OOZIE-3390?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andras Piros reassigned OOZIE-3390:
---

Assignee: Andras Piros

>  shell action's stderr contains a bogus error message (since 5.0.0)
> ---
>
> Key: OOZIE-3390
> URL: https://issues.apache.org/jira/browse/OOZIE-3390
> Project: Oozie
>  Issue Type: Bug
>Affects Versions: 5.1.0
>Reporter: Attila Sasvari
>Assignee: Andras Piros
>Priority: Blocker
>
> Shell action's stderr contains an error message about path.
> I executed shell action example to test Oozie 5.1 rc1, the job finished with 
> success, but stderr of the job contained "Path echo doesn't appear to exist".
> https://github.com/apache/oozie/blob/27e4bf1688a6a7750b9c8454de5021337696fd61/sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ShellContentWriter.java#L91
> {code}
> writeLine(errorStream, "Path " + filename + " doesn't appear 
> to exist");
> {code}
> The problem is that we ignore the PATH of the process.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Assigned] (OOZIE-3389) NPE after OOZIE-3370

2018-11-22 Thread Andras Piros (JIRA)



 [ 
https://issues.apache.org/jira/browse/OOZIE-3389?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andras Piros reassigned OOZIE-3389:
---

Assignee: Andras Piros

> NPE after OOZIE-3370  
> --
>
> Key: OOZIE-3389
> URL: https://issues.apache.org/jira/browse/OOZIE-3389
> Project: Oozie
>  Issue Type: Bug
>Affects Versions: 5.1.0
>Reporter: Attila Sasvari
>Assignee: Andras Piros
>Priority: Blocker
>
> I was testing 5.1 rc1. I submitted the cron-schedule example coordinator, 
> clicked on one of the coordinator jobs on the Oozie Web UI, but the details 
> of the job were not displayed. I noticed the following NPE in the server log:
> {code}
> 2018-11-21 15:45:00,019 ERROR CoordActionMissingDependenciesXCommand:517 - 
> SERVER[Budapests-MacBook-Pro-10.local] USER[-] GROUP[-] TOKEN[-] APP[-] 
> JOB[-] ACTION[-] XException, 
> org.apache.oozie.command.CommandException: E1028: Coord input logic error. 
> null
> at 
> org.apache.oozie.command.coord.CoordActionMissingDependenciesXCommand.execute(CoordActionMissingDependenciesXCommand.java:126)
> at 
> org.apache.oozie.command.coord.CoordActionMissingDependenciesXCommand.execute(CoordActionMissingDependenciesXCommand.java:38)
> at org.apache.oozie.command.XCommand.call(XCommand.java:291)
> at 
> org.apache.oozie.CoordinatorEngine.getCoordActionMissingDependencies(CoordinatorEngine.java:910)
> at 
> org.apache.oozie.servlet.V2JobServlet.getCoordActionMissingDependencies(V2JobServlet.java:336)
> at org.apache.oozie.servlet.BaseJobServlet.doGet(BaseJobServlet.java:370)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:687)
> at 
> org.apache.oozie.servlet.JsonRestServlet.service(JsonRestServlet.java:305)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
> at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:848)
> at 
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1772)
> at org.apache.oozie.servlet.AuthFilter$2.doFilter(AuthFilter.java:171)
> at 
> org.apache.hadoop.security.authentication.server.AuthenticationFilter.doFilter(AuthenticationFilter.java:594)
> at 
> org.apache.hadoop.security.authentication.server.AuthenticationFilter.doFilter(AuthenticationFilter.java:553)
> at org.apache.oozie.servlet.AuthFilter.doFilter(AuthFilter.java:176)
> at 
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
> at 
> org.apache.oozie.servlet.HostnameFilter.doFilter(HostnameFilter.java:86)
> at 
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
> at 
> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582)
> at 
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
> at 
> org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)
> at 
> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:226)
> at 
> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)
> at 
> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512)
> at 
> org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
> at 
> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112)
> at 
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
> at 
> org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:119)
> at 
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
> at org.eclipse.jetty.server.Server.handle(Server.java:534)
> at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:320)
> at 
> org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251)
> at 
> org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:283)
> at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:108)
> at 
> org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
> at 
> org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303)
> at 
> org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
> at 
> org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: [VOTE] Release Oozie 5.1.0 (candidate 1)

2018-11-22 Thread Attila Sasvari

 Hi,

Thanks for driving the release Andras.

-1

Here is what I did:
- Tested with Hadoop 2.7.5, Java 8
- signature, checksum is fine
- rat report is ok
- distro builds
- I submitted example workflows and coordinators and examined jobs via the
Oozie Web UI, I have found and filed the following issues:
https://issues.apache.org/jira/browse/OOZIE-3386
https://issues.apache.org/jira/browse/OOZIE-3389
https://issues.apache.org/jira/browse/OOZIE-3390

- I ran mkdistro.sh, then executed "mvn site" that failed with the following
[ERROR] Failed to execute goal on project oozie-client: Could not resolve
dependencies for project org.apache.oozie:oozie-client:jar:5.1.0: Could not
find artifact org.apache.oozie:oozie-fluent-job-api:jar:5.1.0 in central (
http://repo1.maven.org/maven2) -> [Help 1]
( Installing dependencies locally solves this problem )

Please fix OOZIE-3386, OOZIE-3389, OOZIE-3390 and create a new RC.

Best regards,
- Attila

On Wed, Nov 14, 2018 at 2:29 PM Andras Piros
 wrote:

> Thanks Gp,
>
> extending the vote by two weeks (Fri Nov 23, 12pm PDT) to see what others
> find.
>
> Andras
>
> On Tue, Nov 13, 2018 at 2:01 PM Peter Cseh 
> wrote:
>
> > Hi!
> >
> > Tested with Hadoop2.7.2
> > - signature, checksum is fine
> > - rat report is ok
> > - distro builds
> > - tried some of the examples (MR, Hive, Spark), they are running fine.
> >
> > +1
> >
> >
> > On Tue, Nov 13, 2018 at 12:59 PM Andras Piros
> >  wrote:
> >
> > > Thanks Kinga for checking out RC1, and good to hear it passed from your
> > > perspective.
> > >
> > > Regards,
> > >
> > > Andras
> > >
> > > On Tue, Nov 13, 2018 at 12:36 PM Julia Kinga Marton
> > >  wrote:
> > >
> > > > Hi,
> > > >
> > > > I made some testing on the RC, and it seems OK.
> > > >
> > > > Here is what I checked:
> > > > - sha512
> > > > - gpg signature
> > > > - rat report
> > > > - built Oozie against Hadoop-2.6.0
> > > > - ran a few examples (wf and coordinator), clicked the Web UI, ran
> some
> > > CLI
> > > > commands
> > > > - checked Fluent Job API by running Java and Shell examples
> > > > - looked briefly at the documentation
> > > >
> > > > +1 from my side (non-binding)
> > > >
> > > > Best regards,
> > > > Kinga
> > > >
> > > > On Mon, Nov 5, 2018 at 8:44 PM Andras Piros 
> wrote:
> > > >
> > > > > Hi there,
> > > > >
> > > > > I have created a build for Oozie 5.1.0, release candidate 1.
> > > > >
> > > > > This release introduces following major changes and fixes:
> > > > > - Fluent Job API
> > > > > - Git action
> > > > > - reduced heap waste
> > > > > - switched documentation from TWiki to Markdown
> > > > > - callable queue improvements
> > > > > - Spark action fixes
> > > > > - SLA improvements
> > > > > - third party dependency revamp
> > > > > - fixed CVE-2018-11799
> > > > >
> > > > > Keys to verify the signature of the release artifact are available
> at
> > > > >
> > > > >  http://www.apache.org/dist/oozie/KEYS
> > > > >
> > > > > Please download, test, and try it out:
> > > > >
> > > > > http://people.apache.org/~andras/oozie-5.1.0-rc1/
> > > > >
> > > > > The release candidate, sha512 signature, gpg signature, and rat
> > report
> > > > can
> > > > > all be found
> > > > > at the above address.
> > > > >
> > > > > The vote is open until Friday, November 9, 2018 at 12:00am PDT.
> > > > >
> > > > > Thanks,
> > > > >
> > > > > Andras Piros
> > > > >
> > > >
> > >
> >
> >
> > --
> > *Peter Cseh *| Software Engineer
> > cloudera.com 
> >
> > [image: Cloudera] 
> >
> > [image: Cloudera on Twitter]  [image:
> > Cloudera on Facebook]  [image:
> Cloudera
> > on LinkedIn] 
> > --
> >
>


-- 
-- 
Attila Sasvari
Software Engineer

[jira] [Created] (OOZIE-3390) shell action's stderr contains a bogus error message (since 5.0.0)

2018-11-22 Thread Attila Sasvari (JIRA)

Attila Sasvari created OOZIE-3390:
-

 Summary:  shell action's stderr contains a bogus error message 
(since 5.0.0)
 Key: OOZIE-3390
 URL: https://issues.apache.org/jira/browse/OOZIE-3390
 Project: Oozie
  Issue Type: Bug
Affects Versions: 5.1.0
Reporter: Attila Sasvari


Shell action's stderr contains an error message about path.
I executed shell action example to test Oozie 5.1 rc1, the job finished with 
success, but stderr of the job contained "Path echo doesn't appear to exist".

https://github.com/apache/oozie/blob/27e4bf1688a6a7750b9c8454de5021337696fd61/sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ShellContentWriter.java#L91
{code}
writeLine(errorStream, "Path " + filename + " doesn't appear to 
exist");
{code}

The problem is that we ignore the PATH of the process.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (OOZIE-3389) NPE after OOZIE-3370

2018-11-22 Thread Attila Sasvari (JIRA)

Attila Sasvari created OOZIE-3389:
-

 Summary: NPE after OOZIE-3370  
 Key: OOZIE-3389
 URL: https://issues.apache.org/jira/browse/OOZIE-3389
 Project: Oozie
  Issue Type: Bug
Affects Versions: 5.1.0
Reporter: Attila Sasvari


I was testing 5.1 rc1. I submitted the cron-schedule example coordinator, 
clicked on one of the coordinator jobs on the Oozie Web UI, but the details of 
the job were not displayed. I noticed the following NPE in the server log:
{code}
2018-11-21 15:45:00,019 ERROR CoordActionMissingDependenciesXCommand:517 - 
SERVER[Budapests-MacBook-Pro-10.local] USER[-] GROUP[-] TOKEN[-] APP[-] JOB[-] 
ACTION[-] XException, 
org.apache.oozie.command.CommandException: E1028: Coord input logic error. null
at 
org.apache.oozie.command.coord.CoordActionMissingDependenciesXCommand.execute(CoordActionMissingDependenciesXCommand.java:126)
at 
org.apache.oozie.command.coord.CoordActionMissingDependenciesXCommand.execute(CoordActionMissingDependenciesXCommand.java:38)
at org.apache.oozie.command.XCommand.call(XCommand.java:291)
at 
org.apache.oozie.CoordinatorEngine.getCoordActionMissingDependencies(CoordinatorEngine.java:910)
at 
org.apache.oozie.servlet.V2JobServlet.getCoordActionMissingDependencies(V2JobServlet.java:336)
at org.apache.oozie.servlet.BaseJobServlet.doGet(BaseJobServlet.java:370)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:687)
at 
org.apache.oozie.servlet.JsonRestServlet.service(JsonRestServlet.java:305)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:848)
at 
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1772)
at org.apache.oozie.servlet.AuthFilter$2.doFilter(AuthFilter.java:171)
at 
org.apache.hadoop.security.authentication.server.AuthenticationFilter.doFilter(AuthenticationFilter.java:594)
at 
org.apache.hadoop.security.authentication.server.AuthenticationFilter.doFilter(AuthenticationFilter.java:553)
at org.apache.oozie.servlet.AuthFilter.doFilter(AuthFilter.java:176)
at 
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
at org.apache.oozie.servlet.HostnameFilter.doFilter(HostnameFilter.java:86)
at 
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
at 
org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582)
at 
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
at 
org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)
at 
org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:226)
at 
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512)
at 
org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
at 
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112)
at 
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
at 
org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:119)
at 
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
at org.eclipse.jetty.server.Server.handle(Server.java:534)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:320)
at 
org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251)
at 
org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:283)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:108)
at 
org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
at 
org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303)
at 
org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
at 
org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)
{code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (OOZIE-3388) When use oozie5.0.0 submit a Action on Yarn,Launcher filed

2018-11-22 Thread Andras Piros (JIRA)



[ 
https://issues.apache.org/jira/browse/OOZIE-3388?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16695803#comment-16695803
 ] 

Andras Piros commented on OOZIE-3388:
-

Thanks for reporting this [~nobigo]! Can you please check whether this issue is 
the same as OOZIE-3307?

> When use oozie5.0.0 submit a  Action on Yarn,Launcher filed
> ---
>
> Key: OOZIE-3388
> URL: https://issues.apache.org/jira/browse/OOZIE-3388
> Project: Oozie
>  Issue Type: Bug
>  Components: core
>Affects Versions: 5.0.0
> Environment: Spark 2.1.0
> Yarn 2.6.0
> Oozie 5.0.0
>Reporter: duan xiong
>Priority: Major
>
> When use oozie5.0.0 submit a  Action on Yarn,Launcher filed
> {color:#FF}Application application_1542858285831_0002 failed 1 times due 
> to AM Container for appattempt_1542858285831_0002_01 exited with 
> exitCode: -103
> For more detailed output, check application tracking 
> page:http://ude242:18088/proxy/application_1542858285831_0002/AThen, click on 
> links to logs of each attempt.
> Diagnostics: Container 
> [pid=1383,containerID=container_1542858285831_0002_01_01] is running 
> beyond virtual memory limits. Current usage: 479.9 MB of 5 GB physical memory 
> used; 33.0 GB of 25 GB virtual memory used. Killing container.
> {color}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (OOZIE-3381) [coordinator] Enhance logging of CoordElFunctions

2018-11-22 Thread Julia Kinga Marton (JIRA)



[ 
https://issues.apache.org/jira/browse/OOZIE-3381?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16695761#comment-16695761
 ] 

Julia Kinga Marton commented on OOZIE-3381:
---

committed to master.

> [coordinator] Enhance logging of CoordElFunctions
> -
>
> Key: OOZIE-3381
> URL: https://issues.apache.org/jira/browse/OOZIE-3381
> Project: Oozie
>  Issue Type: Task
>  Components: coordinator
>Affects Versions: 5.1.0
>Reporter: Andras Piros
>Assignee: Andras Piros
>Priority: Major
> Attachments: OOZIE-3381.001.patch, OOZIE-3381.002.patch
>
>
> When using [{{$\{coord:latest\(n\)}}} coordinator EL 
> function|https://oozie.apache.org/docs/5.0.0/CoordinatorFunctionalSpec.html#a6.6.6._coord:latestint_n_EL_Function_for_Synchronous_Datasets]
>  inside an input dataset dependency, it's often the case that more 
> information is needed how many HDFS URIs are being checked for each 
> {{}}.
> Right now we don't have this information. While debugging and fine tuning 
> parameters like {{dataset frequency}}, {{initial-instance}}, and {{data-in}} 
> {{instance}}, it would be very useful to know how many HDFS roundtrips are 
> issues by the current settings {{CoordELFunctions#coord_latestRange_sync()}} 
> and {{CoordELFunctions#coord_futureRange_sync()}} having called 
> {{DFSClient#exists()}}. We need appropriate logging there.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (OOZIE-3381) [coordinator] Enhance logging of CoordElFunctions

2018-11-22 Thread Julia Kinga Marton (JIRA)



[ 
https://issues.apache.org/jira/browse/OOZIE-3381?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16695758#comment-16695758
 ] 

Julia Kinga Marton commented on OOZIE-3381:
---

Thank you [~andras.piros] for the patch! +1

> [coordinator] Enhance logging of CoordElFunctions
> -
>
> Key: OOZIE-3381
> URL: https://issues.apache.org/jira/browse/OOZIE-3381
> Project: Oozie
>  Issue Type: Task
>  Components: coordinator
>Affects Versions: 5.1.0
>Reporter: Andras Piros
>Assignee: Andras Piros
>Priority: Major
> Attachments: OOZIE-3381.001.patch, OOZIE-3381.002.patch
>
>
> When using [{{$\{coord:latest\(n\)}}} coordinator EL 
> function|https://oozie.apache.org/docs/5.0.0/CoordinatorFunctionalSpec.html#a6.6.6._coord:latestint_n_EL_Function_for_Synchronous_Datasets]
>  inside an input dataset dependency, it's often the case that more 
> information is needed how many HDFS URIs are being checked for each 
> {{}}.
> Right now we don't have this information. While debugging and fine tuning 
> parameters like {{dataset frequency}}, {{initial-instance}}, and {{data-in}} 
> {{instance}}, it would be very useful to know how many HDFS roundtrips are 
> issues by the current settings {{CoordELFunctions#coord_latestRange_sync()}} 
> and {{CoordELFunctions#coord_futureRange_sync()}} having called 
> {{DFSClient#exists()}}. We need appropriate logging there.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: Review Request 69348: OOZIE-3381 [coordinator] Enhance logging of CoordElFunctions

2018-11-22 Thread Kinga Marton via Review Board


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/69348/#review210789
---


Ship it!




Ship It!

- Kinga Marton


On Nov. 20, 2018, 10:54 a.m., András Piros wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/69348/
> ---
> 
> (Updated Nov. 20, 2018, 10:54 a.m.)
> 
> 
> Review request for oozie and Kinga Marton.
> 
> 
> Repository: oozie-git
> 
> 
> Description
> ---
> 
> OOZIE-3381 [coordinator] Enhance logging of CoordElFunctions
> 
> 
> Diffs
> -
> 
>   core/src/main/java/org/apache/oozie/command/XCommand.java 
> a80444e14134ec4234e6edfef0888cf1e76566df 
>   core/src/main/java/org/apache/oozie/coord/CoordELFunctions.java 
> 10f4f0d8d14fbb8d60a2d09a45b8f3b3b089f461 
>   core/src/test/java/org/apache/oozie/command/coord/CoordELExtensions.java 
> 796d19cfc9cbd03211746f9b94d0f67371249c7e 
>   core/src/test/java/org/apache/oozie/coord/TestOozieTimeUnitConverter.java 
> PRE-CREATION 
> 
> 
> Diff: https://reviews.apache.org/r/69348/diff/2/
> 
> 
> Testing
> ---
> 
> `TestCoordELFunctions`
> 
> 
> Thanks,
> 
> András Piros
> 
>

[jira] Subscription: Oozie Patch Available

Re: [VOTE] Release Oozie 5.1.0 (candidate 1)

[jira] [Assigned] (OOZIE-3390) shell action's stderr contains a bogus error message (since 5.0.0)

[jira] [Assigned] (OOZIE-3389) NPE after OOZIE-3370

Re: [VOTE] Release Oozie 5.1.0 (candidate 1)

[jira] [Created] (OOZIE-3390) shell action's stderr contains a bogus error message (since 5.0.0)

[jira] [Created] (OOZIE-3389) NPE after OOZIE-3370

[jira] [Commented] (OOZIE-3388) When use oozie5.0.0 submit a Action on Yarn,Launcher filed

[jira] [Commented] (OOZIE-3381) [coordinator] Enhance logging of CoordElFunctions

[jira] [Commented] (OOZIE-3381) [coordinator] Enhance logging of CoordElFunctions

Re: Review Request 69348: OOZIE-3381 [coordinator] Enhance logging of CoordElFunctions

11 matches

Site Navigation

Mail list logo

Footer information