[jira] [Updated] (OOZIE-2944) Shell action example does not work with Oozie on Yarn on hadoop 2.6
[
https://issues.apache.org/jira/browse/OOZIE-2944?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Attila Sasvari updated OOZIE-2944:
--
Issue Type: Sub-task (was: Bug)
Parent: OOZIE-2889
> Shell action example does not work with Oozie on Yarn on hadoop 2.6
> ---
>
> Key: OOZIE-2944
> URL: https://issues.apache.org/jira/browse/OOZIE-2944
> Project: Oozie
> Issue Type: Sub-task
>Reporter: Attila Sasvari
>Assignee: Attila Sasvari
> Attachments: OOZIE-2944-01.patch
>
>
> I tried to execute example shell action with a pseudo hadoop 2.6.0. The
> workflow failed with the exception below :
> {code}
> $ ~/workspace/pseudo/hadoop-2.6.0/bin/yarn logs --applicationId
> application_1497357254962_0005
> 17/06/13 15:59:09 INFO client.RMProxy: Connecting to ResourceManager at
> /0.0.0.0:8032
> 17/06/13 15:59:09 WARN util.NativeCodeLoader: Unable to load native-hadoop
> library for your platform... using builtin-java classes where applicable
> Container: container_1497357254962_0005_01_01 on 172.30.64.221_55437
> ==
> LogType:stderr
> Log Upload Time:13-Jun-2017 15:58:45
> LogLength:1397
> Log Contents:
> Exception in thread "main" java.lang.NoClassDefFoundError:
> org/apache/hadoop/mapred/Mapper
> at java.lang.ClassLoader.defineClass1(Native Method)
> at java.lang.ClassLoader.defineClass(ClassLoader.java:763)
> at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
> at java.net.URLClassLoader.defineClass(URLClassLoader.java:467)
> at java.net.URLClassLoader.access$100(URLClassLoader.java:73)
> at java.net.URLClassLoader$1.run(URLClassLoader.java:368)
> at java.net.URLClassLoader$1.run(URLClassLoader.java:362)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:361)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
> at
> org.apache.oozie.action.hadoop.LauncherAM.getMainArguments(LauncherAM.java:540)
> at org.apache.oozie.action.hadoop.LauncherAM.run(LauncherAM.java:170)
> at org.apache.oozie.action.hadoop.LauncherAM.main(LauncherAM.java:148)
> Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.mapred.Mapper
> at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
> ... 15 more
> LogType:stdout
> Log Upload Time:13-Jun-2017 15:58:45
> LogLength:2152
> Log Contents:
> Submitter user is: asasvari
> log4j: Trying to find [container-log4j.properties] using context classloader
> sun.misc.Launcher$AppClassLoader@6bc7c054.
> log4j: Using URL
> [jar:file:/Users/asasvari/workspace/pseudo/hadoop-2.6.0/share/hadoop/yarn/hadoop-yarn-server-nodemanager-2.6.0.jar!/container-log4j.properties]
> for automatic log4j configuration.
> log4j: Reading configuration from URL
> jar:file:/Users/asasvari/workspace/pseudo/hadoop-2.6.0/share/hadoop/yarn/hadoop-yarn-server-nodemanager-2.6.0.jar!/container-log4j.properties
> log4j: Hierarchy threshold set to [ALL].
> log4j: Parsing for [root] with value=[INFO,CLA, EventCounter].
> log4j: Level token is [INFO].
> log4j: Category root set to INFO
> log4j: Parsing appender named "CLA".
> log4j: Parsing layout options for "CLA".
> log4j: Setting property [conversionPattern] to [%d{ISO8601} %p [%t] %c: %m%n].
> log4j: End of parsing for "CLA".
> log4j: Setting property [containerLogDir] to
> [/Users/asasvari/workspace/pseudo/hadoop-2.6.0/logs/userlogs/application_1497357254962_0005/container_1497357254962_0005_01_01].
> log4j: Setting property [totalLogFileSize] to [0].
> log4j: setFile called:
> /Users/asasvari/workspace/pseudo/hadoop-2.6.0/logs/userlogs/application_1497357254962_0005/container_1497357254962_0005_01_01/syslog,
> true
> log4j: setFile ended
> log4j: Parsed "CLA" options.
> log4j: Parsing appender named "EventCounter".
> log4j: Parsed "EventCounter" options.
> log4j: Finished configuring.
> Using login user for UGI
> Launcher AM configuration loaded
> Starting the execution of prepare actions
> Completed the execution of prepare actions successfully
> Failing Oozie Launcher, null
> Oozie Launcher, uploading action data to HDFS sequence file:
> hdfs://localhost:9000/user/asasvari/oozie-asas/002-170613150915408-oozie-asas-W/shell-node--shell/action-data.seq
> Stopping AM
> Callback notification attempts left 0
> Callback notification trying
> http://0.0.0.0:11000/oozie/callback?id=002-170613150915408-oozie-asas-W@shell-node&status=FAILED
> Call
[jira] [Updated] (OOZIE-2944) Shell action example does not work with Oozie on Yarn on hadoop 2.6
[
https://issues.apache.org/jira/browse/OOZIE-2944?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Attila Sasvari updated OOZIE-2944:
--
Attachment: OOZIE-2944-01.patch
Tests:
- {{mvn test -DtargetJavaVersion=1.8 -DjavaVersion=1.8
-Dtest=TestLauncher,TestWorkflowActionKillXCommand,TestHadoopAccessorService
-Dhadoop.version-2.6.0}} passed
- shell example succeeded
{code}
application_1497357254962_0071 asasvari
oozie:launcher:T=shell:W=shell-wf:A=shell-node:ID=002-170615122337884-oozie-asas-W
Oozie Launcher default Thu, 15 Jun 2017 10:26:30 GMT Thu, 15 Jun 2017
10:26:32 GMT FINISHEDSUCCEEDED
{code}
> Shell action example does not work with Oozie on Yarn on hadoop 2.6
> ---
>
> Key: OOZIE-2944
> URL: https://issues.apache.org/jira/browse/OOZIE-2944
> Project: Oozie
> Issue Type: Bug
>Reporter: Attila Sasvari
>Assignee: Attila Sasvari
> Attachments: OOZIE-2944-01.patch
>
>
> I tried to execute example shell action with a pseudo hadoop 2.6.0. The
> workflow failed with the exception below :
> {code}
> $ ~/workspace/pseudo/hadoop-2.6.0/bin/yarn logs --applicationId
> application_1497357254962_0005
> 17/06/13 15:59:09 INFO client.RMProxy: Connecting to ResourceManager at
> /0.0.0.0:8032
> 17/06/13 15:59:09 WARN util.NativeCodeLoader: Unable to load native-hadoop
> library for your platform... using builtin-java classes where applicable
> Container: container_1497357254962_0005_01_01 on 172.30.64.221_55437
> ==
> LogType:stderr
> Log Upload Time:13-Jun-2017 15:58:45
> LogLength:1397
> Log Contents:
> Exception in thread "main" java.lang.NoClassDefFoundError:
> org/apache/hadoop/mapred/Mapper
> at java.lang.ClassLoader.defineClass1(Native Method)
> at java.lang.ClassLoader.defineClass(ClassLoader.java:763)
> at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
> at java.net.URLClassLoader.defineClass(URLClassLoader.java:467)
> at java.net.URLClassLoader.access$100(URLClassLoader.java:73)
> at java.net.URLClassLoader$1.run(URLClassLoader.java:368)
> at java.net.URLClassLoader$1.run(URLClassLoader.java:362)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:361)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
> at
> org.apache.oozie.action.hadoop.LauncherAM.getMainArguments(LauncherAM.java:540)
> at org.apache.oozie.action.hadoop.LauncherAM.run(LauncherAM.java:170)
> at org.apache.oozie.action.hadoop.LauncherAM.main(LauncherAM.java:148)
> Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.mapred.Mapper
> at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
> ... 15 more
> LogType:stdout
> Log Upload Time:13-Jun-2017 15:58:45
> LogLength:2152
> Log Contents:
> Submitter user is: asasvari
> log4j: Trying to find [container-log4j.properties] using context classloader
> sun.misc.Launcher$AppClassLoader@6bc7c054.
> log4j: Using URL
> [jar:file:/Users/asasvari/workspace/pseudo/hadoop-2.6.0/share/hadoop/yarn/hadoop-yarn-server-nodemanager-2.6.0.jar!/container-log4j.properties]
> for automatic log4j configuration.
> log4j: Reading configuration from URL
> jar:file:/Users/asasvari/workspace/pseudo/hadoop-2.6.0/share/hadoop/yarn/hadoop-yarn-server-nodemanager-2.6.0.jar!/container-log4j.properties
> log4j: Hierarchy threshold set to [ALL].
> log4j: Parsing for [root] with value=[INFO,CLA, EventCounter].
> log4j: Level token is [INFO].
> log4j: Category root set to INFO
> log4j: Parsing appender named "CLA".
> log4j: Parsing layout options for "CLA".
> log4j: Setting property [conversionPattern] to [%d{ISO8601} %p [%t] %c: %m%n].
> log4j: End of parsing for "CLA".
> log4j: Setting property [containerLogDir] to
> [/Users/asasvari/workspace/pseudo/hadoop-2.6.0/logs/userlogs/application_1497357254962_0005/container_1497357254962_0005_01_01].
> log4j: Setting property [totalLogFileSize] to [0].
> log4j: setFile called:
> /Users/asasvari/workspace/pseudo/hadoop-2.6.0/logs/userlogs/application_1497357254962_0005/container_1497357254962_0005_01_01/syslog,
> true
> log4j: setFile ended
> log4j: Parsed "CLA" options.
> log4j: Parsing appender named "EventCounter".
> log4j: Parsed "EventCounter" options.
> log4j: Finished configuring.
> Using login user for UGI
> Launcher AM configuration loaded
> Starting the execution of prepare actions
> Completed the execution of
[jira] [Commented] (OOZIE-2944) Shell action example does not work with Oozie on Yarn on hadoop 2.6
[
https://issues.apache.org/jira/browse/OOZIE-2944?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16050307#comment-16050307
]
Attila Sasvari commented on OOZIE-2944:
---
[~pbacsko] [~rkanter] [~gezapeti] can you have a look at the patch? Thanks
> Shell action example does not work with Oozie on Yarn on hadoop 2.6
> ---
>
> Key: OOZIE-2944
> URL: https://issues.apache.org/jira/browse/OOZIE-2944
> Project: Oozie
> Issue Type: Bug
>Reporter: Attila Sasvari
>Assignee: Attila Sasvari
> Attachments: OOZIE-2944-01.patch
>
>
> I tried to execute example shell action with a pseudo hadoop 2.6.0. The
> workflow failed with the exception below :
> {code}
> $ ~/workspace/pseudo/hadoop-2.6.0/bin/yarn logs --applicationId
> application_1497357254962_0005
> 17/06/13 15:59:09 INFO client.RMProxy: Connecting to ResourceManager at
> /0.0.0.0:8032
> 17/06/13 15:59:09 WARN util.NativeCodeLoader: Unable to load native-hadoop
> library for your platform... using builtin-java classes where applicable
> Container: container_1497357254962_0005_01_01 on 172.30.64.221_55437
> ==
> LogType:stderr
> Log Upload Time:13-Jun-2017 15:58:45
> LogLength:1397
> Log Contents:
> Exception in thread "main" java.lang.NoClassDefFoundError:
> org/apache/hadoop/mapred/Mapper
> at java.lang.ClassLoader.defineClass1(Native Method)
> at java.lang.ClassLoader.defineClass(ClassLoader.java:763)
> at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
> at java.net.URLClassLoader.defineClass(URLClassLoader.java:467)
> at java.net.URLClassLoader.access$100(URLClassLoader.java:73)
> at java.net.URLClassLoader$1.run(URLClassLoader.java:368)
> at java.net.URLClassLoader$1.run(URLClassLoader.java:362)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:361)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
> at
> org.apache.oozie.action.hadoop.LauncherAM.getMainArguments(LauncherAM.java:540)
> at org.apache.oozie.action.hadoop.LauncherAM.run(LauncherAM.java:170)
> at org.apache.oozie.action.hadoop.LauncherAM.main(LauncherAM.java:148)
> Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.mapred.Mapper
> at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
> ... 15 more
> LogType:stdout
> Log Upload Time:13-Jun-2017 15:58:45
> LogLength:2152
> Log Contents:
> Submitter user is: asasvari
> log4j: Trying to find [container-log4j.properties] using context classloader
> sun.misc.Launcher$AppClassLoader@6bc7c054.
> log4j: Using URL
> [jar:file:/Users/asasvari/workspace/pseudo/hadoop-2.6.0/share/hadoop/yarn/hadoop-yarn-server-nodemanager-2.6.0.jar!/container-log4j.properties]
> for automatic log4j configuration.
> log4j: Reading configuration from URL
> jar:file:/Users/asasvari/workspace/pseudo/hadoop-2.6.0/share/hadoop/yarn/hadoop-yarn-server-nodemanager-2.6.0.jar!/container-log4j.properties
> log4j: Hierarchy threshold set to [ALL].
> log4j: Parsing for [root] with value=[INFO,CLA, EventCounter].
> log4j: Level token is [INFO].
> log4j: Category root set to INFO
> log4j: Parsing appender named "CLA".
> log4j: Parsing layout options for "CLA".
> log4j: Setting property [conversionPattern] to [%d{ISO8601} %p [%t] %c: %m%n].
> log4j: End of parsing for "CLA".
> log4j: Setting property [containerLogDir] to
> [/Users/asasvari/workspace/pseudo/hadoop-2.6.0/logs/userlogs/application_1497357254962_0005/container_1497357254962_0005_01_01].
> log4j: Setting property [totalLogFileSize] to [0].
> log4j: setFile called:
> /Users/asasvari/workspace/pseudo/hadoop-2.6.0/logs/userlogs/application_1497357254962_0005/container_1497357254962_0005_01_01/syslog,
> true
> log4j: setFile ended
> log4j: Parsed "CLA" options.
> log4j: Parsing appender named "EventCounter".
> log4j: Parsed "EventCounter" options.
> log4j: Finished configuring.
> Using login user for UGI
> Launcher AM configuration loaded
> Starting the execution of prepare actions
> Completed the execution of prepare actions successfully
> Failing Oozie Launcher, null
> Oozie Launcher, uploading action data to HDFS sequence file:
> hdfs://localhost:9000/user/asasvari/oozie-asas/002-170613150915408-oozie-asas-W/shell-node--shell/action-data.seq
> Stopping AM
> Callback notification attempts left 0
> Callback notification trying
> http://0.0.0.0:11000/oozie/callback?id=
[jira] [Assigned] (OOZIE-2944) Shell action example does not work with Oozie on Yarn on hadoop 2.6
[
https://issues.apache.org/jira/browse/OOZIE-2944?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Attila Sasvari reassigned OOZIE-2944:
-
Assignee: Attila Sasvari
> Shell action example does not work with Oozie on Yarn on hadoop 2.6
> ---
>
> Key: OOZIE-2944
> URL: https://issues.apache.org/jira/browse/OOZIE-2944
> Project: Oozie
> Issue Type: Bug
>Reporter: Attila Sasvari
>Assignee: Attila Sasvari
>
> I tried to execute example shell action with a pseudo hadoop 2.6.0. The
> workflow failed with the exception below :
> {code}
> $ ~/workspace/pseudo/hadoop-2.6.0/bin/yarn logs --applicationId
> application_1497357254962_0005
> 17/06/13 15:59:09 INFO client.RMProxy: Connecting to ResourceManager at
> /0.0.0.0:8032
> 17/06/13 15:59:09 WARN util.NativeCodeLoader: Unable to load native-hadoop
> library for your platform... using builtin-java classes where applicable
> Container: container_1497357254962_0005_01_01 on 172.30.64.221_55437
> ==
> LogType:stderr
> Log Upload Time:13-Jun-2017 15:58:45
> LogLength:1397
> Log Contents:
> Exception in thread "main" java.lang.NoClassDefFoundError:
> org/apache/hadoop/mapred/Mapper
> at java.lang.ClassLoader.defineClass1(Native Method)
> at java.lang.ClassLoader.defineClass(ClassLoader.java:763)
> at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
> at java.net.URLClassLoader.defineClass(URLClassLoader.java:467)
> at java.net.URLClassLoader.access$100(URLClassLoader.java:73)
> at java.net.URLClassLoader$1.run(URLClassLoader.java:368)
> at java.net.URLClassLoader$1.run(URLClassLoader.java:362)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:361)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
> at
> org.apache.oozie.action.hadoop.LauncherAM.getMainArguments(LauncherAM.java:540)
> at org.apache.oozie.action.hadoop.LauncherAM.run(LauncherAM.java:170)
> at org.apache.oozie.action.hadoop.LauncherAM.main(LauncherAM.java:148)
> Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.mapred.Mapper
> at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
> ... 15 more
> LogType:stdout
> Log Upload Time:13-Jun-2017 15:58:45
> LogLength:2152
> Log Contents:
> Submitter user is: asasvari
> log4j: Trying to find [container-log4j.properties] using context classloader
> sun.misc.Launcher$AppClassLoader@6bc7c054.
> log4j: Using URL
> [jar:file:/Users/asasvari/workspace/pseudo/hadoop-2.6.0/share/hadoop/yarn/hadoop-yarn-server-nodemanager-2.6.0.jar!/container-log4j.properties]
> for automatic log4j configuration.
> log4j: Reading configuration from URL
> jar:file:/Users/asasvari/workspace/pseudo/hadoop-2.6.0/share/hadoop/yarn/hadoop-yarn-server-nodemanager-2.6.0.jar!/container-log4j.properties
> log4j: Hierarchy threshold set to [ALL].
> log4j: Parsing for [root] with value=[INFO,CLA, EventCounter].
> log4j: Level token is [INFO].
> log4j: Category root set to INFO
> log4j: Parsing appender named "CLA".
> log4j: Parsing layout options for "CLA".
> log4j: Setting property [conversionPattern] to [%d{ISO8601} %p [%t] %c: %m%n].
> log4j: End of parsing for "CLA".
> log4j: Setting property [containerLogDir] to
> [/Users/asasvari/workspace/pseudo/hadoop-2.6.0/logs/userlogs/application_1497357254962_0005/container_1497357254962_0005_01_01].
> log4j: Setting property [totalLogFileSize] to [0].
> log4j: setFile called:
> /Users/asasvari/workspace/pseudo/hadoop-2.6.0/logs/userlogs/application_1497357254962_0005/container_1497357254962_0005_01_01/syslog,
> true
> log4j: setFile ended
> log4j: Parsed "CLA" options.
> log4j: Parsing appender named "EventCounter".
> log4j: Parsed "EventCounter" options.
> log4j: Finished configuring.
> Using login user for UGI
> Launcher AM configuration loaded
> Starting the execution of prepare actions
> Completed the execution of prepare actions successfully
> Failing Oozie Launcher, null
> Oozie Launcher, uploading action data to HDFS sequence file:
> hdfs://localhost:9000/user/asasvari/oozie-asas/002-170613150915408-oozie-asas-W/shell-node--shell/action-data.seq
> Stopping AM
> Callback notification attempts left 0
> Callback notification trying
> http://0.0.0.0:11000/oozie/callback?id=002-170613150915408-oozie-asas-W@shell-node&status=FAILED
> Callback notification to
> http://0.0.0.0:11000/oozie/callback?id=002-1706131
[jira] [Commented] (OOZIE-2944) Shell action example does not work with Oozie on Yarn on hadoop 2.6
[
https://issues.apache.org/jira/browse/OOZIE-2944?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16050216#comment-16050216
]
Attila Sasvari commented on OOZIE-2944:
---
Right now JavaActionExecutor creates jobConf at multiple places:
-[createBaseHadoopConf()|https://github.com/apache/oozie/blob/83d4ddf45aa16649bd9fae367fa915379d5781cd/core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java#L249]
-[parseJobXmlAndConfiguration()
|https://github.com/apache/oozie/blob/83d4ddf45aa16649bd9fae367fa915379d5781cd/core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java#L342]
As a result, actionConf and launcherJobConf are JobConf objects in
{{createAppSubmissionContext()}}.
> Shell action example does not work with Oozie on Yarn on hadoop 2.6
> ---
>
> Key: OOZIE-2944
> URL: https://issues.apache.org/jira/browse/OOZIE-2944
> Project: Oozie
> Issue Type: Bug
>Reporter: Attila Sasvari
>
> I tried to execute example shell action with a pseudo hadoop 2.6.0. The
> workflow failed with the exception below :
> {code}
> $ ~/workspace/pseudo/hadoop-2.6.0/bin/yarn logs --applicationId
> application_1497357254962_0005
> 17/06/13 15:59:09 INFO client.RMProxy: Connecting to ResourceManager at
> /0.0.0.0:8032
> 17/06/13 15:59:09 WARN util.NativeCodeLoader: Unable to load native-hadoop
> library for your platform... using builtin-java classes where applicable
> Container: container_1497357254962_0005_01_01 on 172.30.64.221_55437
> ==
> LogType:stderr
> Log Upload Time:13-Jun-2017 15:58:45
> LogLength:1397
> Log Contents:
> Exception in thread "main" java.lang.NoClassDefFoundError:
> org/apache/hadoop/mapred/Mapper
> at java.lang.ClassLoader.defineClass1(Native Method)
> at java.lang.ClassLoader.defineClass(ClassLoader.java:763)
> at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
> at java.net.URLClassLoader.defineClass(URLClassLoader.java:467)
> at java.net.URLClassLoader.access$100(URLClassLoader.java:73)
> at java.net.URLClassLoader$1.run(URLClassLoader.java:368)
> at java.net.URLClassLoader$1.run(URLClassLoader.java:362)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:361)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
> at
> org.apache.oozie.action.hadoop.LauncherAM.getMainArguments(LauncherAM.java:540)
> at org.apache.oozie.action.hadoop.LauncherAM.run(LauncherAM.java:170)
> at org.apache.oozie.action.hadoop.LauncherAM.main(LauncherAM.java:148)
> Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.mapred.Mapper
> at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
> ... 15 more
> LogType:stdout
> Log Upload Time:13-Jun-2017 15:58:45
> LogLength:2152
> Log Contents:
> Submitter user is: asasvari
> log4j: Trying to find [container-log4j.properties] using context classloader
> sun.misc.Launcher$AppClassLoader@6bc7c054.
> log4j: Using URL
> [jar:file:/Users/asasvari/workspace/pseudo/hadoop-2.6.0/share/hadoop/yarn/hadoop-yarn-server-nodemanager-2.6.0.jar!/container-log4j.properties]
> for automatic log4j configuration.
> log4j: Reading configuration from URL
> jar:file:/Users/asasvari/workspace/pseudo/hadoop-2.6.0/share/hadoop/yarn/hadoop-yarn-server-nodemanager-2.6.0.jar!/container-log4j.properties
> log4j: Hierarchy threshold set to [ALL].
> log4j: Parsing for [root] with value=[INFO,CLA, EventCounter].
> log4j: Level token is [INFO].
> log4j: Category root set to INFO
> log4j: Parsing appender named "CLA".
> log4j: Parsing layout options for "CLA".
> log4j: Setting property [conversionPattern] to [%d{ISO8601} %p [%t] %c: %m%n].
> log4j: End of parsing for "CLA".
> log4j: Setting property [containerLogDir] to
> [/Users/asasvari/workspace/pseudo/hadoop-2.6.0/logs/userlogs/application_1497357254962_0005/container_1497357254962_0005_01_01].
> log4j: Setting property [totalLogFileSize] to [0].
> log4j: setFile called:
> /Users/asasvari/workspace/pseudo/hadoop-2.6.0/logs/userlogs/application_1497357254962_0005/container_1497357254962_0005_01_01/syslog,
> true
> log4j: setFile ended
> log4j: Parsed "CLA" options.
> log4j: Parsing appender named "EventCounter".
> log4j: Parsed "EventCounter" options.
> log4j: Finished configuring.
> Using login user for UGI
> Launcher AM configuration loaded
> Starting the execution of prepare actions
> Completed the exe
[jira] [Commented] (OOZIE-2944) Shell action example does not work with Oozie on Yarn on hadoop 2.6
[
https://issues.apache.org/jira/browse/OOZIE-2944?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16048160#comment-16048160
]
Attila Sasvari commented on OOZIE-2944:
---
I changed a LauncherMapper so that it does not extend the Mapper class and
rebuilt Oozie. I submitted the shell example that now failed with another, but
similar, reason:
{code}
Exception in thread "main" java.lang.NoClassDefFoundError:
org/apache/hadoop/mapred/JobConf
at
org.apache.oozie.action.hadoop.LauncherAM.getMainArguments(LauncherAM.java:540)
at org.apache.oozie.action.hadoop.LauncherAM.run(LauncherAM.java:170)
at org.apache.oozie.action.hadoop.LauncherAM.main(LauncherAM.java:148)
Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.mapred.JobConf
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
... 3 more
{code}
If I override {{needToAddMapReduceToClassPath()}} in {{ShellActionExecutor}},
so that it returns true, shell example workflow passes (verified).
>From this, I suppose each ActionExecutor that extends JavaActionExecutor will
>fail if they do not override {{needToAddMapReduceToClassPath()}} and MR is not
>in the class path.
> Shell action example does not work with Oozie on Yarn on hadoop 2.6
> ---
>
> Key: OOZIE-2944
> URL: https://issues.apache.org/jira/browse/OOZIE-2944
> Project: Oozie
> Issue Type: Bug
>Reporter: Attila Sasvari
>
> I tried to execute example shell action with a pseudo hadoop 2.6.0. The
> workflow failed with the exception below :
> {code}
> $ ~/workspace/pseudo/hadoop-2.6.0/bin/yarn logs --applicationId
> application_1497357254962_0005
> 17/06/13 15:59:09 INFO client.RMProxy: Connecting to ResourceManager at
> /0.0.0.0:8032
> 17/06/13 15:59:09 WARN util.NativeCodeLoader: Unable to load native-hadoop
> library for your platform... using builtin-java classes where applicable
> Container: container_1497357254962_0005_01_01 on 172.30.64.221_55437
> ==
> LogType:stderr
> Log Upload Time:13-Jun-2017 15:58:45
> LogLength:1397
> Log Contents:
> Exception in thread "main" java.lang.NoClassDefFoundError:
> org/apache/hadoop/mapred/Mapper
> at java.lang.ClassLoader.defineClass1(Native Method)
> at java.lang.ClassLoader.defineClass(ClassLoader.java:763)
> at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
> at java.net.URLClassLoader.defineClass(URLClassLoader.java:467)
> at java.net.URLClassLoader.access$100(URLClassLoader.java:73)
> at java.net.URLClassLoader$1.run(URLClassLoader.java:368)
> at java.net.URLClassLoader$1.run(URLClassLoader.java:362)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:361)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
> at
> org.apache.oozie.action.hadoop.LauncherAM.getMainArguments(LauncherAM.java:540)
> at org.apache.oozie.action.hadoop.LauncherAM.run(LauncherAM.java:170)
> at org.apache.oozie.action.hadoop.LauncherAM.main(LauncherAM.java:148)
> Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.mapred.Mapper
> at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
> ... 15 more
> LogType:stdout
> Log Upload Time:13-Jun-2017 15:58:45
> LogLength:2152
> Log Contents:
> Submitter user is: asasvari
> log4j: Trying to find [container-log4j.properties] using context classloader
> sun.misc.Launcher$AppClassLoader@6bc7c054.
> log4j: Using URL
> [jar:file:/Users/asasvari/workspace/pseudo/hadoop-2.6.0/share/hadoop/yarn/hadoop-yarn-server-nodemanager-2.6.0.jar!/container-log4j.properties]
> for automatic log4j configuration.
> log4j: Reading configuration from URL
> jar:file:/Users/asasvari/workspace/pseudo/hadoop-2.6.0/share/hadoop/yarn/hadoop-yarn-server-nodemanager-2.6.0.jar!/container-log4j.properties
> log4j: Hierarchy threshold set to [ALL].
> log4j: Parsing for [root] with value=[INFO,CLA, EventCounter].
> log4j: Level token is [INFO].
> log4j: Category root set to INFO
> log4j: Parsing appender named "CLA".
> log4j: Parsing layout options for "CLA".
> log4j: Setting property [conversionPattern] to [%d{ISO8601} %p [%t] %c: %m%n].
> log4j: End of parsing for "CLA".
> log4j: Setting pro
[jira] [Created] (OOZIE-2944) Shell action example does not work with Oozie on Yarn on hadoop 2.6
Attila Sasvari created OOZIE-2944:
-
Summary: Shell action example does not work with Oozie on Yarn on
hadoop 2.6
Key: OOZIE-2944
URL: https://issues.apache.org/jira/browse/OOZIE-2944
Project: Oozie
Issue Type: Bug
Reporter: Attila Sasvari
I tried to execute example shell action with a pseudo hadoop 2.6.0. The
workflow failed with the exception below :
{code}
$ ~/workspace/pseudo/hadoop-2.6.0/bin/yarn logs --applicationId
application_1497357254962_0005
17/06/13 15:59:09 INFO client.RMProxy: Connecting to ResourceManager at
/0.0.0.0:8032
17/06/13 15:59:09 WARN util.NativeCodeLoader: Unable to load native-hadoop
library for your platform... using builtin-java classes where applicable
Container: container_1497357254962_0005_01_01 on 172.30.64.221_55437
==
LogType:stderr
Log Upload Time:13-Jun-2017 15:58:45
LogLength:1397
Log Contents:
Exception in thread "main" java.lang.NoClassDefFoundError:
org/apache/hadoop/mapred/Mapper
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClass(ClassLoader.java:763)
at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
at java.net.URLClassLoader.defineClass(URLClassLoader.java:467)
at java.net.URLClassLoader.access$100(URLClassLoader.java:73)
at java.net.URLClassLoader$1.run(URLClassLoader.java:368)
at java.net.URLClassLoader$1.run(URLClassLoader.java:362)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:361)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
at
org.apache.oozie.action.hadoop.LauncherAM.getMainArguments(LauncherAM.java:540)
at org.apache.oozie.action.hadoop.LauncherAM.run(LauncherAM.java:170)
at org.apache.oozie.action.hadoop.LauncherAM.main(LauncherAM.java:148)
Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.mapred.Mapper
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
... 15 more
LogType:stdout
Log Upload Time:13-Jun-2017 15:58:45
LogLength:2152
Log Contents:
Submitter user is: asasvari
log4j: Trying to find [container-log4j.properties] using context classloader
sun.misc.Launcher$AppClassLoader@6bc7c054.
log4j: Using URL
[jar:file:/Users/asasvari/workspace/pseudo/hadoop-2.6.0/share/hadoop/yarn/hadoop-yarn-server-nodemanager-2.6.0.jar!/container-log4j.properties]
for automatic log4j configuration.
log4j: Reading configuration from URL
jar:file:/Users/asasvari/workspace/pseudo/hadoop-2.6.0/share/hadoop/yarn/hadoop-yarn-server-nodemanager-2.6.0.jar!/container-log4j.properties
log4j: Hierarchy threshold set to [ALL].
log4j: Parsing for [root] with value=[INFO,CLA, EventCounter].
log4j: Level token is [INFO].
log4j: Category root set to INFO
log4j: Parsing appender named "CLA".
log4j: Parsing layout options for "CLA".
log4j: Setting property [conversionPattern] to [%d{ISO8601} %p [%t] %c: %m%n].
log4j: End of parsing for "CLA".
log4j: Setting property [containerLogDir] to
[/Users/asasvari/workspace/pseudo/hadoop-2.6.0/logs/userlogs/application_1497357254962_0005/container_1497357254962_0005_01_01].
log4j: Setting property [totalLogFileSize] to [0].
log4j: setFile called:
/Users/asasvari/workspace/pseudo/hadoop-2.6.0/logs/userlogs/application_1497357254962_0005/container_1497357254962_0005_01_01/syslog,
true
log4j: setFile ended
log4j: Parsed "CLA" options.
log4j: Parsing appender named "EventCounter".
log4j: Parsed "EventCounter" options.
log4j: Finished configuring.
Using login user for UGI
Launcher AM configuration loaded
Starting the execution of prepare actions
Completed the execution of prepare actions successfully
Failing Oozie Launcher, null
Oozie Launcher, uploading action data to HDFS sequence file:
hdfs://localhost:9000/user/asasvari/oozie-asas/002-170613150915408-oozie-asas-W/shell-node--shell/action-data.seq
Stopping AM
Callback notification attempts left 0
Callback notification trying
http://0.0.0.0:11000/oozie/callback?id=002-170613150915408-oozie-asas-W@shell-node&status=FAILED
Callback notification to
http://0.0.0.0:11000/oozie/callback?id=002-170613150915408-oozie-asas-W@shell-node&status=FAILED
succeeded
Callback notification succeeded
LogType:syslog
Log Upload Time:13-Jun-2017 15:58:45
LogLength:1215
Log Contents:
2017-06-13 15:58:43,105 WARN [main] org.apache.hadoop.util.NativeCodeLoader:
Unable to load native-hadoop library for your platform... using builtin-java
classes where applicable
2017-06-13 15:58:43,271 INFO [main] org.apache.hadoo
[jira] [Commented] (OOZIE-2941) Old map-reduce workflow example can't work with OYA
[ https://issues.apache.org/jira/browse/OOZIE-2941?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16048022#comment-16048022 ] Attila Sasvari commented on OOZIE-2941: --- [~Dongying Jiao] I re-tested the mapreduce example in my local environment, and the workflow successfully finished. Before running the MR workflow, you should first install Oozie sharelib and make sure jhs is running. As per your previous commend, I suspect sharelib was not installed in your case. So I believe this issue is not the same as my experience with the shell action example. > Old map-reduce workflow example can't work with OYA > --- > > Key: OOZIE-2941 > URL: https://issues.apache.org/jira/browse/OOZIE-2941 > Project: Oozie > Issue Type: Sub-task > Components: workflow >Affects Versions: trunk >Reporter: Dongying Jiao > Attachments: RM_UI.jpg > > > map-reduce workflow in Oozie examples run failed after OYA merged. The > exception in yarn log is: > Exception in thread "main" java.lang.NoClassDefFoundError: > org/apache/hadoop/mapred/Mapper > at java.lang.ClassLoader.defineClass1(Native Method) > at java.lang.ClassLoader.defineClass(ClassLoader.java:763) > at > java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142) > at java.net.URLClassLoader.defineClass(URLClassLoader.java:467) > at java.net.URLClassLoader.access$100(URLClassLoader.java:73) > at java.net.URLClassLoader$1.run(URLClassLoader.java:368) > at java.net.URLClassLoader$1.run(URLClassLoader.java:362) > at java.security.AccessController.doPrivileged(Native Method) > at java.net.URLClassLoader.findClass(URLClassLoader.java:361) > at java.lang.ClassLoader.loadClass(ClassLoader.java:424) > at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331) > at java.lang.ClassLoader.loadClass(ClassLoader.java:357) > at > org.apache.oozie.action.hadoop.LauncherAM.getMainArguments(LauncherAM.java:540) > at org.apache.oozie.action.hadoop.LauncherAM.run(LauncherAM.java:170) > at org.apache.oozie.action.hadoop.LauncherAM.main(LauncherAM.java:148) > Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.mapred.Mapper > at java.net.URLClassLoader.findClass(URLClassLoader.java:381) > at java.lang.ClassLoader.loadClass(ClassLoader.java:424) > at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331) > at java.lang.ClassLoader.loadClass(ClassLoader.java:357) > ... 15 more -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (OOZIE-2941) Old map-reduce workflow example can't work with OYA
[
https://issues.apache.org/jira/browse/OOZIE-2941?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16047989#comment-16047989
]
Attila Sasvari commented on OOZIE-2941:
---
[~Dongying Jiao], [~gezapeti] Today I experienced the same issue with the shell
action example (I was using a pseudo hadoop 2.6.0 for my test).
{code}
$ ~/workspace/pseudo/hadoop-2.6.0/bin/yarn logs --applicationId
application_1497357254962_0005
17/06/13 15:59:09 INFO client.RMProxy: Connecting to ResourceManager at
/0.0.0.0:8032
17/06/13 15:59:09 WARN util.NativeCodeLoader: Unable to load native-hadoop
library for your platform... using builtin-java classes where applicable
Container: container_1497357254962_0005_01_01 on 172.30.64.221_55437
==
LogType:stderr
Log Upload Time:13-Jun-2017 15:58:45
LogLength:1397
Log Contents:
Exception in thread "main" java.lang.NoClassDefFoundError:
org/apache/hadoop/mapred/Mapper
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClass(ClassLoader.java:763)
at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
at java.net.URLClassLoader.defineClass(URLClassLoader.java:467)
at java.net.URLClassLoader.access$100(URLClassLoader.java:73)
at java.net.URLClassLoader$1.run(URLClassLoader.java:368)
at java.net.URLClassLoader$1.run(URLClassLoader.java:362)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:361)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
at
org.apache.oozie.action.hadoop.LauncherAM.getMainArguments(LauncherAM.java:540)
at org.apache.oozie.action.hadoop.LauncherAM.run(LauncherAM.java:170)
at org.apache.oozie.action.hadoop.LauncherAM.main(LauncherAM.java:148)
Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.mapred.Mapper
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
... 15 more
LogType:stdout
Log Upload Time:13-Jun-2017 15:58:45
LogLength:2152
Log Contents:
Submitter user is: asasvari
log4j: Trying to find [container-log4j.properties] using context classloader
sun.misc.Launcher$AppClassLoader@6bc7c054.
log4j: Using URL
[jar:file:/Users/asasvari/workspace/pseudo/hadoop-2.6.0/share/hadoop/yarn/hadoop-yarn-server-nodemanager-2.6.0.jar!/container-log4j.properties]
for automatic log4j configuration.
log4j: Reading configuration from URL
jar:file:/Users/asasvari/workspace/pseudo/hadoop-2.6.0/share/hadoop/yarn/hadoop-yarn-server-nodemanager-2.6.0.jar!/container-log4j.properties
log4j: Hierarchy threshold set to [ALL].
log4j: Parsing for [root] with value=[INFO,CLA, EventCounter].
log4j: Level token is [INFO].
log4j: Category root set to INFO
log4j: Parsing appender named "CLA".
log4j: Parsing layout options for "CLA".
log4j: Setting property [conversionPattern] to [%d{ISO8601} %p [%t] %c: %m%n].
log4j: End of parsing for "CLA".
log4j: Setting property [containerLogDir] to
[/Users/asasvari/workspace/pseudo/hadoop-2.6.0/logs/userlogs/application_1497357254962_0005/container_1497357254962_0005_01_01].
log4j: Setting property [totalLogFileSize] to [0].
log4j: setFile called:
/Users/asasvari/workspace/pseudo/hadoop-2.6.0/logs/userlogs/application_1497357254962_0005/container_1497357254962_0005_01_01/syslog,
true
log4j: setFile ended
log4j: Parsed "CLA" options.
log4j: Parsing appender named "EventCounter".
log4j: Parsed "EventCounter" options.
log4j: Finished configuring.
Using login user for UGI
Launcher AM configuration loaded
Starting the execution of prepare actions
Completed the execution of prepare actions successfully
Failing Oozie Launcher, null
Oozie Launcher, uploading action data to HDFS sequence file:
hdfs://localhost:9000/user/asasvari/oozie-asas/002-170613150915408-oozie-asas-W/shell-node--shell/action-data.seq
Stopping AM
Callback notification attempts left 0
Callback notification trying
http://0.0.0.0:11000/oozie/callback?id=002-170613150915408-oozie-asas-W@shell-node&status=FAILED
Callback notification to
http://0.0.0.0:11000/oozie/callback?id=002-170613150915408-oozie-asas-W@shell-node&status=FAILED
succeeded
Callback notification succeeded
LogType:syslog
Log Upload Time:13-Jun-2017 15:58:45
LogLength:1215
Log Contents:
2017-06-13 15:58:43,105 WARN [main] org.apache.hadoop.util.NativeCodeLoader:
Unable to load native-hadoop library for your platform... using builtin-java
classes where applicable
2017-06-13 15:58:43,271 INFO [main] org.apache.hadoop.yarn.client.RMProxy:
Connecting to ResourceManager at /0.0.0.0:8030
2017-0
[jira] [Updated] (OOZIE-2796) oozie.action.keep.action.dir not getting noticed
[
https://issues.apache.org/jira/browse/OOZIE-2796?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Attila Sasvari updated OOZIE-2796:
--
Attachment: OOZIE-2796-6.patch
rebased patch - tests of TestJavaActionExecutor locally passed
> oozie.action.keep.action.dir not getting noticed
>
>
> Key: OOZIE-2796
> URL: https://issues.apache.org/jira/browse/OOZIE-2796
> Project: Oozie
> Issue Type: Bug
>Affects Versions: 4.3.0
>Reporter: Andras Piros
>Assignee: Xiaobin Zheng
> Fix For: trunk
>
> Attachments: OOZIE-2796-1.patch, OOZIE-2796-2.patch,
> OOZIE-2796-3.patch, OOZIE-2796-4.patch, OOZIE-2796-5.patch, OOZIE-2796-6.patch
>
>
> When trying to add following configuration entry:
> {code:xml}
>
> oozie.action.keep.action.dir
> true
>
> {code}
> to the application level configuration of {{workflow.xml}}, the workflow
> directories are removed just after the workflow has finished.
> This should not be the case.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (OOZIE-2796) oozie.action.keep.action.dir not getting noticed
[
https://issues.apache.org/jira/browse/OOZIE-2796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16038335#comment-16038335
]
Attila Sasvari commented on OOZIE-2796:
---
[~zhengxb2005] could you please rebase the patch? It does not apply on latest
master.
{code}
git apply -p 0 OOZIE-2796-5.patch
error: patch failed:
core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java:65
error:
core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java:
patch does not apply
{code}
> oozie.action.keep.action.dir not getting noticed
>
>
> Key: OOZIE-2796
> URL: https://issues.apache.org/jira/browse/OOZIE-2796
> Project: Oozie
> Issue Type: Bug
>Affects Versions: 4.3.0
>Reporter: Andras Piros
>Assignee: Xiaobin Zheng
> Fix For: trunk
>
> Attachments: OOZIE-2796-1.patch, OOZIE-2796-2.patch,
> OOZIE-2796-3.patch, OOZIE-2796-4.patch, OOZIE-2796-5.patch
>
>
> When trying to add following configuration entry:
> {code:xml}
>
> oozie.action.keep.action.dir
> true
>
> {code}
> to the application level configuration of {{workflow.xml}}, the workflow
> directories are removed just after the workflow has finished.
> This should not be the case.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (OOZIE-2829) Improve sharelib upload to accept multiple source folders
[
https://issues.apache.org/jira/browse/OOZIE-2829?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Attila Sasvari updated OOZIE-2829:
--
Attachment: OOZIE-2829-05.patch
> Improve sharelib upload to accept multiple source folders
> -
>
> Key: OOZIE-2829
> URL: https://issues.apache.org/jira/browse/OOZIE-2829
> Project: Oozie
> Issue Type: Bug
> Components: tools
>Reporter: Peter Cseh
>Assignee: Attila Sasvari
> Attachments: OOZIE-2829-01.patch, OOZIE-2829-02.patch,
> OOZIE-2829-03.patch, OOZIE-2829-04.patch, OOZIE-2829-05.patch
>
>
> Right now sharelib can be created via {{sharelib create -fs FS_URI -locallib
> SHARED_LIBRARY}} where the SHARED_LIBRARY can be a tarbal or a folder.
> It would be nice to have the possibility to define additional folders to be
> uploaded into the sharelib, so the users don't have to copy or link the files
> together on their machine.
> The syntax could be something like -additional-lib
> sharelibName=/path/to/source/;/path/to/some/file,sharelibName2=/path/to/some/folder
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (OOZIE-2829) Improve sharelib upload to accept multiple source folders
[
https://issues.apache.org/jira/browse/OOZIE-2829?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Attila Sasvari updated OOZIE-2829:
--
Attachment: OOZIE-2829-04.patch
> Improve sharelib upload to accept multiple source folders
> -
>
> Key: OOZIE-2829
> URL: https://issues.apache.org/jira/browse/OOZIE-2829
> Project: Oozie
> Issue Type: Bug
> Components: tools
>Reporter: Peter Cseh
>Assignee: Attila Sasvari
> Attachments: OOZIE-2829-01.patch, OOZIE-2829-02.patch,
> OOZIE-2829-03.patch, OOZIE-2829-04.patch
>
>
> Right now sharelib can be created via {{sharelib create -fs FS_URI -locallib
> SHARED_LIBRARY}} where the SHARED_LIBRARY can be a tarbal or a folder.
> It would be nice to have the possibility to define additional folders to be
> uploaded into the sharelib, so the users don't have to copy or link the files
> together on their machine.
> The syntax could be something like -additional-lib
> sharelibName=/path/to/source/;/path/to/some/file,sharelibName2=/path/to/some/folder
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (OOZIE-2829) Improve sharelib upload to accept multiple source folders
[
https://issues.apache.org/jira/browse/OOZIE-2829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16029306#comment-16029306
]
Attila Sasvari commented on OOZIE-2829:
---
[~pbacsko] thanks for the review.
Regarding {{SECURITY_MANAGER}} - thanks for pointing this out, it came from
the existing code. This naming issue is currently present in multiple classes:
--
https://github.com/apache/oozie/search?utf8=%E2%9C%93&q=SECURITY_MANAGER&type=
--
https://github.com/apache/oozie/blob/branch-4.3/tools/src/test/java/org/apache/oozie/tools/TestOozieSharelibCLI.java#L46
-- anyway I will fix these
I will attach a new patch addressing the issues above soon.
> Improve sharelib upload to accept multiple source folders
> -
>
> Key: OOZIE-2829
> URL: https://issues.apache.org/jira/browse/OOZIE-2829
> Project: Oozie
> Issue Type: Bug
> Components: tools
>Reporter: Peter Cseh
>Assignee: Attila Sasvari
> Attachments: OOZIE-2829-01.patch, OOZIE-2829-02.patch,
> OOZIE-2829-03.patch
>
>
> Right now sharelib can be created via {{sharelib create -fs FS_URI -locallib
> SHARED_LIBRARY}} where the SHARED_LIBRARY can be a tarbal or a folder.
> It would be nice to have the possibility to define additional folders to be
> uploaded into the sharelib, so the users don't have to copy or link the files
> together on their machine.
> The syntax could be something like -additional-lib
> sharelibName=/path/to/source/;/path/to/some/file,sharelibName2=/path/to/some/folder
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (OOZIE-2387) Oozie is Unable to handle Spaces in file/archive tag.
[
https://issues.apache.org/jira/browse/OOZIE-2387?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Attila Sasvari updated OOZIE-2387:
--
Attachment: OOZIE-2387-04.patch
[~rkanter] Many thanks for the review, I have attached the rebased patch.
> Oozie is Unable to handle Spaces in file/archive tag.
> -
>
> Key: OOZIE-2387
> URL: https://issues.apache.org/jira/browse/OOZIE-2387
> Project: Oozie
> Issue Type: Bug
> Components: action
>Affects Versions: 4.1.0, 4.2.0
> Environment: Hadoop 2.6,Oozie 4.1, Hortonworks2.2
>Reporter: shiv pratap singh
>Assignee: Attila Sasvari
> Attachments: OOZIE-2387-01.patch, OOZIE-2387-02.patch,
> OOZIE-2387-03.patch, OOZIE-2387-04.patch
>
>
> Oozie file tag is unable to handle white spaces and other special characters.
> Error -
> Caused by: java.lang.NullPointerException
> at
> org.apache.oozie.action.hadoop.JavaActionExecutor.addToCache(JavaActionExecutor.java:529)
> at
> org.apache.oozie.action.hadoop.JavaActionExecutor.setLibFilesArchives(JavaActionExecutor.java:678)
> at
> org.apache.oozie.action.hadoop.JavaActionExecutor.submitLauncher(JavaActionExecutor.java:884)
> Sample XML File --
>
> xmlns:ns5="uri:oozie:shell-action:0.2"
> xmlns:ns4="uri:oozie:workflow:0.4"
> xmlns:ns3="uri:oozie:sqoop-action:0.2"
> xmlns:ns2="uri:oozie:hive-action:0.3" name="test-shell-space">
>
>
>
> ${wf_hadoop_resourceManager}
> ${wf_hadoop_nameNode}
> shell 1.sh
> /user/map-dev/test-case-dir/shell/script/shell
> 1.sh
>
>
>
>
>
> shell-decision job failed
>
>
>
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Created] (OOZIE-2882) Rerun workflow fails Error: E0404
Attila Sasvari created OOZIE-2882:
-
Summary: Rerun workflow fails Error: E0404
Key: OOZIE-2882
URL: https://issues.apache.org/jira/browse/OOZIE-2882
Project: Oozie
Issue Type: Improvement
Reporter: Attila Sasvari
Only one of the properties are allowed [oozie.wf.rerun.skip.nodes OR
oozie.wf.rerun.failnodes]
Reproduction:
1. Create a workflow with more than 1 node. Eg: Fork - with three parallel
shell actions. Make sure one of them fails
2. Rerun with 'oozie.wf.rerun.failnodes' set.
3. Rerun again with 'oozie.wf.rerun.skip.nodes' and check 'Skip all successful
nodes'.
You will get the following error.
Error: E0404 : E0404: Only one of the properties are allowed
[oozie.wf.rerun.skip.nodes OR oozie.wf.rerun.failnodes]
When a user reruns a workflow job with oozie.wf.rerun.failnode=true and if the
job fails in subsequent steps, we do not have an option to resubmit the
workflow using oozie.wf.rerun.skip.node=action1,action2 to allow submission
from predecessor steps.
Currently, once the workflow fails and one of the rerun options is used for job
rerun it gets merged and there is no way to override like regular oozie
configurations or variables.
We have a few options:
1. If fail.nodes and skip.nodes are specified at the same time (or one of them
was carried over from a previous wf run), we can add {generate skip.nodes by
discovering nodes that did not fail} union {skip.nodes}
2. Add a way to remove properties (this is also is potentially helpful for
other use cases)
3. The "newest" property (oozie.wf.rerun.skip.nodes or
oozie.wf.rerun.failnodes) takes priority and the previous is ignored
4. Make oozie.wf.rerun.skip.nodes or oozie.wf.rerun.failnodes somehow not
persist in the DB
Part of this JIRA would be to figure out which is the best option.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (OOZIE-2854) Oozie should handle transient DB problems
[ https://issues.apache.org/jira/browse/OOZIE-2854?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15997108#comment-15997108 ] Attila Sasvari commented on OOZIE-2854: --- [~pbacsko] I see your points, thanks for the clarification > Oozie should handle transient DB problems > - > > Key: OOZIE-2854 > URL: https://issues.apache.org/jira/browse/OOZIE-2854 > Project: Oozie > Issue Type: Improvement > Components: core >Reporter: Peter Bacsko >Assignee: Peter Bacsko > Attachments: OOZIE-2854-POC-001.patch > > > There can be problems when Oozie cannot update the database properly. > Recently, we have experienced erratic behavior with two setups: > * MySQL with the Galera cluster manager. Galera uses cluster-wide optimistic > locking which might cause a transaction to rollback if there are two or more > parallel transaction running and one of them cannot complete because of a > conflict. > * MySQL with Percona XtraDB Cluster. If one of the MySQL instances is killed, > Oozie might get "Communications link failure" exception during the failover. > The problem is that failed DB transactions later might cause a workflow > (which are started/re-started by RecoveryService) to get stuck. It's not > clear to us how this happens but it has to do with the fact that certain DB > updates are not executed. > The solution is to use some sort of retry logic with exponential backoff if > the DB update fails. We could start with a 100ms wait time which is doubled > at every retry. The operation can be considered a failure if it still fails > after 10 attempts. These values could be configurable. We should discuss > initial values in the scope of this JIRA. > Note that this solution is to handle *transient* failures. If the DB is down > for a longer period of time, we have to accept that the internal state of > Oozie is corrupted. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (OOZIE-2854) Oozie should handle transient DB problems
[
https://issues.apache.org/jira/browse/OOZIE-2854?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15996974#comment-15996974
]
Attila Sasvari commented on OOZIE-2854:
---
[~pbacsko] thanks for the POC, I would also vote for having max retries. I also
agree to handle the last exception.
I only have minor comments so far
* I would extract the anonymous classes passed to
{{retryHandler.executeWithRetry}} into a separate classes
* {{em.getTransaction()}} calls could be assigned to local variable.
* There are call() methods that could throw less general exceptions
(JPAExecutorException is enough in some cases)
* To me DBOperationRetryHandler looks like a general retryhandler (nothing DB
specific, except name and package).
> Oozie should handle transient DB problems
> -
>
> Key: OOZIE-2854
> URL: https://issues.apache.org/jira/browse/OOZIE-2854
> Project: Oozie
> Issue Type: Improvement
> Components: core
>Reporter: Peter Bacsko
>Assignee: Peter Bacsko
> Attachments: OOZIE-2854-POC-001.patch
>
>
> There can be problems when Oozie cannot update the database properly.
> Recently, we have experienced erratic behavior with two setups:
> * MySQL with the Galera cluster manager. Galera uses cluster-wide optimistic
> locking which might cause a transaction to rollback if there are two or more
> parallel transaction running and one of them cannot complete because of a
> conflict.
> * MySQL with Percona XtraDB Cluster. If one of the MySQL instances is killed,
> Oozie might get "Communications link failure" exception during the failover.
> The problem is that failed DB transactions later might cause a workflow
> (which are started/re-started by RecoveryService) to get stuck. It's not
> clear to us how this happens but it has to do with the fact that certain DB
> updates are not executed.
> The solution is to use some sort of retry logic with exponential backoff if
> the DB update fails. We could start with a 100ms wait time which is doubled
> at every retry. The operation can be considered a failure if it still fails
> after 10 attempts. These values could be configurable. We should discuss
> initial values in the scope of this JIRA.
> Note that this solution is to handle *transient* failures. If the DB is down
> for a longer period of time, we have to accept that the internal state of
> Oozie is corrupted.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Created] (OOZIE-2880) Improve documentation on Oozie authentication and authorization configuration
Attila Sasvari created OOZIE-2880:
-
Summary: Improve documentation on Oozie authentication and
authorization configuration
Key: OOZIE-2880
URL: https://issues.apache.org/jira/browse/OOZIE-2880
Project: Oozie
Issue Type: Improvement
Reporter: Attila Sasvari
h4. Authentication
[Documentation of Oozie
authentication|https://oozie.apache.org/docs/4.3.0/AG_Install.html#Oozie_User_Authentication_Configuration]
is slightly incorrect.
{quote}
Pseudo/simple authentication requires the user to specify the user name on the
request, this is done by the PseudoAuthenticator class by injecting the
user.name parameter in the query string of all requests. The user.name
parameter value is taken from the client process Java System property user.name
.
{quote}
Actually, when someone performs an Oozie operation using the CLI, a hadoop auth
token is created and saved to {{~/.oozie-auth-token}}. In subsequent actions,
the token is retrieved from this cache file (until the token is expired). In
other words, passing {{user.name}} as system property to the Oozie CLI in an
unsecure environment (or using kinit -kt in a Kerberized cluster) takes no
effect if {{.oozie-auth-token}} is present in the user's home and the token is
still valid.
With {{simple}} authentication type pseudo/simple authentication is used.
However, in an unsecure environment I tested with hadoop 2.4 (default hadoop
version) that
[KerberosAuthenticator|https://github.com/apache/hadoop/blob/branch-2.4.0/hadoop-common-project/hadoop-auth/src/main/java/org/apache/hadoop/security/authentication/client/KerberosAuthenticator.java#L188]
returns a null token. In turn, admin operations cannot be used if there is no
{{~/.oozie-auth-token}} with a valid token. Note: this was fixed by
[HADOOP-11467|https://github.com/apache/hadoop/commit/875256834b892b574499d5fe68f95a9aed244f7d#diff-28df14cad207bee984f5ca4820bacabcR198].
h4. Authorization
Regarding [authorization configuration |
https://oozie.apache.org/docs/4.3.0/AG_Install.html#User_Authorization_Configuration]
the current documentation mentions a deprecated configuration property
{{oozie.service.AuthorizationService.security.enabled}}. If a user specifies
it, the following warning is logged in Oozie server log.
{noformat}
2017-05-02 03:30:59,578 WARN org.apache.oozie.util.ConfigUtils:
SERVER[myserver.com] Using a deprecated configuration property
[oozie.service.Authorization
Service.security.enabled], should use
[oozie.service.AuthorizationService.authorization.enabled]. Please delete the
deprecated property in order for the new property to take effect.
{noformat}
{{oozie.service.AuthorizationService.authorization.enabled}} should be used.
Oozie authorization only make sense if *authentication (kerberos) is enabled*.
Otherwise any user can claim any identity (as "simple" authentication type uses
Pseudo authentication).
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (OOZIE-2829) Improve sharelib upload to accept multiple source folders
[
https://issues.apache.org/jira/browse/OOZIE-2829?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Attila Sasvari updated OOZIE-2829:
--
Attachment: OOZIE-2829-03.patch
[~pbacsko], [~abhishekbafna] could you take a look? Thanks in advance.
> Improve sharelib upload to accept multiple source folders
> -
>
> Key: OOZIE-2829
> URL: https://issues.apache.org/jira/browse/OOZIE-2829
> Project: Oozie
> Issue Type: Bug
> Components: tools
>Reporter: Peter Cseh
>Assignee: Attila Sasvari
> Attachments: OOZIE-2829-01.patch, OOZIE-2829-02.patch,
> OOZIE-2829-03.patch
>
>
> Right now sharelib can be created via {{sharelib create -fs FS_URI -locallib
> SHARED_LIBRARY}} where the SHARED_LIBRARY can be a tarbal or a folder.
> It would be nice to have the possibility to define additional folders to be
> uploaded into the sharelib, so the users don't have to copy or link the files
> together on their machine.
> The syntax could be something like -additional-lib
> sharelibName=/path/to/source/;/path/to/some/file,sharelibName2=/path/to/some/folder
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (OOZIE-2829) Improve sharelib upload to accept multiple source folders
[
https://issues.apache.org/jira/browse/OOZIE-2829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15986331#comment-15986331
]
Attila Sasvari commented on OOZIE-2829:
---
It might make sense to finish, review, fix, commit OOZIE-2629 and OOZIE-2791
prior to this.
> Improve sharelib upload to accept multiple source folders
> -
>
> Key: OOZIE-2829
> URL: https://issues.apache.org/jira/browse/OOZIE-2829
> Project: Oozie
> Issue Type: Bug
> Components: tools
>Reporter: Peter Cseh
>Assignee: Attila Sasvari
> Attachments: OOZIE-2829-01.patch, OOZIE-2829-02.patch
>
>
> Right now sharelib can be created via {{sharelib create -fs FS_URI -locallib
> SHARED_LIBRARY}} where the SHARED_LIBRARY can be a tarbal or a folder.
> It would be nice to have the possibility to define additional folders to be
> uploaded into the sharelib, so the users don't have to copy or link the files
> together on their machine.
> The syntax could be something like -additional-lib
> sharelibName=/path/to/source/;/path/to/some/file,sharelibName2=/path/to/some/folder
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (OOZIE-2829) Improve sharelib upload to accept multiple source folders
[
https://issues.apache.org/jira/browse/OOZIE-2829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15986320#comment-15986320
]
Attila Sasvari commented on OOZIE-2829:
---
Re-executed tests in TestTimestampedMessageParser and each of them passed.
Attaching new patch that fixes new findbugs error soon.
> Improve sharelib upload to accept multiple source folders
> -
>
> Key: OOZIE-2829
> URL: https://issues.apache.org/jira/browse/OOZIE-2829
> Project: Oozie
> Issue Type: Bug
> Components: tools
>Reporter: Peter Cseh
>Assignee: Attila Sasvari
> Attachments: OOZIE-2829-01.patch, OOZIE-2829-02.patch
>
>
> Right now sharelib can be created via {{sharelib create -fs FS_URI -locallib
> SHARED_LIBRARY}} where the SHARED_LIBRARY can be a tarbal or a folder.
> It would be nice to have the possibility to define additional folders to be
> uploaded into the sharelib, so the users don't have to copy or link the files
> together on their machine.
> The syntax could be something like -additional-lib
> sharelibName=/path/to/source/;/path/to/some/file,sharelibName2=/path/to/some/folder
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (OOZIE-2829) Improve sharelib upload to accept multiple source folders
[
https://issues.apache.org/jira/browse/OOZIE-2829?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Attila Sasvari updated OOZIE-2829:
--
Attachment: OOZIE-2829-02.patch
> Improve sharelib upload to accept multiple source folders
> -
>
> Key: OOZIE-2829
> URL: https://issues.apache.org/jira/browse/OOZIE-2829
> Project: Oozie
> Issue Type: Bug
> Components: tools
>Reporter: Peter Cseh
>Assignee: Attila Sasvari
> Attachments: OOZIE-2829-01.patch, OOZIE-2829-02.patch
>
>
> Right now sharelib can be created via {{sharelib create -fs FS_URI -locallib
> SHARED_LIBRARY}} where the SHARED_LIBRARY can be a tarbal or a folder.
> It would be nice to have the possibility to define additional folders to be
> uploaded into the sharelib, so the users don't have to copy or link the files
> together on their machine.
> The syntax could be something like -additional-lib
> sharelibName=/path/to/source/;/path/to/some/file,sharelibName2=/path/to/some/folder
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (OOZIE-2829) Improve sharelib upload to accept multiple source folders
[
https://issues.apache.org/jira/browse/OOZIE-2829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15981110#comment-15981110
]
Attila Sasvari commented on OOZIE-2829:
---
[~gezapeti] thanks for the review comments.
1. fixed
2. fixed - also added more tests to test invalid inputs. I also extracted two
existing integration tests to a new test class. I wanted to use JUnit 4
annotations to check exception is thrown when the input is invalid.
3. OK
4. I adjusted help messages of CLI command to include more details (also
increased width in CLIParser):
{noformat}
usage:
OozieSharelibCLI creates or upgrade sharelib for oozie
oozie-setup.sh create : create a new timestamped version of
oozie sharelib
-additional_libsAdditional sharelib
resources. This option requires a colon-separated list of pairs of
sharelibname and
coma-sperated list of pathnames in the following format:
"sharelib_name=pathname[,pathname...][,sharelib_name2=pathname[,pathname...]...]"
Caveats:
* Each pathname is either
a directory or a regular file (compressed files are not
extracted prior to the
upload operation).
* Sharelibnames shall be
specified only once.
* Do not upload multiple
conflicting library versions for an additional sharelib
directory as it may cause
runtime issues.
Example command:
$ oozie-setup.sh sharelib
create -fs hdfs://localhost:9000 -locallib \
oozie-sharelib.tar.gz
-additional_libs share2=dir2,file2:share3=file3
-concurrencyNumber of threads to be
used for copy operations. (default=1)
-fs URI of the fileSystem to
create oozie share library
-locallib Local share library
directory
{noformat}
5. I adjusted help messages of CLI command (see previous)
> Improve sharelib upload to accept multiple source folders
> -
>
> Key: OOZIE-2829
> URL: https://issues.apache.org/jira/browse/OOZIE-2829
> Project: Oozie
> Issue Type: Bug
> Components: tools
>Reporter: Peter Cseh
>Assignee: Attila Sasvari
> Attachments: OOZIE-2829-01.patch
>
>
> Right now sharelib can be created via {{sharelib create -fs FS_URI -locallib
> SHARED_LIBRARY}} where the SHARED_LIBRARY can be a tarbal or a folder.
> It would be nice to have the possibility to define additional folders to be
> uploaded into the sharelib, so the users don't have to copy or link the files
> together on their machine.
> The syntax could be something like -additional-lib
> sharelibName=/path/to/source/;/path/to/some/file,sharelibName2=/path/to/some/folder
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (OOZIE-2829) Improve sharelib upload to accept multiple source folders
[
https://issues.apache.org/jira/browse/OOZIE-2829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15976377#comment-15976377
]
Attila Sasvari commented on OOZIE-2829:
---
[~gezapeti] could you take a look? Thanks!
> Improve sharelib upload to accept multiple source folders
> -
>
> Key: OOZIE-2829
> URL: https://issues.apache.org/jira/browse/OOZIE-2829
> Project: Oozie
> Issue Type: Bug
> Components: tools
>Reporter: Peter Cseh
>Assignee: Attila Sasvari
> Attachments: OOZIE-2829-01.patch
>
>
> Right now sharelib can be created via {{sharelib create -fs FS_URI -locallib
> SHARED_LIBRARY}} where the SHARED_LIBRARY can be a tarbal or a folder.
> It would be nice to have the possibility to define additional folders to be
> uploaded into the sharelib, so the users don't have to copy or link the files
> together on their machine.
> The syntax could be something like -additional-lib
> sharelibName=/path/to/source/;/path/to/some/file,sharelibName2=/path/to/some/folder
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Created] (OOZIE-2868) Oozie server cannot start if the oozie user does not have permission on oozie.jsp.tmp.dir
Attila Sasvari created OOZIE-2868:
-
Summary: Oozie server cannot start if the oozie user does not have
permission on oozie.jsp.tmp.dir
Key: OOZIE-2868
URL: https://issues.apache.org/jira/browse/OOZIE-2868
Project: Oozie
Issue Type: Bug
Reporter: Attila Sasvari
Oozie fails to start if the directory used for jsp classes is not writable by
the oozie user
{noformat}
java.lang.IllegalStateException: Failed to delete temp dir
/tmp/embedded-jetty-jsp
at
org.eclipse.jetty.webapp.WebInfConfiguration.configureTempDirectory(WebInfConfiguration.java:376)
at
org.eclipse.jetty.webapp.WebInfConfiguration.resolveTempDirectory(WebInfConfiguration.java:275)
at
org.eclipse.jetty.webapp.WebInfConfiguration.preConfigure(WebInfConfiguration.java:69)
at
org.eclipse.jetty.webapp.WebAppContext.preConfigure(WebAppContext.java:483)
at
org.eclipse.jetty.webapp.WebAppContext.doStart(WebAppContext.java:519)
at
org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
at
org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:131)
at
org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:105)
at
org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:61)
at
org.eclipse.jetty.security.SecurityHandler.doStart(SecurityHandler.java:361)
at
org.eclipse.jetty.security.ConstraintSecurityHandler.doStart(ConstraintSecurityHandler.java:448)
at
org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
at
org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:131)
at
org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:113)
at
org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:61)
at
org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
at
org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:131)
at org.eclipse.jetty.server.Server.start(Server.java:411)
at
org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:105)
at
org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:61)
at org.eclipse.jetty.server.Server.doStart(Server.java:378)
at
org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
at
org.apache.oozie.server.EmbeddedOozieServer.start(EmbeddedOozieServer.java:189)
at
org.apache.oozie.server.EmbeddedOozieServer.main(EmbeddedOozieServer.java:215)
{noformat}
It would be better to print out a proper error message and exit if we detect
Oozie has insufficient permission on the directory pointed by
{{oozie.jsp.tmp.dir}}.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (OOZIE-2829) Improve sharelib upload to accept multiple source folders
[
https://issues.apache.org/jira/browse/OOZIE-2829?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Attila Sasvari updated OOZIE-2829:
--
Attachment: OOZIE-2829-01.patch
Besides unit tests, I tested manually with the following:
{code}
$ mvn clean package assembly:single -DskipTests=true -DskipTests
-Dhadoop.version-2.6.0 -Puber
$ cd distro/target/oozie-4.4.0-SNAPSHOT-distro/oozie-4.4.0-SNAPSHOT/
$ mkdir -p spark2
$ touch spark2/2
$ touch 2_1
$ mkdir -p spark3
$ touch spark3/3
$ bin/oozied.sh start
$ bin/oozie-setup.sh sharelib create -fs hdfs://localhost:9000 -locallib
oozie-sharelib-4.4.0-SNAPSHOT.tar.gz -additional_libs
"spark2=spark2,2_1;spark3=spark3" -concurrency 4
{code}
Note: if a source directory is empty, it will be not copied to HDFS.
> Improve sharelib upload to accept multiple source folders
> -
>
> Key: OOZIE-2829
> URL: https://issues.apache.org/jira/browse/OOZIE-2829
> Project: Oozie
> Issue Type: Bug
> Components: tools
>Reporter: Peter Cseh
>Assignee: Attila Sasvari
> Attachments: OOZIE-2829-01.patch
>
>
> Right now sharelib can be created via {{sharelib create -fs FS_URI -locallib
> SHARED_LIBRARY}} where the SHARED_LIBRARY can be a tarbal or a folder.
> It would be nice to have the possibility to define additional folders to be
> uploaded into the sharelib, so the users don't have to copy or link the files
> together on their machine.
> The syntax could be something like -additional-lib
> sharelibName=/path/to/source/;/path/to/some/file,sharelibName2=/path/to/some/folder
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (OOZIE-2829) Improve sharelib upload to accept multiple source folders
[
https://issues.apache.org/jira/browse/OOZIE-2829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15972573#comment-15972573
]
Attila Sasvari commented on OOZIE-2829:
---
[~gezapeti] using semicolon as separator might not be a good a idea in Linux.
It can work if you surround option args with double quotes, but probably it
might better to use colon as sharelibname separator. I am also not sure that we
want to allow adding individual files instead of directories when specifying
the sources.
> Improve sharelib upload to accept multiple source folders
> -
>
> Key: OOZIE-2829
> URL: https://issues.apache.org/jira/browse/OOZIE-2829
> Project: Oozie
> Issue Type: Bug
> Components: tools
>Reporter: Peter Cseh
>Assignee: Attila Sasvari
>
> Right now sharelib can be created via {{sharelib create -fs FS_URI -locallib
> SHARED_LIBRARY}} where the SHARED_LIBRARY can be a tarbal or a folder.
> It would be nice to have the possibility to define additional folders to be
> uploaded into the sharelib, so the users don't have to copy or link the files
> together on their machine.
> The syntax could be something like -additional-lib
> sharelibName=/path/to/source/;/path/to/some/file,sharelibName2=/path/to/some/folder
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (OOZIE-2795) Create lib directory or symlink for Oozie CLI during packaging
[
https://issues.apache.org/jira/browse/OOZIE-2795?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15972559#comment-15972559
]
Attila Sasvari commented on OOZIE-2795:
---
Ran into this again: I attempted to install sharelib before starting the Oozie
server.
{noformat}
Exception in thread "main" java.lang.NoClassDefFoundError:
org/apache/commons/cli/ParseException
at java.lang.Class.getDeclaredMethods0(Native Method)
at java.lang.Class.privateGetDeclaredMethods(Class.java:2701)
at java.lang.Class.privateGetMethodRecursive(Class.java:3048)
at java.lang.Class.getMethod0(Class.java:3018)
at java.lang.Class.getMethod(Class.java:1784)
at
sun.launcher.LauncherHelper.validateMainClass(LauncherHelper.java:544)
at sun.launcher.LauncherHelper.checkAndLoadMain(LauncherHelper.java:526)
Caused by: java.lang.ClassNotFoundException:
org.apache.commons.cli.ParseException
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
... 7 more
{noformat}
> Create lib directory or symlink for Oozie CLI during packaging
> --
>
> Key: OOZIE-2795
> URL: https://issues.apache.org/jira/browse/OOZIE-2795
> Project: Oozie
> Issue Type: Bug
>Reporter: Attila Sasvari
>Assignee: Peter Cseh
> Attachments: OOZIE-2795-001.patch, OOZIE-2795-002.patch,
> OOZIE-2795-003.patch
>
>
> After creating an Oozie distro, lib directory is not present in the distro
> directory (e.g. oozie-4.4.0-SNAPSHOT) in the generated tar.gz .
>
> As a consequence, first run of theOozie CLI will fail if one tries to execute
> it prior to server start:
> {noformat}
> $ bin/oozie
> unzip: cannot find or open
> /Users/asasvari/workspace/apache/oozie/distro/target/oozie-4.4.0-SNAPSHOT/oozie.war,
>
> /Users/asasvari/workspace/apache/oozie/distro/target/oozie-4.4.0-SNAPSHOT/oozie.war.zip
> or
> /Users/asasvari/workspace/apache/oozie/distro/target/oozie-4.4.0-SNAPSHOT/oozie.war.ZIP.
> mv: rename
> /Users/asasvari/workspace/apache/oozie/distro/target/oozie-4.4.0-SNAPSHOT/lib/WEB-INF/lib/*.jar
> to
> /Users/asasvari/workspace/apache/oozie/distro/target/oozie-4.4.0-SNAPSHOT/lib/*.jar:
> No such file or directory
> rmdir:
> /Users/asasvari/workspace/apache/oozie/distro/target/oozie-4.4.0-SNAPSHOT/lib/WEB-INF/lib:
> No such file or directory
> rmdir:
> /Users/asasvari/workspace/apache/oozie/distro/target/oozie-4.4.0-SNAPSHOT/lib/WEB-INF:
> No such file or directory
> Error: Could not find or load main class org.apache.oozie.cli.OozieCLI
> {noformat}
> If you start the server first however, a symlink is created (see
> {{bin/oozie-jetty-server.sh}}), and this error will not be visible.
> Earlier {{bin/oozie}} created this library by extracting it from the WAR file
> used for Tomcat. Comment in the script:
> {noformat}
> #Create lib directory from war if lib doesn't exist
> {noformat}
> We could create a symlink to embedded-oozie-server/webapp/WEB-INF/lib/ or
> copy that directory.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (OOZIE-2829) Improve sharelib upload to accept multiple source folders
[
https://issues.apache.org/jira/browse/OOZIE-2829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15972487#comment-15972487
]
Attila Sasvari commented on OOZIE-2829:
---
{{org.apache.commons.cli.Option}} does not allow "-" in option => use
{{additional_libs}}.
Instead of {{-additional-lib
sharelibName=/path/to/source/;/path/to/some/file,sharelibName2=/path/to/some/folder}},
I would use {{-additional-lib
sharelibName=/path/to/source/,/path/to/some/file;sharelibName2=/path/to/some/folder}}
> Improve sharelib upload to accept multiple source folders
> -
>
> Key: OOZIE-2829
> URL: https://issues.apache.org/jira/browse/OOZIE-2829
> Project: Oozie
> Issue Type: Bug
> Components: tools
>Reporter: Peter Cseh
>Assignee: Attila Sasvari
>
> Right now sharelib can be created via {{sharelib create -fs FS_URI -locallib
> SHARED_LIBRARY}} where the SHARED_LIBRARY can be a tarbal or a folder.
> It would be nice to have the possibility to define additional folders to be
> uploaded into the sharelib, so the users don't have to copy or link the files
> together on their machine.
> The syntax could be something like -additional-lib
> sharelibName=/path/to/source/;/path/to/some/file,sharelibName2=/path/to/some/folder
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (OOZIE-2818) Can't overwrite oozie.action.max.output.data on a per-workflow basis
[
https://issues.apache.org/jira/browse/OOZIE-2818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15972196#comment-15972196
]
Attila Sasvari commented on OOZIE-2818:
---
[~pbacsko] could you review the patch set? thanks
> Can't overwrite oozie.action.max.output.data on a per-workflow basis
>
>
> Key: OOZIE-2818
> URL: https://issues.apache.org/jira/browse/OOZIE-2818
> Project: Oozie
> Issue Type: Bug
>Reporter: Peter Cseh
>Assignee: Attila Sasvari
> Attachments: OOZIE-2818-01.patch
>
>
> It's not possible to raise the limit of the output on a per-action basis.
> With the workflow:
> {quote}
>
>
>
> Action failed, error
> message\[$\{wf:errorMessage(wf:lastErrorNode())}]
>
>
>
> $\{jobTracker}
> $\{nameNode}
>
>
> oozie.action.max.output.data
> 10
>
>
> oozie.launcher.oozie.action.max.output.data
> 10
>
>
> /user/admin/large_output.sh
> 10
> /user/admin/large_output.sh#large_output.sh
>
>
>
>
>
>
>
> {quote}
> In the localized files in the job.xml the value of this property gets
> overwritten:
> {quote}
> # cat
> /yarn/nm/usercache/admin/appcache/application_1489145190857_0001/container_1489145190857_0001_01_02/job.xml
> |grep max.outp
> oozie.launcher.oozie.action.max.output.data10programaticallyjob.xml
> oozie.action.max.output.data2048programaticallyjob.xml
> # cat
> /yarn/nm/usercache/admin/appcache/application_1489145190857_0001/container_1489145190857_0001_01_02/action.xml
> |grep max.outp
> oozie.launcher.oozie.action.max.output.data10programatically
> oozie.action.max.output.data10programatically
> {quote}
> I could not find a reason for this behavior, so I think it's a bug.
> The value is read from
> [ConfigurationService|https://github.com/apache/oozie/blob/master/core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java#L175]
> and set in
> [createLauncherConf|https://github.com/apache/oozie/blob/master/core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java#L986]
> in JavaActionExecutor. There could be a check to see if it's already set.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (OOZIE-2387) Oozie is Unable to handle Spaces in file/archive tag.
[
https://issues.apache.org/jira/browse/OOZIE-2387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15972195#comment-15972195
]
Attila Sasvari commented on OOZIE-2387:
---
[~rkanter], [~gezapeti] can you take a look at the new patch? thanks
> Oozie is Unable to handle Spaces in file/archive tag.
> -
>
> Key: OOZIE-2387
> URL: https://issues.apache.org/jira/browse/OOZIE-2387
> Project: Oozie
> Issue Type: Bug
> Components: action
>Affects Versions: 4.1.0, 4.2.0
> Environment: Hadoop 2.6,Oozie 4.1, Hortonworks2.2
>Reporter: shiv pratap singh
>Assignee: Attila Sasvari
> Attachments: OOZIE-2387-01.patch, OOZIE-2387-02.patch,
> OOZIE-2387-03.patch
>
>
> Oozie file tag is unable to handle white spaces and other special characters.
> Error -
> Caused by: java.lang.NullPointerException
> at
> org.apache.oozie.action.hadoop.JavaActionExecutor.addToCache(JavaActionExecutor.java:529)
> at
> org.apache.oozie.action.hadoop.JavaActionExecutor.setLibFilesArchives(JavaActionExecutor.java:678)
> at
> org.apache.oozie.action.hadoop.JavaActionExecutor.submitLauncher(JavaActionExecutor.java:884)
> Sample XML File --
>
> xmlns:ns5="uri:oozie:shell-action:0.2"
> xmlns:ns4="uri:oozie:workflow:0.4"
> xmlns:ns3="uri:oozie:sqoop-action:0.2"
> xmlns:ns2="uri:oozie:hive-action:0.3" name="test-shell-space">
>
>
>
> ${wf_hadoop_resourceManager}
> ${wf_hadoop_nameNode}
> shell 1.sh
> /user/map-dev/test-case-dir/shell/script/shell
> 1.sh
>
>
>
>
>
> shell-decision job failed
>
>
>
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (OOZIE-2833) when using uber mode the regex pattern used in the extractHeapSizeMB method does not allow heap sizes specified in bytes.
[ https://issues.apache.org/jira/browse/OOZIE-2833?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15972190#comment-15972190 ] Attila Sasvari commented on OOZIE-2833: --- [~pbacsko] Thanks for the review. I used the same magic numbers as of the existing implementation, but you are right. I will attach a new patch addressing the issues you mentioned soon. > when using uber mode the regex pattern used in the extractHeapSizeMB method > does not allow heap sizes specified in bytes. > - > > Key: OOZIE-2833 > URL: https://issues.apache.org/jira/browse/OOZIE-2833 > Project: Oozie > Issue Type: Bug >Reporter: Thomas Scott >Assignee: Attila Sasvari >Priority: Minor > Attachments: OOZIE-2833.01.patch, OOZIE-2833.02.patch > > > The extractHeapSizeMB method used here: > https://github.com/apache/oozie/blob/9c0516608bdd0787b1fac3a69ccfe9d36cbb2d15/core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java#L401 > Uses the following pattern: > -Xmx(([0-9]+)[mMgG]) > This requires that the -Xmx configuration option end in "m" or "g". However, > it is valid to configure this option in bytes without the "m" or "g" and this > will not be picked up. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (OOZIE-2860) Improve Jetty logging
[
https://issues.apache.org/jira/browse/OOZIE-2860?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15964009#comment-15964009
]
Attila Sasvari commented on OOZIE-2860:
---
[~andras.piros] thanks for the patch!
Could you please explain a bit why {{log4j.rootLogger=NONE, none}} is necessary?
You forget to define the {{none}} appender in
{{core/src/main/resources/oozie-log4j.properties}}.
Otherwise, it looks good to me. (verified logging on my local machine)
> Improve Jetty logging
> -
>
> Key: OOZIE-2860
> URL: https://issues.apache.org/jira/browse/OOZIE-2860
> Project: Oozie
> Issue Type: Improvement
> Components: core
>Affects Versions: 5.0.0
>Reporter: Andras Piros
>Assignee: Andras Piros
> Fix For: 5.0.0
>
> Attachments: jetty-logs.zip, OOZIE-2860.001.patch,
> OOZIE-2860.002.patch
>
>
> At the moment only Oozie server logs are written by Jetty, but not its own
> [*server
> logs*|https://www.eclipse.org/jetty/documentation/9.3.x/configuring-logging.html].
> The Oozie 4.3.0 Tomcat logs are written to
> {{$\{OOZIE_DIR\}/logs/catalina--mm-dd.log}}. It's necessary to have the
> same log location and file pattern (rollover each day), but not necessary to
> have the same filename prefix. It's also a must to have the same log format
> like the Tomcat one. It's nice to have log levels per package configurable
> via {{oozie-default.xml}}.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (OOZIE-2836) Fix ps1 and cmd scripts to make Oozie runnable on Windows
[
https://issues.apache.org/jira/browse/OOZIE-2836?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Attila Sasvari updated OOZIE-2836:
--
Attachment: OOZIE-2836-01.patch
> Fix ps1 and cmd scripts to make Oozie runnable on Windows
> -
>
> Key: OOZIE-2836
> URL: https://issues.apache.org/jira/browse/OOZIE-2836
> Project: Oozie
> Issue Type: Bug
>Reporter: Attila Sasvari
>Priority: Minor
> Attachments: OOZIE-2836-01.patch
>
>
> With OOZIE-2666 Oozie was migrated from Tomcat to embedded jetty. Distro
> shell scripts were updated, but Windows scripts were not adjusted in
> {{distro/src/main/bin}}.
> It would be nice if someone with a Windows machine could fix the following
> scripts:
> - oozie-setup.cmd
> - oozie-setup.ps1
> - oozie-sys.cmd
> - oozie.cmd
> - oozied.cmd
> and verify that Oozie works on Windows.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Assigned] (OOZIE-2836) Fix ps1 and cmd scripts to make Oozie runnable on Windows
[
https://issues.apache.org/jira/browse/OOZIE-2836?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Attila Sasvari reassigned OOZIE-2836:
-
Assignee: Attila Sasvari
> Fix ps1 and cmd scripts to make Oozie runnable on Windows
> -
>
> Key: OOZIE-2836
> URL: https://issues.apache.org/jira/browse/OOZIE-2836
> Project: Oozie
> Issue Type: Bug
>Reporter: Attila Sasvari
>Assignee: Attila Sasvari
>Priority: Minor
> Attachments: OOZIE-2836-01.patch
>
>
> With OOZIE-2666 Oozie was migrated from Tomcat to embedded jetty. Distro
> shell scripts were updated, but Windows scripts were not adjusted in
> {{distro/src/main/bin}}.
> It would be nice if someone with a Windows machine could fix the following
> scripts:
> - oozie-setup.cmd
> - oozie-setup.ps1
> - oozie-sys.cmd
> - oozie.cmd
> - oozied.cmd
> and verify that Oozie works on Windows.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (OOZIE-2846) TestSSLServerConnectorFactory fails with Jetty 9.3 and 9.4
[ https://issues.apache.org/jira/browse/OOZIE-2846?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15956974#comment-15956974 ] Attila Sasvari commented on OOZIE-2846: --- [~gezapeti] thanks a lot for the patch, +1 (non-binding) > TestSSLServerConnectorFactory fails with Jetty 9.3 and 9.4 > -- > > Key: OOZIE-2846 > URL: https://issues.apache.org/jira/browse/OOZIE-2846 > Project: Oozie > Issue Type: Bug >Reporter: Peter Cseh >Assignee: Peter Cseh > Attachments: OOZIE-2846.001.patch, OOZIE-2846.002.patch > > > We've tested using Oozie with different Jetty versions and found that the > test TestSSLServerConnectorFactory is failing for newer Jetty versions. > Changing the @Mock to @Spy on the Server solves the issue. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (OOZIE-2850) Fix default callback notifications
[
https://issues.apache.org/jira/browse/OOZIE-2850?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Attila Sasvari updated OOZIE-2850:
--
Attachment: OOZIE-2850-07.patch
@rkanter thanks for the reivew
Attaching new patch because my IDE tricked me, and the \t remained there.
> Fix default callback notifications
> --
>
> Key: OOZIE-2850
> URL: https://issues.apache.org/jira/browse/OOZIE-2850
> Project: Oozie
> Issue Type: Bug
>Reporter: Attila Sasvari
>Assignee: Attila Sasvari
>Priority: Trivial
> Attachments: OOZIE-2850-01.patch, OOZIE-2850-02.patch,
> OOZIE-2850-03.patch, OOZIE-2850-04.patch, OOZIE-2850-05.patch,
> OOZIE-2850-06.patch, OOZIE-2850-07.patch
>
>
> Currently oozie.service.CallbackService.base.url in oozie-default.xml is set
> to use
> [oozie.base.url/callback|
> https://github.com/apache/oozie/blob/7c404ad0ea4c61e90e8c86015de25ef196168c29/core/src/main/resources/oozie-default.xml#L1639]
> which defaults to
> [http://localhost:8080/oozie|https://github.com/apache/oozie/blob/7c404ad0ea4c61e90e8c86015de25ef196168c29/core/src/main/resources/oozie-default.xml#L105].
> As a result, actions finished are not detected by default. (Jetty listens on
> HTTP port 11000, see [port in oozie-default.xml |
> https://github.com/apache/oozie/blob/7c404ad0ea4c61e90e8c86015de25ef196168c29/core/src/main/resources/oozie-default.xml#L2535])
> {{oozie.http.hostname}} and {{oozie.http.port}} could be used in
> {{oozie.base.url}}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Assigned] (OOZIE-2829) Improve sharelib upload to accept multiple source folders
[
https://issues.apache.org/jira/browse/OOZIE-2829?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Attila Sasvari reassigned OOZIE-2829:
-
Assignee: Attila Sasvari
> Improve sharelib upload to accept multiple source folders
> -
>
> Key: OOZIE-2829
> URL: https://issues.apache.org/jira/browse/OOZIE-2829
> Project: Oozie
> Issue Type: Bug
> Components: tools
>Reporter: Peter Cseh
>Assignee: Attila Sasvari
>
> Right now sharelib can be created via {{sharelib create -fs FS_URI -locallib
> SHARED_LIBRARY}} where the SHARED_LIBRARY can be a tarbal or a folder.
> It would be nice to have the possibility to define additional folders to be
> uploaded into the sharelib, so the users don't have to copy or link the files
> together on their machine.
> The syntax could be something like -additional-lib
> sharelibName=/path/to/source/;/path/to/some/file,sharelibName2=/path/to/some/folder
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (OOZIE-2850) Fix default callback notifications
[
https://issues.apache.org/jira/browse/OOZIE-2850?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Attila Sasvari updated OOZIE-2850:
--
Attachment: OOZIE-2850-06.patch
[~gezapeti] thanks for reviewing and testing it on a secure cluster. tab
removed in the new patch
(would be nice to improve test-patch to show here the location)
> Fix default callback notifications
> --
>
> Key: OOZIE-2850
> URL: https://issues.apache.org/jira/browse/OOZIE-2850
> Project: Oozie
> Issue Type: Bug
>Reporter: Attila Sasvari
>Assignee: Attila Sasvari
>Priority: Trivial
> Attachments: OOZIE-2850-01.patch, OOZIE-2850-02.patch,
> OOZIE-2850-03.patch, OOZIE-2850-04.patch, OOZIE-2850-05.patch,
> OOZIE-2850-06.patch
>
>
> Currently oozie.service.CallbackService.base.url in oozie-default.xml is set
> to use
> [oozie.base.url/callback|
> https://github.com/apache/oozie/blob/7c404ad0ea4c61e90e8c86015de25ef196168c29/core/src/main/resources/oozie-default.xml#L1639]
> which defaults to
> [http://localhost:8080/oozie|https://github.com/apache/oozie/blob/7c404ad0ea4c61e90e8c86015de25ef196168c29/core/src/main/resources/oozie-default.xml#L105].
> As a result, actions finished are not detected by default. (Jetty listens on
> HTTP port 11000, see [port in oozie-default.xml |
> https://github.com/apache/oozie/blob/7c404ad0ea4c61e90e8c86015de25ef196168c29/core/src/main/resources/oozie-default.xml#L2535])
> {{oozie.http.hostname}} and {{oozie.http.port}} could be used in
> {{oozie.base.url}}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (OOZIE-2850) Fix default callback notifications
[
https://issues.apache.org/jira/browse/OOZIE-2850?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Attila Sasvari updated OOZIE-2850:
--
Attachment: OOZIE-2850-05.patch
[~pbacsko] thanks for the review
> Fix default callback notifications
> --
>
> Key: OOZIE-2850
> URL: https://issues.apache.org/jira/browse/OOZIE-2850
> Project: Oozie
> Issue Type: Bug
>Reporter: Attila Sasvari
>Assignee: Attila Sasvari
>Priority: Trivial
> Attachments: OOZIE-2850-01.patch, OOZIE-2850-02.patch,
> OOZIE-2850-03.patch, OOZIE-2850-04.patch, OOZIE-2850-05.patch
>
>
> Currently oozie.service.CallbackService.base.url in oozie-default.xml is set
> to use
> [oozie.base.url/callback|
> https://github.com/apache/oozie/blob/7c404ad0ea4c61e90e8c86015de25ef196168c29/core/src/main/resources/oozie-default.xml#L1639]
> which defaults to
> [http://localhost:8080/oozie|https://github.com/apache/oozie/blob/7c404ad0ea4c61e90e8c86015de25ef196168c29/core/src/main/resources/oozie-default.xml#L105].
> As a result, actions finished are not detected by default. (Jetty listens on
> HTTP port 11000, see [port in oozie-default.xml |
> https://github.com/apache/oozie/blob/7c404ad0ea4c61e90e8c86015de25ef196168c29/core/src/main/resources/oozie-default.xml#L2535])
> {{oozie.http.hostname}} and {{oozie.http.port}} could be used in
> {{oozie.base.url}}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Created] (OOZIE-2857) oozie-trunk-precommit-build tries to delete non-existing downloads directory
Attila Sasvari created OOZIE-2857:
-
Summary: oozie-trunk-precommit-build tries to delete non-existing
downloads directory
Key: OOZIE-2857
URL: https://issues.apache.org/jira/browse/OOZIE-2857
Project: Oozie
Issue Type: Improvement
Reporter: Attila Sasvari
Pre-commit build logs contain errors like the following
{code}
[oozie-trunk-precommit-build] $ /bin/bash /tmp/hudson4474802453375480488.sh
rm: cannot remove 'distro/downloads/*': No such file or directory
rm: invalid option -- '.'
Try 'rm --help' for more information.
{code}
As Tomcat is not downloaded anymore, oozie-trunk-precommit-build jenkins job
should be updated.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Created] (OOZIE-2856) Show in JIRA comment exactly which patch was tested by jenkins job
Attila Sasvari created OOZIE-2856:
-
Summary: Show in JIRA comment exactly which patch was tested by
jenkins job
Key: OOZIE-2856
URL: https://issues.apache.org/jira/browse/OOZIE-2856
Project: Oozie
Issue Type: Improvement
Reporter: Attila Sasvari
Priority: Minor
Currently you cannot see in the JIRA comments which patch was applied
{code}
Testing JIRA OOZIE-2850
Cleaning local git workspace
+1 PATCH_APPLIES
+1 CLEAN
-1 RAW_PATCH_ANALYSIS
. +1 the patch does not introduce any @author tags
{code}
test-patch could updated so that it also writes out the exact patchfile's name.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (OOZIE-2850) Fix default callback notifications
[
https://issues.apache.org/jira/browse/OOZIE-2850?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Attila Sasvari updated OOZIE-2850:
--
Attachment: OOZIE-2850-04.patch
rebased patch
> Fix default callback notifications
> --
>
> Key: OOZIE-2850
> URL: https://issues.apache.org/jira/browse/OOZIE-2850
> Project: Oozie
> Issue Type: Bug
>Reporter: Attila Sasvari
>Assignee: Attila Sasvari
>Priority: Trivial
> Attachments: OOZIE-2850-01.patch, OOZIE-2850-02.patch,
> OOZIE-2850-03.patch, OOZIE-2850-04.patch
>
>
> Currently oozie.service.CallbackService.base.url in oozie-default.xml is set
> to use
> [oozie.base.url/callback|
> https://github.com/apache/oozie/blob/7c404ad0ea4c61e90e8c86015de25ef196168c29/core/src/main/resources/oozie-default.xml#L1639]
> which defaults to
> [http://localhost:8080/oozie|https://github.com/apache/oozie/blob/7c404ad0ea4c61e90e8c86015de25ef196168c29/core/src/main/resources/oozie-default.xml#L105].
> As a result, actions finished are not detected by default. (Jetty listens on
> HTTP port 11000, see [port in oozie-default.xml |
> https://github.com/apache/oozie/blob/7c404ad0ea4c61e90e8c86015de25ef196168c29/core/src/main/resources/oozie-default.xml#L2535])
> {{oozie.http.hostname}} and {{oozie.http.port}} could be used in
> {{oozie.base.url}}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (OOZIE-2849) Fix build warnings when assembling directories
[
https://issues.apache.org/jira/browse/OOZIE-2849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15954766#comment-15954766
]
Attila Sasvari commented on OOZIE-2849:
---
[~gezapeti] can you take a look? thanks!
> Fix build warnings when assembling directories
> --
>
> Key: OOZIE-2849
> URL: https://issues.apache.org/jira/browse/OOZIE-2849
> Project: Oozie
> Issue Type: Sub-task
>Reporter: Attila Sasvari
>Assignee: Attila Sasvari
>Priority: Trivial
> Attachments: OOZIE-2849-01.patch, OOZIE-2849-02.patch
>
>
> When building oozie we can observe a lot of warning
> {code}
> [WARNING] Assembly file:
> /Users/asasvari/workspace/apache/oozie_dup/tools/target/oozie-tools-4.4.0-SNAPSHOT-tools
> is not a regular file (it may be a directory). It cannot be attached to the
> project build for installation or deployment.
> {code}
> This behaviour can be explained by
> https://issues.apache.org/jira/browse/MASSEMBLY-628
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (OOZIE-2849) Fix build warnings when assembling directories
[
https://issues.apache.org/jira/browse/OOZIE-2849?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Attila Sasvari updated OOZIE-2849:
--
Attachment: OOZIE-2849-02.patch
[~rkanter] you are right, I should have not removed the {{dir}} format in
client.xml (oozie script only got into the tar.gz). Added it back. What about
the others do we need sharelib and examples in dir format? (I always used the
sharelib and examples as tar.gz files)
> Fix build warnings when assembling directories
> --
>
> Key: OOZIE-2849
> URL: https://issues.apache.org/jira/browse/OOZIE-2849
> Project: Oozie
> Issue Type: Sub-task
>Reporter: Attila Sasvari
>Assignee: Attila Sasvari
>Priority: Trivial
> Attachments: OOZIE-2849-01.patch, OOZIE-2849-02.patch
>
>
> When building oozie we can observe a lot of warning
> {code}
> [WARNING] Assembly file:
> /Users/asasvari/workspace/apache/oozie_dup/tools/target/oozie-tools-4.4.0-SNAPSHOT-tools
> is not a regular file (it may be a directory). It cannot be attached to the
> project build for installation or deployment.
> {code}
> This behaviour can be explained by
> https://issues.apache.org/jira/browse/MASSEMBLY-628
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (OOZIE-2850) Fix default callback notifications
[
https://issues.apache.org/jira/browse/OOZIE-2850?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15954619#comment-15954619
]
Attila Sasvari commented on OOZIE-2850:
---
[~pbacsko], [~abhishekbafna] can you take a look? Thanks!
> Fix default callback notifications
> --
>
> Key: OOZIE-2850
> URL: https://issues.apache.org/jira/browse/OOZIE-2850
> Project: Oozie
> Issue Type: Bug
>Reporter: Attila Sasvari
>Assignee: Attila Sasvari
>Priority: Trivial
> Attachments: OOZIE-2850-01.patch, OOZIE-2850-02.patch,
> OOZIE-2850-03.patch
>
>
> Currently oozie.service.CallbackService.base.url in oozie-default.xml is set
> to use
> [oozie.base.url/callback|
> https://github.com/apache/oozie/blob/7c404ad0ea4c61e90e8c86015de25ef196168c29/core/src/main/resources/oozie-default.xml#L1639]
> which defaults to
> [http://localhost:8080/oozie|https://github.com/apache/oozie/blob/7c404ad0ea4c61e90e8c86015de25ef196168c29/core/src/main/resources/oozie-default.xml#L105].
> As a result, actions finished are not detected by default. (Jetty listens on
> HTTP port 11000, see [port in oozie-default.xml |
> https://github.com/apache/oozie/blob/7c404ad0ea4c61e90e8c86015de25ef196168c29/core/src/main/resources/oozie-default.xml#L2535])
> {{oozie.http.hostname}} and {{oozie.http.port}} could be used in
> {{oozie.base.url}}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (OOZIE-2850) Fix default callback notifications
[
https://issues.apache.org/jira/browse/OOZIE-2850?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Attila Sasvari updated OOZIE-2850:
--
Attachment: OOZIE-2850-03.patch
fix rat warning - add license to new test class
> Fix default callback notifications
> --
>
> Key: OOZIE-2850
> URL: https://issues.apache.org/jira/browse/OOZIE-2850
> Project: Oozie
> Issue Type: Bug
>Reporter: Attila Sasvari
>Assignee: Attila Sasvari
>Priority: Trivial
> Attachments: OOZIE-2850-01.patch, OOZIE-2850-02.patch,
> OOZIE-2850-03.patch
>
>
> Currently oozie.service.CallbackService.base.url in oozie-default.xml is set
> to use
> [oozie.base.url/callback|
> https://github.com/apache/oozie/blob/7c404ad0ea4c61e90e8c86015de25ef196168c29/core/src/main/resources/oozie-default.xml#L1639]
> which defaults to
> [http://localhost:8080/oozie|https://github.com/apache/oozie/blob/7c404ad0ea4c61e90e8c86015de25ef196168c29/core/src/main/resources/oozie-default.xml#L105].
> As a result, actions finished are not detected by default. (Jetty listens on
> HTTP port 11000, see [port in oozie-default.xml |
> https://github.com/apache/oozie/blob/7c404ad0ea4c61e90e8c86015de25ef196168c29/core/src/main/resources/oozie-default.xml#L2535])
> {{oozie.http.hostname}} and {{oozie.http.port}} could be used in
> {{oozie.base.url}}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (OOZIE-2850) Fix default callback notifications
[
https://issues.apache.org/jira/browse/OOZIE-2850?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Attila Sasvari updated OOZIE-2850:
--
Attachment: OOZIE-2850-02.patch
fix findbugs warnings
> Fix default callback notifications
> --
>
> Key: OOZIE-2850
> URL: https://issues.apache.org/jira/browse/OOZIE-2850
> Project: Oozie
> Issue Type: Bug
>Reporter: Attila Sasvari
>Assignee: Attila Sasvari
>Priority: Trivial
> Attachments: OOZIE-2850-01.patch, OOZIE-2850-02.patch
>
>
> Currently oozie.service.CallbackService.base.url in oozie-default.xml is set
> to use
> [oozie.base.url/callback|
> https://github.com/apache/oozie/blob/7c404ad0ea4c61e90e8c86015de25ef196168c29/core/src/main/resources/oozie-default.xml#L1639]
> which defaults to
> [http://localhost:8080/oozie|https://github.com/apache/oozie/blob/7c404ad0ea4c61e90e8c86015de25ef196168c29/core/src/main/resources/oozie-default.xml#L105].
> As a result, actions finished are not detected by default. (Jetty listens on
> HTTP port 11000, see [port in oozie-default.xml |
> https://github.com/apache/oozie/blob/7c404ad0ea4c61e90e8c86015de25ef196168c29/core/src/main/resources/oozie-default.xml#L2535])
> {{oozie.http.hostname}} and {{oozie.http.port}} could be used in
> {{oozie.base.url}}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Created] (OOZIE-2853) Printing out Sqoop Configuration Properties may reveal passwords
Attila Sasvari created OOZIE-2853:
-
Summary: Printing out Sqoop Configuration Properties may reveal
passwords
Key: OOZIE-2853
URL: https://issues.apache.org/jira/browse/OOZIE-2853
Project: Oozie
Issue Type: Bug
Reporter: Attila Sasvari
Sample from Oozie launcher's log:
{code}
Sqoop Configuration Properties:
...
oozie.sqoop.args.5=--password
oozie.sqoop.args.6=examplPassword
{code}
[SqoopMain|https://github.com/apache/oozie/blob/branch-4.3/sharelib/sqoop/src/main/java/org/apache/oozie/action/hadoop/SqoopMain.java#L156]
prints out hadoop configuration that may contain sensitive information.
Note: Sqoop command arguments are redacted.
{code}
Sqoop command arguments :
...
--password
{code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (OOZIE-2850) Fix default callback notifications
[
https://issues.apache.org/jira/browse/OOZIE-2850?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Attila Sasvari updated OOZIE-2850:
--
Attachment: (was: OOZIE-2850-01.patch)
> Fix default callback notifications
> --
>
> Key: OOZIE-2850
> URL: https://issues.apache.org/jira/browse/OOZIE-2850
> Project: Oozie
> Issue Type: Bug
>Reporter: Attila Sasvari
>Assignee: Attila Sasvari
>Priority: Trivial
> Attachments: OOZIE-2850-01.patch
>
>
> Currently oozie.service.CallbackService.base.url in oozie-default.xml is set
> to use
> [oozie.base.url/callback|
> https://github.com/apache/oozie/blob/7c404ad0ea4c61e90e8c86015de25ef196168c29/core/src/main/resources/oozie-default.xml#L1639]
> which defaults to
> [http://localhost:8080/oozie|https://github.com/apache/oozie/blob/7c404ad0ea4c61e90e8c86015de25ef196168c29/core/src/main/resources/oozie-default.xml#L105].
> As a result, actions finished are not detected by default. (Jetty listens on
> HTTP port 11000, see [port in oozie-default.xml |
> https://github.com/apache/oozie/blob/7c404ad0ea4c61e90e8c86015de25ef196168c29/core/src/main/resources/oozie-default.xml#L2535])
> {{oozie.http.hostname}} and {{oozie.http.port}} could be used in
> {{oozie.base.url}}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (OOZIE-2850) Fix default callback notifications
[
https://issues.apache.org/jira/browse/OOZIE-2850?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Attila Sasvari updated OOZIE-2850:
--
Attachment: OOZIE-2850-01.patch
> Fix default callback notifications
> --
>
> Key: OOZIE-2850
> URL: https://issues.apache.org/jira/browse/OOZIE-2850
> Project: Oozie
> Issue Type: Bug
>Reporter: Attila Sasvari
>Assignee: Attila Sasvari
>Priority: Trivial
> Attachments: OOZIE-2850-01.patch
>
>
> Currently oozie.service.CallbackService.base.url in oozie-default.xml is set
> to use
> [oozie.base.url/callback|
> https://github.com/apache/oozie/blob/7c404ad0ea4c61e90e8c86015de25ef196168c29/core/src/main/resources/oozie-default.xml#L1639]
> which defaults to
> [http://localhost:8080/oozie|https://github.com/apache/oozie/blob/7c404ad0ea4c61e90e8c86015de25ef196168c29/core/src/main/resources/oozie-default.xml#L105].
> As a result, actions finished are not detected by default. (Jetty listens on
> HTTP port 11000, see [port in oozie-default.xml |
> https://github.com/apache/oozie/blob/7c404ad0ea4c61e90e8c86015de25ef196168c29/core/src/main/resources/oozie-default.xml#L2535])
> {{oozie.http.hostname}} and {{oozie.http.port}} could be used in
> {{oozie.base.url}}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (OOZIE-2850) Fix default callback notifications
[
https://issues.apache.org/jira/browse/OOZIE-2850?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15952705#comment-15952705
]
Attila Sasvari commented on OOZIE-2850:
---
[~rkanter] I see your point but currently there is no reference to
{{oozie.service.CallbackService.base.url}} in the code ({{git grep
oozie.service.CallbackService.base.url}} returns only oozie-default.xml).
This part (callback handling in secure mode) is handled in
{{server/src/main/java/org/apache/oozie/server/guice/ConstraintSecurityHandlerProvider.java}};
the idea was to not limit callback servlet endpoint to HTTPS requests. Here I
have just noticed a bug. Currently if HTTPS is enabled, a call to
{{/oozie/callback?id=008-170318223842826-oozie-asas-W&status=SUCCEEDED}}
via HTTP will be rejected with {{HTTP ERROR: 403}}
{code}
Problem accessing /oozie/versions. Reason:
!Secure
{code}
The problem is the root context path (/oozie) is not added as prefix to the
unsecure contraint mapping when
[{{setPathSpec()}}|https://github.com/apache/oozie/blob/e3590b420a0914d959f6cd555185ef04640c5801/server/src/main/java/org/apache/oozie/server/guice/ConstraintSecurityHandlerProvider.java#L32]
is called on the ConstraintMapping object.
In the latest patch I changed the server code so that it reads context path
from oozie-default (from oozie.base.url).
> Fix default callback notifications
> --
>
> Key: OOZIE-2850
> URL: https://issues.apache.org/jira/browse/OOZIE-2850
> Project: Oozie
> Issue Type: Bug
>Reporter: Attila Sasvari
>Assignee: Attila Sasvari
>Priority: Trivial
> Attachments: OOZIE-2850-01.patch
>
>
> Currently oozie.service.CallbackService.base.url in oozie-default.xml is set
> to use
> [oozie.base.url/callback|
> https://github.com/apache/oozie/blob/7c404ad0ea4c61e90e8c86015de25ef196168c29/core/src/main/resources/oozie-default.xml#L1639]
> which defaults to
> [http://localhost:8080/oozie|https://github.com/apache/oozie/blob/7c404ad0ea4c61e90e8c86015de25ef196168c29/core/src/main/resources/oozie-default.xml#L105].
> As a result, actions finished are not detected by default. (Jetty listens on
> HTTP port 11000, see [port in oozie-default.xml |
> https://github.com/apache/oozie/blob/7c404ad0ea4c61e90e8c86015de25ef196168c29/core/src/main/resources/oozie-default.xml#L2535])
> {{oozie.http.hostname}} and {{oozie.http.port}} could be used in
> {{oozie.base.url}}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (OOZIE-2850) Fix default callback notifications
[
https://issues.apache.org/jira/browse/OOZIE-2850?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Attila Sasvari updated OOZIE-2850:
--
Attachment: OOZIE-2850-01.patch
> Fix default callback notifications
> --
>
> Key: OOZIE-2850
> URL: https://issues.apache.org/jira/browse/OOZIE-2850
> Project: Oozie
> Issue Type: Bug
>Reporter: Attila Sasvari
>Assignee: Attila Sasvari
>Priority: Trivial
> Attachments: OOZIE-2850-01.patch
>
>
> Currently oozie.service.CallbackService.base.url in oozie-default.xml is set
> to use
> [oozie.base.url/callback|
> https://github.com/apache/oozie/blob/7c404ad0ea4c61e90e8c86015de25ef196168c29/core/src/main/resources/oozie-default.xml#L1639]
> which defaults to
> [http://localhost:8080/oozie|https://github.com/apache/oozie/blob/7c404ad0ea4c61e90e8c86015de25ef196168c29/core/src/main/resources/oozie-default.xml#L105].
> As a result, actions finished are not detected by default. (Jetty listens on
> HTTP port 11000, see [port in oozie-default.xml |
> https://github.com/apache/oozie/blob/7c404ad0ea4c61e90e8c86015de25ef196168c29/core/src/main/resources/oozie-default.xml#L2535])
> {{oozie.http.hostname}} and {{oozie.http.port}} could be used in
> {{oozie.base.url}}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (OOZIE-2387) Oozie is Unable to handle Spaces in file/archive tag.
[
https://issues.apache.org/jira/browse/OOZIE-2387?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Attila Sasvari updated OOZIE-2387:
--
Attachment: OOZIE-2387-03.patch
removed change in pom.xml
> Oozie is Unable to handle Spaces in file/archive tag.
> -
>
> Key: OOZIE-2387
> URL: https://issues.apache.org/jira/browse/OOZIE-2387
> Project: Oozie
> Issue Type: Bug
> Components: action
>Affects Versions: 4.1.0, 4.2.0
> Environment: Hadoop 2.6,Oozie 4.1, Hortonworks2.2
>Reporter: shiv pratap singh
>Assignee: Attila Sasvari
> Attachments: OOZIE-2387-01.patch, OOZIE-2387-02.patch,
> OOZIE-2387-03.patch
>
>
> Oozie file tag is unable to handle white spaces and other special characters.
> Error -
> Caused by: java.lang.NullPointerException
> at
> org.apache.oozie.action.hadoop.JavaActionExecutor.addToCache(JavaActionExecutor.java:529)
> at
> org.apache.oozie.action.hadoop.JavaActionExecutor.setLibFilesArchives(JavaActionExecutor.java:678)
> at
> org.apache.oozie.action.hadoop.JavaActionExecutor.submitLauncher(JavaActionExecutor.java:884)
> Sample XML File --
>
> xmlns:ns5="uri:oozie:shell-action:0.2"
> xmlns:ns4="uri:oozie:workflow:0.4"
> xmlns:ns3="uri:oozie:sqoop-action:0.2"
> xmlns:ns2="uri:oozie:hive-action:0.3" name="test-shell-space">
>
>
>
> ${wf_hadoop_resourceManager}
> ${wf_hadoop_nameNode}
> shell 1.sh
> /user/map-dev/test-case-dir/shell/script/shell
> 1.sh
>
>
>
>
>
> shell-decision job failed
>
>
>
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (OOZIE-2833) when using uber mode the regex pattern used in the extractHeapSizeMB method does not allow heap sizes specified in bytes.
[ https://issues.apache.org/jira/browse/OOZIE-2833?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15950741#comment-15950741 ] Attila Sasvari commented on OOZIE-2833: --- [~pbacsko] can you take a look? thanks. > when using uber mode the regex pattern used in the extractHeapSizeMB method > does not allow heap sizes specified in bytes. > - > > Key: OOZIE-2833 > URL: https://issues.apache.org/jira/browse/OOZIE-2833 > Project: Oozie > Issue Type: Bug >Reporter: Thomas Scott >Assignee: Attila Sasvari >Priority: Minor > Attachments: OOZIE-2833.01.patch, OOZIE-2833.02.patch > > > The extractHeapSizeMB method used here: > https://github.com/apache/oozie/blob/9c0516608bdd0787b1fac3a69ccfe9d36cbb2d15/core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java#L401 > Uses the following pattern: > -Xmx(([0-9]+)[mMgG]) > This requires that the -Xmx configuration option end in "m" or "g". However, > it is valid to configure this option in bytes without the "m" or "g" and this > will not be picked up. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (OOZIE-2818) Can't overwrite oozie.action.max.output.data on a per-workflow basis
[
https://issues.apache.org/jira/browse/OOZIE-2818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15950740#comment-15950740
]
Attila Sasvari commented on OOZIE-2818:
---
[~abhishekbafna] can you take a look?
> Can't overwrite oozie.action.max.output.data on a per-workflow basis
>
>
> Key: OOZIE-2818
> URL: https://issues.apache.org/jira/browse/OOZIE-2818
> Project: Oozie
> Issue Type: Bug
>Reporter: Peter Cseh
>Assignee: Attila Sasvari
> Attachments: OOZIE-2818-01.patch
>
>
> It's not possible to raise the limit of the output on a per-action basis.
> With the workflow:
> {quote}
>
>
>
> Action failed, error
> message\[$\{wf:errorMessage(wf:lastErrorNode())}]
>
>
>
> $\{jobTracker}
> $\{nameNode}
>
>
> oozie.action.max.output.data
> 10
>
>
> oozie.launcher.oozie.action.max.output.data
> 10
>
>
> /user/admin/large_output.sh
> 10
> /user/admin/large_output.sh#large_output.sh
>
>
>
>
>
>
>
> {quote}
> In the localized files in the job.xml the value of this property gets
> overwritten:
> {quote}
> # cat
> /yarn/nm/usercache/admin/appcache/application_1489145190857_0001/container_1489145190857_0001_01_02/job.xml
> |grep max.outp
> oozie.launcher.oozie.action.max.output.data10programaticallyjob.xml
> oozie.action.max.output.data2048programaticallyjob.xml
> # cat
> /yarn/nm/usercache/admin/appcache/application_1489145190857_0001/container_1489145190857_0001_01_02/action.xml
> |grep max.outp
> oozie.launcher.oozie.action.max.output.data10programatically
> oozie.action.max.output.data10programatically
> {quote}
> I could not find a reason for this behavior, so I think it's a bug.
> The value is read from
> [ConfigurationService|https://github.com/apache/oozie/blob/master/core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java#L175]
> and set in
> [createLauncherConf|https://github.com/apache/oozie/blob/master/core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java#L986]
> in JavaActionExecutor. There could be a check to see if it's already set.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (OOZIE-2848) Override sharelib.tgz in distro when rebuilding Oozie
[
https://issues.apache.org/jira/browse/OOZIE-2848?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15950739#comment-15950739
]
Attila Sasvari commented on OOZIE-2848:
---
[~pbacsko] can you have a look?
> Override sharelib.tgz in distro when rebuilding Oozie
> -
>
> Key: OOZIE-2848
> URL: https://issues.apache.org/jira/browse/OOZIE-2848
> Project: Oozie
> Issue Type: Improvement
>Reporter: Attila Sasvari
>Assignee: Attila Sasvari
>Priority: Trivial
> Attachments: OOZIE-2848-01.patch
>
>
> Building Oozie with {{mvn package assembly:single -Dfindbugs.skip=true
> -DskipTests -DjavaVersion=1.8 -DtargetVersion=1.7 -Puber}} twice in a row
> results in the following error:
> {code}
> [ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-assembly-plugin:2.2.1:single (default-cli) on
> project oozie-distro: Failed to create assembly: Error creating assembly
> archive distro: Problem copying files :
> /Users/asasvari/workspace/apache/oozie_dup/distro/target/oozie-4.4.0-SNAPSHOT-distro/oozie-4.4.0-SNAPSHOT/oozie-sharelib-4.4.0-SNAPSHOT.tar.gz
> (Permission denied) -> [Help 1]
> {code}
> We can override the target like it is done for Oozie Client and TAR.GZ-s.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (OOZIE-2387) Oozie is Unable to handle Spaces in file/archive tag.
[
https://issues.apache.org/jira/browse/OOZIE-2387?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Attila Sasvari updated OOZIE-2387:
--
Attachment: OOZIE-2387-02.patch
[~rkanter] thanks for the review, you are right. It would make sense to do it
inside {{addToCache()}}. Added test.
> Oozie is Unable to handle Spaces in file/archive tag.
> -
>
> Key: OOZIE-2387
> URL: https://issues.apache.org/jira/browse/OOZIE-2387
> Project: Oozie
> Issue Type: Bug
> Components: action
>Affects Versions: 4.1.0, 4.2.0
> Environment: Hadoop 2.6,Oozie 4.1, Hortonworks2.2
>Reporter: shiv pratap singh
>Assignee: Attila Sasvari
> Attachments: OOZIE-2387-01.patch, OOZIE-2387-02.patch
>
>
> Oozie file tag is unable to handle white spaces and other special characters.
> Error -
> Caused by: java.lang.NullPointerException
> at
> org.apache.oozie.action.hadoop.JavaActionExecutor.addToCache(JavaActionExecutor.java:529)
> at
> org.apache.oozie.action.hadoop.JavaActionExecutor.setLibFilesArchives(JavaActionExecutor.java:678)
> at
> org.apache.oozie.action.hadoop.JavaActionExecutor.submitLauncher(JavaActionExecutor.java:884)
> Sample XML File --
>
> xmlns:ns5="uri:oozie:shell-action:0.2"
> xmlns:ns4="uri:oozie:workflow:0.4"
> xmlns:ns3="uri:oozie:sqoop-action:0.2"
> xmlns:ns2="uri:oozie:hive-action:0.3" name="test-shell-space">
>
>
>
> ${wf_hadoop_resourceManager}
> ${wf_hadoop_nameNode}
> shell 1.sh
> /user/map-dev/test-case-dir/shell/script/shell
> 1.sh
>
>
>
>
>
> shell-decision job failed
>
>
>
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (OOZIE-2850) Fix default callback notifications
[
https://issues.apache.org/jira/browse/OOZIE-2850?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15950503#comment-15950503
]
Attila Sasvari commented on OOZIE-2850:
---
[~rkanter] thanks for the explanation. I tested that if I start a shell action
(shell action in the examples), it gets into SUCCEEDED state in 10 minutes
instead of the real runtime that is just some seconds. I will spend some more
time in the debugger to double check my assumption.
Yeah {{oozie.base.url} should be fixed and
{{oozie.service.CallbackService.base.url}} should be removed.
> Fix default callback notifications
> --
>
> Key: OOZIE-2850
> URL: https://issues.apache.org/jira/browse/OOZIE-2850
> Project: Oozie
> Issue Type: Bug
>Reporter: Attila Sasvari
>Assignee: Attila Sasvari
>Priority: Trivial
>
> Currently oozie.service.CallbackService.base.url in oozie-default.xml is set
> to use
> [oozie.base.url/callback|
> https://github.com/apache/oozie/blob/7c404ad0ea4c61e90e8c86015de25ef196168c29/core/src/main/resources/oozie-default.xml#L1639]
> which defaults to
> [http://localhost:8080/oozie|https://github.com/apache/oozie/blob/7c404ad0ea4c61e90e8c86015de25ef196168c29/core/src/main/resources/oozie-default.xml#L105].
> As a result, actions finished are not detected by default. (Jetty listens on
> HTTP port 11000, see [port in oozie-default.xml |
> https://github.com/apache/oozie/blob/7c404ad0ea4c61e90e8c86015de25ef196168c29/core/src/main/resources/oozie-default.xml#L2535])
> {{oozie.http.hostname}} and {{oozie.http.port}} could be used in
> {{oozie.base.url}}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Assigned] (OOZIE-2850) Fix default callback notifications
[
https://issues.apache.org/jira/browse/OOZIE-2850?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Attila Sasvari reassigned OOZIE-2850:
-
Assignee: Attila Sasvari
> Fix default callback notifications
> --
>
> Key: OOZIE-2850
> URL: https://issues.apache.org/jira/browse/OOZIE-2850
> Project: Oozie
> Issue Type: Bug
>Reporter: Attila Sasvari
>Assignee: Attila Sasvari
>Priority: Trivial
>
> Currently oozie.service.CallbackService.base.url in oozie-default.xml is set
> to use
> [oozie.base.url/callback|
> https://github.com/apache/oozie/blob/7c404ad0ea4c61e90e8c86015de25ef196168c29/core/src/main/resources/oozie-default.xml#L1639]
> which defaults to
> [http://localhost:8080/oozie|https://github.com/apache/oozie/blob/7c404ad0ea4c61e90e8c86015de25ef196168c29/core/src/main/resources/oozie-default.xml#L105].
> As a result, actions finished are not detected by default. (Jetty listens on
> HTTP port 11000, see [port in oozie-default.xml |
> https://github.com/apache/oozie/blob/7c404ad0ea4c61e90e8c86015de25ef196168c29/core/src/main/resources/oozie-default.xml#L2535])
> {{oozie.http.hostname}} and {{oozie.http.port}} could be used in
> {{oozie.base.url}}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (OOZIE-2850) Fix default callback notifications
[
https://issues.apache.org/jira/browse/OOZIE-2850?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15948926#comment-15948926
]
Attila Sasvari commented on OOZIE-2850:
---
OOZIE-2648 removed the call to {{injectCallback(context, actionConf);}} in
{{JavaActionExecutor}}, see [this |
https://github.com/apache/oozie/blob/88fc6e554ca7e73dba89d40aa87b97bfb5a3e881/core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java#L1054],
this is probably we get no notifications when an action finishes.
[~abhishekbafna] can you agree that's why a shell action does not send back
notification to the Oozie server when it finished?
In the worfklow log you can see
{code}
2017-03-30 13:53:32,203 INFO WorkflowNotificationXCommand:520 - SERVER[]
USER[-] GROUP[-] TOKEN[-] APP[-] JOB[004-170330133327713-oozie-asas-W]
ACTION[004-170330133327713-oozie-asas-W@shell-node] No Notification URL is
defined. Therefore nothing to notify for job
004-170330133327713-oozie-asas-W@shell-node
{code}
Now I manually invoke
http://localhost:11000/oozie/callback?id=005-170330133327713-oozie-asas-W@shell-node&status=SUCCEEDED
to notify Oozie.
> Fix default callback notifications
> --
>
> Key: OOZIE-2850
> URL: https://issues.apache.org/jira/browse/OOZIE-2850
> Project: Oozie
> Issue Type: Bug
>Reporter: Attila Sasvari
>Priority: Trivial
>
> Currently oozie.service.CallbackService.base.url in oozie-default.xml is set
> to use
> [oozie.base.url/callback|
> https://github.com/apache/oozie/blob/7c404ad0ea4c61e90e8c86015de25ef196168c29/core/src/main/resources/oozie-default.xml#L1639]
> which defaults to [ http://localhost:8080/oozie |
> https://github.com/apache/oozie/blob/7c404ad0ea4c61e90e8c86015de25ef196168c29/core/src/main/resources/oozie-default.xml#L105].
> As a result, actions finished are not detected by default. (Jetty listens on
> HTTP port 11000, see [port in oozie-default.xml |
> https://github.com/apache/oozie/blob/7c404ad0ea4c61e90e8c86015de25ef196168c29/core/src/main/resources/oozie-default.xml#L2535])
> {{oozie.http.hostname}} and {{oozie.http.port}} could be used in
> {{oozie.base.url}}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (OOZIE-2850) Fix default callback notifications
[
https://issues.apache.org/jira/browse/OOZIE-2850?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15948886#comment-15948886
]
Attila Sasvari commented on OOZIE-2850:
---
Hmm, this might be an invalid one as "oozie.service.CallbackService.base.url"
is not referenced in the code anymore.
Callback handling is set by [org.apache.oozie.server.ServletMapper |
https://github.com/apache/oozie/blob/master/server/src/main/java/org/apache/oozie/server/ServletMapper.java#L63].
Still, callback notification is not sent back when an action has been finished.
10 minutes needed to detect that my shell action had been finished.
> Fix default callback notifications
> --
>
> Key: OOZIE-2850
> URL: https://issues.apache.org/jira/browse/OOZIE-2850
> Project: Oozie
> Issue Type: Bug
>Reporter: Attila Sasvari
>Priority: Trivial
>
> Currently oozie.service.CallbackService.base.url in oozie-default.xml is set
> to use
> [oozie.base.url/callback|
> https://github.com/apache/oozie/blob/7c404ad0ea4c61e90e8c86015de25ef196168c29/core/src/main/resources/oozie-default.xml#L1639]
> which defaults to [ http://localhost:8080/oozie |
> https://github.com/apache/oozie/blob/7c404ad0ea4c61e90e8c86015de25ef196168c29/core/src/main/resources/oozie-default.xml#L105].
> As a result, actions finished are not detected by default. (Jetty listens on
> HTTP port 11000, see [port in oozie-default.xml |
> https://github.com/apache/oozie/blob/7c404ad0ea4c61e90e8c86015de25ef196168c29/core/src/main/resources/oozie-default.xml#L2535])
> {{oozie.http.hostname}} and {{oozie.http.port}} could be used in
> {{oozie.base.url}}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Created] (OOZIE-2850) Fix default callback notifications
Attila Sasvari created OOZIE-2850:
-
Summary: Fix default callback notifications
Key: OOZIE-2850
URL: https://issues.apache.org/jira/browse/OOZIE-2850
Project: Oozie
Issue Type: Bug
Reporter: Attila Sasvari
Priority: Trivial
Currently oozie.service.CallbackService.base.url in oozie-default.xml is set
to use
[oozie.base.url/callback|
https://github.com/apache/oozie/blob/7c404ad0ea4c61e90e8c86015de25ef196168c29/core/src/main/resources/oozie-default.xml#L1639]
which defaults to [ http://localhost:8080/oozie |
https://github.com/apache/oozie/blob/7c404ad0ea4c61e90e8c86015de25ef196168c29/core/src/main/resources/oozie-default.xml#L105].
As a result, actions finished are not detected by default. (Jetty listens on
HTTP port 11000, see [port in oozie-default.xml |
https://github.com/apache/oozie/blob/7c404ad0ea4c61e90e8c86015de25ef196168c29/core/src/main/resources/oozie-default.xml#L2535])
{{oozie.http.hostname}} and {{oozie.http.port}} could be used in
{{oozie.base.url}}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (OOZIE-2387) Oozie is Unable to handle Spaces in file/archive tag.
[
https://issues.apache.org/jira/browse/OOZIE-2387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15947620#comment-15947620
]
Attila Sasvari commented on OOZIE-2387:
---
Regarding HDFS filenames: there is an open ticket to restrict filenames:
https://issues.apache.org/jira/browse/HDFS-240
> Oozie is Unable to handle Spaces in file/archive tag.
> -
>
> Key: OOZIE-2387
> URL: https://issues.apache.org/jira/browse/OOZIE-2387
> Project: Oozie
> Issue Type: Bug
> Components: action
>Affects Versions: 4.1.0, 4.2.0
> Environment: Hadoop 2.6,Oozie 4.1, Hortonworks2.2
>Reporter: shiv pratap singh
>Assignee: Attila Sasvari
> Attachments: OOZIE-2387-01.patch
>
>
> Oozie file tag is unable to handle white spaces and other special characters.
> Error -
> Caused by: java.lang.NullPointerException
> at
> org.apache.oozie.action.hadoop.JavaActionExecutor.addToCache(JavaActionExecutor.java:529)
> at
> org.apache.oozie.action.hadoop.JavaActionExecutor.setLibFilesArchives(JavaActionExecutor.java:678)
> at
> org.apache.oozie.action.hadoop.JavaActionExecutor.submitLauncher(JavaActionExecutor.java:884)
> Sample XML File --
>
> xmlns:ns5="uri:oozie:shell-action:0.2"
> xmlns:ns4="uri:oozie:workflow:0.4"
> xmlns:ns3="uri:oozie:sqoop-action:0.2"
> xmlns:ns2="uri:oozie:hive-action:0.3" name="test-shell-space">
>
>
>
> ${wf_hadoop_resourceManager}
> ${wf_hadoop_nameNode}
> shell 1.sh
> /user/map-dev/test-case-dir/shell/script/shell
> 1.sh
>
>
>
>
>
> shell-decision job failed
>
>
>
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (OOZIE-2387) Oozie is Unable to handle Spaces in file/archive tag.
[
https://issues.apache.org/jira/browse/OOZIE-2387?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Attila Sasvari updated OOZIE-2387:
--
Attachment: OOZIE-2387-01.patch
> Oozie is Unable to handle Spaces in file/archive tag.
> -
>
> Key: OOZIE-2387
> URL: https://issues.apache.org/jira/browse/OOZIE-2387
> Project: Oozie
> Issue Type: Bug
> Components: action
>Affects Versions: 4.1.0, 4.2.0
> Environment: Hadoop 2.6,Oozie 4.1, Hortonworks2.2
>Reporter: shiv pratap singh
>Assignee: Attila Sasvari
> Attachments: OOZIE-2387-01.patch
>
>
> Oozie file tag is unable to handle white spaces and other special characters.
> Error -
> Caused by: java.lang.NullPointerException
> at
> org.apache.oozie.action.hadoop.JavaActionExecutor.addToCache(JavaActionExecutor.java:529)
> at
> org.apache.oozie.action.hadoop.JavaActionExecutor.setLibFilesArchives(JavaActionExecutor.java:678)
> at
> org.apache.oozie.action.hadoop.JavaActionExecutor.submitLauncher(JavaActionExecutor.java:884)
> Sample XML File --
>
> xmlns:ns5="uri:oozie:shell-action:0.2"
> xmlns:ns4="uri:oozie:workflow:0.4"
> xmlns:ns3="uri:oozie:sqoop-action:0.2"
> xmlns:ns2="uri:oozie:hive-action:0.3" name="test-shell-space">
>
>
>
> ${wf_hadoop_resourceManager}
> ${wf_hadoop_nameNode}
> shell 1.sh
> /user/map-dev/test-case-dir/shell/script/shell
> 1.sh
>
>
>
>
>
> shell-decision job failed
>
>
>
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (OOZIE-2387) Oozie is Unable to handle Spaces in file/archive tag.
[
https://issues.apache.org/jira/browse/OOZIE-2387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15947560#comment-15947560
]
Attila Sasvari commented on OOZIE-2387:
---
It is the {{new URI(filepath)}} call that results in an exception in
{{addToCache()}} in {{JavaActionExecutor}}.
> Oozie is Unable to handle Spaces in file/archive tag.
> -
>
> Key: OOZIE-2387
> URL: https://issues.apache.org/jira/browse/OOZIE-2387
> Project: Oozie
> Issue Type: Bug
> Components: action
>Affects Versions: 4.1.0, 4.2.0
> Environment: Hadoop 2.6,Oozie 4.1, Hortonworks2.2
>Reporter: shiv pratap singh
>Assignee: Attila Sasvari
>
> Oozie file tag is unable to handle white spaces and other special characters.
> Error -
> Caused by: java.lang.NullPointerException
> at
> org.apache.oozie.action.hadoop.JavaActionExecutor.addToCache(JavaActionExecutor.java:529)
> at
> org.apache.oozie.action.hadoop.JavaActionExecutor.setLibFilesArchives(JavaActionExecutor.java:678)
> at
> org.apache.oozie.action.hadoop.JavaActionExecutor.submitLauncher(JavaActionExecutor.java:884)
> Sample XML File --
>
> xmlns:ns5="uri:oozie:shell-action:0.2"
> xmlns:ns4="uri:oozie:workflow:0.4"
> xmlns:ns3="uri:oozie:sqoop-action:0.2"
> xmlns:ns2="uri:oozie:hive-action:0.3" name="test-shell-space">
>
>
>
> ${wf_hadoop_resourceManager}
> ${wf_hadoop_nameNode}
> shell 1.sh
> /user/map-dev/test-case-dir/shell/script/shell
> 1.sh
>
>
>
>
>
> shell-decision job failed
>
>
>
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Assigned] (OOZIE-2387) Oozie is Unable to handle Spaces in file/archive tag.
[
https://issues.apache.org/jira/browse/OOZIE-2387?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Attila Sasvari reassigned OOZIE-2387:
-
Assignee: Attila Sasvari
> Oozie is Unable to handle Spaces in file/archive tag.
> -
>
> Key: OOZIE-2387
> URL: https://issues.apache.org/jira/browse/OOZIE-2387
> Project: Oozie
> Issue Type: Bug
> Components: action
>Affects Versions: 4.1.0, 4.2.0
> Environment: Hadoop 2.6,Oozie 4.1, Hortonworks2.2
>Reporter: shiv pratap singh
>Assignee: Attila Sasvari
>
> Oozie file tag is unable to handle white spaces and other special characters.
> Error -
> Caused by: java.lang.NullPointerException
> at
> org.apache.oozie.action.hadoop.JavaActionExecutor.addToCache(JavaActionExecutor.java:529)
> at
> org.apache.oozie.action.hadoop.JavaActionExecutor.setLibFilesArchives(JavaActionExecutor.java:678)
> at
> org.apache.oozie.action.hadoop.JavaActionExecutor.submitLauncher(JavaActionExecutor.java:884)
> Sample XML File --
>
> xmlns:ns5="uri:oozie:shell-action:0.2"
> xmlns:ns4="uri:oozie:workflow:0.4"
> xmlns:ns3="uri:oozie:sqoop-action:0.2"
> xmlns:ns2="uri:oozie:hive-action:0.3" name="test-shell-space">
>
>
>
> ${wf_hadoop_resourceManager}
> ${wf_hadoop_nameNode}
> shell 1.sh
> /user/map-dev/test-case-dir/shell/script/shell
> 1.sh
>
>
>
>
>
> shell-decision job failed
>
>
>
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (OOZIE-2818) Can't overwrite oozie.action.max.output.data on a per-workflow basis
[
https://issues.apache.org/jira/browse/OOZIE-2818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15946789#comment-15946789
]
Attila Sasvari commented on OOZIE-2818:
---
[~gezapeti] can you take a look?
> Can't overwrite oozie.action.max.output.data on a per-workflow basis
>
>
> Key: OOZIE-2818
> URL: https://issues.apache.org/jira/browse/OOZIE-2818
> Project: Oozie
> Issue Type: Bug
>Reporter: Peter Cseh
>Assignee: Attila Sasvari
> Attachments: OOZIE-2818-01.patch
>
>
> It's not possible to raise the limit of the output on a per-action basis.
> With the workflow:
> {quote}
>
>
>
> Action failed, error
> message\[$\{wf:errorMessage(wf:lastErrorNode())}]
>
>
>
> $\{jobTracker}
> $\{nameNode}
>
>
> oozie.action.max.output.data
> 10
>
>
> oozie.launcher.oozie.action.max.output.data
> 10
>
>
> /user/admin/large_output.sh
> 10
> /user/admin/large_output.sh#large_output.sh
>
>
>
>
>
>
>
> {quote}
> In the localized files in the job.xml the value of this property gets
> overwritten:
> {quote}
> # cat
> /yarn/nm/usercache/admin/appcache/application_1489145190857_0001/container_1489145190857_0001_01_02/job.xml
> |grep max.outp
> oozie.launcher.oozie.action.max.output.data10programaticallyjob.xml
> oozie.action.max.output.data2048programaticallyjob.xml
> # cat
> /yarn/nm/usercache/admin/appcache/application_1489145190857_0001/container_1489145190857_0001_01_02/action.xml
> |grep max.outp
> oozie.launcher.oozie.action.max.output.data10programatically
> oozie.action.max.output.data10programatically
> {quote}
> I could not find a reason for this behavior, so I think it's a bug.
> The value is read from
> [ConfigurationService|https://github.com/apache/oozie/blob/master/core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java#L175]
> and set in
> [createLauncherConf|https://github.com/apache/oozie/blob/master/core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java#L986]
> in JavaActionExecutor. There could be a check to see if it's already set.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (OOZIE-2818) Can't overwrite oozie.action.max.output.data on a per-workflow basis
[
https://issues.apache.org/jira/browse/OOZIE-2818?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Attila Sasvari updated OOZIE-2818:
--
Attachment: OOZIE-2818-01.patch
> Can't overwrite oozie.action.max.output.data on a per-workflow basis
>
>
> Key: OOZIE-2818
> URL: https://issues.apache.org/jira/browse/OOZIE-2818
> Project: Oozie
> Issue Type: Bug
>Reporter: Peter Cseh
>Assignee: Attila Sasvari
> Attachments: OOZIE-2818-01.patch
>
>
> It's not possible to raise the limit of the output on a per-action basis.
> With the workflow:
> {quote}
>
>
>
> Action failed, error
> message\[$\{wf:errorMessage(wf:lastErrorNode())}]
>
>
>
> $\{jobTracker}
> $\{nameNode}
>
>
> oozie.action.max.output.data
> 10
>
>
> oozie.launcher.oozie.action.max.output.data
> 10
>
>
> /user/admin/large_output.sh
> 10
> /user/admin/large_output.sh#large_output.sh
>
>
>
>
>
>
>
> {quote}
> In the localized files in the job.xml the value of this property gets
> overwritten:
> {quote}
> # cat
> /yarn/nm/usercache/admin/appcache/application_1489145190857_0001/container_1489145190857_0001_01_02/job.xml
> |grep max.outp
> oozie.launcher.oozie.action.max.output.data10programaticallyjob.xml
> oozie.action.max.output.data2048programaticallyjob.xml
> # cat
> /yarn/nm/usercache/admin/appcache/application_1489145190857_0001/container_1489145190857_0001_01_02/action.xml
> |grep max.outp
> oozie.launcher.oozie.action.max.output.data10programatically
> oozie.action.max.output.data10programatically
> {quote}
> I could not find a reason for this behavior, so I think it's a bug.
> The value is read from
> [ConfigurationService|https://github.com/apache/oozie/blob/master/core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java#L175]
> and set in
> [createLauncherConf|https://github.com/apache/oozie/blob/master/core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java#L986]
> in JavaActionExecutor. There could be a check to see if it's already set.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (OOZIE-2849) Fix build warnings when assembling directories
[
https://issues.apache.org/jira/browse/OOZIE-2849?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Attila Sasvari updated OOZIE-2849:
--
Attachment: OOZIE-2849-01.patch
> Fix build warnings when assembling directories
> --
>
> Key: OOZIE-2849
> URL: https://issues.apache.org/jira/browse/OOZIE-2849
> Project: Oozie
> Issue Type: Improvement
>Reporter: Attila Sasvari
>Assignee: Attila Sasvari
>Priority: Trivial
> Attachments: OOZIE-2849-01.patch
>
>
> When building oozie we can observe a lot of warning
> {code}
> [WARNING] Assembly file:
> /Users/asasvari/workspace/apache/oozie_dup/tools/target/oozie-tools-4.4.0-SNAPSHOT-tools
> is not a regular file (it may be a directory). It cannot be attached to the
> project build for installation or deployment.
> {code}
> This behaviour can be explained by
> https://issues.apache.org/jira/browse/MASSEMBLY-628
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (OOZIE-2849) Fix build warnings when assembling directories
[
https://issues.apache.org/jira/browse/OOZIE-2849?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Attila Sasvari updated OOZIE-2849:
--
Attachment: (was: OOZIE-2849-01.patch)
> Fix build warnings when assembling directories
> --
>
> Key: OOZIE-2849
> URL: https://issues.apache.org/jira/browse/OOZIE-2849
> Project: Oozie
> Issue Type: Improvement
>Reporter: Attila Sasvari
>Assignee: Attila Sasvari
>Priority: Trivial
> Attachments: OOZIE-2849-01.patch
>
>
> When building oozie we can observe a lot of warning
> {code}
> [WARNING] Assembly file:
> /Users/asasvari/workspace/apache/oozie_dup/tools/target/oozie-tools-4.4.0-SNAPSHOT-tools
> is not a regular file (it may be a directory). It cannot be attached to the
> project build for installation or deployment.
> {code}
> This behaviour can be explained by
> https://issues.apache.org/jira/browse/MASSEMBLY-628
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (OOZIE-2849) Fix build warnings when assembling directories
[
https://issues.apache.org/jira/browse/OOZIE-2849?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Attila Sasvari updated OOZIE-2849:
--
Attachment: (was: OOZIE-2849-01.patch)
> Fix build warnings when assembling directories
> --
>
> Key: OOZIE-2849
> URL: https://issues.apache.org/jira/browse/OOZIE-2849
> Project: Oozie
> Issue Type: Improvement
>Reporter: Attila Sasvari
>Assignee: Attila Sasvari
>Priority: Trivial
> Attachments: OOZIE-2849-01.patch
>
>
> When building oozie we can observe a lot of warning
> {code}
> [WARNING] Assembly file:
> /Users/asasvari/workspace/apache/oozie_dup/tools/target/oozie-tools-4.4.0-SNAPSHOT-tools
> is not a regular file (it may be a directory). It cannot be attached to the
> project build for installation or deployment.
> {code}
> This behaviour can be explained by
> https://issues.apache.org/jira/browse/MASSEMBLY-628
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (OOZIE-2849) Fix build warnings when assembling directories
[
https://issues.apache.org/jira/browse/OOZIE-2849?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Attila Sasvari updated OOZIE-2849:
--
Attachment: OOZIE-2849-01.patch
> Fix build warnings when assembling directories
> --
>
> Key: OOZIE-2849
> URL: https://issues.apache.org/jira/browse/OOZIE-2849
> Project: Oozie
> Issue Type: Improvement
>Reporter: Attila Sasvari
>Assignee: Attila Sasvari
>Priority: Trivial
> Attachments: OOZIE-2849-01.patch, OOZIE-2849-01.patch
>
>
> When building oozie we can observe a lot of warning
> {code}
> [WARNING] Assembly file:
> /Users/asasvari/workspace/apache/oozie_dup/tools/target/oozie-tools-4.4.0-SNAPSHOT-tools
> is not a regular file (it may be a directory). It cannot be attached to the
> project build for installation or deployment.
> {code}
> This behaviour can be explained by
> https://issues.apache.org/jira/browse/MASSEMBLY-628
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (OOZIE-2849) Fix build warnings when assembling directories
[
https://issues.apache.org/jira/browse/OOZIE-2849?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Attila Sasvari updated OOZIE-2849:
--
Attachment: OOZIE-2849-01.patch
> Fix build warnings when assembling directories
> --
>
> Key: OOZIE-2849
> URL: https://issues.apache.org/jira/browse/OOZIE-2849
> Project: Oozie
> Issue Type: Improvement
>Reporter: Attila Sasvari
>Assignee: Attila Sasvari
>Priority: Trivial
> Attachments: OOZIE-2849-01.patch, OOZIE-2849-01.patch
>
>
> When building oozie we can observe a lot of warning
> {code}
> [WARNING] Assembly file:
> /Users/asasvari/workspace/apache/oozie_dup/tools/target/oozie-tools-4.4.0-SNAPSHOT-tools
> is not a regular file (it may be a directory). It cannot be attached to the
> project build for installation or deployment.
> {code}
> This behaviour can be explained by
> https://issues.apache.org/jira/browse/MASSEMBLY-628
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Created] (OOZIE-2849) Fix build warnings when assembling directories
Attila Sasvari created OOZIE-2849:
-
Summary: Fix build warnings when assembling directories
Key: OOZIE-2849
URL: https://issues.apache.org/jira/browse/OOZIE-2849
Project: Oozie
Issue Type: Improvement
Reporter: Attila Sasvari
Assignee: Attila Sasvari
Priority: Trivial
When building oozie we can observe a lot of warning
{code}
[WARNING] Assembly file:
/Users/asasvari/workspace/apache/oozie_dup/tools/target/oozie-tools-4.4.0-SNAPSHOT-tools
is not a regular file (it may be a directory). It cannot be attached to the
project build for installation or deployment.
{code}
This behaviour can be explained by
https://issues.apache.org/jira/browse/MASSEMBLY-628
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (OOZIE-2848) Override sharelib.tgz in distro when rebuilding Oozie
[
https://issues.apache.org/jira/browse/OOZIE-2848?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Attila Sasvari updated OOZIE-2848:
--
Attachment: OOZIE-2848-01.patch
> Override sharelib.tgz in distro when rebuilding Oozie
> -
>
> Key: OOZIE-2848
> URL: https://issues.apache.org/jira/browse/OOZIE-2848
> Project: Oozie
> Issue Type: Improvement
>Reporter: Attila Sasvari
>Assignee: Attila Sasvari
>Priority: Trivial
> Attachments: OOZIE-2848-01.patch
>
>
> Building Oozie with {{mvn package assembly:single -Dfindbugs.skip=true
> -DskipTests -DjavaVersion=1.8 -DtargetVersion=1.7 -Puber}} twice in a row
> results in the following error:
> {code}
> [ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-assembly-plugin:2.2.1:single (default-cli) on
> project oozie-distro: Failed to create assembly: Error creating assembly
> archive distro: Problem copying files :
> /Users/asasvari/workspace/apache/oozie_dup/distro/target/oozie-4.4.0-SNAPSHOT-distro/oozie-4.4.0-SNAPSHOT/oozie-sharelib-4.4.0-SNAPSHOT.tar.gz
> (Permission denied) -> [Help 1]
> {code}
> We can override the target like it is done for Oozie Client and TAR.GZ-s.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Created] (OOZIE-2848) Override sharelib.tgz in distro when rebuilding Oozie
Attila Sasvari created OOZIE-2848:
-
Summary: Override sharelib.tgz in distro when rebuilding Oozie
Key: OOZIE-2848
URL: https://issues.apache.org/jira/browse/OOZIE-2848
Project: Oozie
Issue Type: Improvement
Reporter: Attila Sasvari
Assignee: Attila Sasvari
Priority: Trivial
Building Oozie with {{mvn package assembly:single -Dfindbugs.skip=true
-DskipTests -DjavaVersion=1.8 -DtargetVersion=1.7 -Puber}} twice in a row
results in the following error:
{code}
[ERROR] Failed to execute goal
org.apache.maven.plugins:maven-assembly-plugin:2.2.1:single (default-cli) on
project oozie-distro: Failed to create assembly: Error creating assembly
archive distro: Problem copying files :
/Users/asasvari/workspace/apache/oozie_dup/distro/target/oozie-4.4.0-SNAPSHOT-distro/oozie-4.4.0-SNAPSHOT/oozie-sharelib-4.4.0-SNAPSHOT.tar.gz
(Permission denied) -> [Help 1]
{code}
We can override the target like it is done for Oozie Client and TAR.GZ-s.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (OOZIE-2753) Update Docs for Jetty Tomcat changes
[ https://issues.apache.org/jira/browse/OOZIE-2753?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Attila Sasvari updated OOZIE-2753: -- Attachment: OOZIE-2753-06.patch > Update Docs for Jetty Tomcat changes > > > Key: OOZIE-2753 > URL: https://issues.apache.org/jira/browse/OOZIE-2753 > Project: Oozie > Issue Type: Sub-task > Components: docs >Affects Versions: 5.0.0 >Reporter: Robert Kanter >Assignee: Attila Sasvari > Fix For: 5.0.0 > > Attachments: OOZIE-2753-01.patch, OOZIE-2753-02.patch, > OOZIE-2753-03.patch, OOZIE-2753-04.patch, OOZIE-2753-05.patch, > OOZIE-2753-06.patch > > > Update the docs based on the changes for Jetty and Tomcat (OOZIE-2317 and > child JIRAs). -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (OOZIE-2753) Update Docs for Jetty Tomcat changes
[
https://issues.apache.org/jira/browse/OOZIE-2753?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15945841#comment-15945841
]
Attila Sasvari commented on OOZIE-2753:
---
[~abhishekbafna] please note {{git grep -i catalina}} still shows references at
various places, e.g
sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/TestPasswordMasker.java
sharelib/oozie/src/test/resources/instrumentation-os-env.json
sharelib/oozie/src/test/resources/instrumentation-system-properties.json
> Update Docs for Jetty Tomcat changes
>
>
> Key: OOZIE-2753
> URL: https://issues.apache.org/jira/browse/OOZIE-2753
> Project: Oozie
> Issue Type: Sub-task
> Components: docs
>Affects Versions: 5.0.0
>Reporter: Robert Kanter
>Assignee: Attila Sasvari
> Fix For: 5.0.0
>
> Attachments: OOZIE-2753-01.patch, OOZIE-2753-02.patch,
> OOZIE-2753-03.patch, OOZIE-2753-04.patch, OOZIE-2753-05.patch,
> OOZIE-2753-06.patch
>
>
> Update the docs based on the changes for Jetty and Tomcat (OOZIE-2317 and
> child JIRAs).
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (OOZIE-2753) Update Docs for Jetty Tomcat changes
[ https://issues.apache.org/jira/browse/OOZIE-2753?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Attila Sasvari updated OOZIE-2753: -- Attachment: OOZIE-2753-05.patch > Update Docs for Jetty Tomcat changes > > > Key: OOZIE-2753 > URL: https://issues.apache.org/jira/browse/OOZIE-2753 > Project: Oozie > Issue Type: Sub-task > Components: docs >Affects Versions: 5.0.0 >Reporter: Robert Kanter >Assignee: Attila Sasvari > Fix For: 5.0.0 > > Attachments: OOZIE-2753-01.patch, OOZIE-2753-02.patch, > OOZIE-2753-03.patch, OOZIE-2753-04.patch, OOZIE-2753-05.patch > > > Update the docs based on the changes for Jetty and Tomcat (OOZIE-2317 and > child JIRAs). -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (OOZIE-2753) Update Docs for Jetty Tomcat changes
[ https://issues.apache.org/jira/browse/OOZIE-2753?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15944809#comment-15944809 ] Attila Sasvari commented on OOZIE-2753: --- [~pbacsko] thanks. Earlier it contained rkanter, but I can change it to oozie. > Update Docs for Jetty Tomcat changes > > > Key: OOZIE-2753 > URL: https://issues.apache.org/jira/browse/OOZIE-2753 > Project: Oozie > Issue Type: Sub-task > Components: docs >Affects Versions: 5.0.0 >Reporter: Robert Kanter >Assignee: Attila Sasvari > Fix For: 5.0.0 > > Attachments: OOZIE-2753-01.patch, OOZIE-2753-02.patch, > OOZIE-2753-03.patch, OOZIE-2753-04.patch > > > Update the docs based on the changes for Jetty and Tomcat (OOZIE-2317 and > child JIRAs). -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (OOZIE-2753) Update Docs for Jetty Tomcat changes
[ https://issues.apache.org/jira/browse/OOZIE-2753?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Attila Sasvari updated OOZIE-2753: -- Attachment: OOZIE-2753-04.patch > Update Docs for Jetty Tomcat changes > > > Key: OOZIE-2753 > URL: https://issues.apache.org/jira/browse/OOZIE-2753 > Project: Oozie > Issue Type: Sub-task > Components: docs >Affects Versions: 5.0.0 >Reporter: Robert Kanter >Assignee: Attila Sasvari > Fix For: 5.0.0 > > Attachments: OOZIE-2753-01.patch, OOZIE-2753-02.patch, > OOZIE-2753-03.patch, OOZIE-2753-04.patch > > > Update the docs based on the changes for Jetty and Tomcat (OOZIE-2317 and > child JIRAs). -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (OOZIE-2753) Update Docs for Jetty Tomcat changes
[
https://issues.apache.org/jira/browse/OOZIE-2753?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15944051#comment-15944051
]
Attila Sasvari commented on OOZIE-2753:
---
[~abhishekbafna] there are things that are still set (unnecessary).
{code}
$ bin/oozie admin -oozie http://localhost:11000/oozie -osenv | egrep -e
"OOZIE|JETTY"
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in
[jar:file:/Users/asasvari/workspace/apache/oozie_dup/distro/target/oozie-4.4.0-SNAPSHOT-distro/oozie-4.4.0-SNAPSHOT/embedded-oozie-server/webapp/WEB-INF/lib/slf4j-log4j12-1.6.6.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in
[jar:file:/Users/asasvari/workspace/apache/oozie_dup/distro/target/oozie-4.4.0-SNAPSHOT-distro/oozie-4.4.0-SNAPSHOT/embedded-oozie-server/webapp/WEB-INF/lib/slf4j-simple-1.6.6.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
log4j:WARN No appenders could be found for logger
(org.apache.hadoop.security.authentication.client.KerberosAuthenticator).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more
info.
JETTY_OPTS : -Doozie.home.dir=/Users/asasvari/dev/oozie
-Doozie.config.dir=/Users/asasvari/dev/oozie/conf
-Doozie.log.dir=/Users/asasvari/dev/oozie/logs
-Doozie.data.dir=/Users/asasvari/dev/oozie/data
-Doozie.config.file=oozie-site.xml -Doozie.log4j.file=oozie-log4j.properties
-Doozie.log4j.reload=10 -Djava.library.path= -cp
/Users/asasvari/dev/oozie/embedded-oozie-server/*:/Users/asasvari/dev/oozie/embedded-oozie-server/dependency/*:/Users/asasvari/dev/oozie/lib/*:/Users/asasvari/dev/oozie/libtools/*:/Users/asasvari/dev/oozie/embedded-oozie-server
JETTY_OUT : /Users/asasvari/dev/oozie/logs/jetty.out
JETTY_PID_FILE : /Users/asasvari/dev/oozie/embedded-oozie-server/oozie.pid
OOZIE_ADMIN_PORT : 11001
OOZIE_BASE_URL : http://Budapests-MacBook-Pro-10.local:11000/oozie
OOZIE_CONFIG : /Users/asasvari/dev/oozie/conf
OOZIE_CONFIG_FILE : oozie-site.xml
OOZIE_DATA : /Users/asasvari/dev/oozie/data
OOZIE_HOME : /Users/asasvari/dev/oozie
OOZIE_HTTPS_KEYSTORE_FILE : /Users/asasvari/.keystore
OOZIE_HTTPS_KEYSTORE_PASS : *
OOZIE_HTTPS_PORT : 11443
OOZIE_HTTP_HOSTNAME : Budapests-MacBook-Pro-10.local
OOZIE_HTTP_PORT : 11000
OOZIE_INSTANCE_ID : Budapests-MacBook-Pro-10.local
OOZIE_LOG : /Users/asasvari/dev/oozie/logs
OOZIE_LOG4J_FILE : oozie-log4j.properties
OOZIE_LOG4J_RELOAD : 10
{code}
HTTP things and OOZIE_BASE_URL should be read from the oozie-site , and
OOZIE_ADMIN_PORT was only used for Tomcat.
> Update Docs for Jetty Tomcat changes
>
>
> Key: OOZIE-2753
> URL: https://issues.apache.org/jira/browse/OOZIE-2753
> Project: Oozie
> Issue Type: Sub-task
> Components: docs
>Affects Versions: 5.0.0
>Reporter: Robert Kanter
>Assignee: Attila Sasvari
> Fix For: 5.0.0
>
> Attachments: OOZIE-2753-01.patch, OOZIE-2753-02.patch,
> OOZIE-2753-03.patch
>
>
> Update the docs based on the changes for Jetty and Tomcat (OOZIE-2317 and
> child JIRAs).
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (OOZIE-2753) Update Docs for Jetty Tomcat changes
[ https://issues.apache.org/jira/browse/OOZIE-2753?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15943397#comment-15943397 ] Attila Sasvari commented on OOZIE-2753: --- Thanks. Created OOZIE-2836 last week, but I did not link it to this. > Update Docs for Jetty Tomcat changes > > > Key: OOZIE-2753 > URL: https://issues.apache.org/jira/browse/OOZIE-2753 > Project: Oozie > Issue Type: Sub-task > Components: docs >Affects Versions: 5.0.0 >Reporter: Robert Kanter >Assignee: Attila Sasvari > Fix For: 5.0.0 > > Attachments: OOZIE-2753-01.patch, OOZIE-2753-02.patch, > OOZIE-2753-03.patch > > > Update the docs based on the changes for Jetty and Tomcat (OOZIE-2317 and > child JIRAs). -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (OOZIE-2796) oozie.action.keep.action.dir not getting noticed
[
https://issues.apache.org/jira/browse/OOZIE-2796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15943163#comment-15943163
]
Attila Sasvari commented on OOZIE-2796:
---
[~pbacsko], [~gezapeti] can you take a look?
> oozie.action.keep.action.dir not getting noticed
>
>
> Key: OOZIE-2796
> URL: https://issues.apache.org/jira/browse/OOZIE-2796
> Project: Oozie
> Issue Type: Bug
>Affects Versions: 4.3.0
>Reporter: Andras Piros
>Assignee: Xiaobin Zheng
> Fix For: trunk
>
> Attachments: OOZIE-2796-1.patch, OOZIE-2796-2.patch,
> OOZIE-2796-3.patch, OOZIE-2796-4.patch, OOZIE-2796-5.patch
>
>
> When trying to add following configuration entry:
> {code:xml}
>
> oozie.action.keep.action.dir
> true
>
> {code}
> to the application level configuration of {{workflow.xml}}, the workflow
> directories are removed just after the workflow has finished.
> This should not be the case.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (OOZIE-2753) Update Docs for Jetty Tomcat changes
[ https://issues.apache.org/jira/browse/OOZIE-2753?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15942969#comment-15942969 ] Attila Sasvari commented on OOZIE-2753: --- [~pbacsko], [~abhishekbafna] can you take a look? thanks. > Update Docs for Jetty Tomcat changes > > > Key: OOZIE-2753 > URL: https://issues.apache.org/jira/browse/OOZIE-2753 > Project: Oozie > Issue Type: Sub-task > Components: docs >Affects Versions: 5.0.0 >Reporter: Robert Kanter >Assignee: Attila Sasvari > Fix For: 5.0.0 > > Attachments: OOZIE-2753-01.patch, OOZIE-2753-02.patch, > OOZIE-2753-03.patch > > > Update the docs based on the changes for Jetty and Tomcat (OOZIE-2317 and > child JIRAs). -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (OOZIE-2833) when using uber mode the regex pattern used in the extractHeapSizeMB method does not allow heap sizes specified in bytes.
[ https://issues.apache.org/jira/browse/OOZIE-2833?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15940478#comment-15940478 ] Attila Sasvari commented on OOZIE-2833: --- [~abhishekbafna], [~gezapeti] can you take a look? Thanks. > when using uber mode the regex pattern used in the extractHeapSizeMB method > does not allow heap sizes specified in bytes. > - > > Key: OOZIE-2833 > URL: https://issues.apache.org/jira/browse/OOZIE-2833 > Project: Oozie > Issue Type: Bug >Reporter: Thomas Scott >Assignee: Attila Sasvari >Priority: Minor > Attachments: OOZIE-2833.01.patch, OOZIE-2833.02.patch > > > The extractHeapSizeMB method used here: > https://github.com/apache/oozie/blob/9c0516608bdd0787b1fac3a69ccfe9d36cbb2d15/core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java#L401 > Uses the following pattern: > -Xmx(([0-9]+)[mMgG]) > This requires that the -Xmx configuration option end in "m" or "g". However, > it is valid to configure this option in bytes without the "m" or "g" and this > will not be picked up. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (OOZIE-2833) when using uber mode the regex pattern used in the extractHeapSizeMB method does not allow heap sizes specified in bytes.
[ https://issues.apache.org/jira/browse/OOZIE-2833?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Attila Sasvari updated OOZIE-2833: -- Attachment: OOZIE-2833.02.patch > when using uber mode the regex pattern used in the extractHeapSizeMB method > does not allow heap sizes specified in bytes. > - > > Key: OOZIE-2833 > URL: https://issues.apache.org/jira/browse/OOZIE-2833 > Project: Oozie > Issue Type: Bug >Reporter: Thomas Scott >Assignee: Attila Sasvari >Priority: Minor > Attachments: OOZIE-2833.01.patch, OOZIE-2833.02.patch > > > The extractHeapSizeMB method used here: > https://github.com/apache/oozie/blob/9c0516608bdd0787b1fac3a69ccfe9d36cbb2d15/core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java#L401 > Uses the following pattern: > -Xmx(([0-9]+)[mMgG]) > This requires that the -Xmx configuration option end in "m" or "g". However, > it is valid to configure this option in bytes without the "m" or "g" and this > will not be picked up. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (OOZIE-2833) when using uber mode the regex pattern used in the extractHeapSizeMB method does not allow heap sizes specified in bytes.
[ https://issues.apache.org/jira/browse/OOZIE-2833?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Attila Sasvari updated OOZIE-2833: -- Attachment: (was: OOZIE-2833.02.patch) > when using uber mode the regex pattern used in the extractHeapSizeMB method > does not allow heap sizes specified in bytes. > - > > Key: OOZIE-2833 > URL: https://issues.apache.org/jira/browse/OOZIE-2833 > Project: Oozie > Issue Type: Bug >Reporter: Thomas Scott >Assignee: Attila Sasvari >Priority: Minor > Attachments: OOZIE-2833.01.patch > > > The extractHeapSizeMB method used here: > https://github.com/apache/oozie/blob/9c0516608bdd0787b1fac3a69ccfe9d36cbb2d15/core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java#L401 > Uses the following pattern: > -Xmx(([0-9]+)[mMgG]) > This requires that the -Xmx configuration option end in "m" or "g". However, > it is valid to configure this option in bytes without the "m" or "g" and this > will not be picked up. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (OOZIE-2833) when using uber mode the regex pattern used in the extractHeapSizeMB method does not allow heap sizes specified in bytes.
[ https://issues.apache.org/jira/browse/OOZIE-2833?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Attila Sasvari updated OOZIE-2833: -- Attachment: OOZIE-2833.02.patch > when using uber mode the regex pattern used in the extractHeapSizeMB method > does not allow heap sizes specified in bytes. > - > > Key: OOZIE-2833 > URL: https://issues.apache.org/jira/browse/OOZIE-2833 > Project: Oozie > Issue Type: Bug >Reporter: Thomas Scott >Assignee: Attila Sasvari >Priority: Minor > Attachments: OOZIE-2833.01.patch, OOZIE-2833.02.patch > > > The extractHeapSizeMB method used here: > https://github.com/apache/oozie/blob/9c0516608bdd0787b1fac3a69ccfe9d36cbb2d15/core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java#L401 > Uses the following pattern: > -Xmx(([0-9]+)[mMgG]) > This requires that the -Xmx configuration option end in "m" or "g". However, > it is valid to configure this option in bytes without the "m" or "g" and this > will not be picked up. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (OOZIE-2833) when using uber mode the regex pattern used in the extractHeapSizeMB method does not allow heap sizes specified in bytes.
[ https://issues.apache.org/jira/browse/OOZIE-2833?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15940177#comment-15940177 ] Attila Sasvari commented on OOZIE-2833: --- [~zhengxb2005] thanks for the review! I see your point, I will upload a new patch soon. > when using uber mode the regex pattern used in the extractHeapSizeMB method > does not allow heap sizes specified in bytes. > - > > Key: OOZIE-2833 > URL: https://issues.apache.org/jira/browse/OOZIE-2833 > Project: Oozie > Issue Type: Bug >Reporter: Thomas Scott >Assignee: Attila Sasvari >Priority: Minor > Attachments: OOZIE-2833.01.patch > > > The extractHeapSizeMB method used here: > https://github.com/apache/oozie/blob/9c0516608bdd0787b1fac3a69ccfe9d36cbb2d15/core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java#L401 > Uses the following pattern: > -Xmx(([0-9]+)[mMgG]) > This requires that the -Xmx configuration option end in "m" or "g". However, > it is valid to configure this option in bytes without the "m" or "g" and this > will not be picked up. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (OOZIE-2838) TestClassUtils,TestJsonUtils,TestWritableUtils,TestXmlUtils shall not be an Oozie XTestCase
[ https://issues.apache.org/jira/browse/OOZIE-2838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15940170#comment-15940170 ] Attila Sasvari commented on OOZIE-2838: --- [~pbacsko] thanks a lot! > TestClassUtils,TestJsonUtils,TestWritableUtils,TestXmlUtils shall not be an > Oozie XTestCase > > > Key: OOZIE-2838 > URL: https://issues.apache.org/jira/browse/OOZIE-2838 > Project: Oozie > Issue Type: Improvement >Reporter: Attila Sasvari >Assignee: Attila Sasvari >Priority: Trivial > Fix For: 5.0.0 > > Attachments: OOZIE-2838.01.patch > > > TestClassUtils,TestIOUtils,TestWritableUtils,TestXmlUtils (like TestIOUtils > covered by OOZIE-2835) unnecessary extend Oozie custom XTestCase / TestCase > that are intended for more complex tests. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Assigned] (OOZIE-2818) Can't overwrite oozie.action.max.output.data on a per-workflow basis
[
https://issues.apache.org/jira/browse/OOZIE-2818?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Attila Sasvari reassigned OOZIE-2818:
-
Assignee: Attila Sasvari
> Can't overwrite oozie.action.max.output.data on a per-workflow basis
>
>
> Key: OOZIE-2818
> URL: https://issues.apache.org/jira/browse/OOZIE-2818
> Project: Oozie
> Issue Type: Bug
>Reporter: Peter Cseh
>Assignee: Attila Sasvari
>
> It's not possible to raise the limit of the output on a per-action basis.
> With the workflow:
> {quote}
>
>
>
> Action failed, error
> message\[$\{wf:errorMessage(wf:lastErrorNode())}]
>
>
>
> $\{jobTracker}
> $\{nameNode}
>
>
> oozie.action.max.output.data
> 10
>
>
> oozie.launcher.oozie.action.max.output.data
> 10
>
>
> /user/admin/large_output.sh
> 10
> /user/admin/large_output.sh#large_output.sh
>
>
>
>
>
>
>
> {quote}
> In the localized files in the job.xml the value of this property gets
> overwritten:
> {quote}
> # cat
> /yarn/nm/usercache/admin/appcache/application_1489145190857_0001/container_1489145190857_0001_01_02/job.xml
> |grep max.outp
> oozie.launcher.oozie.action.max.output.data10programaticallyjob.xml
> oozie.action.max.output.data2048programaticallyjob.xml
> # cat
> /yarn/nm/usercache/admin/appcache/application_1489145190857_0001/container_1489145190857_0001_01_02/action.xml
> |grep max.outp
> oozie.launcher.oozie.action.max.output.data10programatically
> oozie.action.max.output.data10programatically
> {quote}
> I could not find a reason for this behavior, so I think it's a bug.
> The value is read from
> [ConfigurationService|https://github.com/apache/oozie/blob/master/core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java#L175]
> and set in
> [createLauncherConf|https://github.com/apache/oozie/blob/master/core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java#L986]
> in JavaActionExecutor. There could be a check to see if it's already set.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (OOZIE-2838) TestClassUtils,TestJsonUtils,TestWritableUtils,TestXmlUtils shall not be an Oozie XTestCase
[ https://issues.apache.org/jira/browse/OOZIE-2838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15940104#comment-15940104 ] Attila Sasvari commented on OOZIE-2838: --- [~pbacsko] could you please take a look? thanks! > TestClassUtils,TestJsonUtils,TestWritableUtils,TestXmlUtils shall not be an > Oozie XTestCase > > > Key: OOZIE-2838 > URL: https://issues.apache.org/jira/browse/OOZIE-2838 > Project: Oozie > Issue Type: Improvement >Reporter: Attila Sasvari >Assignee: Attila Sasvari >Priority: Trivial > Attachments: OOZIE-2838.01.patch > > > TestClassUtils,TestIOUtils,TestWritableUtils,TestXmlUtils (like TestIOUtils > covered by OOZIE-2835) unnecessary extend Oozie custom XTestCase / TestCase > that are intended for more complex tests. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (OOZIE-2701) Oozie to support Multiple HCatalog URIs
[ https://issues.apache.org/jira/browse/OOZIE-2701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15940056#comment-15940056 ] Attila Sasvari commented on OOZIE-2701: --- +1 (non-binding) > Oozie to support Multiple HCatalog URIs > --- > > Key: OOZIE-2701 > URL: https://issues.apache.org/jira/browse/OOZIE-2701 > Project: Oozie > Issue Type: New Feature > Components: core >Reporter: Abhishek Bafna >Assignee: Abhishek Bafna > Fix For: 5.0.0 > > Attachments: OOZIE-2701-00.patch, OOZIE-2701-01.patch, > OOZIE-2701-02.patch, OOZIE-2701-03.patch, OOZIE-2701-04.patch, > OOZIE-2701-05.patch, OOZIE-2701-06.patch, OOZIE-2701-07.patch > > -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (OOZIE-2701) Oozie to support Multiple HCatalog URIs
[
https://issues.apache.org/jira/browse/OOZIE-2701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15940037#comment-15940037
]
Attila Sasvari commented on OOZIE-2701:
---
[~abhishekbafna] thanks a lot for the updated patch, in the regex
{code}
[\w+\.\-]
{code}
I guess + after \w is not needed here. Other than that, I don't have other
comments.
> Oozie to support Multiple HCatalog URIs
> ---
>
> Key: OOZIE-2701
> URL: https://issues.apache.org/jira/browse/OOZIE-2701
> Project: Oozie
> Issue Type: New Feature
> Components: core
>Reporter: Abhishek Bafna
>Assignee: Abhishek Bafna
> Fix For: 5.0.0
>
> Attachments: OOZIE-2701-00.patch, OOZIE-2701-01.patch,
> OOZIE-2701-02.patch, OOZIE-2701-03.patch, OOZIE-2701-04.patch,
> OOZIE-2701-05.patch, OOZIE-2701-06.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (OOZIE-2701) Oozie to support Multiple HCatalog URIs
[
https://issues.apache.org/jira/browse/OOZIE-2701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15939388#comment-15939388
]
Attila Sasvari commented on OOZIE-2701:
---
* Looking at
{code}
\[\w+\.\-\w+\]+ = Hostname or IP address.
\[/\w+=;\-\]* = partition information
{code}
these are character classes (that use other character classes like \w) .
Invalid hostnames would be also matched like "asdac."
* {{splitHCatUris}} creates a new Pattern object each time it is called, it
should be created only once (HCAT_URI_REGEX_CONFIG only set once at startup).
* I don't think URI's getScheme() should be tested (e.g. assertEquals("hcat",
uri.getScheme()) )
> Oozie to support Multiple HCatalog URIs
> ---
>
> Key: OOZIE-2701
> URL: https://issues.apache.org/jira/browse/OOZIE-2701
> Project: Oozie
> Issue Type: New Feature
> Components: core
>Reporter: Abhishek Bafna
>Assignee: Abhishek Bafna
> Fix For: 5.0.0
>
> Attachments: OOZIE-2701-00.patch, OOZIE-2701-01.patch,
> OOZIE-2701-02.patch, OOZIE-2701-03.patch, OOZIE-2701-04.patch,
> OOZIE-2701-05.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (OOZIE-2701) Oozie to support Multiple HCatalog URIs
[ https://issues.apache.org/jira/browse/OOZIE-2701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15938041#comment-15938041 ] Attila Sasvari commented on OOZIE-2701: --- [~abhishekbafna] you are right, thanks for the clarification. I overlooked. > Oozie to support Multiple HCatalog URIs > --- > > Key: OOZIE-2701 > URL: https://issues.apache.org/jira/browse/OOZIE-2701 > Project: Oozie > Issue Type: New Feature > Components: core >Reporter: Abhishek Bafna >Assignee: Abhishek Bafna > Fix For: 5.0.0 > > Attachments: OOZIE-2701-00.patch, OOZIE-2701-01.patch, > OOZIE-2701-02.patch, OOZIE-2701-03.patch, OOZIE-2701-04.patch, > OOZIE-2701-05.patch > > -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (OOZIE-2701) Oozie to support Multiple HCatalog URIs
[
https://issues.apache.org/jira/browse/OOZIE-2701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15937936#comment-15937936
]
Attila Sasvari commented on OOZIE-2701:
---
{code}
([a-z]+://[\w+\.\-\w+]+:\d+[,]*)+/\w+/\w+[/\w+=;\-]*
{code} can be rewrited as
{code}
([a-z]+://[\w+\.\-\w+]+:\d+[,]*)+/\w/\w+[/\w+=;\-]*
{code}
Note: x+x+ can be a really slow regexp. If you need something like it, just use
xx+ .
> Oozie to support Multiple HCatalog URIs
> ---
>
> Key: OOZIE-2701
> URL: https://issues.apache.org/jira/browse/OOZIE-2701
> Project: Oozie
> Issue Type: New Feature
> Components: core
>Reporter: Abhishek Bafna
>Assignee: Abhishek Bafna
> Fix For: 5.0.0
>
> Attachments: OOZIE-2701-00.patch, OOZIE-2701-01.patch,
> OOZIE-2701-02.patch, OOZIE-2701-03.patch, OOZIE-2701-04.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
