[jira] [Commented] (OOZIE-3071) Oozie 4.3 Spark sharelib ueses a different version of commons-lang3 than Spark 2.2.0
[ https://issues.apache.org/jira/browse/OOZIE-3071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17031509#comment-17031509 ] Andras Salamon commented on OOZIE-3071: --- [~matijhs] Thanks for the bump. (To repeat [~gezapeti]'s question): What will happen with Spark1 if we bump commons-lang3 version? It would be great if you could test it. We already have problems with the different requirements of spark-1 and spark-2. It's not too nice, but we already have spark-1 and spark-2 profiles, maybe we could try to update commons-lang3 and we could specify the old one in spark-1 if it's required. > Oozie 4.3 Spark sharelib ueses a different version of commons-lang3 than > Spark 2.2.0 > > > Key: OOZIE-3071 > URL: https://issues.apache.org/jira/browse/OOZIE-3071 > Project: Oozie > Issue Type: Bug > Components: core >Affects Versions: 4.3.0 >Reporter: Ran Gu >Assignee: Artem Ervits >Priority: Major > Labels: patch > Attachments: OOZIE-3071-1.patch, OOZIE-3071.patch > > > Currently Oozie 4.3.0 uses commons-lang3 version 3.3.2 in Spark sharelib. > Spark 2.2.0 uses commons-lang3 version 3.5. > This causes Oozie(/Spark) job failures on EMR-5.8.0 clusters. > Error message: > 17/08/22 00:22:43 ERROR ApplicationMaster: User class threw exception: > java.lang.IllegalArgumentException: Illegal pattern component: XXX > java.lang.IllegalArgumentException: Illegal pattern component: XXX > at > org.apache.commons.lang3.time.FastDatePrinter.parsePattern(FastDatePrinter.java:282) > at > org.apache.commons.lang3.time.FastDatePrinter.init(FastDatePrinter.java:149) > at > org.apache.commons.lang3.time.FastDatePrinter.(FastDatePrinter.java:142) > at > org.apache.commons.lang3.time.FastDateFormat.(FastDateFormat.java:384) > at > org.apache.commons.lang3.time.FastDateFormat.(FastDateFormat.java:369) > at > org.apache.commons.lang3.time.FastDateFormat$1.createInstance(FastDateFormat.java:91) > at > org.apache.commons.lang3.time.FastDateFormat$1.createInstance(FastDateFormat.java:88) > at org.apache.commons.lang3.time.FormatCache.getInstance(FormatCache.java:82) -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (OOZIE-3071) Oozie 4.3 Spark sharelib ueses a different version of commons-lang3 than Spark 2.2.0
[
https://issues.apache.org/jira/browse/OOZIE-3071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17031482#comment-17031482
]
Hadoop QA commented on OOZIE-3071:
--
Testing JIRA OOZIE-3071
Cleaning local git workspace
{color:red}-1{color} Patch failed to apply to head of branch
> Oozie 4.3 Spark sharelib ueses a different version of commons-lang3 than
> Spark 2.2.0
>
>
> Key: OOZIE-3071
> URL: https://issues.apache.org/jira/browse/OOZIE-3071
> Project: Oozie
> Issue Type: Bug
> Components: core
>Affects Versions: 4.3.0
>Reporter: Ran Gu
>Assignee: Artem Ervits
>Priority: Major
> Labels: patch
> Attachments: OOZIE-3071-1.patch, OOZIE-3071.patch
>
>
> Currently Oozie 4.3.0 uses commons-lang3 version 3.3.2 in Spark sharelib.
> Spark 2.2.0 uses commons-lang3 version 3.5.
> This causes Oozie(/Spark) job failures on EMR-5.8.0 clusters.
> Error message:
> 17/08/22 00:22:43 ERROR ApplicationMaster: User class threw exception:
> java.lang.IllegalArgumentException: Illegal pattern component: XXX
> java.lang.IllegalArgumentException: Illegal pattern component: XXX
> at
> org.apache.commons.lang3.time.FastDatePrinter.parsePattern(FastDatePrinter.java:282)
> at
> org.apache.commons.lang3.time.FastDatePrinter.init(FastDatePrinter.java:149)
> at
> org.apache.commons.lang3.time.FastDatePrinter.(FastDatePrinter.java:142)
> at
> org.apache.commons.lang3.time.FastDateFormat.(FastDateFormat.java:384)
> at
> org.apache.commons.lang3.time.FastDateFormat.(FastDateFormat.java:369)
> at
> org.apache.commons.lang3.time.FastDateFormat$1.createInstance(FastDateFormat.java:91)
> at
> org.apache.commons.lang3.time.FastDateFormat$1.createInstance(FastDateFormat.java:88)
> at org.apache.commons.lang3.time.FormatCache.getInstance(FormatCache.java:82)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (OOZIE-3071) Oozie 4.3 Spark sharelib ueses a different version of commons-lang3 than Spark 2.2.0
[ https://issues.apache.org/jira/browse/OOZIE-3071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17031480#comment-17031480 ] Hadoop QA commented on OOZIE-3071: -- PreCommit-OOZIE-Build started > Oozie 4.3 Spark sharelib ueses a different version of commons-lang3 than > Spark 2.2.0 > > > Key: OOZIE-3071 > URL: https://issues.apache.org/jira/browse/OOZIE-3071 > Project: Oozie > Issue Type: Bug > Components: core >Affects Versions: 4.3.0 >Reporter: Ran Gu >Assignee: Artem Ervits >Priority: Major > Labels: patch > Attachments: OOZIE-3071-1.patch, OOZIE-3071.patch > > > Currently Oozie 4.3.0 uses commons-lang3 version 3.3.2 in Spark sharelib. > Spark 2.2.0 uses commons-lang3 version 3.5. > This causes Oozie(/Spark) job failures on EMR-5.8.0 clusters. > Error message: > 17/08/22 00:22:43 ERROR ApplicationMaster: User class threw exception: > java.lang.IllegalArgumentException: Illegal pattern component: XXX > java.lang.IllegalArgumentException: Illegal pattern component: XXX > at > org.apache.commons.lang3.time.FastDatePrinter.parsePattern(FastDatePrinter.java:282) > at > org.apache.commons.lang3.time.FastDatePrinter.init(FastDatePrinter.java:149) > at > org.apache.commons.lang3.time.FastDatePrinter.(FastDatePrinter.java:142) > at > org.apache.commons.lang3.time.FastDateFormat.(FastDateFormat.java:384) > at > org.apache.commons.lang3.time.FastDateFormat.(FastDateFormat.java:369) > at > org.apache.commons.lang3.time.FastDateFormat$1.createInstance(FastDateFormat.java:91) > at > org.apache.commons.lang3.time.FastDateFormat$1.createInstance(FastDateFormat.java:88) > at org.apache.commons.lang3.time.FormatCache.getInstance(FormatCache.java:82) -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (OOZIE-3071) Oozie 4.3 Spark sharelib ueses a different version of commons-lang3 than Spark 2.2.0
[
https://issues.apache.org/jira/browse/OOZIE-3071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17031477#comment-17031477
]
Mate Juhasz commented on OOZIE-3071:
Bumping up the topic again... commons-lang3 3.3.2 does not support JDK 11, the
below error can be observed when running spark actions on Java 11.
{noformat}
Caused by: java.lang.NullPointerException
at
org.apache.commons.lang3.SystemUtils.isJavaVersionAtLeast(SystemUtils.java:1654)
{noformat}
commons-lang3:3.8.1 fixes it, but we should use the same 3.9 as in SPARK-30035,
which is the first release to support JDK9+.
> Oozie 4.3 Spark sharelib ueses a different version of commons-lang3 than
> Spark 2.2.0
>
>
> Key: OOZIE-3071
> URL: https://issues.apache.org/jira/browse/OOZIE-3071
> Project: Oozie
> Issue Type: Bug
> Components: core
>Affects Versions: 4.3.0
>Reporter: Ran Gu
>Assignee: Artem Ervits
>Priority: Major
> Labels: patch
> Attachments: OOZIE-3071-1.patch, OOZIE-3071.patch
>
>
> Currently Oozie 4.3.0 uses commons-lang3 version 3.3.2 in Spark sharelib.
> Spark 2.2.0 uses commons-lang3 version 3.5.
> This causes Oozie(/Spark) job failures on EMR-5.8.0 clusters.
> Error message:
> 17/08/22 00:22:43 ERROR ApplicationMaster: User class threw exception:
> java.lang.IllegalArgumentException: Illegal pattern component: XXX
> java.lang.IllegalArgumentException: Illegal pattern component: XXX
> at
> org.apache.commons.lang3.time.FastDatePrinter.parsePattern(FastDatePrinter.java:282)
> at
> org.apache.commons.lang3.time.FastDatePrinter.init(FastDatePrinter.java:149)
> at
> org.apache.commons.lang3.time.FastDatePrinter.(FastDatePrinter.java:142)
> at
> org.apache.commons.lang3.time.FastDateFormat.(FastDateFormat.java:384)
> at
> org.apache.commons.lang3.time.FastDateFormat.(FastDateFormat.java:369)
> at
> org.apache.commons.lang3.time.FastDateFormat$1.createInstance(FastDateFormat.java:91)
> at
> org.apache.commons.lang3.time.FastDateFormat$1.createInstance(FastDateFormat.java:88)
> at org.apache.commons.lang3.time.FormatCache.getInstance(FormatCache.java:82)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (OOZIE-3071) Oozie 4.3 Spark sharelib ueses a different version of commons-lang3 than Spark 2.2.0
[ https://issues.apache.org/jira/browse/OOZIE-3071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16279152#comment-16279152 ] Artem Ervits commented on OOZIE-3071: - [~rangu] no I meant 3.5, the concern was that 3.5 would break the Spark action for Spark 2.2 and only fix for 1.6. I see that branch 1.6 is on 3.3.2 https://github.com/apache/spark/blob/branch-1.6/pom.xml#L179 branch 2.0 is on 3.3.2 https://github.com/apache/spark/blob/branch-2.0/pom.xml#L173 branch 2.1 and 2.2 are on 3.5 https://github.com/apache/spark/blob/branch-2.1/pom.xml#L171 and https://github.com/apache/spark/blob/branch-2.2/pom.xml#L171 respectively so we either wait or take risk updating to 3.6. > Oozie 4.3 Spark sharelib ueses a different version of commons-lang3 than > Spark 2.2.0 > > > Key: OOZIE-3071 > URL: https://issues.apache.org/jira/browse/OOZIE-3071 > Project: Oozie > Issue Type: Bug > Components: core >Affects Versions: 4.3.0 >Reporter: Ran Gu >Assignee: Artem Ervits > Labels: patch > Attachments: OOZIE-3071-1.patch, OOZIE-3071.patch > > > Currently Oozie 4.3.0 uses commons-lang3 version 3.3.2 in Spark sharelib. > Spark 2.2.0 uses commons-lang3 version 3.5. > This causes Oozie(/Spark) job failures on EMR-5.8.0 clusters. > Error message: > 17/08/22 00:22:43 ERROR ApplicationMaster: User class threw exception: > java.lang.IllegalArgumentException: Illegal pattern component: XXX > java.lang.IllegalArgumentException: Illegal pattern component: XXX > at > org.apache.commons.lang3.time.FastDatePrinter.parsePattern(FastDatePrinter.java:282) > at > org.apache.commons.lang3.time.FastDatePrinter.init(FastDatePrinter.java:149) > at > org.apache.commons.lang3.time.FastDatePrinter.(FastDatePrinter.java:142) > at > org.apache.commons.lang3.time.FastDateFormat.(FastDateFormat.java:384) > at > org.apache.commons.lang3.time.FastDateFormat.(FastDateFormat.java:369) > at > org.apache.commons.lang3.time.FastDateFormat$1.createInstance(FastDateFormat.java:91) > at > org.apache.commons.lang3.time.FastDateFormat$1.createInstance(FastDateFormat.java:88) > at org.apache.commons.lang3.time.FormatCache.getInstance(FormatCache.java:82) -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (OOZIE-3071) Oozie 4.3 Spark sharelib ueses a different version of commons-lang3 than Spark 2.2.0
[ https://issues.apache.org/jira/browse/OOZIE-3071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16279034#comment-16279034 ] Ran Gu commented on OOZIE-3071: --- [~dbist13]Just to clarify you meant 3.6 not 3.5 right? The patch you attached is using 3.6 > Oozie 4.3 Spark sharelib ueses a different version of commons-lang3 than > Spark 2.2.0 > > > Key: OOZIE-3071 > URL: https://issues.apache.org/jira/browse/OOZIE-3071 > Project: Oozie > Issue Type: Bug > Components: core >Affects Versions: 4.3.0 >Reporter: Ran Gu > Labels: patch > Attachments: OOZIE-3071-1.patch, OOZIE-3071.patch > > > Currently Oozie 4.3.0 uses commons-lang3 version 3.3.2 in Spark sharelib. > Spark 2.2.0 uses commons-lang3 version 3.5. > This causes Oozie(/Spark) job failures on EMR-5.8.0 clusters. > Error message: > 17/08/22 00:22:43 ERROR ApplicationMaster: User class threw exception: > java.lang.IllegalArgumentException: Illegal pattern component: XXX > java.lang.IllegalArgumentException: Illegal pattern component: XXX > at > org.apache.commons.lang3.time.FastDatePrinter.parsePattern(FastDatePrinter.java:282) > at > org.apache.commons.lang3.time.FastDatePrinter.init(FastDatePrinter.java:149) > at > org.apache.commons.lang3.time.FastDatePrinter.(FastDatePrinter.java:142) > at > org.apache.commons.lang3.time.FastDateFormat.(FastDateFormat.java:384) > at > org.apache.commons.lang3.time.FastDateFormat.(FastDateFormat.java:369) > at > org.apache.commons.lang3.time.FastDateFormat$1.createInstance(FastDateFormat.java:91) > at > org.apache.commons.lang3.time.FastDateFormat$1.createInstance(FastDateFormat.java:88) > at org.apache.commons.lang3.time.FormatCache.getInstance(FormatCache.java:82) -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (OOZIE-3071) Oozie 4.3 Spark sharelib ueses a different version of commons-lang3 than Spark 2.2.0
[ https://issues.apache.org/jira/browse/OOZIE-3071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16278984#comment-16278984 ] Artem Ervits commented on OOZIE-3071: - [~rangu] it's really up to the community to decide and get +1s. [~gezapeti] what did you think about my comment on 3.5 being the version in both 1.6 and 2.2 branches? > Oozie 4.3 Spark sharelib ueses a different version of commons-lang3 than > Spark 2.2.0 > > > Key: OOZIE-3071 > URL: https://issues.apache.org/jira/browse/OOZIE-3071 > Project: Oozie > Issue Type: Bug > Components: core >Affects Versions: 4.3.0 >Reporter: Ran Gu > Labels: patch > Attachments: OOZIE-3071-1.patch, OOZIE-3071.patch > > > Currently Oozie 4.3.0 uses commons-lang3 version 3.3.2 in Spark sharelib. > Spark 2.2.0 uses commons-lang3 version 3.5. > This causes Oozie(/Spark) job failures on EMR-5.8.0 clusters. > Error message: > 17/08/22 00:22:43 ERROR ApplicationMaster: User class threw exception: > java.lang.IllegalArgumentException: Illegal pattern component: XXX > java.lang.IllegalArgumentException: Illegal pattern component: XXX > at > org.apache.commons.lang3.time.FastDatePrinter.parsePattern(FastDatePrinter.java:282) > at > org.apache.commons.lang3.time.FastDatePrinter.init(FastDatePrinter.java:149) > at > org.apache.commons.lang3.time.FastDatePrinter.(FastDatePrinter.java:142) > at > org.apache.commons.lang3.time.FastDateFormat.(FastDateFormat.java:384) > at > org.apache.commons.lang3.time.FastDateFormat.(FastDateFormat.java:369) > at > org.apache.commons.lang3.time.FastDateFormat$1.createInstance(FastDateFormat.java:91) > at > org.apache.commons.lang3.time.FastDateFormat$1.createInstance(FastDateFormat.java:88) > at org.apache.commons.lang3.time.FormatCache.getInstance(FormatCache.java:82) -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (OOZIE-3071) Oozie 4.3 Spark sharelib ueses a different version of commons-lang3 than Spark 2.2.0
[ https://issues.apache.org/jira/browse/OOZIE-3071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16278961#comment-16278961 ] Ran Gu commented on OOZIE-3071: --- Ping > Oozie 4.3 Spark sharelib ueses a different version of commons-lang3 than > Spark 2.2.0 > > > Key: OOZIE-3071 > URL: https://issues.apache.org/jira/browse/OOZIE-3071 > Project: Oozie > Issue Type: Bug > Components: core >Affects Versions: 4.3.0 >Reporter: Ran Gu > Labels: patch > Attachments: OOZIE-3071-1.patch, OOZIE-3071.patch > > > Currently Oozie 4.3.0 uses commons-lang3 version 3.3.2 in Spark sharelib. > Spark 2.2.0 uses commons-lang3 version 3.5. > This causes Oozie(/Spark) job failures on EMR-5.8.0 clusters. > Error message: > 17/08/22 00:22:43 ERROR ApplicationMaster: User class threw exception: > java.lang.IllegalArgumentException: Illegal pattern component: XXX > java.lang.IllegalArgumentException: Illegal pattern component: XXX > at > org.apache.commons.lang3.time.FastDatePrinter.parsePattern(FastDatePrinter.java:282) > at > org.apache.commons.lang3.time.FastDatePrinter.init(FastDatePrinter.java:149) > at > org.apache.commons.lang3.time.FastDatePrinter.(FastDatePrinter.java:142) > at > org.apache.commons.lang3.time.FastDateFormat.(FastDateFormat.java:384) > at > org.apache.commons.lang3.time.FastDateFormat.(FastDateFormat.java:369) > at > org.apache.commons.lang3.time.FastDateFormat$1.createInstance(FastDateFormat.java:91) > at > org.apache.commons.lang3.time.FastDateFormat$1.createInstance(FastDateFormat.java:88) > at org.apache.commons.lang3.time.FormatCache.getInstance(FormatCache.java:82) -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (OOZIE-3071) Oozie 4.3 Spark sharelib ueses a different version of commons-lang3 than Spark 2.2.0
[
https://issues.apache.org/jira/browse/OOZIE-3071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16277657#comment-16277657
]
Hadoop QA commented on OOZIE-3071:
--
Testing JIRA OOZIE-3071
Cleaning local git workspace
{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:red}-1 RAW_PATCH_ANALYSIS{color}
.{color:green}+1{color} the patch does not introduce any @author tags
.{color:green}+1{color} the patch does not introduce any tabs
.{color:green}+1{color} the patch does not introduce any trailing spaces
.{color:green}+1{color} the patch does not introduce any line longer than
132
.{color:red}-1{color} the patch does not add/modify any testcase
{color:green}+1 RAT{color}
.{color:green}+1{color} the patch does not seem to introduce new RAT
warnings
{color:green}+1 JAVADOC{color}
.{color:green}+1{color} the patch does not seem to introduce new Javadoc
warnings
{color:green}+1 COMPILE{color}
.{color:green}+1{color} HEAD compiles
.{color:green}+1{color} patch compiles
.{color:green}+1{color} the patch does not seem to introduce new javac
warnings
{color:green}+1{color} There are no new bugs found in total.
. {color:green}+1{color} There are no new bugs found in [core].
. {color:green}+1{color} There are no new bugs found in [tools].
. {color:green}+1{color} There are no new bugs found in [sharelib/hive2].
. {color:green}+1{color} There are no new bugs found in [sharelib/distcp].
. {color:green}+1{color} There are no new bugs found in [sharelib/hcatalog].
. {color:green}+1{color} There are no new bugs found in [sharelib/streaming].
. {color:green}+1{color} There are no new bugs found in [sharelib/sqoop].
. {color:green}+1{color} There are no new bugs found in [sharelib/oozie].
. {color:green}+1{color} There are no new bugs found in [sharelib/pig].
. {color:green}+1{color} There are no new bugs found in [sharelib/hive].
. {color:green}+1{color} There are no new bugs found in [sharelib/spark].
. {color:green}+1{color} There are no new bugs found in [client].
. {color:green}+1{color} There are no new bugs found in [examples].
. {color:green}+1{color} There are no new bugs found in [docs].
. {color:green}+1{color} There are no new bugs found in [server].
{color:green}+1 BACKWARDS_COMPATIBILITY{color}
.{color:green}+1{color} the patch does not change any JPA
Entity/Colum/Basic/Lob/Transient annotations
.{color:green}+1{color} the patch does not modify JPA files
{color:green}+1 TESTS{color}
.Tests run: 2075
.Tests rerun: 96
.Tests failed at first run:
org.apache.oozie.action.hadoop.TestJavaActionExecutor,org.apache.oozie.service.TestStatusTransitService,org.apache.oozie.command.coord.TestCoordActionsKillXCommand,
{color:green}+1 DISTRO{color}
.{color:green}+1{color} distro tarball builds with the patch
{color:red}*-1 Overall result, please check the reported -1(s)*{color}
The full output of the test-patch run is available at
. https://builds.apache.org/job/PreCommit-OOZIE-Build/272/
> Oozie 4.3 Spark sharelib ueses a different version of commons-lang3 than
> Spark 2.2.0
>
>
> Key: OOZIE-3071
> URL: https://issues.apache.org/jira/browse/OOZIE-3071
> Project: Oozie
> Issue Type: Bug
> Components: core
>Affects Versions: 4.3.0
>Reporter: Ran Gu
> Labels: patch
> Attachments: OOZIE-3071-1.patch, OOZIE-3071.patch
>
>
> Currently Oozie 4.3.0 uses commons-lang3 version 3.3.2 in Spark sharelib.
> Spark 2.2.0 uses commons-lang3 version 3.5.
> This causes Oozie(/Spark) job failures on EMR-5.8.0 clusters.
> Error message:
> 17/08/22 00:22:43 ERROR ApplicationMaster: User class threw exception:
> java.lang.IllegalArgumentException: Illegal pattern component: XXX
> java.lang.IllegalArgumentException: Illegal pattern component: XXX
> at
> org.apache.commons.lang3.time.FastDatePrinter.parsePattern(FastDatePrinter.java:282)
> at
> org.apache.commons.lang3.time.FastDatePrinter.init(FastDatePrinter.java:149)
> at
> org.apache.commons.lang3.time.FastDatePrinter.(FastDatePrinter.java:142)
> at
> org.apache.commons.lang3.time.FastDateFormat.(FastDateFormat.java:384)
> at
> org.apache.commons.lang3.time.FastDateFormat.(FastDateFormat.java:369)
> at
> org.apache.commons.lang3.time.FastDateFormat$1.createInstance(FastDateFormat.java:91)
> at
> org.apache.commons.lang3.time.FastDateFormat$1.createInstance(FastDateFormat.java:88)
> at org.apache.commons.lang3.time.FormatCache.getInstance(FormatCache.java:82)
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (OOZIE-3071) Oozie 4.3 Spark sharelib ueses a different version of commons-lang3 than Spark 2.2.0
[ https://issues.apache.org/jira/browse/OOZIE-3071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16277412#comment-16277412 ] Ran Gu commented on OOZIE-3071: --- Hello, any updates on this? > Oozie 4.3 Spark sharelib ueses a different version of commons-lang3 than > Spark 2.2.0 > > > Key: OOZIE-3071 > URL: https://issues.apache.org/jira/browse/OOZIE-3071 > Project: Oozie > Issue Type: Bug > Components: core >Affects Versions: 4.3.0 >Reporter: Ran Gu > Labels: patch > Attachments: OOZIE-3071-1.patch, OOZIE-3071.patch > > > Currently Oozie 4.3.0 uses commons-lang3 version 3.3.2 in Spark sharelib. > Spark 2.2.0 uses commons-lang3 version 3.5. > This causes Oozie(/Spark) job failures on EMR-5.8.0 clusters. > Error message: > 17/08/22 00:22:43 ERROR ApplicationMaster: User class threw exception: > java.lang.IllegalArgumentException: Illegal pattern component: XXX > java.lang.IllegalArgumentException: Illegal pattern component: XXX > at > org.apache.commons.lang3.time.FastDatePrinter.parsePattern(FastDatePrinter.java:282) > at > org.apache.commons.lang3.time.FastDatePrinter.init(FastDatePrinter.java:149) > at > org.apache.commons.lang3.time.FastDatePrinter.(FastDatePrinter.java:142) > at > org.apache.commons.lang3.time.FastDateFormat.(FastDateFormat.java:384) > at > org.apache.commons.lang3.time.FastDateFormat.(FastDateFormat.java:369) > at > org.apache.commons.lang3.time.FastDateFormat$1.createInstance(FastDateFormat.java:91) > at > org.apache.commons.lang3.time.FastDateFormat$1.createInstance(FastDateFormat.java:88) > at org.apache.commons.lang3.time.FormatCache.getInstance(FormatCache.java:82) -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (OOZIE-3071) Oozie 4.3 Spark sharelib ueses a different version of commons-lang3 than Spark 2.2.0
[ https://issues.apache.org/jira/browse/OOZIE-3071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16193235#comment-16193235 ] Artem Ervits commented on OOZIE-3071: - [~gezapeti] I checked spark 1.6 branch and I see it is compiled with 3.5 just as Spark 2.2 branch. Compiling with either spark version on 3.6 works fine. Is there anything you would propose to do in terms of pulling out the dep out of main pom for example? > Oozie 4.3 Spark sharelib ueses a different version of commons-lang3 than > Spark 2.2.0 > > > Key: OOZIE-3071 > URL: https://issues.apache.org/jira/browse/OOZIE-3071 > Project: Oozie > Issue Type: Bug > Components: core >Affects Versions: 4.3.0 >Reporter: Ran Gu > Labels: patch > Attachments: OOZIE-3071-1.patch, OOZIE-3071.patch > > > Currently Oozie 4.3.0 uses commons-lang3 version 3.3.2 in Spark sharelib. > Spark 2.2.0 uses commons-lang3 version 3.5. > This causes Oozie(/Spark) job failures on EMR-5.8.0 clusters. > Error message: > 17/08/22 00:22:43 ERROR ApplicationMaster: User class threw exception: > java.lang.IllegalArgumentException: Illegal pattern component: XXX > java.lang.IllegalArgumentException: Illegal pattern component: XXX > at > org.apache.commons.lang3.time.FastDatePrinter.parsePattern(FastDatePrinter.java:282) > at > org.apache.commons.lang3.time.FastDatePrinter.init(FastDatePrinter.java:149) > at > org.apache.commons.lang3.time.FastDatePrinter.(FastDatePrinter.java:142) > at > org.apache.commons.lang3.time.FastDateFormat.(FastDateFormat.java:384) > at > org.apache.commons.lang3.time.FastDateFormat.(FastDateFormat.java:369) > at > org.apache.commons.lang3.time.FastDateFormat$1.createInstance(FastDateFormat.java:91) > at > org.apache.commons.lang3.time.FastDateFormat$1.createInstance(FastDateFormat.java:88) > at org.apache.commons.lang3.time.FormatCache.getInstance(FormatCache.java:82) -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (OOZIE-3071) Oozie 4.3 Spark sharelib ueses a different version of commons-lang3 than Spark 2.2.0
[ https://issues.apache.org/jira/browse/OOZIE-3071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16193223#comment-16193223 ] Peter Cseh commented on OOZIE-3071: --- My understanding that this dependency caused a runtime issue when Oozie was compiled and run with Spark2. My guess is that bumping this version could cause runtime issues with Spark1.6.3. > Oozie 4.3 Spark sharelib ueses a different version of commons-lang3 than > Spark 2.2.0 > > > Key: OOZIE-3071 > URL: https://issues.apache.org/jira/browse/OOZIE-3071 > Project: Oozie > Issue Type: Bug > Components: core >Affects Versions: 4.3.0 >Reporter: Ran Gu > Labels: patch > Attachments: OOZIE-3071-1.patch, OOZIE-3071.patch > > > Currently Oozie 4.3.0 uses commons-lang3 version 3.3.2 in Spark sharelib. > Spark 2.2.0 uses commons-lang3 version 3.5. > This causes Oozie(/Spark) job failures on EMR-5.8.0 clusters. > Error message: > 17/08/22 00:22:43 ERROR ApplicationMaster: User class threw exception: > java.lang.IllegalArgumentException: Illegal pattern component: XXX > java.lang.IllegalArgumentException: Illegal pattern component: XXX > at > org.apache.commons.lang3.time.FastDatePrinter.parsePattern(FastDatePrinter.java:282) > at > org.apache.commons.lang3.time.FastDatePrinter.init(FastDatePrinter.java:149) > at > org.apache.commons.lang3.time.FastDatePrinter.(FastDatePrinter.java:142) > at > org.apache.commons.lang3.time.FastDateFormat.(FastDateFormat.java:384) > at > org.apache.commons.lang3.time.FastDateFormat.(FastDateFormat.java:369) > at > org.apache.commons.lang3.time.FastDateFormat$1.createInstance(FastDateFormat.java:91) > at > org.apache.commons.lang3.time.FastDateFormat$1.createInstance(FastDateFormat.java:88) > at org.apache.commons.lang3.time.FormatCache.getInstance(FormatCache.java:82) -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (OOZIE-3071) Oozie 4.3 Spark sharelib ueses a different version of commons-lang3 than Spark 2.2.0
[
https://issues.apache.org/jira/browse/OOZIE-3071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16193074#comment-16193074
]
Artem Ervits commented on OOZIE-3071:
-
[~gezapeti] I was able to compile my change with Spark 1.6.3, unless I'm
missing something, please let me know and I'll pick up the jira to work further
on.
{noformat}
bin/mkdistro.sh -DjavaVersion=1.8 -fae -Dmaven.test.failure.ignore=true
-Dspark.version=1.6.3 -DskipTests=true
{noformat}
> Oozie 4.3 Spark sharelib ueses a different version of commons-lang3 than
> Spark 2.2.0
>
>
> Key: OOZIE-3071
> URL: https://issues.apache.org/jira/browse/OOZIE-3071
> Project: Oozie
> Issue Type: Bug
> Components: core
>Affects Versions: 4.3.0
>Reporter: Ran Gu
> Labels: patch
> Attachments: OOZIE-3071-1.patch, OOZIE-3071.patch
>
>
> Currently Oozie 4.3.0 uses commons-lang3 version 3.3.2 in Spark sharelib.
> Spark 2.2.0 uses commons-lang3 version 3.5.
> This causes Oozie(/Spark) job failures on EMR-5.8.0 clusters.
> Error message:
> 17/08/22 00:22:43 ERROR ApplicationMaster: User class threw exception:
> java.lang.IllegalArgumentException: Illegal pattern component: XXX
> java.lang.IllegalArgumentException: Illegal pattern component: XXX
> at
> org.apache.commons.lang3.time.FastDatePrinter.parsePattern(FastDatePrinter.java:282)
> at
> org.apache.commons.lang3.time.FastDatePrinter.init(FastDatePrinter.java:149)
> at
> org.apache.commons.lang3.time.FastDatePrinter.(FastDatePrinter.java:142)
> at
> org.apache.commons.lang3.time.FastDateFormat.(FastDateFormat.java:384)
> at
> org.apache.commons.lang3.time.FastDateFormat.(FastDateFormat.java:369)
> at
> org.apache.commons.lang3.time.FastDateFormat$1.createInstance(FastDateFormat.java:91)
> at
> org.apache.commons.lang3.time.FastDateFormat$1.createInstance(FastDateFormat.java:88)
> at org.apache.commons.lang3.time.FormatCache.getInstance(FormatCache.java:82)
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (OOZIE-3071) Oozie 4.3 Spark sharelib ueses a different version of commons-lang3 than Spark 2.2.0
[
https://issues.apache.org/jira/browse/OOZIE-3071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16190470#comment-16190470
]
Hadoop QA commented on OOZIE-3071:
--
Testing JIRA OOZIE-3071
Cleaning local git workspace
{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:red}-1 RAW_PATCH_ANALYSIS{color}
.{color:green}+1{color} the patch does not introduce any @author tags
.{color:green}+1{color} the patch does not introduce any tabs
.{color:green}+1{color} the patch does not introduce any trailing spaces
.{color:green}+1{color} the patch does not introduce any line longer than
132
.{color:red}-1{color} the patch does not add/modify any testcase
{color:green}+1 RAT{color}
.{color:green}+1{color} the patch does not seem to introduce new RAT
warnings
{color:green}+1 JAVADOC{color}
.{color:green}+1{color} the patch does not seem to introduce new Javadoc
warnings
.{color:red}WARNING{color}: the current HEAD has 77 Javadoc warning(s)
{color:green}+1 COMPILE{color}
.{color:green}+1{color} HEAD compiles
.{color:green}+1{color} patch compiles
.{color:green}+1{color} the patch does not seem to introduce new javac
warnings
{color:green}+1{color} There are no new bugs found in total.
. {color:green}+1{color} There are no new bugs found in [server].
. {color:green}+1{color} There are no new bugs found in [client].
. {color:green}+1{color} There are no new bugs found in [core].
. {color:green}+1{color} There are no new bugs found in [docs].
. {color:green}+1{color} There are no new bugs found in [tools].
. {color:green}+1{color} There are no new bugs found in [examples].
. {color:green}+1{color} There are no new bugs found in [sharelib/streaming].
. {color:green}+1{color} There are no new bugs found in [sharelib/sqoop].
. {color:green}+1{color} There are no new bugs found in [sharelib/distcp].
. {color:green}+1{color} There are no new bugs found in [sharelib/oozie].
. {color:green}+1{color} There are no new bugs found in [sharelib/hcatalog].
. {color:green}+1{color} There are no new bugs found in [sharelib/hive].
. {color:green}+1{color} There are no new bugs found in [sharelib/hive2].
. {color:green}+1{color} There are no new bugs found in [sharelib/pig].
. {color:green}+1{color} There are no new bugs found in [sharelib/spark].
{color:green}+1 BACKWARDS_COMPATIBILITY{color}
.{color:green}+1{color} the patch does not change any JPA
Entity/Colum/Basic/Lob/Transient annotations
.{color:green}+1{color} the patch does not modify JPA files
{color:red}-1 TESTS{color} - patch does not compile, cannot run testcases
{color:green}+1 DISTRO{color}
.{color:green}+1{color} distro tarball builds with the patch
{color:red}*-1 Overall result, please check the reported -1(s)*{color}
{color:red}. There is at least one warning, please check{color}
The full output of the test-patch run is available at
. https://builds.apache.org/job/PreCommit-OOZIE-Build/72/
> Oozie 4.3 Spark sharelib ueses a different version of commons-lang3 than
> Spark 2.2.0
>
>
> Key: OOZIE-3071
> URL: https://issues.apache.org/jira/browse/OOZIE-3071
> Project: Oozie
> Issue Type: Bug
> Components: core
>Affects Versions: 4.3.0
>Reporter: Ran Gu
> Labels: patch
> Attachments: OOZIE-3071-1.patch, OOZIE-3071.patch
>
>
> Currently Oozie 4.3.0 uses commons-lang3 version 3.3.2 in Spark sharelib.
> Spark 2.2.0 uses commons-lang3 version 3.5.
> This causes Oozie(/Spark) job failures on EMR-5.8.0 clusters.
> Error message:
> 17/08/22 00:22:43 ERROR ApplicationMaster: User class threw exception:
> java.lang.IllegalArgumentException: Illegal pattern component: XXX
> java.lang.IllegalArgumentException: Illegal pattern component: XXX
> at
> org.apache.commons.lang3.time.FastDatePrinter.parsePattern(FastDatePrinter.java:282)
> at
> org.apache.commons.lang3.time.FastDatePrinter.init(FastDatePrinter.java:149)
> at
> org.apache.commons.lang3.time.FastDatePrinter.(FastDatePrinter.java:142)
> at
> org.apache.commons.lang3.time.FastDateFormat.(FastDateFormat.java:384)
> at
> org.apache.commons.lang3.time.FastDateFormat.(FastDateFormat.java:369)
> at
> org.apache.commons.lang3.time.FastDateFormat$1.createInstance(FastDateFormat.java:91)
> at
> org.apache.commons.lang3.time.FastDateFormat$1.createInstance(FastDateFormat.java:88)
> at org.apache.commons.lang3.time.FormatCache.getInstance(FormatCache.java:82)
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (OOZIE-3071) Oozie 4.3 Spark sharelib ueses a different version of commons-lang3 than Spark 2.2.0
[ https://issues.apache.org/jira/browse/OOZIE-3071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16190240#comment-16190240 ] Peter Cseh commented on OOZIE-3071: --- While this version bump solves the issue with Spark 2.2, won't it break Spark1.6 runs from Oozie? We should find a way to separate out server dependencies from the Sharelib dependencies in general so we can avoid problems like this. > Oozie 4.3 Spark sharelib ueses a different version of commons-lang3 than > Spark 2.2.0 > > > Key: OOZIE-3071 > URL: https://issues.apache.org/jira/browse/OOZIE-3071 > Project: Oozie > Issue Type: Bug > Components: core >Affects Versions: 4.3.0 >Reporter: Ran Gu > Labels: patch > Attachments: OOZIE-3071-1.patch, OOZIE-3071.patch > > > Currently Oozie 4.3.0 uses commons-lang3 version 3.3.2 in Spark sharelib. > Spark 2.2.0 uses commons-lang3 version 3.5. > This causes Oozie(/Spark) job failures on EMR-5.8.0 clusters. > Error message: > 17/08/22 00:22:43 ERROR ApplicationMaster: User class threw exception: > java.lang.IllegalArgumentException: Illegal pattern component: XXX > java.lang.IllegalArgumentException: Illegal pattern component: XXX > at > org.apache.commons.lang3.time.FastDatePrinter.parsePattern(FastDatePrinter.java:282) > at > org.apache.commons.lang3.time.FastDatePrinter.init(FastDatePrinter.java:149) > at > org.apache.commons.lang3.time.FastDatePrinter.(FastDatePrinter.java:142) > at > org.apache.commons.lang3.time.FastDateFormat.(FastDateFormat.java:384) > at > org.apache.commons.lang3.time.FastDateFormat.(FastDateFormat.java:369) > at > org.apache.commons.lang3.time.FastDateFormat$1.createInstance(FastDateFormat.java:91) > at > org.apache.commons.lang3.time.FastDateFormat$1.createInstance(FastDateFormat.java:88) > at org.apache.commons.lang3.time.FormatCache.getInstance(FormatCache.java:82) -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (OOZIE-3071) Oozie 4.3 Spark sharelib ueses a different version of commons-lang3 than Spark 2.2.0
[ https://issues.apache.org/jira/browse/OOZIE-3071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16190159#comment-16190159 ] Ran Gu commented on OOZIE-3071: --- Sorry for the late response. I have verified that updating commons-lang3 version to 3.6 in top level pom.xml worked properly. Are you going to make that change? And will that be included in the next Oozie release? > Oozie 4.3 Spark sharelib ueses a different version of commons-lang3 than > Spark 2.2.0 > > > Key: OOZIE-3071 > URL: https://issues.apache.org/jira/browse/OOZIE-3071 > Project: Oozie > Issue Type: Bug > Components: core >Affects Versions: 4.3.0 >Reporter: Ran Gu > Labels: patch > Attachments: OOZIE-3071.patch > > > Currently Oozie 4.3.0 uses commons-lang3 version 3.3.2 in Spark sharelib. > Spark 2.2.0 uses commons-lang3 version 3.5. > This causes Oozie(/Spark) job failures on EMR-5.8.0 clusters. > Error message: > 17/08/22 00:22:43 ERROR ApplicationMaster: User class threw exception: > java.lang.IllegalArgumentException: Illegal pattern component: XXX > java.lang.IllegalArgumentException: Illegal pattern component: XXX > at > org.apache.commons.lang3.time.FastDatePrinter.parsePattern(FastDatePrinter.java:282) > at > org.apache.commons.lang3.time.FastDatePrinter.init(FastDatePrinter.java:149) > at > org.apache.commons.lang3.time.FastDatePrinter.(FastDatePrinter.java:142) > at > org.apache.commons.lang3.time.FastDateFormat.(FastDateFormat.java:384) > at > org.apache.commons.lang3.time.FastDateFormat.(FastDateFormat.java:369) > at > org.apache.commons.lang3.time.FastDateFormat$1.createInstance(FastDateFormat.java:91) > at > org.apache.commons.lang3.time.FastDateFormat$1.createInstance(FastDateFormat.java:88) > at org.apache.commons.lang3.time.FormatCache.getInstance(FormatCache.java:82) -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (OOZIE-3071) Oozie 4.3 Spark sharelib ueses a different version of commons-lang3 than Spark 2.2.0
[
https://issues.apache.org/jira/browse/OOZIE-3071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16185847#comment-16185847
]
Artem Ervits commented on OOZIE-3071:
-
[~rangu] not sure why you're getting failures, I bumped the commons-lang3 to
3.6 in pom.xml only and compiled with
{code}
bin/mkdistro.sh -DjavaVersion=1.8 -fae -Dmaven.test.failure.ignore=true
-Dspark.version=2.2.0 -DskipTests=true
{code}
I didn't get any failures. Perhaps you'd want to try that? Btw, I'm using the
master branch.
My only change is
{noformat}
diff --git a/pom.xml b/pom.xml
index efccc346..492e7f62 100644
--- a/pom.xml
+++ b/pom.xml
@@ -1308,7 +1308,7 @@
org.apache.commons
commons-lang3
-3.3.2
+3.6
{noformat}
> Oozie 4.3 Spark sharelib ueses a different version of commons-lang3 than
> Spark 2.2.0
>
>
> Key: OOZIE-3071
> URL: https://issues.apache.org/jira/browse/OOZIE-3071
> Project: Oozie
> Issue Type: Bug
> Components: core
>Affects Versions: 4.3.0
>Reporter: Ran Gu
> Labels: patch
> Attachments: OOZIE-3071.patch
>
>
> Currently Oozie 4.3.0 uses commons-lang3 version 3.3.2 in Spark sharelib.
> Spark 2.2.0 uses commons-lang3 version 3.5.
> This causes Oozie(/Spark) job failures on EMR-5.8.0 clusters.
> Error message:
> 17/08/22 00:22:43 ERROR ApplicationMaster: User class threw exception:
> java.lang.IllegalArgumentException: Illegal pattern component: XXX
> java.lang.IllegalArgumentException: Illegal pattern component: XXX
> at
> org.apache.commons.lang3.time.FastDatePrinter.parsePattern(FastDatePrinter.java:282)
> at
> org.apache.commons.lang3.time.FastDatePrinter.init(FastDatePrinter.java:149)
> at
> org.apache.commons.lang3.time.FastDatePrinter.(FastDatePrinter.java:142)
> at
> org.apache.commons.lang3.time.FastDateFormat.(FastDateFormat.java:384)
> at
> org.apache.commons.lang3.time.FastDateFormat.(FastDateFormat.java:369)
> at
> org.apache.commons.lang3.time.FastDateFormat$1.createInstance(FastDateFormat.java:91)
> at
> org.apache.commons.lang3.time.FastDateFormat$1.createInstance(FastDateFormat.java:88)
> at org.apache.commons.lang3.time.FormatCache.getInstance(FormatCache.java:82)
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (OOZIE-3071) Oozie 4.3 Spark sharelib ueses a different version of commons-lang3 than Spark 2.2.0
[
https://issues.apache.org/jira/browse/OOZIE-3071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16185100#comment-16185100
]
Hadoop QA commented on OOZIE-3071:
--
Testing JIRA OOZIE-3071
Cleaning local git workspace
{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:red}-1 RAW_PATCH_ANALYSIS{color}
.{color:green}+1{color} the patch does not introduce any @author tags
.{color:green}+1{color} the patch does not introduce any tabs
.{color:green}+1{color} the patch does not introduce any trailing spaces
.{color:green}+1{color} the patch does not introduce any line longer than
132
.{color:red}-1{color} the patch does not add/modify any testcase
{color:green}+1 RAT{color}
.{color:green}+1{color} the patch does not seem to introduce new RAT
warnings
{color:green}+1 JAVADOC{color}
.{color:green}+1{color} the patch does not seem to introduce new Javadoc
warnings
.{color:red}WARNING{color}: the current HEAD has 77 Javadoc warning(s)
{color:red}-1 COMPILE{color}
.{color:green}+1{color} HEAD compiles
.{color:red}-1{color} patch does not compile
.{color:green}+1{color} the patch does not seem to introduce new javac
warnings
{color:green}+1{color} There are no new bugs found in total.
. {color:green}+1{color} There are no new bugs found in [core].
. {color:green}+1{color} There are no new bugs found in [client].
. {color:green}+1{color} There are no new bugs found in [docs].
. {color:green}+1{color} There are no new bugs found in [sharelib/streaming].
. {color:green}+1{color} There are no new bugs found in [sharelib/hive].
. {color:green}+1{color} There are no new bugs found in [sharelib/hive2].
. {color:green}+1{color} There are no new bugs found in [sharelib/sqoop].
. {color:green}+1{color} There are no new bugs found in [sharelib/oozie].
. {color:green}+1{color} There are no new bugs found in [sharelib/hcatalog].
. {color:green}+1{color} There are no new bugs found in [sharelib/pig].
. {color:green}+1{color} There are no new bugs found in [sharelib/spark].
. {color:green}+1{color} There are no new bugs found in [sharelib/distcp].
. {color:green}+1{color} There are no new bugs found in [examples].
. {color:green}+1{color} There are no new bugs found in [tools].
. {color:green}+1{color} There are no new bugs found in [server].
{color:green}+1 BACKWARDS_COMPATIBILITY{color}
.{color:green}+1{color} the patch does not change any JPA
Entity/Colum/Basic/Lob/Transient annotations
.{color:green}+1{color} the patch does not modify JPA files
{color:red}-1 TESTS{color} - patch does not compile, cannot run testcases
{color:red}-1 DISTRO{color}
.{color:red}-1{color} distro tarball fails with the patch
{color:red}*-1 Overall result, please check the reported -1(s)*{color}
{color:red}. There is at least one warning, please check{color}
The full output of the test-patch run is available at
. https://builds.apache.org/job/PreCommit-OOZIE-Build/57/
> Oozie 4.3 Spark sharelib ueses a different version of commons-lang3 than
> Spark 2.2.0
>
>
> Key: OOZIE-3071
> URL: https://issues.apache.org/jira/browse/OOZIE-3071
> Project: Oozie
> Issue Type: Bug
> Components: core
>Affects Versions: 4.3.0
>Reporter: Ran Gu
> Labels: patch
> Attachments: OOZIE-3071.patch
>
>
> Currently Oozie 4.3.0 uses commons-lang3 version 3.3.2 in Spark sharelib.
> Spark 2.2.0 uses commons-lang3 version 3.5.
> This causes Oozie(/Spark) job failures on EMR-5.8.0 clusters.
> Error message:
> 17/08/22 00:22:43 ERROR ApplicationMaster: User class threw exception:
> java.lang.IllegalArgumentException: Illegal pattern component: XXX
> java.lang.IllegalArgumentException: Illegal pattern component: XXX
> at
> org.apache.commons.lang3.time.FastDatePrinter.parsePattern(FastDatePrinter.java:282)
> at
> org.apache.commons.lang3.time.FastDatePrinter.init(FastDatePrinter.java:149)
> at
> org.apache.commons.lang3.time.FastDatePrinter.(FastDatePrinter.java:142)
> at
> org.apache.commons.lang3.time.FastDateFormat.(FastDateFormat.java:384)
> at
> org.apache.commons.lang3.time.FastDateFormat.(FastDateFormat.java:369)
> at
> org.apache.commons.lang3.time.FastDateFormat$1.createInstance(FastDateFormat.java:91)
> at
> org.apache.commons.lang3.time.FastDateFormat$1.createInstance(FastDateFormat.java:88)
> at org.apache.commons.lang3.time.FormatCache.getInstance(FormatCache.java:82)
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (OOZIE-3071) Oozie 4.3 Spark sharelib ueses a different version of commons-lang3 than Spark 2.2.0
[ https://issues.apache.org/jira/browse/OOZIE-3071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16185085#comment-16185085 ] Ran Gu commented on OOZIE-3071: --- Not sure why "patch failed to apply to head of branch". Recreated patch against Oozie branch-4.3 > Oozie 4.3 Spark sharelib ueses a different version of commons-lang3 than > Spark 2.2.0 > > > Key: OOZIE-3071 > URL: https://issues.apache.org/jira/browse/OOZIE-3071 > Project: Oozie > Issue Type: Bug > Components: core >Affects Versions: 4.3.0 >Reporter: Ran Gu > Labels: patch > Attachments: OOZIE-3071.patch > > > Currently Oozie 4.3.0 uses commons-lang3 version 3.3.2 in Spark sharelib. > Spark 2.2.0 uses commons-lang3 version 3.5. > This causes Oozie(/Spark) job failures on EMR-5.8.0 clusters. > Error message: > 17/08/22 00:22:43 ERROR ApplicationMaster: User class threw exception: > java.lang.IllegalArgumentException: Illegal pattern component: XXX > java.lang.IllegalArgumentException: Illegal pattern component: XXX > at > org.apache.commons.lang3.time.FastDatePrinter.parsePattern(FastDatePrinter.java:282) > at > org.apache.commons.lang3.time.FastDatePrinter.init(FastDatePrinter.java:149) > at > org.apache.commons.lang3.time.FastDatePrinter.(FastDatePrinter.java:142) > at > org.apache.commons.lang3.time.FastDateFormat.(FastDateFormat.java:384) > at > org.apache.commons.lang3.time.FastDateFormat.(FastDateFormat.java:369) > at > org.apache.commons.lang3.time.FastDateFormat$1.createInstance(FastDateFormat.java:91) > at > org.apache.commons.lang3.time.FastDateFormat$1.createInstance(FastDateFormat.java:88) > at org.apache.commons.lang3.time.FormatCache.getInstance(FormatCache.java:82) -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (OOZIE-3071) Oozie 4.3 Spark sharelib ueses a different version of commons-lang3 than Spark 2.2.0
[
https://issues.apache.org/jira/browse/OOZIE-3071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16184875#comment-16184875
]
Hadoop QA commented on OOZIE-3071:
--
Testing JIRA OOZIE-3071
Cleaning local git workspace
{color:red}-1{color} Patch failed to apply to head of branch
> Oozie 4.3 Spark sharelib ueses a different version of commons-lang3 than
> Spark 2.2.0
>
>
> Key: OOZIE-3071
> URL: https://issues.apache.org/jira/browse/OOZIE-3071
> Project: Oozie
> Issue Type: Bug
> Components: core
>Affects Versions: 4.3.0
>Reporter: Ran Gu
> Labels: patch
> Attachments: OOZIE-3071.patch
>
>
> Currently Oozie 4.3.0 uses commons-lang3 version 3.3.2 in Spark sharelib.
> Spark 2.2.0 uses commons-lang3 version 3.5.
> This causes Oozie(/Spark) job failures on EMR-5.8.0 clusters.
> Error message:
> 17/08/22 00:22:43 ERROR ApplicationMaster: User class threw exception:
> java.lang.IllegalArgumentException: Illegal pattern component: XXX
> java.lang.IllegalArgumentException: Illegal pattern component: XXX
> at
> org.apache.commons.lang3.time.FastDatePrinter.parsePattern(FastDatePrinter.java:282)
> at
> org.apache.commons.lang3.time.FastDatePrinter.init(FastDatePrinter.java:149)
> at
> org.apache.commons.lang3.time.FastDatePrinter.(FastDatePrinter.java:142)
> at
> org.apache.commons.lang3.time.FastDateFormat.(FastDateFormat.java:384)
> at
> org.apache.commons.lang3.time.FastDateFormat.(FastDateFormat.java:369)
> at
> org.apache.commons.lang3.time.FastDateFormat$1.createInstance(FastDateFormat.java:91)
> at
> org.apache.commons.lang3.time.FastDateFormat$1.createInstance(FastDateFormat.java:88)
> at org.apache.commons.lang3.time.FormatCache.getInstance(FormatCache.java:82)
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (OOZIE-3071) Oozie 4.3 Spark sharelib ueses a different version of commons-lang3 than Spark 2.2.0
[ https://issues.apache.org/jira/browse/OOZIE-3071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16184852#comment-16184852 ] Ran Gu commented on OOZIE-3071: --- [~dbist13] Updating commons-lang3 to 3.5 in Oozie top level pom.xml breaks hive2 (mvn clean install would fail). Seems to me Hive2 is using commons-lang3 version 3.3.2 where Spark is using commons-lang3 version 3.5. > Oozie 4.3 Spark sharelib ueses a different version of commons-lang3 than > Spark 2.2.0 > > > Key: OOZIE-3071 > URL: https://issues.apache.org/jira/browse/OOZIE-3071 > Project: Oozie > Issue Type: Bug > Components: core >Affects Versions: 4.3.0 >Reporter: Ran Gu > Attachments: OOZIE-3071.patch > > > Currently Oozie 4.3.0 uses commons-lang3 version 3.3.2 in Spark sharelib. > Spark 2.2.0 uses commons-lang3 version 3.5. > This causes Oozie(/Spark) job failures on EMR-5.8.0 clusters. > Error message: > 17/08/22 00:22:43 ERROR ApplicationMaster: User class threw exception: > java.lang.IllegalArgumentException: Illegal pattern component: XXX > java.lang.IllegalArgumentException: Illegal pattern component: XXX > at > org.apache.commons.lang3.time.FastDatePrinter.parsePattern(FastDatePrinter.java:282) > at > org.apache.commons.lang3.time.FastDatePrinter.init(FastDatePrinter.java:149) > at > org.apache.commons.lang3.time.FastDatePrinter.(FastDatePrinter.java:142) > at > org.apache.commons.lang3.time.FastDateFormat.(FastDateFormat.java:384) > at > org.apache.commons.lang3.time.FastDateFormat.(FastDateFormat.java:369) > at > org.apache.commons.lang3.time.FastDateFormat$1.createInstance(FastDateFormat.java:91) > at > org.apache.commons.lang3.time.FastDateFormat$1.createInstance(FastDateFormat.java:88) > at org.apache.commons.lang3.time.FormatCache.getInstance(FormatCache.java:82) -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (OOZIE-3071) Oozie 4.3 Spark sharelib ueses a different version of commons-lang3 than Spark 2.2.0
[ https://issues.apache.org/jira/browse/OOZIE-3071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16184823#comment-16184823 ] Artem Ervits commented on OOZIE-3071: - [~rangu] why not just bump commons-lang3 to 3.5 and not add a second one in hive sharelib? In case it does need it, why not just remove the version all together from sharelib pom and leave it only in main? > Oozie 4.3 Spark sharelib ueses a different version of commons-lang3 than > Spark 2.2.0 > > > Key: OOZIE-3071 > URL: https://issues.apache.org/jira/browse/OOZIE-3071 > Project: Oozie > Issue Type: Bug > Components: core >Affects Versions: 4.3.0 >Reporter: Ran Gu > Attachments: OOZIE-3071.patch > > > Currently Oozie 4.3.0 uses commons-lang3 version 3.3.2 in Spark sharelib. > Spark 2.2.0 uses commons-lang3 version 3.5. > This causes Oozie(/Spark) job failures on EMR-5.8.0 clusters. > Error message: > 17/08/22 00:22:43 ERROR ApplicationMaster: User class threw exception: > java.lang.IllegalArgumentException: Illegal pattern component: XXX > java.lang.IllegalArgumentException: Illegal pattern component: XXX > at > org.apache.commons.lang3.time.FastDatePrinter.parsePattern(FastDatePrinter.java:282) > at > org.apache.commons.lang3.time.FastDatePrinter.init(FastDatePrinter.java:149) > at > org.apache.commons.lang3.time.FastDatePrinter.(FastDatePrinter.java:142) > at > org.apache.commons.lang3.time.FastDateFormat.(FastDateFormat.java:384) > at > org.apache.commons.lang3.time.FastDateFormat.(FastDateFormat.java:369) > at > org.apache.commons.lang3.time.FastDateFormat$1.createInstance(FastDateFormat.java:91) > at > org.apache.commons.lang3.time.FastDateFormat$1.createInstance(FastDateFormat.java:88) > at org.apache.commons.lang3.time.FormatCache.getInstance(FormatCache.java:82) -- This message was sent by Atlassian JIRA (v6.4.14#64029)
