[jira] [Commented] (HADOOP-18033) Upgrade fasterxml Jackson to 2.13.0
[ https://issues.apache.org/jira/browse/HADOOP-18033?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17735643#comment-17735643 ] Steve Loughran commented on HADOOP-18033: - java11, please please please > Upgrade fasterxml Jackson to 2.13.0 > --- > > Key: HADOOP-18033 > URL: https://issues.apache.org/jira/browse/HADOOP-18033 > Project: Hadoop Common > Issue Type: Improvement > Components: build >Reporter: Akira Ajisaka >Assignee: Viraj Jasani >Priority: Major > Labels: pull-request-available > Fix For: 3.3.2 > > Time Spent: 6.5h > Remaining Estimate: 0h > > Spark 3.2.0 depends on Jackson 2.12.3. Let's upgrade to 2.12.5 (2.12.x latest > as of now) or upper. > h2. this has been reverted. > we had to revert this as it broke tez. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18033) Upgrade fasterxml Jackson to 2.13.0
[ https://issues.apache.org/jira/browse/HADOOP-18033?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17731687#comment-17731687 ] Ayush Saxena commented on HADOOP-18033: --- I have almost lost track of this and honestly didn't return back once Tez-Hive upgrade got sorted. :( Jersey upgrade is the best thing to do, but that is stuck, we need that anyway for JDK-11 compile time support as well. but if thats not working, HADOOP-18619 could be a way out, forking would be a trouble during next upgrades and all. > Upgrade fasterxml Jackson to 2.13.0 > --- > > Key: HADOOP-18033 > URL: https://issues.apache.org/jira/browse/HADOOP-18033 > Project: Hadoop Common > Issue Type: Improvement > Components: build >Reporter: Akira Ajisaka >Assignee: Viraj Jasani >Priority: Major > Labels: pull-request-available > Fix For: 3.3.2 > > Time Spent: 6.5h > Remaining Estimate: 0h > > Spark 3.2.0 depends on Jackson 2.12.3. Let's upgrade to 2.12.5 (2.12.x latest > as of now) or upper. > h2. this has been reverted. > we had to revert this as it broke tez. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18033) Upgrade fasterxml Jackson to 2.13.0
[ https://issues.apache.org/jira/browse/HADOOP-18033?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17731654#comment-17731654 ] PJ Fanning commented on HADOOP-18033: - We're stuck on Jackson 2.12 because of jersey v1. Jackson 2.13 has a change that drops support for jersey v1. Options include: * forking the jackson module for jaxrs to undo the change that drops jersey v1 support * or removing the need for that jackson module - which I think might require the removal of the io.swagger code in the yarn modules * or completing the move to jersey 2 (https://issues.apache.org/jira/browse/HADOOP-15984) > Upgrade fasterxml Jackson to 2.13.0 > --- > > Key: HADOOP-18033 > URL: https://issues.apache.org/jira/browse/HADOOP-18033 > Project: Hadoop Common > Issue Type: Improvement > Components: build >Reporter: Akira Ajisaka >Assignee: Viraj Jasani >Priority: Major > Labels: pull-request-available > Fix For: 3.3.2 > > Time Spent: 6.5h > Remaining Estimate: 0h > > Spark 3.2.0 depends on Jackson 2.12.3. Let's upgrade to 2.12.5 (2.12.x latest > as of now) or upper. > h2. this has been reverted. > we had to revert this as it broke tez. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18033) Upgrade fasterxml Jackson to 2.13.0
[ https://issues.apache.org/jira/browse/HADOOP-18033?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17731624#comment-17731624 ] Steve Loughran commented on HADOOP-18033: - transient CVE issues (snakeyaml) are generating motivation for upgrading hadoop jackson. anyone got an idea about how to do this in a way which could work downstream? > Upgrade fasterxml Jackson to 2.13.0 > --- > > Key: HADOOP-18033 > URL: https://issues.apache.org/jira/browse/HADOOP-18033 > Project: Hadoop Common > Issue Type: Improvement > Components: build >Reporter: Akira Ajisaka >Assignee: Viraj Jasani >Priority: Major > Labels: pull-request-available > Fix For: 3.3.2 > > Time Spent: 6.5h > Remaining Estimate: 0h > > Spark 3.2.0 depends on Jackson 2.12.3. Let's upgrade to 2.12.5 (2.12.x latest > as of now) or upper. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18033) Upgrade fasterxml Jackson to 2.13.0
[ https://issues.apache.org/jira/browse/HADOOP-18033?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17566593#comment-17566593 ] Viraj Jasani commented on HADOOP-18033: --- Thanks [~ayushtkn], HADOOP-18332 PRs (trunk and 3.3) seem good enough to unblock 3.3.4. > Upgrade fasterxml Jackson to 2.13.0 > --- > > Key: HADOOP-18033 > URL: https://issues.apache.org/jira/browse/HADOOP-18033 > Project: Hadoop Common > Issue Type: Improvement > Components: build >Reporter: Akira Ajisaka >Assignee: Viraj Jasani >Priority: Major > Labels: pull-request-available > Fix For: 3.4.0, 3.3.2 > > Time Spent: 6.5h > Remaining Estimate: 0h > > Spark 3.2.0 depends on Jackson 2.12.3. Let's upgrade to 2.12.5 (2.12.x latest > as of now) or upper. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18033) Upgrade fasterxml Jackson to 2.13.0
[ https://issues.apache.org/jira/browse/HADOOP-18033?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17566573#comment-17566573 ] Ayush Saxena commented on HADOOP-18033: --- I have approved the PR at HADOOP-18332, Tried the two Tez tests which failed with 3.3.3. They pass locally with those changes. Haven't run all the tests though... It is kind of revert of these changes, So, once folks involved here are convinced with the new changes. It can be merged. Nothing blocking from my side now. > Upgrade fasterxml Jackson to 2.13.0 > --- > > Key: HADOOP-18033 > URL: https://issues.apache.org/jira/browse/HADOOP-18033 > Project: Hadoop Common > Issue Type: Improvement > Components: build >Reporter: Akira Ajisaka >Assignee: Viraj Jasani >Priority: Major > Labels: pull-request-available > Fix For: 3.4.0, 3.3.2 > > Time Spent: 6.5h > Remaining Estimate: 0h > > Spark 3.2.0 depends on Jackson 2.12.3. Let's upgrade to 2.12.5 (2.12.x latest > as of now) or upper. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18033) Upgrade fasterxml Jackson to 2.13.0
[
https://issues.apache.org/jira/browse/HADOOP-18033?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17565986#comment-17565986
]
Ayush Saxena commented on HADOOP-18033:
---
[~ste...@apache.org] I have a PR which reverts the two commits here:
[https://github.com/apache/hadoop/pull/4544]
So, initial thought was to revert those commits and unblock the releases.
Then HADOOP-18332 came up with revert 2 + move to Jackson 2.12.7, so we don't
expose the CVE as well and remove the new jar which is creating problems. (Let
me know if need separate commits, like 2 different revert commits & one
upgrade, will do some CLI stuff with HADOOP-18332)
Both revert PR & the new PR have green builds, unfortunately I have a review
comment on the new one but that is no big stuff and to me that is the final
solution, unless other people come and block us. The plan was to try the Tez
stuff as well with that change & ask the other folks who flagged Spark issues
to try that as well, but considering the timelines, lets not spend too much
time there...
{*}So, in best case should unblock the release by day after{*}, considering the
build will take some 24 hours, if updated tomorrow.
Regarding trunk vs only branch-3.3, in favour of keeping all the branches in
sync for now, otherwise if some change comes in trunk which uses this new jar,
then we would be doing this revert exercise again and with new set of problems.
Moreover no point in keeping the trunk also in broken state.
[~vjasani] regarding the effort due to this revert activity and so. The best
offer I have is "I can help or worst get some help", may be with some rebase
effort, so this revert activity doesn't become an overhead for you.
> Upgrade fasterxml Jackson to 2.13.0
> ---
>
> Key: HADOOP-18033
> URL: https://issues.apache.org/jira/browse/HADOOP-18033
> Project: Hadoop Common
> Issue Type: Improvement
> Components: build
>Reporter: Akira Ajisaka
>Assignee: Viraj Jasani
>Priority: Major
> Labels: pull-request-available
> Fix For: 3.4.0, 3.3.2
>
> Time Spent: 6.5h
> Remaining Estimate: 0h
>
> Spark 3.2.0 depends on Jackson 2.12.3. Let's upgrade to 2.12.5 (2.12.x latest
> as of now) or upper.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18033) Upgrade fasterxml Jackson to 2.13.0
[ https://issues.apache.org/jira/browse/HADOOP-18033?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17565978#comment-17565978 ] PJ Fanning commented on HADOOP-18033: - [~ste...@apache.org] I've had to make a change to https://github.com/apache/hadoop/pull/4547 - there is also https://github.com/apache/hadoop/pull/4544 (which builds ok). The difference is that 4544 uses an older version of Jackson - but both PRs involve downgrading Jackson. Is there a branch you would prefer us to target for your 3.3.3+ cve-only release? > Upgrade fasterxml Jackson to 2.13.0 > --- > > Key: HADOOP-18033 > URL: https://issues.apache.org/jira/browse/HADOOP-18033 > Project: Hadoop Common > Issue Type: Improvement > Components: build >Reporter: Akira Ajisaka >Assignee: Viraj Jasani >Priority: Major > Labels: pull-request-available > Fix For: 3.4.0, 3.3.2 > > Time Spent: 6.5h > Remaining Estimate: 0h > > Spark 3.2.0 depends on Jackson 2.12.3. Let's upgrade to 2.12.5 (2.12.x latest > as of now) or upper. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18033) Upgrade fasterxml Jackson to 2.13.0
[ https://issues.apache.org/jira/browse/HADOOP-18033?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17565967#comment-17565967 ] Steve Loughran commented on HADOOP-18033: - i want to kick off a 3.3 3+ cve-only release this week, with the real "branch-3.3" coming later. what do we do here? as the longer we think about this the more PRs to update even more jars will surface > Upgrade fasterxml Jackson to 2.13.0 > --- > > Key: HADOOP-18033 > URL: https://issues.apache.org/jira/browse/HADOOP-18033 > Project: Hadoop Common > Issue Type: Improvement > Components: build >Reporter: Akira Ajisaka >Assignee: Viraj Jasani >Priority: Major > Labels: pull-request-available > Fix For: 3.4.0, 3.3.2 > > Time Spent: 6.5h > Remaining Estimate: 0h > > Spark 3.2.0 depends on Jackson 2.12.3. Let's upgrade to 2.12.5 (2.12.x latest > as of now) or upper. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18033) Upgrade fasterxml Jackson to 2.13.0
[
https://issues.apache.org/jira/browse/HADOOP-18033?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17565145#comment-17565145
]
Viraj Jasani commented on HADOOP-18033:
---
{quote}Currently I recommend downgrading to 2.12.7 in both trunk and
branch-3.3. That way we don't need to treat HADOOP-15984 as a blocker for 3.4.0.
{quote}
I understand that if we are doing the revert with a new Jira, the new Jira
should ideally land on trunk before making it's way to active release branches,
but Jackson downgrade to 2.12.7 and removal of javax.ws.rs-api would also
likely need to be reverted as part of HADOOP-15984, so for HADOOP-15984 it will
be too much work staying upto date with trunk (it's already struggling to do so
btw with whatever progress is made), and now it will have to reintroduce
javax.ws.rs-api and remove jsr311-api. So far I have jsr311-api removed from
the current local patch, but if trunk removes javax.ws.rs-api as part of revert
of HADOOP-18033 on trunk, there will be rework (basically, revert of revert of
HADOOP-18033 for HADOOP-15984 to make progress).
> Upgrade fasterxml Jackson to 2.13.0
> ---
>
> Key: HADOOP-18033
> URL: https://issues.apache.org/jira/browse/HADOOP-18033
> Project: Hadoop Common
> Issue Type: Improvement
> Components: build
>Reporter: Akira Ajisaka
>Assignee: Viraj Jasani
>Priority: Major
> Labels: pull-request-available
> Fix For: 3.4.0, 3.3.2
>
> Time Spent: 6h 20m
> Remaining Estimate: 0h
>
> Spark 3.2.0 depends on Jackson 2.12.3. Let's upgrade to 2.12.5 (2.12.x latest
> as of now) or upper.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18033) Upgrade fasterxml Jackson to 2.13.0
[
https://issues.apache.org/jira/browse/HADOOP-18033?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17565079#comment-17565079
]
Ayush Saxena commented on HADOOP-18033:
---
{quote}is it ok to downgrade jackson to 2.12.7? - has latest CVE fixes but not
this change
{quote}
Sounds good to me , if we get rid of javax.ws.rs-api dependency without
compromising on the CVE, I think there isn't anything better which we can think
of.
[~aajisaka] too pointed that we can explore moving to 2.12.7. Initially this
Jira too was raised to move Jackson to 2.12.x latest. I think if the build
doesn't complain post removing javax.ws.rs-api and moving to 2.12.7, then we
are sorted
> Upgrade fasterxml Jackson to 2.13.0
> ---
>
> Key: HADOOP-18033
> URL: https://issues.apache.org/jira/browse/HADOOP-18033
> Project: Hadoop Common
> Issue Type: Improvement
> Components: build
>Reporter: Akira Ajisaka
>Assignee: Viraj Jasani
>Priority: Major
> Labels: pull-request-available
> Fix For: 3.4.0, 3.3.2
>
> Time Spent: 6h
> Remaining Estimate: 0h
>
> Spark 3.2.0 depends on Jackson 2.12.3. Let's upgrade to 2.12.5 (2.12.x latest
> as of now) or upper.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18033) Upgrade fasterxml Jackson to 2.13.0
[ https://issues.apache.org/jira/browse/HADOOP-18033?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17565050#comment-17565050 ] PJ Fanning commented on HADOOP-18033: - So the Tez issue seems (possibly) to be caused by https://github.com/FasterXML/jackson-jaxrs-providers/issues/134 - is it ok to downgrade jackson to 2.12.7? - has latest CVE fixes but not this change > Upgrade fasterxml Jackson to 2.13.0 > --- > > Key: HADOOP-18033 > URL: https://issues.apache.org/jira/browse/HADOOP-18033 > Project: Hadoop Common > Issue Type: Improvement > Components: build >Reporter: Akira Ajisaka >Assignee: Viraj Jasani >Priority: Major > Labels: pull-request-available > Fix For: 3.4.0, 3.3.2 > > Time Spent: 6h > Remaining Estimate: 0h > > Spark 3.2.0 depends on Jackson 2.12.3. Let's upgrade to 2.12.5 (2.12.x latest > as of now) or upper. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18033) Upgrade fasterxml Jackson to 2.13.0
[ https://issues.apache.org/jira/browse/HADOOP-18033?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17564983#comment-17564983 ] Ayush Saxena commented on HADOOP-18033: --- [~pj.fanning] I think that was tried: https://github.com/apache/hadoop/pull/4460 Try TestHttpServer#testJersey if it passes with your code change > Upgrade fasterxml Jackson to 2.13.0 > --- > > Key: HADOOP-18033 > URL: https://issues.apache.org/jira/browse/HADOOP-18033 > Project: Hadoop Common > Issue Type: Improvement > Components: build >Reporter: Akira Ajisaka >Assignee: Viraj Jasani >Priority: Major > Labels: pull-request-available > Fix For: 3.4.0, 3.3.2 > > Time Spent: 5.5h > Remaining Estimate: 0h > > Spark 3.2.0 depends on Jackson 2.12.3. Let's upgrade to 2.12.5 (2.12.x latest > as of now) or upper. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18033) Upgrade fasterxml Jackson to 2.13.0
[ https://issues.apache.org/jira/browse/HADOOP-18033?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17564966#comment-17564966 ] PJ Fanning commented on HADOOP-18033: - [~ayushtkn] would https://issues.apache.org/jira/browse/HADOOP-18332 be worth trying first before looking to undo jackson/rs-api changes? I've been doing a build locally and so far, at least, things look ok (that jsr311-api dependency can be removed). > Upgrade fasterxml Jackson to 2.13.0 > --- > > Key: HADOOP-18033 > URL: https://issues.apache.org/jira/browse/HADOOP-18033 > Project: Hadoop Common > Issue Type: Improvement > Components: build >Reporter: Akira Ajisaka >Assignee: Viraj Jasani >Priority: Major > Labels: pull-request-available > Fix For: 3.4.0, 3.3.2 > > Time Spent: 5.5h > Remaining Estimate: 0h > > Spark 3.2.0 depends on Jackson 2.12.3. Let's upgrade to 2.12.5 (2.12.x latest > as of now) or upper. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18033) Upgrade fasterxml Jackson to 2.13.0
[ https://issues.apache.org/jira/browse/HADOOP-18033?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17564957#comment-17564957 ] Steve Loughran commented on HADOOP-18033: - ok. someone do the 3.3.x revert and i will get it into the 3.3.4 release i will kick off once it is in > Upgrade fasterxml Jackson to 2.13.0 > --- > > Key: HADOOP-18033 > URL: https://issues.apache.org/jira/browse/HADOOP-18033 > Project: Hadoop Common > Issue Type: Improvement > Components: build >Reporter: Akira Ajisaka >Assignee: Viraj Jasani >Priority: Major > Labels: pull-request-available > Fix For: 3.4.0, 3.3.2 > > Time Spent: 5.5h > Remaining Estimate: 0h > > Spark 3.2.0 depends on Jackson 2.12.3. Let's upgrade to 2.12.5 (2.12.x latest > as of now) or upper. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18033) Upgrade fasterxml Jackson to 2.13.0
[ https://issues.apache.org/jira/browse/HADOOP-18033?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17564839#comment-17564839 ] Ayush Saxena commented on HADOOP-18033: --- Well, Jersey upgrade is good to have, for Java-11, kind of new feature support. If we have it in a good and safe manor, But yes I too believe it will have a bunch of incompatible changes, need to see, how to handle that... Reverting for now in both trunk & branch-3.3, makes sense, it will allow us time and won't be blocking any of the release lines. I reverted locally these two commits and pushed a PR. Hopefully it shouldn't break any tests. https://github.com/apache/hadoop/pull/4544 > Upgrade fasterxml Jackson to 2.13.0 > --- > > Key: HADOOP-18033 > URL: https://issues.apache.org/jira/browse/HADOOP-18033 > Project: Hadoop Common > Issue Type: Improvement > Components: build >Reporter: Akira Ajisaka >Assignee: Viraj Jasani >Priority: Major > Labels: pull-request-available > Fix For: 3.4.0, 3.3.2 > > Time Spent: 5.5h > Remaining Estimate: 0h > > Spark 3.2.0 depends on Jackson 2.12.3. Let's upgrade to 2.12.5 (2.12.x latest > as of now) or upper. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18033) Upgrade fasterxml Jackson to 2.13.0
[ https://issues.apache.org/jira/browse/HADOOP-18033?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17564792#comment-17564792 ] Akira Ajisaka commented on HADOOP-18033: [~vjasani] Currently I recommend downgrading to 2.12.7 in both trunk and branch-3.3. That way we don't need to treat HADOOP-15984 as a blocker for 3.4.0. > Upgrade fasterxml Jackson to 2.13.0 > --- > > Key: HADOOP-18033 > URL: https://issues.apache.org/jira/browse/HADOOP-18033 > Project: Hadoop Common > Issue Type: Improvement > Components: build >Reporter: Akira Ajisaka >Assignee: Viraj Jasani >Priority: Major > Labels: pull-request-available > Fix For: 3.4.0, 3.3.2 > > Time Spent: 5.5h > Remaining Estimate: 0h > > Spark 3.2.0 depends on Jackson 2.12.3. Let's upgrade to 2.12.5 (2.12.x latest > as of now) or upper. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18033) Upgrade fasterxml Jackson to 2.13.0
[ https://issues.apache.org/jira/browse/HADOOP-18033?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17564787#comment-17564787 ] Viraj Jasani commented on HADOOP-18033: --- [~aajisaka] do you recommend downgrading to 2.12.7 only for branch-3.3 or for trunk as well? Can trunk still be as is because Jersey upgrade work is in progress? For 3.4.0 release, HADOOP-15984 can be treated as blocker? > Upgrade fasterxml Jackson to 2.13.0 > --- > > Key: HADOOP-18033 > URL: https://issues.apache.org/jira/browse/HADOOP-18033 > Project: Hadoop Common > Issue Type: Improvement > Components: build >Reporter: Akira Ajisaka >Assignee: Viraj Jasani >Priority: Major > Labels: pull-request-available > Fix For: 3.4.0, 3.3.2 > > Time Spent: 5.5h > Remaining Estimate: 0h > > Spark 3.2.0 depends on Jackson 2.12.3. Let's upgrade to 2.12.5 (2.12.x latest > as of now) or upper. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18033) Upgrade fasterxml Jackson to 2.13.0
[ https://issues.apache.org/jira/browse/HADOOP-18033?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17564780#comment-17564780 ] Akira Ajisaka commented on HADOOP-18033: Note: When reverting this issue, I recommend to use a separate JIRA because the change was released in 3.3.2 and 3.3.3. That way we can easily track what change is in the specific release. > Upgrade fasterxml Jackson to 2.13.0 > --- > > Key: HADOOP-18033 > URL: https://issues.apache.org/jira/browse/HADOOP-18033 > Project: Hadoop Common > Issue Type: Improvement > Components: build >Reporter: Akira Ajisaka >Assignee: Viraj Jasani >Priority: Major > Labels: pull-request-available > Fix For: 3.4.0, 3.3.2 > > Time Spent: 5.5h > Remaining Estimate: 0h > > Spark 3.2.0 depends on Jackson 2.12.3. Let's upgrade to 2.12.5 (2.12.x latest > as of now) or upper. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18033) Upgrade fasterxml Jackson to 2.13.0
[ https://issues.apache.org/jira/browse/HADOOP-18033?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17564761#comment-17564761 ] Akira Ajisaka commented on HADOOP-18033: bq. we might have to call out on the Jackson CVE The CVE is fixed in 2.12.6.1 or upper (https://github.com/FasterXML/jackson-databind/issues/2816), therefore we should change the version to 2.12.7 (the latest 2.12.x as of now). > Upgrade fasterxml Jackson to 2.13.0 > --- > > Key: HADOOP-18033 > URL: https://issues.apache.org/jira/browse/HADOOP-18033 > Project: Hadoop Common > Issue Type: Improvement > Components: build >Reporter: Akira Ajisaka >Assignee: Viraj Jasani >Priority: Major > Labels: pull-request-available > Fix For: 3.4.0, 3.3.2 > > Time Spent: 5.5h > Remaining Estimate: 0h > > Spark 3.2.0 depends on Jackson 2.12.3. Let's upgrade to 2.12.5 (2.12.x latest > as of now) or upper. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18033) Upgrade fasterxml Jackson to 2.13.0
[
https://issues.apache.org/jira/browse/HADOOP-18033?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17564712#comment-17564712
]
Viraj Jasani commented on HADOOP-18033:
---
{quote}In my past experience, Jersey 2.x upgrade takes a lot of time and I
think it will cause some incompatible changes.
{quote}
I agree that 3.3 subsequent releases should not wait for Jersey 2 because of
the sheer volume of changes and incompatibility with Jersey 1.
>From my previous comment:
{quote}FWIW, although Hadoop 3.3 could revert this for 3.3.4 release but from
security viewpoint, staying up with latest Jackson2 is also in good favour of
3.3 release line, given that 3.3 is the latest release line.
{quote}
we might have to call out on the Jackson CVE that we claimed to have fixed with
3.3.2 and 3.3.3 and now 3.3.4 would get it exposed with the revert.
IIRC, Jersey 1.19 is not flagged by security for active CVEs but Jackson
versions <= 2.12 are?
> Upgrade fasterxml Jackson to 2.13.0
> ---
>
> Key: HADOOP-18033
> URL: https://issues.apache.org/jira/browse/HADOOP-18033
> Project: Hadoop Common
> Issue Type: Improvement
> Components: build
>Reporter: Akira Ajisaka
>Assignee: Viraj Jasani
>Priority: Major
> Labels: pull-request-available
> Fix For: 3.4.0, 3.3.2
>
> Time Spent: 5.5h
> Remaining Estimate: 0h
>
> Spark 3.2.0 depends on Jackson 2.12.3. Let's upgrade to 2.12.5 (2.12.x latest
> as of now) or upper.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18033) Upgrade fasterxml Jackson to 2.13.0
[ https://issues.apache.org/jira/browse/HADOOP-18033?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17564702#comment-17564702 ] Cheng Pan commented on HADOOP-18033: Reverting Jackson in the 3.3 branch looks reasonable to me, since Kyuubi and Spark use Hadoop shaded client, downgrading Jackson from 2.13 to 2.12 should not cause another dependency issue. > Upgrade fasterxml Jackson to 2.13.0 > --- > > Key: HADOOP-18033 > URL: https://issues.apache.org/jira/browse/HADOOP-18033 > Project: Hadoop Common > Issue Type: Improvement > Components: build >Reporter: Akira Ajisaka >Assignee: Viraj Jasani >Priority: Major > Labels: pull-request-available > Fix For: 3.4.0, 3.3.2 > > Time Spent: 5.5h > Remaining Estimate: 0h > > Spark 3.2.0 depends on Jackson 2.12.3. Let's upgrade to 2.12.5 (2.12.x latest > as of now) or upper. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18033) Upgrade fasterxml Jackson to 2.13.0
[ https://issues.apache.org/jira/browse/HADOOP-18033?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17564665#comment-17564665 ] Ayush Saxena commented on HADOOP-18033: --- Thanx [~aajisaka] for checking, I am +1 on revert > Upgrade fasterxml Jackson to 2.13.0 > --- > > Key: HADOOP-18033 > URL: https://issues.apache.org/jira/browse/HADOOP-18033 > Project: Hadoop Common > Issue Type: Improvement > Components: build >Reporter: Akira Ajisaka >Assignee: Viraj Jasani >Priority: Major > Labels: pull-request-available > Fix For: 3.4.0, 3.3.2 > > Time Spent: 5.5h > Remaining Estimate: 0h > > Spark 3.2.0 depends on Jackson 2.12.3. Let's upgrade to 2.12.5 (2.12.x latest > as of now) or upper. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18033) Upgrade fasterxml Jackson to 2.13.0
[ https://issues.apache.org/jira/browse/HADOOP-18033?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17564664#comment-17564664 ] Akira Ajisaka commented on HADOOP-18033: Thank you [~ayushtkn] [~pan3793] [~vjasani] for your discussion. I'm really sorry for not caching up with. In my past experience, Jersey 2.x upgrade takes a lot of time and I think it will cause some incompatible changes. Therefore I think we should revert the patch and update to 2.12.x latest to avoid the above issue for (at least) Hadoop 3.3.x. What do you think? > Upgrade fasterxml Jackson to 2.13.0 > --- > > Key: HADOOP-18033 > URL: https://issues.apache.org/jira/browse/HADOOP-18033 > Project: Hadoop Common > Issue Type: Improvement > Components: build >Reporter: Akira Ajisaka >Assignee: Viraj Jasani >Priority: Major > Labels: pull-request-available > Fix For: 3.4.0, 3.3.2 > > Time Spent: 5.5h > Remaining Estimate: 0h > > Spark 3.2.0 depends on Jackson 2.12.3. Let's upgrade to 2.12.5 (2.12.x latest > as of now) or upper. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18033) Upgrade fasterxml Jackson to 2.13.0
[
https://issues.apache.org/jira/browse/HADOOP-18033?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17556118#comment-17556118
]
Viraj Jasani commented on HADOOP-18033:
---
{quote}Excluding/Removing javax.ws.rs-api will make Jackson cry.
{quote}
{quote}Excluding/Removing jsr311-api makes Jersey cry.
{quote}
That is so true.
Unfortunately we have not received a single QA result on PR#4460 so far.
Although Jenkins is getting shut down it seems, at least we have bunch of test
results available on PR#4461, hence perhaps some test failures are making
builds difficult on 4460. Anyways, we might have to spend another day to see
the full results. But we are sure we will have test failures (likely more than
expected, as Ayush already mentioned about TestHttpServer#testJersey).
Overall, it seems we are back to HADOOP-15984 (upgrading Jersey to 2.x) because
so long as we have "com.sun.jersey:jersey-core", things will stay complicated.
> Upgrade fasterxml Jackson to 2.13.0
> ---
>
> Key: HADOOP-18033
> URL: https://issues.apache.org/jira/browse/HADOOP-18033
> Project: Hadoop Common
> Issue Type: Improvement
> Components: build
>Reporter: Akira Ajisaka
>Assignee: Viraj Jasani
>Priority: Major
> Labels: pull-request-available
> Fix For: 3.4.0, 3.3.2
>
> Time Spent: 5h 10m
> Remaining Estimate: 0h
>
> Spark 3.2.0 depends on Jackson 2.12.3. Let's upgrade to 2.12.5 (2.12.x latest
> as of now) or upper.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18033) Upgrade fasterxml Jackson to 2.13.0
[
https://issues.apache.org/jira/browse/HADOOP-18033?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17556066#comment-17556066
]
Ayush Saxena commented on HADOOP-18033:
---
Shading issue we might get rid of, because the issue wasn't in the same jar, so
that isn't a problem & that might solve problem for some. Tez doesn't pull in
hadoop-client dependency AFAIK,
Now after this it is pulling in both javax.ws.rs-api and jsr311-api which is
creating runtime issues like URIBuilder Class is there in both the packages and
so.
Excluding/Removing javax.ws.rs-api will make Jackson cry. I tried this as well
a lot of test fails.
Excluding/Removing jsr311-api makes Jersey cry. Like TestHttpServer#testJersey
fails if we exclude it. I tried this only but some more can fail as well.
This gives the same error what Tez is getting:
{noformat}
2022-06-19 17:53:24,623 WARN server.HttpChannel
(HttpChannel.java:handleException(689)) - /jersey/foo
java.lang.AbstractMethodError:
javax.ws.rs.core.UriBuilder.uri(Ljava/lang/String;)Ljavax/ws/rs/core/UriBuilder;
{noformat}
> Upgrade fasterxml Jackson to 2.13.0
> ---
>
> Key: HADOOP-18033
> URL: https://issues.apache.org/jira/browse/HADOOP-18033
> Project: Hadoop Common
> Issue Type: Improvement
> Components: build
>Reporter: Akira Ajisaka
>Assignee: Viraj Jasani
>Priority: Major
> Labels: pull-request-available
> Fix For: 3.4.0, 3.3.2
>
> Time Spent: 4h 20m
> Remaining Estimate: 0h
>
> Spark 3.2.0 depends on Jackson 2.12.3. Let's upgrade to 2.12.5 (2.12.x latest
> as of now) or upper.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18033) Upgrade fasterxml Jackson to 2.13.0
[ https://issues.apache.org/jira/browse/HADOOP-18033?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17556004#comment-17556004 ] Cheng Pan commented on HADOOP-18033: Looks reasonable, thanks [~ayushtkn] and [~vjasani] for investigating this issue. cc [~csun] > Upgrade fasterxml Jackson to 2.13.0 > --- > > Key: HADOOP-18033 > URL: https://issues.apache.org/jira/browse/HADOOP-18033 > Project: Hadoop Common > Issue Type: Improvement > Components: build >Reporter: Akira Ajisaka >Assignee: Viraj Jasani >Priority: Major > Labels: pull-request-available > Fix For: 3.4.0, 3.3.2 > > Time Spent: 4h 20m > Remaining Estimate: 0h > > Spark 3.2.0 depends on Jackson 2.12.3. Let's upgrade to 2.12.5 (2.12.x latest > as of now) or upper. -- This message was sent by Atlassian Jira (v8.20.7#820007) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18033) Upgrade fasterxml Jackson to 2.13.0
[ https://issues.apache.org/jira/browse/HADOOP-18033?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17556003#comment-17556003 ] Viraj Jasani commented on HADOOP-18033: --- Once QA results are available, will create new Jira to link both PRs #4460 and #4461. > Upgrade fasterxml Jackson to 2.13.0 > --- > > Key: HADOOP-18033 > URL: https://issues.apache.org/jira/browse/HADOOP-18033 > Project: Hadoop Common > Issue Type: Improvement > Components: build >Reporter: Akira Ajisaka >Assignee: Viraj Jasani >Priority: Major > Labels: pull-request-available > Fix For: 3.4.0, 3.3.2 > > Time Spent: 4h 20m > Remaining Estimate: 0h > > Spark 3.2.0 depends on Jackson 2.12.3. Let's upgrade to 2.12.5 (2.12.x latest > as of now) or upper. -- This message was sent by Atlassian Jira (v8.20.7#820007) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18033) Upgrade fasterxml Jackson to 2.13.0
[
https://issues.apache.org/jira/browse/HADOOP-18033?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17556002#comment-17556002
]
Viraj Jasani commented on HADOOP-18033:
---
Yeah that problem should be at least resolved by #4461 even if #4460 turns out
to be complicated. If #4460 is good, maybe we can pursue some testing in ATSv2
and Yarn to verify the basic functionalities with REST APIs.
{quote}The duplicate error that you were quoting was conflict in between
hadoop-client-runtime vs hadoop-minicluster
{quote}
Unfortunately, yes i also realized this now :(
> Upgrade fasterxml Jackson to 2.13.0
> ---
>
> Key: HADOOP-18033
> URL: https://issues.apache.org/jira/browse/HADOOP-18033
> Project: Hadoop Common
> Issue Type: Improvement
> Components: build
>Reporter: Akira Ajisaka
>Assignee: Viraj Jasani
>Priority: Major
> Labels: pull-request-available
> Fix For: 3.4.0, 3.3.2
>
> Time Spent: 4h 20m
> Remaining Estimate: 0h
>
> Spark 3.2.0 depends on Jackson 2.12.3. Let's upgrade to 2.12.5 (2.12.x latest
> as of now) or upper.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18033) Upgrade fasterxml Jackson to 2.13.0
[
https://issues.apache.org/jira/browse/HADOOP-18033?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17556001#comment-17556001
]
Ayush Saxena commented on HADOOP-18033:
---
{quote}Actually, javax.ws.rs-api is not even clashing with jsr311-api (weird,
didn't expect this).
{quote}
The duplicate error that you were quoting was conflict in between
hadoop-client-runtime vs hadoop-minicluster :(
Looks like yes:
{noformat}
Duplicate classes found: Found in:
org.apache.hadoop:hadoop-client-minicluster:jar:3.4.0-SNAPSHOT:compile
org.apache.hadoop:hadoop-client-runtime:jar:3.4.0-SNAPSHOT:compile{noformat}
AFAIK Whatever gets shaded in hadoop-client-runtime we anyway have to exclude
from hadoop-minicluster, we can't shade it in both, there are bunch of lines
like. : {{exclude everything that comes in via the shaded runtime and api}}
{{exclude things that came in via transitive in shaded runtime and api}}
the problem that [~pan3793] quoted for spark & kyubi should get sorted atleast
by this
> Upgrade fasterxml Jackson to 2.13.0
> ---
>
> Key: HADOOP-18033
> URL: https://issues.apache.org/jira/browse/HADOOP-18033
> Project: Hadoop Common
> Issue Type: Improvement
> Components: build
>Reporter: Akira Ajisaka
>Assignee: Viraj Jasani
>Priority: Major
> Labels: pull-request-available
> Fix For: 3.4.0, 3.3.2
>
> Time Spent: 4h 20m
> Remaining Estimate: 0h
>
> Spark 3.2.0 depends on Jackson 2.12.3. Let's upgrade to 2.12.5 (2.12.x latest
> as of now) or upper.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18033) Upgrade fasterxml Jackson to 2.13.0
[
https://issues.apache.org/jira/browse/HADOOP-18033?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17556000#comment-17556000
]
Viraj Jasani commented on HADOOP-18033:
---
Actually, javax.ws.rs-api is not even clashing with jsr311-api (weird, didn't
expect this).
I just applied this patch, and the build is successful:
{code:java}
diff --git a/hadoop-client-modules/hadoop-client-runtime/pom.xml
b/hadoop-client-modules/hadoop-client-runtime/pom.xml
index 35fbd7665fb..0879ce1e3bc 100644
--- a/hadoop-client-modules/hadoop-client-runtime/pom.xml
+++ b/hadoop-client-modules/hadoop-client-runtime/pom.xml
@@ -163,7 +163,6 @@
org.bouncycastle:*
org.xerial.snappy:*
- javax.ws.rs:javax.ws.rs-api
{code}
Created PR [https://github.com/apache/hadoop/pull/4461] for full build results.
We can compare QA results for both PRs tomorrow.
> Upgrade fasterxml Jackson to 2.13.0
> ---
>
> Key: HADOOP-18033
> URL: https://issues.apache.org/jira/browse/HADOOP-18033
> Project: Hadoop Common
> Issue Type: Improvement
> Components: build
>Reporter: Akira Ajisaka
>Assignee: Viraj Jasani
>Priority: Major
> Labels: pull-request-available
> Fix For: 3.4.0, 3.3.2
>
> Time Spent: 4h 20m
> Remaining Estimate: 0h
>
> Spark 3.2.0 depends on Jackson 2.12.3. Let's upgrade to 2.12.5 (2.12.x latest
> as of now) or upper.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18033) Upgrade fasterxml Jackson to 2.13.0
[
https://issues.apache.org/jira/browse/HADOOP-18033?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17555999#comment-17555999
]
Viraj Jasani commented on HADOOP-18033:
---
Thanks [~ayushtkn], yes we should hopefully get the full build QA results in
~24 hrs.
On the shading side, I meant:
{quote}I don't think it was as simple as "completely removing jsr311-api from
Hadoop" would allow us to exclude shading javax.ws.rs-api from both
hadoop-client-minicluster and hadoop-client-runtime
{quote}
I tried this again and the build fails with the same error that I faced earlier:
{code:java}
[INFO] ---< org.apache.hadoop:hadoop-client-check-test-invariants >
[INFO] Building Apache Hadoop Client Packaging Invariants for Test
3.4.0-SNAPSHOT [105/112]
[INFO] [ pom ]-
[INFO]
[INFO] --- maven-clean-plugin:3.1.0:clean (default-clean) @
hadoop-client-check-test-invariants ---
[INFO] Deleting
/Users/vjasani/Documents/src/hadoop-trunk/hadoop/hadoop-client-modules/hadoop-client-check-test-invariants/target
[INFO] Deleting
/Users/vjasani/Documents/src/hadoop-trunk/hadoop/hadoop-client-modules/hadoop-client-check-test-invariants
(includes = [dependency-reduced-pom.xml], excludes = [])
[INFO]
[INFO] --- maven-antrun-plugin:1.7:run (create-testdirs) @
hadoop-client-check-test-invariants ---
[INFO] Executing tasks
main:
[mkdir] Created dir:
/Users/vjasani/Documents/src/hadoop-trunk/hadoop/hadoop-client-modules/hadoop-client-check-test-invariants/target/test-dir
[INFO] Executed tasks
[INFO]
[INFO] --- maven-enforcer-plugin:3.0.0:enforce (enforce-banned-dependencies) @
hadoop-client-check-test-invariants ---
[INFO] Adding ignore: module-info
[INFO] Adding ignore: META-INF/versions/*/module-info
[INFO] Adding ignorable dependency: org.apache.hadoop:hadoop-annotations:null
[INFO] Adding ignore: *
[WARNING] Rule 1: org.apache.maven.plugins.enforcer.BanDuplicateClasses failed
with message:
Duplicate classes found:
Found in:
org.apache.hadoop:hadoop-client-minicluster:jar:3.4.0-SNAPSHOT:compile
org.apache.hadoop:hadoop-client-runtime:jar:3.4.0-SNAPSHOT:compile
Duplicate classes:
org/apache/hadoop/shaded/javax/ws/rs/POST.class
org/apache/hadoop/shaded/javax/ws/rs/core/Link$JaxbLink.class
org/apache/hadoop/shaded/javax/ws/rs/NotFoundException.class
org/apache/hadoop/shaded/javax/ws/rs/container/PreMatching.class
org/apache/hadoop/shaded/javax/ws/rs/container/ContainerRequestContext.class
org/apache/hadoop/shaded/javax/ws/rs/core/FeatureContext.class
org/apache/hadoop/shaded/javax/ws/rs/core/HttpHeaders.class
org/apache/hadoop/shaded/javax/ws/rs/PATCH.class
org/apache/hadoop/shaded/javax/ws/rs/sse/OutboundSseEvent$Builder.class
org/apache/hadoop/shaded/javax/ws/rs/core/GenericType.class
org/apache/hadoop/shaded/javax/ws/rs/sse/SseBroadcaster.class
org/apache/hadoop/shaded/javax/ws/rs/core/MediaType$2.class
org/apache/hadoop/shaded/javax/ws/rs/core/StreamingOutput.class
...
...
... {code}
Hence, with the above PR, I have removed exclusion only from
hadoop-client-runtime shade. Now we can confirm that these classes are present
in hadoop-client-runtime but not on hadoop-client-minicluster jar:
{code:java}
$ jar tf
hadoop-client-modules/hadoop-client-runtime/target/hadoop-client-runtime-3.4.0-SNAPSHOT.jar
| grep "AbstractMultivaluedMap"
org/apache/hadoop/shaded/javax/ws/rs/core/AbstractMultivaluedMap.class
$ jar tf
hadoop-client-modules/hadoop-client-minicluster/target/hadoop-client-minicluster-3.4.0-SNAPSHOT.jar
| grep "AbstractMultivaluedMap" {code}
> Upgrade fasterxml Jackson to 2.13.0
> ---
>
> Key: HADOOP-18033
> URL: https://issues.apache.org/jira/browse/HADOOP-18033
> Project: Hadoop Common
> Issue Type: Improvement
> Components: build
>Reporter: Akira Ajisaka
>Assignee: Viraj Jasani
>Priority: Major
> Labels: pull-request-available
> Fix For: 3.4.0, 3.3.2
>
> Time Spent: 4h 10m
> Remaining Estimate: 0h
>
> Spark 3.2.0 depends on Jackson 2.12.3. Let's upgrade to 2.12.5 (2.12.x latest
> as of now) or upper.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18033) Upgrade fasterxml Jackson to 2.13.0
[ https://issues.apache.org/jira/browse/HADOOP-18033?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17555993#comment-17555993 ] Ayush Saxena commented on HADOOP-18033: --- Removing jsr311-api from hadoop, will not cause duplicate file exception in shading, because when I tried it didn't... But whether we can do that? because jsr311-api & javax.ws.rs-api aren't compatible with each other. That is one thing I am sure, because that only caused Tez to give that AbstractMethod Error... The build will pass I think, If test fails it is good, at least we will come to know what is broken and what needs to be fixed and we can some how figure out how. It might come green as well, because couple of tests which I tried were passing.(Running whole Hadoop test suite isn't easy to run locally) Else if jersey is using jsr311-api, that can create runtime issues, earlier we had jsr311-api in our client jar, now we won't be having that, what impact to downstream projects, that will stay a mystery... Atleast we have to try we don't need have any transitive dependency of jsr311-api I think downstream projects have to get rid of jsr311-api and upgrade Jackson to adapt to this even if we sort this FWIW. Tez is already going ahead with 3.3.1 for the current release: [https://lists.apache.org/thread/7sw84rcc729fgw31g0w9h9y9r61tok9d] > Upgrade fasterxml Jackson to 2.13.0 > --- > > Key: HADOOP-18033 > URL: https://issues.apache.org/jira/browse/HADOOP-18033 > Project: Hadoop Common > Issue Type: Improvement > Components: build >Reporter: Akira Ajisaka >Assignee: Viraj Jasani >Priority: Major > Labels: pull-request-available > Fix For: 3.4.0, 3.3.2 > > Time Spent: 4h 10m > Remaining Estimate: 0h > > Spark 3.2.0 depends on Jackson 2.12.3. Let's upgrade to 2.12.5 (2.12.x latest > as of now) or upper. -- This message was sent by Atlassian Jira (v8.20.7#820007) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18033) Upgrade fasterxml Jackson to 2.13.0
[ https://issues.apache.org/jira/browse/HADOOP-18033?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17555982#comment-17555982 ] Viraj Jasani commented on HADOOP-18033: --- Remove jsr311-api dependency: [https://github.com/apache/hadoop/pull/4460] (to see how QA results go for now) > Upgrade fasterxml Jackson to 2.13.0 > --- > > Key: HADOOP-18033 > URL: https://issues.apache.org/jira/browse/HADOOP-18033 > Project: Hadoop Common > Issue Type: Improvement > Components: build >Reporter: Akira Ajisaka >Assignee: Viraj Jasani >Priority: Major > Labels: pull-request-available > Fix For: 3.4.0, 3.3.2 > > Time Spent: 4h 10m > Remaining Estimate: 0h > > Spark 3.2.0 depends on Jackson 2.12.3. Let's upgrade to 2.12.5 (2.12.x latest > as of now) or upper. -- This message was sent by Atlassian Jira (v8.20.7#820007) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18033) Upgrade fasterxml Jackson to 2.13.0
[ https://issues.apache.org/jira/browse/HADOOP-18033?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17555980#comment-17555980 ] Viraj Jasani commented on HADOOP-18033: --- [~pan3793] HADOOP-15984 has migration to Jersey 2.x related efforts going on. > Upgrade fasterxml Jackson to 2.13.0 > --- > > Key: HADOOP-18033 > URL: https://issues.apache.org/jira/browse/HADOOP-18033 > Project: Hadoop Common > Issue Type: Improvement > Components: build >Reporter: Akira Ajisaka >Assignee: Viraj Jasani >Priority: Major > Labels: pull-request-available > Fix For: 3.4.0, 3.3.2 > > Time Spent: 4h 10m > Remaining Estimate: 0h > > Spark 3.2.0 depends on Jackson 2.12.3. Let's upgrade to 2.12.5 (2.12.x latest > as of now) or upper. -- This message was sent by Atlassian Jira (v8.20.7#820007) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18033) Upgrade fasterxml Jackson to 2.13.0
[
https://issues.apache.org/jira/browse/HADOOP-18033?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17555979#comment-17555979
]
Viraj Jasani commented on HADOOP-18033:
---
[~pan3793] Thanks for your comments.
{quote}And I also see that Hadoop 3.3.2 mixed use jersey 1.x and 2.x in module
hadoop-yarn-applications-catalog-webapp, not sure if it's a good practice.
{quote}
Do you mean jersey-json or jersey-media-json-jackson? HADOOP-15983 has been a
recent work and jersey-media-json-jackson is a test dependency.
> Upgrade fasterxml Jackson to 2.13.0
> ---
>
> Key: HADOOP-18033
> URL: https://issues.apache.org/jira/browse/HADOOP-18033
> Project: Hadoop Common
> Issue Type: Improvement
> Components: build
>Reporter: Akira Ajisaka
>Assignee: Viraj Jasani
>Priority: Major
> Labels: pull-request-available
> Fix For: 3.4.0, 3.3.2
>
> Time Spent: 4h 10m
> Remaining Estimate: 0h
>
> Spark 3.2.0 depends on Jackson 2.12.3. Let's upgrade to 2.12.5 (2.12.x latest
> as of now) or upper.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18033) Upgrade fasterxml Jackson to 2.13.0
[ https://issues.apache.org/jira/browse/HADOOP-18033?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17555977#comment-17555977 ] Cheng Pan commented on HADOOP-18033: The mvnrepository[1] tips that com.sun.jersey:jersey-core was moved to [org.glassfish.jersey.core|https://mvnrepository.com/artifact/org.glassfish.jersey.core] Upgrading(or migrating) glassfish jersey 2.x should help. [1] https://mvnrepository.com/artifact/com.sun.jersey/jersey-core > Upgrade fasterxml Jackson to 2.13.0 > --- > > Key: HADOOP-18033 > URL: https://issues.apache.org/jira/browse/HADOOP-18033 > Project: Hadoop Common > Issue Type: Improvement > Components: build >Reporter: Akira Ajisaka >Assignee: Viraj Jasani >Priority: Major > Labels: pull-request-available > Fix For: 3.4.0, 3.3.2 > > Time Spent: 4h 10m > Remaining Estimate: 0h > > Spark 3.2.0 depends on Jackson 2.12.3. Let's upgrade to 2.12.5 (2.12.x latest > as of now) or upper. -- This message was sent by Atlassian Jira (v8.20.7#820007) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18033) Upgrade fasterxml Jackson to 2.13.0
[
https://issues.apache.org/jira/browse/HADOOP-18033?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17555973#comment-17555973
]
Viraj Jasani commented on HADOOP-18033:
---
{quote}The duplicate class exception that we saw here was actually an alarm
here that these two dependencies can't stay in peace together, but we got away
with that by an exclude...
{quote}
I don't think it was as simple as "completely removing jsr311-api from Hadoop"
would allow us to exclude shading javax.ws.rs-api from both
hadoop-client-minicluster and hadoop-client-runtime. I have already tried this
before, it doesn't work AFAIK. At least, one of them would have to keep the
exclusion on (just like say for instance, hadoop-annotation I guess).
HADOOP-15983 has upgraded all com.sun.jersey dependencies (jersey-core,
jersey-servlet etc) to the latest version and still the latest version of
jersey-core pulls-in jsr311-api with it:
{code:java}
[INFO] | +- com.sun.jersey:jersey-core:jar:1.19.4:compile
[INFO] | | \- javax.ws.rs:jsr311-api:jar:1.1.1:compile {code}
I don't think without exclusion (and maybe some additional code change, if
JAX-RS 1.x and 2.x incompatibilities are in use), we might be able to get rid
of jsr311-api. I would expect the same for Tez and other dependencies as well.
Tez and other dependencies also can explore the similar path of excluding
jsr311-api completely and only rely on JAX-RS 2.x based javax.ws.rs-api.
jsr311-api is the official spec jar for the JAX-RS 1.x line and the latest
central release available is from Nov, 2009 (too old)
[https://mvnrepository.com/artifact/javax.ws.rs/jsr311-api]
whereas javax.ws.rs-api is jar for JAX-RS 2.x line
[https://mvnrepository.com/artifact/javax.ws.rs/javax.ws.rs-api] (latest
version from 2018), hence we can expect more upgraded thirdparty libraries
(just like Jackson2) having dependency on javax.ws.rs-api and less on
jsr311-api. So all downstreamers (Tez, Hadoop, Spark) should try to get rid of
jsr311-api anyways, totally agree here. One dependency doesn't necessarily have
to wait for another to remove it, for instance, Tez can go ahead with exclusion
of jsr311-api even before upgrading to Hadoop 3.3.4 because if not Hadoop, some
of it's other dependencies (like jersey-core latest version, as mentioned
above) would likely anyways pull it in transitively.
On the other hand, let Hadoop also get rid of jsr311-api. But I am pretty sure,
removing it won't solve shading issue completely. Will come up with patch soon
because I do recall I have already tried this as part of this Jira only. We can
also run full build QA (all modules) and I can manually verify HDFS, MapReduce
and ATSv2 working on pseudo-distributed mode.
> Upgrade fasterxml Jackson to 2.13.0
> ---
>
> Key: HADOOP-18033
> URL: https://issues.apache.org/jira/browse/HADOOP-18033
> Project: Hadoop Common
> Issue Type: Improvement
> Components: build
>Reporter: Akira Ajisaka
>Assignee: Viraj Jasani
>Priority: Major
> Labels: pull-request-available
> Fix For: 3.4.0, 3.3.2
>
> Time Spent: 4h 10m
> Remaining Estimate: 0h
>
> Spark 3.2.0 depends on Jackson 2.12.3. Let's upgrade to 2.12.5 (2.12.x latest
> as of now) or upper.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18033) Upgrade fasterxml Jackson to 2.13.0
[ https://issues.apache.org/jira/browse/HADOOP-18033?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17555953#comment-17555953 ] Cheng Pan commented on HADOOP-18033: > BTW. I am not sure what Spark and Kyuubi issues are exactly, that also seems > class conflicts may be Yes, I think it's because Jackson requires some classes which only exist in javax.ws.rs-api, which does not bundle into the shaded client. Have a brief look, js311-api is only required by jersey 1.x? If yes, I think upgrading the jersey to 2.x which depends on javax.ws.rs-api and dropping js311-api may be the right direction. And I also see that Hadoop 3.3.2 mixed use jersey 1.x and 2.x in module hadoop-yarn-applications-catalog-webapp, not sure if it's a good practice. > Upgrade fasterxml Jackson to 2.13.0 > --- > > Key: HADOOP-18033 > URL: https://issues.apache.org/jira/browse/HADOOP-18033 > Project: Hadoop Common > Issue Type: Improvement > Components: build >Reporter: Akira Ajisaka >Assignee: Viraj Jasani >Priority: Major > Labels: pull-request-available > Fix For: 3.4.0, 3.3.2 > > Time Spent: 4h 10m > Remaining Estimate: 0h > > Spark 3.2.0 depends on Jackson 2.12.3. Let's upgrade to 2.12.5 (2.12.x latest > as of now) or upper. -- This message was sent by Atlassian Jira (v8.20.7#820007) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18033) Upgrade fasterxml Jackson to 2.13.0
[
https://issues.apache.org/jira/browse/HADOOP-18033?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17555916#comment-17555916
]
Ayush Saxena commented on HADOOP-18033:
---
Spent some time checking if we have any quick solution or not and see how
things are:
Both {{jsr311-api}} and {{javax.ws.rs-api}} have couple of similar classes and
different implementations, That is why this duplicate classes issue started
surfacing, I guess Jackson 2 requires implementation classes from
{{javax.ws.rs-api}} at runtime or so. In ideal situation we should either have
{{javax.ws.rs-api}} or {{js311-api}} in our code, when adding
{{javax.ws.rs-api}} if we could have got rid of {{js311-api}} then everything
would have been sorted for the shading part. But I guess we have some
dependencies on {{{}js311-api{}}}, and it is coming from some other thirdparty
libs as well, so may be we have to explore and upgrade them to a version, where
they ditch {{js311-api}} for {{{}javax.ws.rs-api{}}}. Then our shading jar
should get sorted. How tough is that we don't know, a normal exclude of
{{js311-api}} as a transitive dependency isn't a solution because
{{javax.ws.rs-api}} has different implementation of methods.
The duplicate class exception that we saw here was actually an alarm here that
these two dependencies can't stay in peace together, but we got away with that
by an exclude...
Now coming for Tez, Tez still has {{js311-api}} as a dependency, if we some how
ditch that and move to {{javax.ws.rs-api}} in hadoop, I am not very sure if Tez
too have to adapt to our Jackson version and do the same to get things working..
{quote}FWIW, although Hadoop 3.3 could revert this for 3.3.4 release but from
security viewpoint, staying up with latest Jackson2 is also in good favour of
3.3 release line
{quote}
Revert isn't an option now, HADOOP-18178 got its way clear only because of
this, else it would have been facing this same issue and would have crashed.
Now we have a CVE fixed in 3.3.2 & 3.3.3, we can't get it back in 3.3.4, We
won't fix a thirdparty CVE we could have said, but after fixing and claiming we
have fixed one, we can't get it back AFAIK, this issue only somehow we have to
fix.
BTW. I am not sure what Spark and Kyubi issues are exactly, that also seems
class conflicts may be.. [~pan3793] can you share some more information about
that here
> Upgrade fasterxml Jackson to 2.13.0
> ---
>
> Key: HADOOP-18033
> URL: https://issues.apache.org/jira/browse/HADOOP-18033
> Project: Hadoop Common
> Issue Type: Improvement
> Components: build
>Reporter: Akira Ajisaka
>Assignee: Viraj Jasani
>Priority: Major
> Labels: pull-request-available
> Fix For: 3.4.0, 3.3.2
>
> Time Spent: 4h 10m
> Remaining Estimate: 0h
>
> Spark 3.2.0 depends on Jackson 2.12.3. Let's upgrade to 2.12.5 (2.12.x latest
> as of now) or upper.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18033) Upgrade fasterxml Jackson to 2.13.0
[
https://issues.apache.org/jira/browse/HADOOP-18033?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17555848#comment-17555848
]
Ayush Saxena commented on HADOOP-18033:
---
{quote}For this one, is it be convenient to include javax.ws.rs-api in
[https://github.com/apache/tez/blob/master/tez-plugins/tez-yarn-timeline-history-with-acls/pom.xml]
? Or does that also conflict with jsr311-api?
{quote}
The actual error is :
java.lang.AbstractMethodError:
javax.ws.rs.core.UriBuilder.uri(Ljava/lang/String;)Ljavax/ws/rs/core/UriBuilder;
and this is due to conflict with jsr311-api, javax.ws.rs-api already got
included as transitive dependency, if I exclude javax.ws.rs-api in Tez it can
make the test pass, but we don't want to play with exclusions as we aren't sure
what runtime issues it can create
> Upgrade fasterxml Jackson to 2.13.0
> ---
>
> Key: HADOOP-18033
> URL: https://issues.apache.org/jira/browse/HADOOP-18033
> Project: Hadoop Common
> Issue Type: Improvement
> Components: build
>Reporter: Akira Ajisaka
>Assignee: Viraj Jasani
>Priority: Major
> Labels: pull-request-available
> Fix For: 3.4.0, 3.3.2
>
> Time Spent: 4h 10m
> Remaining Estimate: 0h
>
> Spark 3.2.0 depends on Jackson 2.12.3. Let's upgrade to 2.12.5 (2.12.x latest
> as of now) or upper.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18033) Upgrade fasterxml Jackson to 2.13.0
[ https://issues.apache.org/jira/browse/HADOOP-18033?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17555805#comment-17555805 ] Viraj Jasani commented on HADOOP-18033: --- Ah I see, HADOOP-18178 has also bumped Jackson to 2.13.2 in light of fixing CVE-2020-36518, and it has made it's way to 3.3.2 release but I guess the pain related to javax.ws.rs-api remains the same. > Upgrade fasterxml Jackson to 2.13.0 > --- > > Key: HADOOP-18033 > URL: https://issues.apache.org/jira/browse/HADOOP-18033 > Project: Hadoop Common > Issue Type: Improvement > Components: build >Reporter: Akira Ajisaka >Assignee: Viraj Jasani >Priority: Major > Labels: pull-request-available > Fix For: 3.4.0, 3.3.2 > > Time Spent: 4h 10m > Remaining Estimate: 0h > > Spark 3.2.0 depends on Jackson 2.12.3. Let's upgrade to 2.12.5 (2.12.x latest > as of now) or upper. -- This message was sent by Atlassian Jira (v8.20.7#820007) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18033) Upgrade fasterxml Jackson to 2.13.0
[
https://issues.apache.org/jira/browse/HADOOP-18033?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17555804#comment-17555804
]
Viraj Jasani commented on HADOOP-18033:
---
[~ayushtkn] Thanks for posting your findings, just had a high level glance at
the above failure stacktrace in Tez.
{quote}at
org.apache.tez.dag.history.ats.acls.ATSHistoryACLPolicyManager.createTimelineDomain(ATSHistoryACLPolicyManager.java:127)
{quote}
For this one, is it be convenient to include javax.ws.rs-api in
[https://github.com/apache/tez/blob/master/tez-plugins/tez-yarn-timeline-history-with-acls/pom.xml]
? Or does that also conflict with jsr311-api?
I understand the pain with minor release upgrade when it has to deal with such
issues. FWIW, although Hadoop 3.3 could revert this for 3.3.4 release but from
security viewpoint, staying up with latest Jackson2 is also in good favour of
3.3 release line, given that 3.3 is the latest release line. Let me also check
if something can be done in the meantime. (As you already noticed, the problem
here is that with shading, we get multiple duplicate class clashes for
javax.ws.rs dependencies and hence we have no choice but to remove it from
shading)
> Upgrade fasterxml Jackson to 2.13.0
> ---
>
> Key: HADOOP-18033
> URL: https://issues.apache.org/jira/browse/HADOOP-18033
> Project: Hadoop Common
> Issue Type: Improvement
> Components: build
>Reporter: Akira Ajisaka
>Assignee: Viraj Jasani
>Priority: Major
> Labels: pull-request-available
> Fix For: 3.4.0, 3.3.2
>
> Time Spent: 4h 10m
> Remaining Estimate: 0h
>
> Spark 3.2.0 depends on Jackson 2.12.3. Let's upgrade to 2.12.5 (2.12.x latest
> as of now) or upper.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18033) Upgrade fasterxml Jackson to 2.13.0
[ https://issues.apache.org/jira/browse/HADOOP-18033?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17555743#comment-17555743 ] Ayush Saxena commented on HADOOP-18033: --- [~ste...@apache.org] It added javax.ws.rs:javax.ws.rs-api and that isn't shaded also and is conflicting with jsr311-api in tez. Can see the error here as well: [https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-213/4/testReport/org.apache.tez.dag.history.ats.acls/TestATSHistoryWithACLs/testDagLoggingDisabled/] and someone quoted some problem here as well(I didn't check what is that though): https://github.com/apache/hadoop/pull/3764#issuecomment-1158641569 > Upgrade fasterxml Jackson to 2.13.0 > --- > > Key: HADOOP-18033 > URL: https://issues.apache.org/jira/browse/HADOOP-18033 > Project: Hadoop Common > Issue Type: Improvement > Components: build >Reporter: Akira Ajisaka >Assignee: Viraj Jasani >Priority: Major > Labels: pull-request-available > Fix For: 3.4.0, 3.3.2 > > Time Spent: 4h 10m > Remaining Estimate: 0h > > Spark 3.2.0 depends on Jackson 2.12.3. Let's upgrade to 2.12.5 (2.12.x latest > as of now) or upper. -- This message was sent by Atlassian Jira (v8.20.7#820007) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18033) Upgrade fasterxml Jackson to 2.13.0
[ https://issues.apache.org/jira/browse/HADOOP-18033?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17555741#comment-17555741 ] Steve Loughran commented on HADOOP-18033: - what is the problem here? > Upgrade fasterxml Jackson to 2.13.0 > --- > > Key: HADOOP-18033 > URL: https://issues.apache.org/jira/browse/HADOOP-18033 > Project: Hadoop Common > Issue Type: Improvement > Components: build >Reporter: Akira Ajisaka >Assignee: Viraj Jasani >Priority: Major > Labels: pull-request-available > Fix For: 3.4.0, 3.3.2 > > Time Spent: 4h 10m > Remaining Estimate: 0h > > Spark 3.2.0 depends on Jackson 2.12.3. Let's upgrade to 2.12.5 (2.12.x latest > as of now) or upper. -- This message was sent by Atlassian Jira (v8.20.7#820007) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18033) Upgrade fasterxml Jackson to 2.13.0
[
https://issues.apache.org/jira/browse/HADOOP-18033?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17555470#comment-17555470
]
Ayush Saxena commented on HADOOP-18033:
---
guess, this broke the way for tez to upgrade post 3.3.1 and there after for
hive as well. Have been discussing the upgrade stuff for Tez internally and I
think this is the one, any way we have decided to settle for at 3.3.1. and that
works
->
{quote}With this upgrade, we will also need to explicitly add new dependency
javax.ws.rs:javax.ws.rs-api.
We also need to exclude it from shading, else we will get multiple duplicate
class clash with existing javax.ws.rs dependencies.
{quote}
Most probably this is the reason, I will create a Jira and try to see what can
be done or if something in trunk already sorted this, I see a couple of Jiras
linked to this ticket
> Upgrade fasterxml Jackson to 2.13.0
> ---
>
> Key: HADOOP-18033
> URL: https://issues.apache.org/jira/browse/HADOOP-18033
> Project: Hadoop Common
> Issue Type: Improvement
> Components: build
>Reporter: Akira Ajisaka
>Assignee: Viraj Jasani
>Priority: Major
> Labels: pull-request-available
> Fix For: 3.4.0, 3.3.2
>
> Time Spent: 3h 40m
> Remaining Estimate: 0h
>
> Spark 3.2.0 depends on Jackson 2.12.3. Let's upgrade to 2.12.5 (2.12.x latest
> as of now) or upper.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18033) Upgrade fasterxml Jackson to 2.13.0
[ https://issues.apache.org/jira/browse/HADOOP-18033?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17455047#comment-17455047 ] Viraj Jasani commented on HADOOP-18033: --- Sure [~aajisaka], I am on it. > Upgrade fasterxml Jackson to 2.13.0 > --- > > Key: HADOOP-18033 > URL: https://issues.apache.org/jira/browse/HADOOP-18033 > Project: Hadoop Common > Issue Type: Improvement > Components: build >Reporter: Akira Ajisaka >Assignee: Viraj Jasani >Priority: Major > Labels: pull-request-available > Fix For: 3.4.0 > > Time Spent: 1h 20m > Remaining Estimate: 0h > > Spark 3.2.0 depends on Jackson 2.12.3. Let's upgrade to 2.12.5 (2.12.x latest > as of now) or upper. -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18033) Upgrade fasterxml Jackson to 2.13.0
[ https://issues.apache.org/jira/browse/HADOOP-18033?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17455021#comment-17455021 ] Akira Ajisaka commented on HADOOP-18033: Merged the PR into trunk. Hi [~vjasani], would you create a PR for branch-3.3? > Upgrade fasterxml Jackson to 2.13.0 > --- > > Key: HADOOP-18033 > URL: https://issues.apache.org/jira/browse/HADOOP-18033 > Project: Hadoop Common > Issue Type: Improvement > Components: build >Reporter: Akira Ajisaka >Assignee: Viraj Jasani >Priority: Major > Labels: pull-request-available > Time Spent: 1h 20m > Remaining Estimate: 0h > > Spark 3.2.0 depends on Jackson 2.12.3. Let's upgrade to 2.12.5 (2.12.x latest > as of now) or upper. -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
