[jira] [Created] (HADOOP-18890) remove okhttp usage
PJ Fanning created HADOOP-18890: --- Summary: remove okhttp usage Key: HADOOP-18890 URL: https://issues.apache.org/jira/browse/HADOOP-18890 Project: Hadoop Common Issue Type: Bug Components: common Reporter: PJ Fanning * relates to HADOOP-18496 * simplifies the dependencies if hadoop doesn't use multiple 3rd party libs to make http calls * okhttp brings in other dependencies like the kotlin runtime * hadoop already uses apache httpclient in some places -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18890) remove okhttp usage
[ https://issues.apache.org/jira/browse/HADOOP-18890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17764306#comment-17764306 ] PJ Fanning commented on HADOOP-18890: - It seems to be used in a few places - notably hadoop-hdfs-client. It should be easy enough to rewrite the code to use apache httpclient. > remove okhttp usage > --- > > Key: HADOOP-18890 > URL: https://issues.apache.org/jira/browse/HADOOP-18890 > Project: Hadoop Common > Issue Type: Bug > Components: common >Reporter: PJ Fanning >Priority: Major > > * relates to HADOOP-18496 > * simplifies the dependencies if hadoop doesn't use multiple 3rd party libs > to make http calls > * okhttp brings in other dependencies like the kotlin runtime > * hadoop already uses apache httpclient in some places -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Created] (HADOOP-18894) upgrade sshd-core due to CVEs
PJ Fanning created HADOOP-18894: --- Summary: upgrade sshd-core due to CVEs Key: HADOOP-18894 URL: https://issues.apache.org/jira/browse/HADOOP-18894 Project: Hadoop Common Issue Type: Bug Reporter: PJ Fanning https://mvnrepository.com/artifact/org.apache.sshd/sshd-core hadoop currently uses v1.7.0 -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-18894) upgrade sshd-core due to CVEs
[ https://issues.apache.org/jira/browse/HADOOP-18894?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] PJ Fanning updated HADOOP-18894: Description: https://mvnrepository.com/artifact/org.apache.sshd/sshd-core hadoop currently uses v1.6.0 was: https://mvnrepository.com/artifact/org.apache.sshd/sshd-core hadoop currently uses v1.7.0 > upgrade sshd-core due to CVEs > - > > Key: HADOOP-18894 > URL: https://issues.apache.org/jira/browse/HADOOP-18894 > Project: Hadoop Common > Issue Type: Bug >Reporter: PJ Fanning >Priority: Major > Labels: pull-request-available > > https://mvnrepository.com/artifact/org.apache.sshd/sshd-core > hadoop currently uses v1.6.0 -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Created] (HADOOP-18895) upgrade to commons-compress 1.24.0
PJ Fanning created HADOOP-18895: --- Summary: upgrade to commons-compress 1.24.0 Key: HADOOP-18895 URL: https://issues.apache.org/jira/browse/HADOOP-18895 Project: Hadoop Common Issue Type: Improvement Reporter: PJ Fanning Includes some important bug fixes -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-18895) upgrade to commons-compress 1.24.0 due to CVE
[ https://issues.apache.org/jira/browse/HADOOP-18895?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] PJ Fanning updated HADOOP-18895: Summary: upgrade to commons-compress 1.24.0 due to CVE (was: upgrade to commons-compress 1.24.0) > upgrade to commons-compress 1.24.0 due to CVE > - > > Key: HADOOP-18895 > URL: https://issues.apache.org/jira/browse/HADOOP-18895 > Project: Hadoop Common > Issue Type: Improvement >Reporter: PJ Fanning >Priority: Major > Labels: pull-request-available > > Includes some important bug fixes -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-18895) upgrade to commons-compress 1.24.0 due to CVE
[ https://issues.apache.org/jira/browse/HADOOP-18895?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] PJ Fanning updated HADOOP-18895: Description: Includes some important bug fixes including https://lists.apache.org/thread/g9lrsz8j9nrgltcoc7v6cpkopg07czc9 (was: Includes some important bug fixes) > upgrade to commons-compress 1.24.0 due to CVE > - > > Key: HADOOP-18895 > URL: https://issues.apache.org/jira/browse/HADOOP-18895 > Project: Hadoop Common > Issue Type: Improvement >Reporter: PJ Fanning >Priority: Major > Labels: pull-request-available > > Includes some important bug fixes including > https://lists.apache.org/thread/g9lrsz8j9nrgltcoc7v6cpkopg07czc9 -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-18895) upgrade to commons-compress 1.24.0 due to CVE
[ https://issues.apache.org/jira/browse/HADOOP-18895?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] PJ Fanning updated HADOOP-18895: Description: Includes some important bug fixes including https://lists.apache.org/thread/g9lrsz8j9nrgltcoc7v6cpkopg07czc9 - CVE-2023-42503 (was: Includes some important bug fixes including https://lists.apache.org/thread/g9lrsz8j9nrgltcoc7v6cpkopg07czc9) > upgrade to commons-compress 1.24.0 due to CVE > - > > Key: HADOOP-18895 > URL: https://issues.apache.org/jira/browse/HADOOP-18895 > Project: Hadoop Common > Issue Type: Improvement >Reporter: PJ Fanning >Priority: Major > Labels: pull-request-available > > Includes some important bug fixes including > https://lists.apache.org/thread/g9lrsz8j9nrgltcoc7v6cpkopg07czc9 - > CVE-2023-42503 -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Created] (HADOOP-18912) upgrade snappy-java to 1.1.10.4 due to CVE
PJ Fanning created HADOOP-18912: --- Summary: upgrade snappy-java to 1.1.10.4 due to CVE Key: HADOOP-18912 URL: https://issues.apache.org/jira/browse/HADOOP-18912 Project: Hadoop Common Issue Type: Bug Components: build Reporter: PJ Fanning follow up to HADOOP-18782 https://github.com/xerial/snappy-java/security/advisories/GHSA-55g7-9cwv-5qfv -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Created] (HADOOP-18619) replace jsr311-api dependency with rs-api
PJ Fanning created HADOOP-18619: --- Summary: replace jsr311-api dependency with rs-api Key: HADOOP-18619 URL: https://issues.apache.org/jira/browse/HADOOP-18619 Project: Hadoop Common Issue Type: Task Components: common Reporter: PJ Fanning [jsr311-api|https://mvnrepository.com/artifact/javax.ws.rs/jsr311-api] is unmaintained and causes issues when jars bring in a dependency on the newer [rs-api|https://mvnrepository.com/artifact/javax.ws.rs/javax.ws.rs-api/2.1.1] jar - that uses the same package name but has incompatible code To make things worse, there is now a jakarta fork of rs-api but I suggest we worry about that later. jersey-core 1.19.x gives us the jsr311-api dependency. The upgrade to HADOOP-15984 is currently blocked and looks hard. HADOOP-15983 is a workaround that allows us to keep jersey 1.x but removes the issue where we end up relying on the unmaintained Jackson 1.9 jars. We may now need a similar fork of jersey-core 1.19 to build a version of that jar that uses rs-api instead of jsr311. The main benefit here is get around the fact that jackson jaxrs 2.13+ has dropped support for jsr311 and now only supports rs-api. (see HADOOP-18332) -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18619) replace jsr311-api dependency with rs-api
[ https://issues.apache.org/jira/browse/HADOOP-18619?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17685255#comment-17685255 ] PJ Fanning commented on HADOOP-18619: - When 3.3.5 is released, jersey-json dependency will be replaced by a forked version that I made. This will likely force any downstream projects that have explicit build dependencies on jersey-json to change their build to use my forked version. For this task, it looks likely that I would need to fork jersey-core in a similar way. And if/when Hadoop switch to that forked version of jersey-core, downstream projects would also need to review any explicit dependencies that they have on jersey-core. > replace jsr311-api dependency with rs-api > - > > Key: HADOOP-18619 > URL: https://issues.apache.org/jira/browse/HADOOP-18619 > Project: Hadoop Common > Issue Type: Task > Components: build, common >Reporter: PJ Fanning >Priority: Major > > [jsr311-api|https://mvnrepository.com/artifact/javax.ws.rs/jsr311-api] is > unmaintained and causes issues when jars bring in a dependency on the newer > [rs-api|https://mvnrepository.com/artifact/javax.ws.rs/javax.ws.rs-api/2.1.1] > jar - that uses the same package name but has incompatible code > To make things worse, there is now a jakarta fork of rs-api but I suggest we > worry about that later. > jersey-core 1.19.x gives us the jsr311-api dependency. > The upgrade to HADOOP-15984 is currently blocked and looks hard. > HADOOP-15983 is a workaround that allows us to keep jersey 1.x but removes > the issue where we end up relying on the unmaintained Jackson 1.9 jars. > We may now need a similar fork of jersey-core 1.19 to build a version of that > jar that uses rs-api instead of jsr311. > The main benefit here is get around the fact that jackson jaxrs 2.13+ has > dropped support for jsr311 and now only supports rs-api. (see HADOOP-18332) > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18619) replace jsr311-api dependency with rs-api
[ https://issues.apache.org/jira/browse/HADOOP-18619?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17687023#comment-17687023 ] PJ Fanning commented on HADOOP-18619: - I haven't tried playing with jersey-core too much yet. I just know it doesn't compile if you switch out jsr311-api and use rs-api jar instead. It might be a few days before I can get back to it but if I can work out what if any code changes I need to make to my local copy of jersey-core to get it to compile and test ok with rs-api jar - then I think I'll have a better idea of whether shading is an option or if we're stuck with forking the code and making some changes to the forked copy (which should be small enough, if needed). > replace jsr311-api dependency with rs-api > - > > Key: HADOOP-18619 > URL: https://issues.apache.org/jira/browse/HADOOP-18619 > Project: Hadoop Common > Issue Type: Task > Components: build, common >Reporter: PJ Fanning >Priority: Major > > [jsr311-api|https://mvnrepository.com/artifact/javax.ws.rs/jsr311-api] is > unmaintained and causes issues when jars bring in a dependency on the newer > [rs-api|https://mvnrepository.com/artifact/javax.ws.rs/javax.ws.rs-api/2.1.1] > jar - that uses the same package name but has incompatible code > To make things worse, there is now a jakarta fork of rs-api but I suggest we > worry about that later. > jersey-core 1.19.x gives us the jsr311-api dependency. > The upgrade to HADOOP-15984 is currently blocked and looks hard. > HADOOP-15983 is a workaround that allows us to keep jersey 1.x but removes > the issue where we end up relying on the unmaintained Jackson 1.9 jars. > We may now need a similar fork of jersey-core 1.19 to build a version of that > jar that uses rs-api instead of jsr311. > The main benefit here is get around the fact that jackson jaxrs 2.13+ has > dropped support for jsr311 and now only supports rs-api. (see HADOOP-18332) > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (HADOOP-18619) replace jsr311-api dependency with rs-api
[ https://issues.apache.org/jira/browse/HADOOP-18619?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17687023#comment-17687023 ] PJ Fanning edited comment on HADOOP-18619 at 2/10/23 10:51 AM: --- I haven't tried playing with jersey-core too much yet. I just know it doesn't compile if you switch out jsr311-api and use rs-api jar instead. It might be a few days before I can get back to it but if I can work out what if any code changes I need to make to my local copy of jersey-core, to get it to compile and test ok with rs-api jar - then I think I'll have a better idea of whether shading is an option or if we're stuck with forking the code and making some changes to the forked copy (which should be small enough, if needed). was (Author: pj.fanning): I haven't tried playing with jersey-core too much yet. I just know it doesn't compile if you switch out jsr311-api and use rs-api jar instead. It might be a few days before I can get back to it but if I can work out what if any code changes I need to make to my local copy of jersey-core to get it to compile and test ok with rs-api jar - then I think I'll have a better idea of whether shading is an option or if we're stuck with forking the code and making some changes to the forked copy (which should be small enough, if needed). > replace jsr311-api dependency with rs-api > - > > Key: HADOOP-18619 > URL: https://issues.apache.org/jira/browse/HADOOP-18619 > Project: Hadoop Common > Issue Type: Task > Components: build, common >Reporter: PJ Fanning >Priority: Major > > [jsr311-api|https://mvnrepository.com/artifact/javax.ws.rs/jsr311-api] is > unmaintained and causes issues when jars bring in a dependency on the newer > [rs-api|https://mvnrepository.com/artifact/javax.ws.rs/javax.ws.rs-api/2.1.1] > jar - that uses the same package name but has incompatible code > To make things worse, there is now a jakarta fork of rs-api but I suggest we > worry about that later. > jersey-core 1.19.x gives us the jsr311-api dependency. > The upgrade to HADOOP-15984 is currently blocked and looks hard. > HADOOP-15983 is a workaround that allows us to keep jersey 1.x but removes > the issue where we end up relying on the unmaintained Jackson 1.9 jars. > We may now need a similar fork of jersey-core 1.19 to build a version of that > jar that uses rs-api instead of jsr311. > The main benefit here is get around the fact that jackson jaxrs 2.13+ has > dropped support for jsr311 and now only supports rs-api. (see HADOOP-18332) > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18619) replace jsr311-api dependency with rs-api
[ https://issues.apache.org/jira/browse/HADOOP-18619?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17687104#comment-17687104 ] PJ Fanning commented on HADOOP-18619: - I had a quick look and getting jersey-core to work with javax rs-api is non-trivial but probably not massive. There are more methods on the interafaces in rs-api (than in jsr311-api). Apache CXF has an implementation of the affected classes and it would probably be feasible to cherry pick some of that code into a forked version of jersey-core. The idea of doing something in hadoop-thirdparty is also feasible. We could take the jersey-core, jersey-server and my existing fork of jersey-json and build a fat jar that shades all the classes and well as jsr311-api. The packages could start with org.apache.hadoop.jersey1 instead of com.sun.jersey and org.apache.hadoop.jsr311 instead of javax.ws.rs. We could then adjust hadoop code to use these classes. Some of the uptake appears in web.xml files. I do suspect that this issue I raised today on the yarn-dev mailing list will bite us with this shading approach because the swagger jaxrs jar is likely to also depend on javax.ws.rs. Modifying or removing this swagger support may reduce the complexity for this issue. https://lists.apache.org/thread/80mf4w6zopxyzp2vc777pq6f1fbt5wjq > replace jsr311-api dependency with rs-api > - > > Key: HADOOP-18619 > URL: https://issues.apache.org/jira/browse/HADOOP-18619 > Project: Hadoop Common > Issue Type: Task > Components: build, common >Reporter: PJ Fanning >Priority: Major > > [jsr311-api|https://mvnrepository.com/artifact/javax.ws.rs/jsr311-api] is > unmaintained and causes issues when jars bring in a dependency on the newer > [rs-api|https://mvnrepository.com/artifact/javax.ws.rs/javax.ws.rs-api/2.1.1] > jar - that uses the same package name but has incompatible code > To make things worse, there is now a jakarta fork of rs-api but I suggest we > worry about that later. > jersey-core 1.19.x gives us the jsr311-api dependency. > The upgrade to HADOOP-15984 is currently blocked and looks hard. > HADOOP-15983 is a workaround that allows us to keep jersey 1.x but removes > the issue where we end up relying on the unmaintained Jackson 1.9 jars. > We may now need a similar fork of jersey-core 1.19 to build a version of that > jar that uses rs-api instead of jsr311. > The main benefit here is get around the fact that jackson jaxrs 2.13+ has > dropped support for jsr311 and now only supports rs-api. (see HADOOP-18332) > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (HADOOP-18619) replace jsr311-api dependency with rs-api
[ https://issues.apache.org/jira/browse/HADOOP-18619?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17687104#comment-17687104 ] PJ Fanning edited comment on HADOOP-18619 at 2/10/23 2:54 PM: -- I had a quick look and getting jersey-core to work with javax rs-api is non-trivial but probably not massive. There are more methods on the interfaces in rs-api (than in jsr311-api). Apache CXF has an implementation of the affected classes and it would probably be feasible to cherry pick some of that code into a forked version of jersey-core. The idea of doing something in hadoop-thirdparty is also feasible. We could take the jersey-core, jersey-server and my existing fork of jersey-json and build a fat jar that shades all the classes and well as jsr311-api. The packages could start with org.apache.hadoop.jersey1 instead of com.sun.jersey and org.apache.hadoop.jsr311 instead of javax.ws.rs. We could then adjust hadoop code to use these classes. Some of the uptake appears in web.xml files. I do suspect that this issue I raised today on the yarn-dev mailing list will bite us with this shading approach because the swagger jaxrs jar is likely to also depend on javax.ws.rs. Modifying or removing this swagger support may reduce the complexity for this issue. [https://lists.apache.org/thread/80mf4w6zopxyzp2vc777pq6f1fbt5wjq] was (Author: pj.fanning): I had a quick look and getting jersey-core to work with javax rs-api is non-trivial but probably not massive. There are more methods on the interafaces in rs-api (than in jsr311-api). Apache CXF has an implementation of the affected classes and it would probably be feasible to cherry pick some of that code into a forked version of jersey-core. The idea of doing something in hadoop-thirdparty is also feasible. We could take the jersey-core, jersey-server and my existing fork of jersey-json and build a fat jar that shades all the classes and well as jsr311-api. The packages could start with org.apache.hadoop.jersey1 instead of com.sun.jersey and org.apache.hadoop.jsr311 instead of javax.ws.rs. We could then adjust hadoop code to use these classes. Some of the uptake appears in web.xml files. I do suspect that this issue I raised today on the yarn-dev mailing list will bite us with this shading approach because the swagger jaxrs jar is likely to also depend on javax.ws.rs. Modifying or removing this swagger support may reduce the complexity for this issue. https://lists.apache.org/thread/80mf4w6zopxyzp2vc777pq6f1fbt5wjq > replace jsr311-api dependency with rs-api > - > > Key: HADOOP-18619 > URL: https://issues.apache.org/jira/browse/HADOOP-18619 > Project: Hadoop Common > Issue Type: Task > Components: build, common >Reporter: PJ Fanning >Priority: Major > > [jsr311-api|https://mvnrepository.com/artifact/javax.ws.rs/jsr311-api] is > unmaintained and causes issues when jars bring in a dependency on the newer > [rs-api|https://mvnrepository.com/artifact/javax.ws.rs/javax.ws.rs-api/2.1.1] > jar - that uses the same package name but has incompatible code > To make things worse, there is now a jakarta fork of rs-api but I suggest we > worry about that later. > jersey-core 1.19.x gives us the jsr311-api dependency. > The upgrade to HADOOP-15984 is currently blocked and looks hard. > HADOOP-15983 is a workaround that allows us to keep jersey 1.x but removes > the issue where we end up relying on the unmaintained Jackson 1.9 jars. > We may now need a similar fork of jersey-core 1.19 to build a version of that > jar that uses rs-api instead of jsr311. > The main benefit here is get around the fact that jackson jaxrs 2.13+ has > dropped support for jsr311 and now only supports rs-api. (see HADOOP-18332) > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (HADOOP-18619) replace jsr311-api dependency with rs-api
[ https://issues.apache.org/jira/browse/HADOOP-18619?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17687104#comment-17687104 ] PJ Fanning edited comment on HADOOP-18619 at 2/10/23 5:40 PM: -- I had a quick look and getting jersey-core to work with javax rs-api is non-trivial but probably not massive. There are more methods on the interfaces in rs-api (than in jsr311-api). Apache CXF has an implementation of the affected classes and it would probably be feasible to cherry pick some of that code into a forked version of jersey-core. The idea of doing something in hadoop-thirdparty is also feasible. We could take the jersey-core, jersey-server and my existing fork of jersey-json and build a fat jar that shades all the classes as well as jsr311-api. The packages could start with org.apache.hadoop.jersey1 instead of com.sun.jersey and org.apache.hadoop.jsr311 instead of javax.ws.rs. We could then adjust hadoop code to use these classes. Some of the uptake appears in web.xml files. I do suspect that this issue I raised today on the yarn-dev mailing list will bite us with this shading approach because the swagger jaxrs jar is likely to also depend on javax.ws.rs. Modifying or removing this swagger support may reduce the complexity for this issue. [https://lists.apache.org/thread/80mf4w6zopxyzp2vc777pq6f1fbt5wjq] was (Author: pj.fanning): I had a quick look and getting jersey-core to work with javax rs-api is non-trivial but probably not massive. There are more methods on the interfaces in rs-api (than in jsr311-api). Apache CXF has an implementation of the affected classes and it would probably be feasible to cherry pick some of that code into a forked version of jersey-core. The idea of doing something in hadoop-thirdparty is also feasible. We could take the jersey-core, jersey-server and my existing fork of jersey-json and build a fat jar that shades all the classes and well as jsr311-api. The packages could start with org.apache.hadoop.jersey1 instead of com.sun.jersey and org.apache.hadoop.jsr311 instead of javax.ws.rs. We could then adjust hadoop code to use these classes. Some of the uptake appears in web.xml files. I do suspect that this issue I raised today on the yarn-dev mailing list will bite us with this shading approach because the swagger jaxrs jar is likely to also depend on javax.ws.rs. Modifying or removing this swagger support may reduce the complexity for this issue. [https://lists.apache.org/thread/80mf4w6zopxyzp2vc777pq6f1fbt5wjq] > replace jsr311-api dependency with rs-api > - > > Key: HADOOP-18619 > URL: https://issues.apache.org/jira/browse/HADOOP-18619 > Project: Hadoop Common > Issue Type: Task > Components: build, common >Reporter: PJ Fanning >Priority: Major > > [jsr311-api|https://mvnrepository.com/artifact/javax.ws.rs/jsr311-api] is > unmaintained and causes issues when jars bring in a dependency on the newer > [rs-api|https://mvnrepository.com/artifact/javax.ws.rs/javax.ws.rs-api/2.1.1] > jar - that uses the same package name but has incompatible code > To make things worse, there is now a jakarta fork of rs-api but I suggest we > worry about that later. > jersey-core 1.19.x gives us the jsr311-api dependency. > The upgrade to HADOOP-15984 is currently blocked and looks hard. > HADOOP-15983 is a workaround that allows us to keep jersey 1.x but removes > the issue where we end up relying on the unmaintained Jackson 1.9 jars. > We may now need a similar fork of jersey-core 1.19 to build a version of that > jar that uses rs-api instead of jsr311. > The main benefit here is get around the fact that jackson jaxrs 2.13+ has > dropped support for jsr311 and now only supports rs-api. (see HADOOP-18332) > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Created] (HADOOP-18658) snakeyaml dependency: upgrade to v2.0
PJ Fanning created HADOOP-18658: --- Summary: snakeyaml dependency: upgrade to v2.0 Key: HADOOP-18658 URL: https://issues.apache.org/jira/browse/HADOOP-18658 Project: Hadoop Common Issue Type: Task Reporter: PJ Fanning * [https://github.com/advisories/GHSA-mjmj-j48q-9wg2] * I don't think this needs to go in v3.3.5 - since this CVE affects part of snakeyaml that hadoop doesn't use -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Created] (HADOOP-18693) upgrade Apache Derby due to CVEs
PJ Fanning created HADOOP-18693: --- Summary: upgrade Apache Derby due to CVEs Key: HADOOP-18693 URL: https://issues.apache.org/jira/browse/HADOOP-18693 Project: Hadoop Common Issue Type: Task Reporter: PJ Fanning [https://github.com/advisories/GHSA-wr69-g62g-2r9h] [https://github.com/advisories/GHSA-42xw-p62x-hwcf] -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-18693) upgrade Apache Derby due to CVEs
[ https://issues.apache.org/jira/browse/HADOOP-18693?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] PJ Fanning updated HADOOP-18693: Description: [https://github.com/advisories/GHSA-wr69-g62g-2r9h] [https://github.com/advisories/GHSA-42xw-p62x-hwcf] [https://github.com/apache/hadoop/pull/5427] was: [https://github.com/advisories/GHSA-wr69-g62g-2r9h] [https://github.com/advisories/GHSA-42xw-p62x-hwcf] > upgrade Apache Derby due to CVEs > > > Key: HADOOP-18693 > URL: https://issues.apache.org/jira/browse/HADOOP-18693 > Project: Hadoop Common > Issue Type: Task >Reporter: PJ Fanning >Priority: Major > > [https://github.com/advisories/GHSA-wr69-g62g-2r9h] > [https://github.com/advisories/GHSA-42xw-p62x-hwcf] > [https://github.com/apache/hadoop/pull/5427] > > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-18693) upgrade Apache Derby due to CVEs
[ https://issues.apache.org/jira/browse/HADOOP-18693?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] PJ Fanning updated HADOOP-18693: Description: [https://github.com/advisories/GHSA-wr69-g62g-2r9h] [https://github.com/advisories/GHSA-42xw-p62x-hwcf] [https://github.com/apache/hadoop/pull/5427] Only seems to be used in test scope but it would be nice to silence the dependabot warnings by merging the PR. was: [https://github.com/advisories/GHSA-wr69-g62g-2r9h] [https://github.com/advisories/GHSA-42xw-p62x-hwcf] [https://github.com/apache/hadoop/pull/5427] > upgrade Apache Derby due to CVEs > > > Key: HADOOP-18693 > URL: https://issues.apache.org/jira/browse/HADOOP-18693 > Project: Hadoop Common > Issue Type: Task >Reporter: PJ Fanning >Priority: Major > > [https://github.com/advisories/GHSA-wr69-g62g-2r9h] > [https://github.com/advisories/GHSA-42xw-p62x-hwcf] > [https://github.com/apache/hadoop/pull/5427] > Only seems to be used in test scope but it would be nice to silence the > dependabot warnings by merging the PR. > > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-17225) Update jackson-mapper-asl-1.9.13 to atlassian version to mitigate: CVE-2019-10172
[ https://issues.apache.org/jira/browse/HADOOP-17225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17770690#comment-17770690 ] PJ Fanning commented on HADOOP-17225: - This can probably be closed because latest hadoop 3.3 releases no longer have jackson 1.9 dependency > Update jackson-mapper-asl-1.9.13 to atlassian version to mitigate: > CVE-2019-10172 > - > > Key: HADOOP-17225 > URL: https://issues.apache.org/jira/browse/HADOOP-17225 > Project: Hadoop Common > Issue Type: Bug >Reporter: Brahma Reddy Battula >Assignee: Brahma Reddy Battula >Priority: Major > Attachments: HADOOP-17225-001.patch, HADOOP-17225-002.patch > > > Currently jersey depends on the jackson, and upgradation of jersey from 1.X > to 2.x looks complicated(see HADOOP-15984 and HADOOP-16485). > Update jackson-mapper-asl-1.9.13 to atlassian version to mitigate: > CVE-2019-10172. > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Created] (HADOOP-18916) module-info classes from external dependencies appearing in uber jars
PJ Fanning created HADOOP-18916: --- Summary: module-info classes from external dependencies appearing in uber jars Key: HADOOP-18916 URL: https://issues.apache.org/jira/browse/HADOOP-18916 Project: Hadoop Common Issue Type: Improvement Components: build Reporter: PJ Fanning hadoop-client-minicluster and hadoop-client-runtime try unsuccessfully to exclude module-info classes from dependencies. Over time, more and more jars have these classes. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-18916) module-info classes from external dependencies appearing in uber jars
[ https://issues.apache.org/jira/browse/HADOOP-18916?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] PJ Fanning updated HADOOP-18916: Description: hadoop-client-minicluster and hadoop-client-runtime try unsuccessfully to exclude module-info classes from dependencies. Over time, more and more jars have these classes. The module-info classes are causing issue with CI builds. Builds can fail if there are more than module-inf class that is not excluded. It seems better to exclude them all, especially since they will be affected by shading anyway. was:hadoop-client-minicluster and hadoop-client-runtime try unsuccessfully to exclude module-info classes from dependencies. Over time, more and more jars have these classes. > module-info classes from external dependencies appearing in uber jars > - > > Key: HADOOP-18916 > URL: https://issues.apache.org/jira/browse/HADOOP-18916 > Project: Hadoop Common > Issue Type: Improvement > Components: build >Reporter: PJ Fanning >Priority: Major > > hadoop-client-minicluster and hadoop-client-runtime try unsuccessfully to > exclude module-info classes from dependencies. Over time, more and more jars > have these classes. > The module-info classes are causing issue with CI builds. Builds can fail if > there are more than module-inf class that is not excluded. > It seems better to exclude them all, especially since they will be affected > by shading anyway. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Created] (HADOOP-18917) upgrade to commons-io 2.14.0
PJ Fanning created HADOOP-18917: --- Summary: upgrade to commons-io 2.14.0 Key: HADOOP-18917 URL: https://issues.apache.org/jira/browse/HADOOP-18917 Project: Hadoop Common Issue Type: Improvement Components: build Reporter: PJ Fanning The release contains some hardening of support in some areas -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Created] (HADOOP-18921) upgrade avro in hadoop-thirdparty to 1.11.3
PJ Fanning created HADOOP-18921: --- Summary: upgrade avro in hadoop-thirdparty to 1.11.3 Key: HADOOP-18921 URL: https://issues.apache.org/jira/browse/HADOOP-18921 Project: Hadoop Common Issue Type: Improvement Reporter: PJ Fanning https://lists.apache.org/thread/wcj1747hvyl7qjhrfr6d6j1l62hvpr5l -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Created] (HADOOP-18924) upgrade grpc jars to v1.53.0 due to CVEs
PJ Fanning created HADOOP-18924: --- Summary: upgrade grpc jars to v1.53.0 due to CVEs Key: HADOOP-18924 URL: https://issues.apache.org/jira/browse/HADOOP-18924 Project: Hadoop Common Issue Type: Improvement Components: build Reporter: PJ Fanning https://mvnrepository.com/artifact/io.grpc/grpc-protobuf -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18929) Build failure while trying to create apache 3.3.7 release locally.
[
https://issues.apache.org/jira/browse/HADOOP-18929?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17773810#comment-17773810
]
PJ Fanning commented on HADOOP-18929:
-
It looks like commons-compress 1.24.0 is the 1st commons-compress jar to have
module-info.class in it.
If you are amenable, I can do a PR that excludes the commons-compress 1.24.0
module-info.class from hadoop-client-minicluster and hadoop-client-runtime jars.
> Build failure while trying to create apache 3.3.7 release locally.
> --
>
> Key: HADOOP-18929
> URL: https://issues.apache.org/jira/browse/HADOOP-18929
> Project: Hadoop Common
> Issue Type: Bug
>Affects Versions: 3.3.6
>Reporter: Mukund Thakur
>Priority: Critical
>
> {noformat}
> [ESC[1;34mINFOESC[m] ESC[1m---<
> ESC[0;36morg.apache.hadoop:hadoop-client-check-test-invariantsESC[0;1m
> >ESC[m
> [ESC[1;34mINFOESC[m] ESC[1mBuilding Apache Hadoop Client Packaging Invariants
> for Test 3.3.9-SNAPSHOT [105/111]ESC[m
> [ESC[1;34mINFOESC[m] ESC[1m[ pom
> ]-ESC[m
> [ESC[1;34mINFOESC[m]
> [ESC[1;34mINFOESC[m] ESC[1m---
> ESC[0;32mmaven-enforcer-plugin:3.0.0-M1:enforceESC[m
> ESC[1m(enforce-banned-dependencies)ESC[m @
> ESC[36mhadoop-client-check-test-invariantsESC[0;1m ---ESC[m
> [ESC[1;34mINFOESC[m] Adding ignorable dependency:
> org.apache.hadoop:hadoop-annotations:null
> [ESC[1;34mINFOESC[m] Adding ignore: *
> [ESC[1;33mWARNINGESC[m] Rule 1:
> org.apache.maven.plugins.enforcer.BanDuplicateClasses failed with message:
> Duplicate classes found:
> Found in:
> org.apache.hadoop:hadoop-client-minicluster:jar:3.3.9-SNAPSHOT:compile
> org.apache.hadoop:hadoop-client-runtime:jar:3.3.9-SNAPSHOT:compile
> Duplicate classes:
> META-INF/versions/9/module-info.class
> {noformat}
> CC [~ste...@apache.org] [~weichu]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18929) Build failure while trying to create apache 3.3.7 release locally.
[
https://issues.apache.org/jira/browse/HADOOP-18929?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17773818#comment-17773818
]
PJ Fanning commented on HADOOP-18929:
-
https://github.com/apache/hadoop/pull/6169
> Build failure while trying to create apache 3.3.7 release locally.
> --
>
> Key: HADOOP-18929
> URL: https://issues.apache.org/jira/browse/HADOOP-18929
> Project: Hadoop Common
> Issue Type: Bug
>Affects Versions: 3.3.6
>Reporter: Mukund Thakur
>Priority: Critical
> Labels: pull-request-available
>
> {noformat}
> [ESC[1;34mINFOESC[m] ESC[1m---<
> ESC[0;36morg.apache.hadoop:hadoop-client-check-test-invariantsESC[0;1m
> >ESC[m
> [ESC[1;34mINFOESC[m] ESC[1mBuilding Apache Hadoop Client Packaging Invariants
> for Test 3.3.9-SNAPSHOT [105/111]ESC[m
> [ESC[1;34mINFOESC[m] ESC[1m[ pom
> ]-ESC[m
> [ESC[1;34mINFOESC[m]
> [ESC[1;34mINFOESC[m] ESC[1m---
> ESC[0;32mmaven-enforcer-plugin:3.0.0-M1:enforceESC[m
> ESC[1m(enforce-banned-dependencies)ESC[m @
> ESC[36mhadoop-client-check-test-invariantsESC[0;1m ---ESC[m
> [ESC[1;34mINFOESC[m] Adding ignorable dependency:
> org.apache.hadoop:hadoop-annotations:null
> [ESC[1;34mINFOESC[m] Adding ignore: *
> [ESC[1;33mWARNINGESC[m] Rule 1:
> org.apache.maven.plugins.enforcer.BanDuplicateClasses failed with message:
> Duplicate classes found:
> Found in:
> org.apache.hadoop:hadoop-client-minicluster:jar:3.3.9-SNAPSHOT:compile
> org.apache.hadoop:hadoop-client-runtime:jar:3.3.9-SNAPSHOT:compile
> Duplicate classes:
> META-INF/versions/9/module-info.class
> {noformat}
> CC [~ste...@apache.org] [~weichu]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Created] (HADOOP-18933) upgrade netty to 4.1.100 due to CVE
PJ Fanning created HADOOP-18933: --- Summary: upgrade netty to 4.1.100 due to CVE Key: HADOOP-18933 URL: https://issues.apache.org/jira/browse/HADOOP-18933 Project: Hadoop Common Issue Type: Improvement Reporter: PJ Fanning follow up to https://issues.apache.org/jira/browse/HADOOP-18783 https://netty.io/news/2023/10/10/4-1-100-Final.html -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Created] (HADOOP-18936) upgrade jetty to 9.4.53 due to CVEs
PJ Fanning created HADOOP-18936: --- Summary: upgrade jetty to 9.4.53 due to CVEs Key: HADOOP-18936 URL: https://issues.apache.org/jira/browse/HADOOP-18936 Project: Hadoop Common Issue Type: Improvement Components: build Reporter: PJ Fanning 2 CVE fixes in https://github.com/jetty/jetty.project/releases/tag/jetty-9.4.53.v20231009 -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-18936) upgrade jetty to 9.4.53 due to CVEs
[ https://issues.apache.org/jira/browse/HADOOP-18936?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] PJ Fanning updated HADOOP-18936: Description: 2 CVE fixes in https://github.com/jetty/jetty.project/releases/tag/jetty-9.4.53.v20231009 4 more security fixes in https://github.com/jetty/jetty.project/releases/tag/jetty-9.4.52.v20230823 was:2 CVE fixes in https://github.com/jetty/jetty.project/releases/tag/jetty-9.4.53.v20231009 > upgrade jetty to 9.4.53 due to CVEs > --- > > Key: HADOOP-18936 > URL: https://issues.apache.org/jira/browse/HADOOP-18936 > Project: Hadoop Common > Issue Type: Improvement > Components: build >Reporter: PJ Fanning >Priority: Major > > 2 CVE fixes in > https://github.com/jetty/jetty.project/releases/tag/jetty-9.4.53.v20231009 > 4 more security fixes in > https://github.com/jetty/jetty.project/releases/tag/jetty-9.4.52.v20230823 -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18359) Update commons-cli from 1.2 to 1.5.
[ https://issues.apache.org/jira/browse/HADOOP-18359?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=1299#comment-1299 ] PJ Fanning commented on HADOOP-18359: - [~coheigea] I have not been involved with this issue. I am not a Hadoop committer. Maybe [~slfan1989] or [~ayushtkn] may be able to help. > Update commons-cli from 1.2 to 1.5. > > > Key: HADOOP-18359 > URL: https://issues.apache.org/jira/browse/HADOOP-18359 > Project: Hadoop Common > Issue Type: Improvement > Components: common >Affects Versions: 3.4.0 >Reporter: Shilun Fan >Assignee: Shilun Fan >Priority: Major > Labels: pull-request-available > Fix For: 3.4.0 > > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Created] (HADOOP-18949) upgrade maven dependency plugin due to security issue
PJ Fanning created HADOOP-18949: --- Summary: upgrade maven dependency plugin due to security issue Key: HADOOP-18949 URL: https://issues.apache.org/jira/browse/HADOOP-18949 Project: Hadoop Common Issue Type: Improvement Components: build Reporter: PJ Fanning https://github.com/advisories/GHSA-2f88-5hg8-9x2x -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Created] (HADOOP-18957) Use StandardCharsets.UTF_8 constant
PJ Fanning created HADOOP-18957: --- Summary: Use StandardCharsets.UTF_8 constant Key: HADOOP-18957 URL: https://issues.apache.org/jira/browse/HADOOP-18957 Project: Hadoop Common Issue Type: Improvement Reporter: PJ Fanning * there are some places in the code that have to check for UnsupportedCharsetException when explicitly using the charset name "UTF-8" * using StandardCharsets.UTF_8 is more efficient because the Java libs usually have to look up the charsets when you provide it as String param instead -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-18957) Use StandardCharsets.UTF_8 constant
[ https://issues.apache.org/jira/browse/HADOOP-18957?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] PJ Fanning updated HADOOP-18957: Description: * there are some places in the code that have to check for UnsupportedCharsetException when explicitly using the charset name "UTF-8" * using StandardCharsets.UTF_8 is more efficient because the Java libs usually have to look up the charsets when you provide it as String param instead * also stop using Guava Charsets and use StandardCharsets was: * there are some places in the code that have to check for UnsupportedCharsetException when explicitly using the charset name "UTF-8" * using StandardCharsets.UTF_8 is more efficient because the Java libs usually have to look up the charsets when you provide it as String param instead > Use StandardCharsets.UTF_8 constant > --- > > Key: HADOOP-18957 > URL: https://issues.apache.org/jira/browse/HADOOP-18957 > Project: Hadoop Common > Issue Type: Improvement >Reporter: PJ Fanning >Priority: Major > > * there are some places in the code that have to check for > UnsupportedCharsetException when explicitly using the charset name "UTF-8" > * using StandardCharsets.UTF_8 is more efficient because the Java libs > usually have to look up the charsets when you provide it as String param > instead > * also stop using Guava Charsets and use StandardCharsets -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18936) Upgrade to jetty 9.4.53
[ https://issues.apache.org/jira/browse/HADOOP-18936?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17781027#comment-17781027 ] PJ Fanning commented on HADOOP-18936: - [~coheigea] [~ayushtkn] I created https://github.com/apache/hadoop/pull/6239 to backport this. > Upgrade to jetty 9.4.53 > --- > > Key: HADOOP-18936 > URL: https://issues.apache.org/jira/browse/HADOOP-18936 > Project: Hadoop Common > Issue Type: Improvement > Components: build >Reporter: PJ Fanning >Assignee: PJ Fanning >Priority: Major > Labels: pull-request-available > Fix For: 3.4.0 > > > 2 CVE fixes in > https://github.com/jetty/jetty.project/releases/tag/jetty-9.4.53.v20231009 > 4 more security fixes in > https://github.com/jetty/jetty.project/releases/tag/jetty-9.4.52.v20230823 -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Created] (HADOOP-19014) use jsr311-compat jar to allow us to use Jackson 2.14.3
PJ Fanning created HADOOP-19014: --- Summary: use jsr311-compat jar to allow us to use Jackson 2.14.3 Key: HADOOP-19014 URL: https://issues.apache.org/jira/browse/HADOOP-19014 Project: Hadoop Common Issue Type: Task Components: common Reporter: PJ Fanning An alternative to HADOOP-18619 See https://github.com/pjfanning/jsr311-compat -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Created] (HADOOP-18711) upgrade nimbus jwt jar due to issues in its embedded shaded json-smart code
PJ Fanning created HADOOP-18711: --- Summary: upgrade nimbus jwt jar due to issues in its embedded shaded json-smart code Key: HADOOP-18711 URL: https://issues.apache.org/jira/browse/HADOOP-18711 Project: Hadoop Common Issue Type: Task Reporter: PJ Fanning https://github.com/apache/hadoop/pull/5549#issuecomment-1515174820 -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Created] (HADOOP-18712) upgrade to jetty 9.4.51 due to cve
PJ Fanning created HADOOP-18712: --- Summary: upgrade to jetty 9.4.51 due to cve Key: HADOOP-18712 URL: https://issues.apache.org/jira/browse/HADOOP-18712 Project: Hadoop Common Issue Type: Task Components: common Reporter: PJ Fanning https://github.com/advisories/GHSA-qw69-rqj8-6qw8 -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Created] (HADOOP-18719) upgrade snakeyaml to 2.0 (fixes CVE-2022-1471)
PJ Fanning created HADOOP-18719: --- Summary: upgrade snakeyaml to 2.0 (fixes CVE-2022-1471) Key: HADOOP-18719 URL: https://issues.apache.org/jira/browse/HADOOP-18719 Project: Hadoop Common Issue Type: Task Reporter: PJ Fanning https://bitbucket.org/snakeyaml/snakeyaml/wiki/Changes -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Resolved] (HADOOP-18719) upgrade snakeyaml to 2.0 (fixes CVE-2022-1471)
[ https://issues.apache.org/jira/browse/HADOOP-18719?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] PJ Fanning resolved HADOOP-18719. - Resolution: Duplicate > upgrade snakeyaml to 2.0 (fixes CVE-2022-1471) > -- > > Key: HADOOP-18719 > URL: https://issues.apache.org/jira/browse/HADOOP-18719 > Project: Hadoop Common > Issue Type: Task >Reporter: PJ Fanning >Priority: Major > > https://bitbucket.org/snakeyaml/snakeyaml/wiki/Changes -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18033) Upgrade fasterxml Jackson to 2.13.0
[ https://issues.apache.org/jira/browse/HADOOP-18033?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17731654#comment-17731654 ] PJ Fanning commented on HADOOP-18033: - We're stuck on Jackson 2.12 because of jersey v1. Jackson 2.13 has a change that drops support for jersey v1. Options include: * forking the jackson module for jaxrs to undo the change that drops jersey v1 support * or removing the need for that jackson module - which I think might require the removal of the io.swagger code in the yarn modules * or completing the move to jersey 2 (https://issues.apache.org/jira/browse/HADOOP-15984) > Upgrade fasterxml Jackson to 2.13.0 > --- > > Key: HADOOP-18033 > URL: https://issues.apache.org/jira/browse/HADOOP-18033 > Project: Hadoop Common > Issue Type: Improvement > Components: build >Reporter: Akira Ajisaka >Assignee: Viraj Jasani >Priority: Major > Labels: pull-request-available > Fix For: 3.3.2 > > Time Spent: 6.5h > Remaining Estimate: 0h > > Spark 3.2.0 depends on Jackson 2.12.3. Let's upgrade to 2.12.5 (2.12.x latest > as of now) or upper. > h2. this has been reverted. > we had to revert this as it broke tez. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (HADOOP-18033) Upgrade fasterxml Jackson to 2.13.0
[ https://issues.apache.org/jira/browse/HADOOP-18033?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17731654#comment-17731654 ] PJ Fanning edited comment on HADOOP-18033 at 6/12/23 3:14 PM: -- We're stuck on Jackson 2.12 because of jersey v1. Jackson 2.13 has a change that drops support for jersey v1. Options include: * forking the jackson module for jaxrs to undo the change that drops jersey v1 support * or removing the dependence on that jackson module by doing https://issues.apache.org/jira/browse/HADOOP-18619 * or completing the move to jersey 2 (https://issues.apache.org/jira/browse/HADOOP-15984) was (Author: pj.fanning): We're stuck on Jackson 2.12 because of jersey v1. Jackson 2.13 has a change that drops support for jersey v1. Options include: * forking the jackson module for jaxrs to undo the change that drops jersey v1 support * or removing the need for that jackson module - which I think might require the removal of the io.swagger code in the yarn modules * or completing the move to jersey 2 (https://issues.apache.org/jira/browse/HADOOP-15984) > Upgrade fasterxml Jackson to 2.13.0 > --- > > Key: HADOOP-18033 > URL: https://issues.apache.org/jira/browse/HADOOP-18033 > Project: Hadoop Common > Issue Type: Improvement > Components: build >Reporter: Akira Ajisaka >Assignee: Viraj Jasani >Priority: Major > Labels: pull-request-available > Fix For: 3.3.2 > > Time Spent: 6.5h > Remaining Estimate: 0h > > Spark 3.2.0 depends on Jackson 2.12.3. Let's upgrade to 2.12.5 (2.12.x latest > as of now) or upper. > h2. this has been reverted. > we had to revert this as it broke tez. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Created] (HADOOP-18782) upgrade to snappy-java 1.1.10.1 due to CVEs
PJ Fanning created HADOOP-18782: --- Summary: upgrade to snappy-java 1.1.10.1 due to CVEs Key: HADOOP-18782 URL: https://issues.apache.org/jira/browse/HADOOP-18782 Project: Hadoop Common Issue Type: Task Components: common Reporter: PJ Fanning see https://mvnrepository.com/artifact/org.xerial.snappy/snappy-java -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Created] (HADOOP-18783) upgrade netty to 4.1.94 due to CVE
PJ Fanning created HADOOP-18783: --- Summary: upgrade netty to 4.1.94 due to CVE Key: HADOOP-18783 URL: https://issues.apache.org/jira/browse/HADOOP-18783 Project: Hadoop Common Issue Type: Task Reporter: PJ Fanning https://github.com/advisories/GHSA-6mjq-h674-j845 -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Created] (HADOOP-19024) change to bouncy castle jdk1.8 jars
PJ Fanning created HADOOP-19024: --- Summary: change to bouncy castle jdk1.8 jars Key: HADOOP-19024 URL: https://issues.apache.org/jira/browse/HADOOP-19024 Project: Hadoop Common Issue Type: Task Reporter: PJ Fanning They have stopped patching the JDK 1.5 jars that Hadoop uses (see https://issues.apache.org/jira/browse/HADOOP-18540). The new artifacts have similar names - but the names are like bcprov-jdk18on as opposed to bcprov-jdk15on. CVE-2023-33201 is an example of a security issue that seems only to be fixed in the JDK 1.8 artifacts (ie no JDK 1.5 jar has the fix). https://www.bouncycastle.org/releasenotes.html#r1rv77 latest current release but the CVE was fixed in 1.74. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Created] (HADOOP-19041) further use of StandardCharsets
PJ Fanning created HADOOP-19041: --- Summary: further use of StandardCharsets Key: HADOOP-19041 URL: https://issues.apache.org/jira/browse/HADOOP-19041 Project: Hadoop Common Issue Type: Task Reporter: PJ Fanning builds on HADOOP-18957 -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18895) upgrade to commons-compress 1.24.0 due to CVE
[ https://issues.apache.org/jira/browse/HADOOP-18895?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17807140#comment-17807140 ] PJ Fanning commented on HADOOP-18895: - [~slfan1989] this was not reverted - it is still fixed in 3.4.0. See https://github.com/apache/hadoop/pull/6169 for the fix for HADOOP-18929 > upgrade to commons-compress 1.24.0 due to CVE > - > > Key: HADOOP-18895 > URL: https://issues.apache.org/jira/browse/HADOOP-18895 > Project: Hadoop Common > Issue Type: Improvement > Components: build >Reporter: PJ Fanning >Assignee: PJ Fanning >Priority: Major > Labels: pull-request-available > Fix For: 3.3.9 > > > Includes some important bug fixes including > https://lists.apache.org/thread/g9lrsz8j9nrgltcoc7v6cpkopg07czc9 - > CVE-2023-42503 -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-15984) Update jersey from 1.19 to 2.x
[ https://issues.apache.org/jira/browse/HADOOP-15984?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17816897#comment-17816897 ] PJ Fanning commented on HADOOP-15984: - Jersey 1 uses jsr311 jar and Jersey2 uses rs-api jar. These 2 jars use exactly the same package names but the code is different. You might get away with forcing the use of just the newer rs-api jar and excluding the older jsr311 api. If not, you would have to look at shading rs-api jar and any code that uses it (ie jersey 2). > Update jersey from 1.19 to 2.x > -- > > Key: HADOOP-15984 > URL: https://issues.apache.org/jira/browse/HADOOP-15984 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Akira Ajisaka >Priority: Major > Labels: pull-request-available > Time Spent: 2h 10m > Remaining Estimate: 0h > > jersey-json 1.19 depends on Jackson 1.9.2. Let's upgrade. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-15984) Update jersey from 1.19 to 2.x
[ https://issues.apache.org/jira/browse/HADOOP-15984?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17817480#comment-17817480 ] PJ Fanning commented on HADOOP-15984: - the jersey dependencies should only be exposed on the small number of MapReduce and Yarn subprojects that expose REST services that need jersey > Update jersey from 1.19 to 2.x > -- > > Key: HADOOP-15984 > URL: https://issues.apache.org/jira/browse/HADOOP-15984 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Akira Ajisaka >Priority: Major > Labels: pull-request-available > Time Spent: 2h 10m > Remaining Estimate: 0h > > jersey-json 1.19 depends on Jackson 1.9.2. Let's upgrade. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-15984) Update jersey from 1.19 to 2.x
[ https://issues.apache.org/jira/browse/HADOOP-15984?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17817492#comment-17817492 ] PJ Fanning commented on HADOOP-15984: - I don't understand why, for instance, hadoop-common exposes jersey 1.x dependencies? https://mvnrepository.com/artifact/org.apache.hadoop/hadoop-common/3.3.6 Jerseu should only be needed by the server side code. Until we do something about exposing Jersey 1.x as a dependency do anything with Hadoop, I think we will really struggle. > Update jersey from 1.19 to 2.x > -- > > Key: HADOOP-15984 > URL: https://issues.apache.org/jira/browse/HADOOP-15984 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Akira Ajisaka >Priority: Major > Labels: pull-request-available > Time Spent: 2h 10m > Remaining Estimate: 0h > > jersey-json 1.19 depends on Jackson 1.9.2. Let's upgrade. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-15984) Update jersey from 1.19 to 2.x
[ https://issues.apache.org/jira/browse/HADOOP-15984?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17817496#comment-17817496 ] PJ Fanning commented on HADOOP-15984: - It does look like we have some client side Jersey code too. It might serve us well to try to remove the client side dependency on Jersey as this is probably the reason why we expose jersey as a dependency in the core modules. It should be feasible to interact with Jersey based REST services without using the Jersey specific client code. It is REST after all and we should be able to choose framework independent ways to interact with the services. > Update jersey from 1.19 to 2.x > -- > > Key: HADOOP-15984 > URL: https://issues.apache.org/jira/browse/HADOOP-15984 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Akira Ajisaka >Priority: Major > Labels: pull-request-available > Time Spent: 2h 10m > Remaining Estimate: 0h > > jersey-json 1.19 depends on Jackson 1.9.2. Let's upgrade. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Created] (HADOOP-19076) move jersey code in hadoop-common jar to a new hadoop-jersey1-common jar
PJ Fanning created HADOOP-19076: --- Summary: move jersey code in hadoop-common jar to a new hadoop-jersey1-common jar Key: HADOOP-19076 URL: https://issues.apache.org/jira/browse/HADOOP-19076 Project: Hadoop Common Issue Type: Task Reporter: PJ Fanning Hadoop's Jersey dependencies are causing us real trouble. I'm wondering if it would be a good idea to take the jersey and javax.ws code out of hadoop-common and move it into a dedicated hadoop-jersey1-common jar. We could later create a hadoop-jersey2-common (or hadoop-jersey3-common - because Jersey 3 is out and maybe better to skip to Jersey 2). hadoop-jersey1-common and hadoop-jersey2-common would have equivalent classes - just depend on different versions of Jersey. Example code: * https://github.com/apache/hadoop/blob/12498b35bbb754225b0b2ca90d5ad4f5cf628d56/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/http/HttpServer2.java#L1030 * https://github.com/apache/hadoop/blob/12498b35bbb754225b0b2ca90d5ad4f5cf628d56/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/HttpExceptionUtils.java#L89 Hadoop modules that need access to the common jersey code could start with depending on hadoop-jersey1-common but later be refactored to use hadoop-jersey2-common. We could do this on a module by module basis (one a time). hadoop-common jar would have its jersey and jsr311-api dependencies removed. Wdyt [~slfan1989], [~steve_l], [~ayushsaxena] ? -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Created] (HADOOP-19077) remove use of javax.ws.rs.core.HttpHeaders
PJ Fanning created HADOOP-19077: --- Summary: remove use of javax.ws.rs.core.HttpHeaders Key: HADOOP-19077 URL: https://issues.apache.org/jira/browse/HADOOP-19077 Project: Hadoop Common Issue Type: Task Components: io Reporter: PJ Fanning One step towards removing Hadoop's dependence on Jersey1 and jsr311-api. We have other classes where we can get HTTP header names. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-19076) move jersey code in hadoop-common jar to a new hadoop-jersey1-common jar
[ https://issues.apache.org/jira/browse/HADOOP-19076?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17817696#comment-17817696 ] PJ Fanning commented on HADOOP-19076: - Thanks [~slfan1989] for the background on Jersey 3. What do you think of the idea of removing the Jersey 1 code from hadoop-common and creating a new hadoop-jersey1-common project for the code we move? > move jersey code in hadoop-common jar to a new hadoop-jersey1-common jar > > > Key: HADOOP-19076 > URL: https://issues.apache.org/jira/browse/HADOOP-19076 > Project: Hadoop Common > Issue Type: Task >Reporter: PJ Fanning >Priority: Major > > Hadoop's Jersey dependencies are causing us real trouble. > I'm wondering if it would be a good idea to take the jersey and javax.ws code > out of hadoop-common and move it into a dedicated hadoop-jersey1-common jar. > We could later create a hadoop-jersey2-common (or hadoop-jersey3-common - > because Jersey 3 is out and maybe better to skip to Jersey 2). > hadoop-jersey1-common and hadoop-jersey2-common would have equivalent classes > - just depend on different versions of Jersey. > Example code: > * > https://github.com/apache/hadoop/blob/12498b35bbb754225b0b2ca90d5ad4f5cf628d56/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/http/HttpServer2.java#L1030 > * > https://github.com/apache/hadoop/blob/12498b35bbb754225b0b2ca90d5ad4f5cf628d56/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/HttpExceptionUtils.java#L89 > Hadoop modules that need access to the common jersey code could start with > depending on hadoop-jersey1-common but later be refactored to use > hadoop-jersey2-common. We could do this on a module by module basis (one a > time). > hadoop-common jar would have its jersey and jsr311-api dependencies removed. > Wdyt [~slfan1989], [~steve_l], [~ayushsaxena] ? -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-19076) move jersey code in hadoop-common jar to a new hadoop-jersey1-common jar
[ https://issues.apache.org/jira/browse/HADOOP-19076?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17817730#comment-17817730 ] PJ Fanning commented on HADOOP-19076: - Thanks [~ste...@apache.org], the idea would be to have 1 jar with jersey1 dependencies and a 2nd jar with jersey2 dependencies - so we need different names for the 2 jars. > move jersey code in hadoop-common jar to a new hadoop-jersey1-common jar > > > Key: HADOOP-19076 > URL: https://issues.apache.org/jira/browse/HADOOP-19076 > Project: Hadoop Common > Issue Type: Task >Reporter: PJ Fanning >Priority: Major > > Hadoop's Jersey dependencies are causing us real trouble. > I'm wondering if it would be a good idea to take the jersey and javax.ws code > out of hadoop-common and move it into a dedicated hadoop-jersey1-common jar. > We could later create a hadoop-jersey2-common (or hadoop-jersey3-common - > because Jersey 3 is out and maybe better to skip to Jersey 2). > hadoop-jersey1-common and hadoop-jersey2-common would have equivalent classes > - just depend on different versions of Jersey. > Example code: > * > https://github.com/apache/hadoop/blob/12498b35bbb754225b0b2ca90d5ad4f5cf628d56/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/http/HttpServer2.java#L1030 > * > https://github.com/apache/hadoop/blob/12498b35bbb754225b0b2ca90d5ad4f5cf628d56/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/HttpExceptionUtils.java#L89 > Hadoop modules that need access to the common jersey code could start with > depending on hadoop-jersey1-common but later be refactored to use > hadoop-jersey2-common. We could do this on a module by module basis (one a > time). > hadoop-common jar would have its jersey and jsr311-api dependencies removed. > Wdyt [~slfan1989], [~steve_l], [~ayushsaxena] ? -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-19076) move jersey code in hadoop-common jar to a new hadoop-jersey1-common jar
[ https://issues.apache.org/jira/browse/HADOOP-19076?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] PJ Fanning updated HADOOP-19076: Description: Hadoop's Jersey dependencies are causing us real trouble. I'm wondering if it would be a good idea to take the jersey and javax.ws code out of hadoop-common and move it into a dedicated hadoop-jersey1-common jar. We could later create a hadoop-jersey2-common. hadoop-jersey1-common and hadoop-jersey2-common would have equivalent classes - just depend on different versions of Jersey. Example code: * https://github.com/apache/hadoop/blob/12498b35bbb754225b0b2ca90d5ad4f5cf628d56/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/http/HttpServer2.java#L1030 * https://github.com/apache/hadoop/blob/12498b35bbb754225b0b2ca90d5ad4f5cf628d56/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/HttpExceptionUtils.java#L89 Hadoop modules that need access to the common jersey code could start with depending on hadoop-jersey1-common but later be refactored to use hadoop-jersey2-common. We could do this on a module by module basis (one a time). hadoop-common jar would have its jersey and jsr311-api dependencies removed. Wdyt [~slfan1989], [~steve_l], [~ayushsaxena] ? was: Hadoop's Jersey dependencies are causing us real trouble. I'm wondering if it would be a good idea to take the jersey and javax.ws code out of hadoop-common and move it into a dedicated hadoop-jersey1-common jar. We could later create a hadoop-jersey2-common (or hadoop-jersey3-common - because Jersey 3 is out and maybe better to skip to Jersey 2). hadoop-jersey1-common and hadoop-jersey2-common would have equivalent classes - just depend on different versions of Jersey. Example code: * https://github.com/apache/hadoop/blob/12498b35bbb754225b0b2ca90d5ad4f5cf628d56/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/http/HttpServer2.java#L1030 * https://github.com/apache/hadoop/blob/12498b35bbb754225b0b2ca90d5ad4f5cf628d56/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/HttpExceptionUtils.java#L89 Hadoop modules that need access to the common jersey code could start with depending on hadoop-jersey1-common but later be refactored to use hadoop-jersey2-common. We could do this on a module by module basis (one a time). hadoop-common jar would have its jersey and jsr311-api dependencies removed. Wdyt [~slfan1989], [~steve_l], [~ayushsaxena] ? > move jersey code in hadoop-common jar to a new hadoop-jersey1-common jar > > > Key: HADOOP-19076 > URL: https://issues.apache.org/jira/browse/HADOOP-19076 > Project: Hadoop Common > Issue Type: Task >Reporter: PJ Fanning >Priority: Major > > Hadoop's Jersey dependencies are causing us real trouble. > I'm wondering if it would be a good idea to take the jersey and javax.ws code > out of hadoop-common and move it into a dedicated hadoop-jersey1-common jar. > We could later create a hadoop-jersey2-common. > hadoop-jersey1-common and hadoop-jersey2-common would have equivalent classes > - just depend on different versions of Jersey. > Example code: > * > https://github.com/apache/hadoop/blob/12498b35bbb754225b0b2ca90d5ad4f5cf628d56/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/http/HttpServer2.java#L1030 > * > https://github.com/apache/hadoop/blob/12498b35bbb754225b0b2ca90d5ad4f5cf628d56/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/HttpExceptionUtils.java#L89 > Hadoop modules that need access to the common jersey code could start with > depending on hadoop-jersey1-common but later be refactored to use > hadoop-jersey2-common. We could do this on a module by module basis (one a > time). > hadoop-common jar would have its jersey and jsr311-api dependencies removed. > Wdyt [~slfan1989], [~steve_l], [~ayushsaxena] ? -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Created] (HADOOP-19078) reduce use of javax.ws.rs.core.MediaType
PJ Fanning created HADOOP-19078: --- Summary: reduce use of javax.ws.rs.core.MediaType Key: HADOOP-19078 URL: https://issues.apache.org/jira/browse/HADOOP-19078 Project: Hadoop Common Issue Type: Task Reporter: PJ Fanning makes it easier to support jersey 1 and 2 -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Created] (HADOOP-19079) check that class that is loaded is really an exception
PJ Fanning created HADOOP-19079: --- Summary: check that class that is loaded is really an exception Key: HADOOP-19079 URL: https://issues.apache.org/jira/browse/HADOOP-19079 Project: Hadoop Common Issue Type: Task Components: common Reporter: PJ Fanning It can be dangerous taking class names as inputs from HTTP messages even if we control the source I can provide a PR that will highlight the issue. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-19079) check that class that is loaded is really an exception
[ https://issues.apache.org/jira/browse/HADOOP-19079?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] PJ Fanning updated HADOOP-19079: Description: It can be dangerous taking class names as inputs from HTTP messages even if we control the source. Issue is in HttpExceptionUtils in hadoop-common (validateResponse method). I can provide a PR that will highlight the issue. was: It can be dangerous taking class names as inputs from HTTP messages even if we control the source I can provide a PR that will highlight the issue. > check that class that is loaded is really an exception > -- > > Key: HADOOP-19079 > URL: https://issues.apache.org/jira/browse/HADOOP-19079 > Project: Hadoop Common > Issue Type: Task > Components: common >Reporter: PJ Fanning >Priority: Major > > It can be dangerous taking class names as inputs from HTTP messages even if > we control the source. Issue is in HttpExceptionUtils in hadoop-common > (validateResponse method). > I can provide a PR that will highlight the issue. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Created] (HADOOP-19081) move ssh/sftp code out of hadoop-common into a dedicated jar
PJ Fanning created HADOOP-19081: --- Summary: move ssh/sftp code out of hadoop-common into a dedicated jar Key: HADOOP-19081 URL: https://issues.apache.org/jira/browse/HADOOP-19081 Project: Hadoop Common Issue Type: Task Reporter: PJ Fanning We could call it hadoop-ssh-common. This code is only used in 1 or 2 other places and it means that hadoop-common (which is used in a lot of places) leaks dependencies on ssh-core and jsch jars to many places. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-19081) move ssh/sftp code out of hadoop-common into a dedicated jar
[ https://issues.apache.org/jira/browse/HADOOP-19081?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] PJ Fanning updated HADOOP-19081: Description: We could call it hadoop-ssh-common. This code is only used in 1 or 2 other places and it means that hadoop-common (which is used in a lot of places) leaks dependencies on ssh-core and jsch jars to many places. See [~steve_l] comments in HADOOP-19076 was: We could call it hadoop-ssh-common. This code is only used in 1 or 2 other places and it means that hadoop-common (which is used in a lot of places) leaks dependencies on ssh-core and jsch jars to many places. > move ssh/sftp code out of hadoop-common into a dedicated jar > > > Key: HADOOP-19081 > URL: https://issues.apache.org/jira/browse/HADOOP-19081 > Project: Hadoop Common > Issue Type: Task >Reporter: PJ Fanning >Priority: Major > > We could call it hadoop-ssh-common. This code is only used in 1 or 2 other > places and it means that hadoop-common (which is used in a lot of places) > leaks dependencies on ssh-core and jsch jars to many places. > See [~steve_l] comments in HADOOP-19076 -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Created] (HADOOP-19088) upgrade to jersey-json 1.22.0
PJ Fanning created HADOOP-19088: --- Summary: upgrade to jersey-json 1.22.0 Key: HADOOP-19088 URL: https://issues.apache.org/jira/browse/HADOOP-19088 Project: Hadoop Common Issue Type: Bug Reporter: PJ Fanning Tidies up support for Jettison and Jackson versions used by Hadoop -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18197) Update protobuf 3.7.1 to a version without CVE-2021-22569
[ https://issues.apache.org/jira/browse/HADOOP-18197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17820707#comment-17820707 ] PJ Fanning commented on HADOOP-18197: - The fix only seems to be in protobuf-java 3.23 and above - https://github.com/protocolbuffers/protobuf/commit/d40aadf823cf7e1e62b65561656f689da8969463 Issue - https://github.com/protocolbuffers/protobuf/issues/11393 The options seem to be * sticking with the shaded protobuf 3_7 jar * upgrading the CI boxes to use JDKs where the issue doesn't happen and adding release notes * trying protobuf-java 3.23 instead of 3.21 * patching our shaded protobuf-java 3.21 jar - we could get the source of protobuf-java 3.21.12, apply the fix above and release a hadoop-shaded-protobuf_3_21 1.2.1 None of these are great but I favour the idea of patching our shaded protobuf-java 3.21.12. > Update protobuf 3.7.1 to a version without CVE-2021-22569 > - > > Key: HADOOP-18197 > URL: https://issues.apache.org/jira/browse/HADOOP-18197 > Project: Hadoop Common > Issue Type: Improvement > Components: hadoop-thirdparty >Affects Versions: thirdparty-1.2.0 >Reporter: Ivan Viaznikov >Assignee: PJ Fanning >Priority: Major > Labels: pull-request-available, security > Fix For: thirdparty-1.2.0 > > Time Spent: 2h 20m > Remaining Estimate: 0h > > The artifact `org.apache.hadoop:hadoop-common` brings in a dependency > `com.google.protobuf:protobuf-java:2.5.0`, which is an outdated version > released in 2013 and it contains a vulnerability > [CVE-2021-22569|https://nvd.nist.gov/vuln/detail/CVE-2021-22569]. > Therefore, requesting you to clarify if this library version is going to be > updated in the following releases -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Created] (HADOOP-19090) Update Protocol Buffers installation to 3.23.4
PJ Fanning created HADOOP-19090: --- Summary: Update Protocol Buffers installation to 3.23.4 Key: HADOOP-19090 URL: https://issues.apache.org/jira/browse/HADOOP-19090 Project: Hadoop Common Issue Type: Improvement Components: build Reporter: PJ Fanning We are seeing issues with Java 8 usage of protobuf-java See https://issues.apache.org/jira/browse/HADOOP-18197 and comments about java.lang.NoSuchMethodError: java.nio.ByteBuffer.position(I)Ljava/nio/ByteBuffer; -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18197) Update protobuf 3.7.1 to a version without CVE-2021-22569
[ https://issues.apache.org/jira/browse/HADOOP-18197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17821186#comment-17821186 ] PJ Fanning commented on HADOOP-18197: - I have https://github.com/apache/hadoop-thirdparty/pull/34 and have done some basic testing with Hadoop. I'm running https://github.com/apache/hadoop/pull/6593 as an experiment. > Update protobuf 3.7.1 to a version without CVE-2021-22569 > - > > Key: HADOOP-18197 > URL: https://issues.apache.org/jira/browse/HADOOP-18197 > Project: Hadoop Common > Issue Type: Improvement > Components: hadoop-thirdparty >Affects Versions: thirdparty-1.2.0 >Reporter: Ivan Viaznikov >Assignee: PJ Fanning >Priority: Major > Labels: pull-request-available, security > Fix For: thirdparty-1.2.0 > > Time Spent: 2h 20m > Remaining Estimate: 0h > > The artifact `org.apache.hadoop:hadoop-common` brings in a dependency > `com.google.protobuf:protobuf-java:2.5.0`, which is an outdated version > released in 2013 and it contains a vulnerability > [CVE-2021-22569|https://nvd.nist.gov/vuln/detail/CVE-2021-22569]. > Therefore, requesting you to clarify if this library version is going to be > updated in the following releases -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-12705) Upgrade Jackson 2.2.3 to 2.5.3 or later
[ https://issues.apache.org/jira/browse/HADOOP-12705?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15629174#comment-15629174 ] PJ Fanning commented on HADOOP-12705: - I think only 2.7.6 and 2.8.x have the XEE fix. > Upgrade Jackson 2.2.3 to 2.5.3 or later > --- > > Key: HADOOP-12705 > URL: https://issues.apache.org/jira/browse/HADOOP-12705 > Project: Hadoop Common > Issue Type: Sub-task > Components: build >Affects Versions: 2.8.0 >Reporter: Steve Loughran > Attachments: HADOOP-12705.002.patch, HADOOP-12705.01.patch, > HADOOP-13050-001.patch > > > There's no rush to do this; this is just the JIRA to track versions. However, > without the upgrade, things written for Jackson 2.4.4 can break ( SPARK-12807) > being Jackson, this is a potentially dangerous update. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-12705) Upgrade Jackson 2.2.3 to 2.5.3 or later
[ https://issues.apache.org/jira/browse/HADOOP-12705?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15355804#comment-15355804 ] PJ Fanning commented on HADOOP-12705: - Can we upgrade to jackson v2.7.6 or v2.8.0 - these versions coming soon, fix an XML Entity Expansion vulnerability? Would it be possible to remove the dependency on jackson 1.9.13 too - this code base is no longer maintained and has the same XML Entity Expansion vulnerability? > Upgrade Jackson 2.2.3 to 2.5.3 or later > --- > > Key: HADOOP-12705 > URL: https://issues.apache.org/jira/browse/HADOOP-12705 > Project: Hadoop Common > Issue Type: Sub-task > Components: build >Affects Versions: 2.8.0 >Reporter: Steve Loughran > Attachments: HADOOP-12705.01.patch, HADOOP-13050-001.patch > > > There's no rush to do this; this is just the JIRA to track versions. However, > without the upgrade, things written for Jackson 2.4.4 can break ( SPARK-12807) > being Jackson, this is a potentially dangerous update. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-12705) Upgrade Jackson 2.2.3 to 2.5.3 or later
[ https://issues.apache.org/jira/browse/HADOOP-12705?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15355860#comment-15355860 ] PJ Fanning commented on HADOOP-12705: - [@akira.ajisaka] https://github.com/FasterXML/jackson-databind/issues/1279 covers the latest XEE bug > Upgrade Jackson 2.2.3 to 2.5.3 or later > --- > > Key: HADOOP-12705 > URL: https://issues.apache.org/jira/browse/HADOOP-12705 > Project: Hadoop Common > Issue Type: Sub-task > Components: build >Affects Versions: 2.8.0 >Reporter: Steve Loughran > Attachments: HADOOP-12705.01.patch, HADOOP-13050-001.patch > > > There's no rush to do this; this is just the JIRA to track versions. However, > without the upgrade, things written for Jackson 2.4.4 can break ( SPARK-12807) > being Jackson, this is a potentially dangerous update. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (HADOOP-12705) Upgrade Jackson 2.2.3 to 2.5.3 or later
[ https://issues.apache.org/jira/browse/HADOOP-12705?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15355860#comment-15355860 ] PJ Fanning edited comment on HADOOP-12705 at 6/29/16 9:58 PM: -- [@aajisaka] https://github.com/FasterXML/jackson-databind/issues/1279 covers the latest XEE bug was (Author: pj.fanning): [@akira.ajisaka] https://github.com/FasterXML/jackson-databind/issues/1279 covers the latest XEE bug > Upgrade Jackson 2.2.3 to 2.5.3 or later > --- > > Key: HADOOP-12705 > URL: https://issues.apache.org/jira/browse/HADOOP-12705 > Project: Hadoop Common > Issue Type: Sub-task > Components: build >Affects Versions: 2.8.0 >Reporter: Steve Loughran > Attachments: HADOOP-12705.01.patch, HADOOP-13050-001.patch > > > There's no rush to do this; this is just the JIRA to track versions. However, > without the upgrade, things written for Jackson 2.4.4 can break ( SPARK-12807) > being Jackson, this is a potentially dangerous update. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (HADOOP-12705) Upgrade Jackson 2.2.3 to 2.5.3 or later
[ https://issues.apache.org/jira/browse/HADOOP-12705?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15355860#comment-15355860 ] PJ Fanning edited comment on HADOOP-12705 at 6/29/16 10:00 PM: --- https://github.com/FasterXML/jackson-databind/issues/1279 covers the latest XEE bug was (Author: pj.fanning): [@aajisaka] https://github.com/FasterXML/jackson-databind/issues/1279 covers the latest XEE bug > Upgrade Jackson 2.2.3 to 2.5.3 or later > --- > > Key: HADOOP-12705 > URL: https://issues.apache.org/jira/browse/HADOOP-12705 > Project: Hadoop Common > Issue Type: Sub-task > Components: build >Affects Versions: 2.8.0 >Reporter: Steve Loughran > Attachments: HADOOP-12705.01.patch, HADOOP-13050-001.patch > > > There's no rush to do this; this is just the JIRA to track versions. However, > without the upgrade, things written for Jackson 2.4.4 can break ( SPARK-12807) > being Jackson, this is a potentially dangerous update. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Created] (HADOOP-13332) Remove jackson 1.9.13 and switch all jackson code to 2.x code line
PJ Fanning created HADOOP-13332: --- Summary: Remove jackson 1.9.13 and switch all jackson code to 2.x code line Key: HADOOP-13332 URL: https://issues.apache.org/jira/browse/HADOOP-13332 Project: Hadoop Common Issue Type: Sub-task Components: build Affects Versions: 2.8.0 Reporter: PJ Fanning There's no rush to do this; this is just the JIRA to track versions. However, without the upgrade, things written for Jackson 2.4.4 can break ( SPARK-12807) being Jackson, this is a potentially dangerous update. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-13332) Remove jackson 1.9.13 and switch all jackson code to 2.x code line
[ https://issues.apache.org/jira/browse/HADOOP-13332?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] PJ Fanning updated HADOOP-13332: Description: This jackson 1.9 code line is no longer maintained and has a number of issues, including some issues related to XML Entity Expansion vulnerabilities. http://stackoverflow.com/questions/38017676/small-fix-for-cve-2016-3720-with-older-versions-of-jackson-all-1-9-11-and-in-ja Most changes from jackson 1.9 to 2.x just involve changing the package name. was: There's no rush to do this; this is just the JIRA to track versions. However, without the upgrade, things written for Jackson 2.4.4 can break ( SPARK-12807) being Jackson, this is a potentially dangerous update. > Remove jackson 1.9.13 and switch all jackson code to 2.x code line > -- > > Key: HADOOP-13332 > URL: https://issues.apache.org/jira/browse/HADOOP-13332 > Project: Hadoop Common > Issue Type: Sub-task > Components: build >Affects Versions: 2.8.0 >Reporter: PJ Fanning > > This jackson 1.9 code line is no longer maintained and has a number of > issues, including some issues related to XML Entity Expansion > vulnerabilities. > http://stackoverflow.com/questions/38017676/small-fix-for-cve-2016-3720-with-older-versions-of-jackson-all-1-9-11-and-in-ja > Most changes from jackson 1.9 to 2.x just involve changing the package name. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Created] (HADOOP-15804) upgrade to commons-compress 1.18
PJ Fanning created HADOOP-15804: --- Summary: upgrade to commons-compress 1.18 Key: HADOOP-15804 URL: https://issues.apache.org/jira/browse/HADOOP-15804 Project: Hadoop Common Issue Type: Improvement Reporter: PJ Fanning [https://github.com/apache/commons-compress/blob/master/RELEASE-NOTES.txt] Some CVEs have been fixed in recent releases -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-15804) upgrade to commons-compress 1.18
[ https://issues.apache.org/jira/browse/HADOOP-15804?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] PJ Fanning updated HADOOP-15804: Description: [https://github.com/apache/commons-compress/blob/master/RELEASE-NOTES.txt] Some CVEs have been fixed in recent releases (https://commons.apache.org/proper/commons-compress/security-reports.html) was: [https://github.com/apache/commons-compress/blob/master/RELEASE-NOTES.txt] Some CVEs have been fixed in recent releases > upgrade to commons-compress 1.18 > > > Key: HADOOP-15804 > URL: https://issues.apache.org/jira/browse/HADOOP-15804 > Project: Hadoop Common > Issue Type: Improvement >Reporter: PJ Fanning >Priority: Major > > [https://github.com/apache/commons-compress/blob/master/RELEASE-NOTES.txt] > Some CVEs have been fixed in recent releases > (https://commons.apache.org/proper/commons-compress/security-reports.html) -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-15804) upgrade to commons-compress 1.18
[ https://issues.apache.org/jira/browse/HADOOP-15804?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] PJ Fanning updated HADOOP-15804: Description: [https://github.com/apache/commons-compress/blob/master/RELEASE-NOTES.txt] Some CVEs have been fixed in recent releases ([https://commons.apache.org/proper/commons-compress/security-reports.html]) [https://mvnrepository.com/artifact/org.apache.hadoop/hadoop-common/3.1.1] depends on commons-compress 1.4.1 was: [https://github.com/apache/commons-compress/blob/master/RELEASE-NOTES.txt] Some CVEs have been fixed in recent releases (https://commons.apache.org/proper/commons-compress/security-reports.html) > upgrade to commons-compress 1.18 > > > Key: HADOOP-15804 > URL: https://issues.apache.org/jira/browse/HADOOP-15804 > Project: Hadoop Common > Issue Type: Improvement >Reporter: PJ Fanning >Priority: Major > > [https://github.com/apache/commons-compress/blob/master/RELEASE-NOTES.txt] > Some CVEs have been fixed in recent releases > ([https://commons.apache.org/proper/commons-compress/security-reports.html]) > [https://mvnrepository.com/artifact/org.apache.hadoop/hadoop-common/3.1.1] > depends on commons-compress 1.4.1 -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Created] (HADOOP-19289) upgrade to protobuf-java 3.25.5 due to CVE-2024-7254
PJ Fanning created HADOOP-19289: --- Summary: upgrade to protobuf-java 3.25.5 due to CVE-2024-7254 Key: HADOOP-19289 URL: https://issues.apache.org/jira/browse/HADOOP-19289 Project: Hadoop Common Issue Type: Task Components: common Reporter: PJ Fanning https://github.com/advisories/GHSA-735f-pc8j-v9w8 Presumably protobuf encoded messages in Hadoop come from trusted sources but it is still useful to upgrade the jar. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-19302) Update rat version in the docker build.sh script
[ https://issues.apache.org/jira/browse/HADOOP-19302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17886996#comment-17886996 ] PJ Fanning commented on HADOOP-19302: - Please use https://archive.apache.org/dist/creadur/ instead of https://dlcdn.apache.org/creadur ASF Infra encourage projects to remove old releases from dlcdn.apache.org but all releases get automatically archived. I would suggest a better solution is to use the Maven Plugin which relies on Maven Central. https://creadur.apache.org/rat/apache-rat-plugin/ > Update rat version in the docker build.sh script > > > Key: HADOOP-19302 > URL: https://issues.apache.org/jira/browse/HADOOP-19302 > Project: Hadoop Common > Issue Type: Bug >Affects Versions: 3.3.7, 3.4.1 >Reporter: Wei-Chiu Chuang >Priority: Major > > The docker build.sh script uses apache rat 0.15 which is removed from Apache > CDN. > https://github.com/apache/hadoop/blob/docker-hadoop-3.4/build.sh#L20 > The build in the DockerHub doesn't fail, probably because there's cache. But > I don't download it locally. > The latest is 0.16.1. Let's update. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (HADOOP-19302) Update rat version in the docker build.sh script
[ https://issues.apache.org/jira/browse/HADOOP-19302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17886996#comment-17886996 ] PJ Fanning edited comment on HADOOP-19302 at 10/4/24 6:09 PM: -- Please use https://archive.apache.org/dist/creadur/ instead of https://dlcdn.apache.org/creadur ASF Infra encourage projects to remove old releases from dlcdn.apache.org but all releases get automatically archived. Using the archive copy means that you don't have to worry about it being removed, I would suggest a better solution is to use the Maven Plugin which relies on Maven Central. https://creadur.apache.org/rat/apache-rat-plugin/ was (Author: fanningpj): Please use https://archive.apache.org/dist/creadur/ instead of https://dlcdn.apache.org/creadur ASF Infra encourage projects to remove old releases from dlcdn.apache.org but all releases get automatically archived. I would suggest a better solution is to use the Maven Plugin which relies on Maven Central. https://creadur.apache.org/rat/apache-rat-plugin/ > Update rat version in the docker build.sh script > > > Key: HADOOP-19302 > URL: https://issues.apache.org/jira/browse/HADOOP-19302 > Project: Hadoop Common > Issue Type: Bug >Affects Versions: 3.3.7, 3.4.1 >Reporter: Wei-Chiu Chuang >Priority: Major > > The docker build.sh script uses apache rat 0.15 which is removed from Apache > CDN. > https://github.com/apache/hadoop/blob/docker-hadoop-3.4/build.sh#L20 > The build in the DockerHub doesn't fail, probably because there's cache. But > I don't download it locally. > The latest is 0.16.1. Let's update. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
