[jira] [Commented] (HADOOP-18950) upgrade avro to 1.11.3 due to CVE
[ https://issues.apache.org/jira/browse/HADOOP-18950?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17826037#comment-17826037 ] ASF GitHub Bot commented on HADOOP-18950: - steveloughran commented on PR #4854: URL: https://github.com/apache/hadoop/pull/4854#issuecomment-1994281106 let's start this as a [DISCUSS] I'd certainly want our internals to be hidden and upgraded; for the other bits we have to choose how safest to update them > upgrade avro to 1.11.3 due to CVE > - > > Key: HADOOP-18950 > URL: https://issues.apache.org/jira/browse/HADOOP-18950 > Project: Hadoop Common > Issue Type: Bug > Components: common >Reporter: Xuze Yang >Priority: Major > Labels: pull-request-available > > [https://nvd.nist.gov/vuln/detail/CVE-2023-39410] > When deserializing untrusted or corrupted data, it is possible for a reader > to consume memory beyond the allowed constraints and thus lead to out of > memory on the system. This issue affects Java applications using Apache Avro > Java SDK up to and including 1.11.2. Users should update to apache-avro > version 1.11.3 which addresses this issue. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18950) upgrade avro to 1.11.3 due to CVE
[ https://issues.apache.org/jira/browse/HADOOP-18950?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17825807#comment-17825807 ] ASF GitHub Bot commented on HADOOP-18950: - pjfanning commented on PR #4854: URL: https://github.com/apache/hadoop/pull/4854#issuecomment-1992503197 I don't know enough about Avro to hack it to work with shaded and non-shaded annotations. I thought all we cared about was how to support the internal Hadoop code and its internal use of Avro. If we need to support users who want to do their own Avro serialization of Hadoop classes, then I think we should abandon this PR. I think it would be far easier to just upgrade the actual Avro jars that Hadoop uses and give up on shading it. > upgrade avro to 1.11.3 due to CVE > - > > Key: HADOOP-18950 > URL: https://issues.apache.org/jira/browse/HADOOP-18950 > Project: Hadoop Common > Issue Type: Bug > Components: common >Reporter: Xuze Yang >Priority: Major > Labels: pull-request-available > > [https://nvd.nist.gov/vuln/detail/CVE-2023-39410] > When deserializing untrusted or corrupted data, it is possible for a reader > to consume memory beyond the allowed constraints and thus lead to out of > memory on the system. This issue affects Java applications using Apache Avro > Java SDK up to and including 1.11.2. Users should update to apache-avro > version 1.11.3 which addresses this issue. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18950) upgrade avro to 1.11.3 due to CVE
[ https://issues.apache.org/jira/browse/HADOOP-18950?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17825784#comment-17825784 ] ASF GitHub Bot commented on HADOOP-18950: - steveloughran commented on code in PR #4854: URL: https://github.com/apache/hadoop/pull/4854#discussion_r1522032834 ## hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/Path.java: ## @@ -27,7 +27,7 @@ import java.util.Optional; import java.util.regex.Pattern; -import org.apache.avro.reflect.Stringable; +import org.apache.hadoop.thirdparty.avro.reflect.Stringable; Review Comment: This could be dangerous, as we are saying that a public class can no longer be serialised through Avro. Do you think it will be possible for us to retain the unshaded annotation as well as adding the new one? And still have everything to work without Avro on the CP? ## hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/AvroFSInput.java: ## @@ -21,7 +21,7 @@ import java.io.Closeable; import java.io.IOException; -import org.apache.avro.file.SeekableInput; +import org.apache.hadoop.thirdparty.avro.file.SeekableInput; Review Comment: again, this is a public class we don't use internally. Should we actually deprecate it? I don't know what uses it? > upgrade avro to 1.11.3 due to CVE > - > > Key: HADOOP-18950 > URL: https://issues.apache.org/jira/browse/HADOOP-18950 > Project: Hadoop Common > Issue Type: Bug > Components: common >Reporter: Xuze Yang >Priority: Major > Labels: pull-request-available > > [https://nvd.nist.gov/vuln/detail/CVE-2023-39410] > When deserializing untrusted or corrupted data, it is possible for a reader > to consume memory beyond the allowed constraints and thus lead to out of > memory on the system. This issue affects Java applications using Apache Avro > Java SDK up to and including 1.11.2. Users should update to apache-avro > version 1.11.3 which addresses this issue. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18950) upgrade avro to 1.11.3 due to CVE
[ https://issues.apache.org/jira/browse/HADOOP-18950?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17825778#comment-17825778 ] ASF GitHub Bot commented on HADOOP-18950: - steveloughran commented on PR #4854: URL: https://github.com/apache/hadoop/pull/4854#issuecomment-1992402992 let's do it in 3.4.1 after a 1.3.0 release, and make the "we've tuned the packaging" a key change along with "we've fixed the bits steve broke". > upgrade avro to 1.11.3 due to CVE > - > > Key: HADOOP-18950 > URL: https://issues.apache.org/jira/browse/HADOOP-18950 > Project: Hadoop Common > Issue Type: Bug > Components: common >Reporter: Xuze Yang >Priority: Major > Labels: pull-request-available > > [https://nvd.nist.gov/vuln/detail/CVE-2023-39410] > When deserializing untrusted or corrupted data, it is possible for a reader > to consume memory beyond the allowed constraints and thus lead to out of > memory on the system. This issue affects Java applications using Apache Avro > Java SDK up to and including 1.11.2. Users should update to apache-avro > version 1.11.3 which addresses this issue. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18950) upgrade avro to 1.11.3 due to CVE
[
https://issues.apache.org/jira/browse/HADOOP-18950?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17824980#comment-17824980
]
ASF GitHub Bot commented on HADOOP-18950:
-
hadoop-yetus commented on PR #4854:
URL: https://github.com/apache/hadoop/pull/4854#issuecomment-1986962381
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 31s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 1s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +0 :ok: | xmllint | 0m 0s | | xmllint was not available. |
| +0 :ok: | shelldocs | 0m 0s | | Shelldocs was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 3 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 15m 30s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 31m 42s | | trunk passed |
| +1 :green_heart: | compile | 17m 12s | | trunk passed with JDK
Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1 |
| +1 :green_heart: | compile | 16m 9s | | trunk passed with JDK
Private Build-1.8.0_392-8u392-ga-1~20.04-b08 |
| +1 :green_heart: | checkstyle | 4m 17s | | trunk passed |
| +1 :green_heart: | mvnsite | 21m 45s | | trunk passed |
| +1 :green_heart: | javadoc | 9m 12s | | trunk passed with JDK
Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1 |
| +1 :green_heart: | javadoc | 7m 52s | | trunk passed with JDK
Private Build-1.8.0_392-8u392-ga-1~20.04-b08 |
| +0 :ok: | spotbugs | 0m 18s | | branch/hadoop-project no spotbugs
output file (spotbugsXml.xml) |
| -1 :x: | spotbugs | 2m 13s |
[/branch-spotbugs-hadoop-common-project_hadoop-common-warnings.html](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4854/10/artifact/out/branch-spotbugs-hadoop-common-project_hadoop-common-warnings.html)
| hadoop-common-project/hadoop-common in trunk has 1 extant spotbugs
warnings. |
| +0 :ok: | spotbugs | 0m 19s | |
branch/hadoop-client-modules/hadoop-client no spotbugs output file
(spotbugsXml.xml) |
| +0 :ok: | spotbugs | 0m 18s | |
branch/hadoop-client-modules/hadoop-client-minicluster no spotbugs output file
(spotbugsXml.xml) |
| -1 :x: | spotbugs | 33m 35s |
[/branch-spotbugs-root-warnings.html](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4854/10/artifact/out/branch-spotbugs-root-warnings.html)
| root in trunk has 5 extant spotbugs warnings. |
| +1 :green_heart: | shadedclient | 69m 39s | | branch has no errors
when building and testing our client artifacts. |
| -0 :warning: | patch | 70m 0s | | Used diff version of patch file.
Binary files and potentially other changes not applied. Please rebase and
squash commits if necessary. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 36s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 52m 38s | | the patch passed |
| +1 :green_heart: | compile | 18m 35s | | the patch passed with JDK
Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1 |
| +1 :green_heart: | javac | 18m 35s | | the patch passed |
| +1 :green_heart: | compile | 17m 4s | | the patch passed with JDK
Private Build-1.8.0_392-8u392-ga-1~20.04-b08 |
| +1 :green_heart: | javac | 17m 4s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 4m 38s | | root: The patch generated
0 new + 445 unchanged - 1 fixed = 445 total (was 446) |
| +1 :green_heart: | mvnsite | 17m 7s | | the patch passed |
| +1 :green_heart: | shellcheck | 0m 0s | | No new issues. |
| +1 :green_heart: | javadoc | 8m 52s | | the patch passed with JDK
Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1 |
| +1 :green_heart: | javadoc | 7m 57s | | the patch passed with JDK
Private Build-1.8.0_392-8u392-ga-1~20.04-b08 |
| +0 :ok: | spotbugs | 0m 17s | | hadoop-project has no data from
spotbugs |
| +0 :ok: | spotbugs | 0m 22s | | hadoop-client-modules/hadoop-client
has no data from spotbugs |
| +0 :ok: | spotbugs | 0m 19s | |
hadoop-client-modules/hadoop-client-minicluster has no data from spotbugs |
| +1 :green_heart: | shadedclient | 68m 18s | | patch has no errors
when building and testing our client artifacts. |
_ Othe
[jira] [Commented] (HADOOP-18950) upgrade avro to 1.11.3 due to CVE
[
https://issues.apache.org/jira/browse/HADOOP-18950?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17820446#comment-17820446
]
ASF GitHub Bot commented on HADOOP-18950:
-
hadoop-yetus commented on PR #4854:
URL: https://github.com/apache/hadoop/pull/4854#issuecomment-1962873755
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 55s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 1s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available.
|
| +0 :ok: | xmllint | 0m 1s | | xmllint was not available. |
| +0 :ok: | shelldocs | 0m 1s | | Shelldocs was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 3 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 27m 25s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 32m 44s | | trunk passed |
| +1 :green_heart: | compile | 17m 36s | | trunk passed with JDK
Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | compile | 16m 9s | | trunk passed with JDK
Private Build-1.8.0_392-8u392-ga-1~20.04-b08 |
| +1 :green_heart: | checkstyle | 4m 36s | | trunk passed |
| +1 :green_heart: | mvnsite | 28m 5s | | trunk passed |
| +1 :green_heart: | javadoc | 12m 43s | | trunk passed with JDK
Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 7m 49s | | trunk passed with JDK
Private Build-1.8.0_392-8u392-ga-1~20.04-b08 |
| +0 :ok: | spotbugs | 0m 20s | | branch/hadoop-project no spotbugs
output file (spotbugsXml.xml) |
| -1 :x: | spotbugs | 2m 13s |
[/branch-spotbugs-hadoop-common-project_hadoop-common-warnings.html](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4854/9/artifact/out/branch-spotbugs-hadoop-common-project_hadoop-common-warnings.html)
| hadoop-common-project/hadoop-common in trunk has 1 extant spotbugs
warnings. |
| +0 :ok: | spotbugs | 0m 23s | |
branch/hadoop-client-modules/hadoop-client no spotbugs output file
(spotbugsXml.xml) |
| +0 :ok: | spotbugs | 0m 23s | |
branch/hadoop-client-modules/hadoop-client-minicluster no spotbugs output file
(spotbugsXml.xml) |
| -1 :x: | spotbugs | 30m 59s |
[/branch-spotbugs-root-warnings.html](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4854/9/artifact/out/branch-spotbugs-root-warnings.html)
| root in trunk has 5 extant spotbugs warnings. |
| +1 :green_heart: | shadedclient | 62m 56s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 47s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 45m 14s | | the patch passed |
| +1 :green_heart: | compile | 16m 58s | | the patch passed with JDK
Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javac | 16m 58s | | the patch passed |
| +1 :green_heart: | compile | 16m 7s | | the patch passed with JDK
Private Build-1.8.0_392-8u392-ga-1~20.04-b08 |
| +1 :green_heart: | javac | 16m 7s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 4m 27s | | root: The patch generated
0 new + 445 unchanged - 1 fixed = 445 total (was 446) |
| +1 :green_heart: | mvnsite | 15m 39s | | the patch passed |
| +1 :green_heart: | shellcheck | 0m 0s | | No new issues. |
| +1 :green_heart: | javadoc | 8m 31s | | the patch passed with JDK
Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 7m 51s | | the patch passed with JDK
Private Build-1.8.0_392-8u392-ga-1~20.04-b08 |
| +0 :ok: | spotbugs | 0m 22s | | hadoop-project has no data from
spotbugs |
| +0 :ok: | spotbugs | 0m 23s | | hadoop-client-modules/hadoop-client
has no data from spotbugs |
| +0 :ok: | spotbugs | 0m 23s | |
hadoop-client-modules/hadoop-client-minicluster has no data from spotbugs |
| +1 :green_heart: | shadedclient | 63m 15s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| -1 :x: | unit | 790m 22s |
[/patch-unit-root.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4854/9/artifact/out/patch-unit-root.txt)
| root in the patch fail
