[
https://issues.apache.org/jira/browse/HADOOP-20049?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18137837#comment-18137837
]
ASF GitHub Bot commented on HADOOP-20049:
-----------------------------------------
joseluisll commented on PR #8903:
URL: https://github.com/apache/hadoop/pull/8903#issuecomment-6625569434
> @joseluisll I mean overriding, not excluding.
>
> > Maven picks the “nearest definition”.
>
>
https://maven.apache.org/guides/introduction/introduction-to-dependency-mechanism.html
>
> So declaring the slf4j version at the top level in the
`dependencyManagement` block overrides the transitive pulled version
Hi!. I only believe in the code I execute and see the results in the logs,
my two proposals work from the point of view of a downstream application, both
declaring in the module and the exclusion. And my tests indicate that under
maven 3.9.11 and 3.9.16 what you say does not work, only with maven 4.
Also, large downstream projects (HBase, Hive, Flink, Tez, Ozone, Phoenix)
depend on hadoop-hdfs directly but pin slf4j-api in their own
dependencyManagement, so they’re unaffected either way; Spark uses the shaded
client. The risk is for smaller consumers that depend on hadoop-hdfs or
hadoop-yarn-client (e.g. for MiniDFSCluster) without managing slf4j themselves.
| Project | Uses `hadoop-hdfs` directly? | Uses `hadoop-yarn-client`? | Own
slf4j-api pin | Hadoop version today |
|---|---|---|---|---|
| **HBase** | **Yes, normal scope** in hbase-server, hbase-asyncfs,
hbase-mapreduce, hbase-balancer. The test-jar is compile scope in
hbase-testing-util. | Only in the bring-your-own-Hadoop assembly, provided
scope | 1.7.30, with log4j2's slf4j 1.x binding | 3.4.3 |
| Hive | Yes: ql, shims, hcatalog, metastore, streaming, plus many test
modules | Yes: ql, shims | 1.7.30, with log4j2's slf4j 1.x binding | 3.4.2 |
| Flink | Yes: flink-runtime, flink-yarn, flink-hadoop-fs (excludes Hadoop's
slf4j-reload4j) | Yes: flink-yarn | **2.0.20**, with log4j2's slf4j 2 binding |
— |
| Tez | Mostly test scope; managed in the root POM with
`org.eclipse.jetty:*` excluded | Yes, in most modules | 1.7.36 | 3.5.0 |
| Ozone | Yes: container-service, cli-debug | No | **2.0.20** (through
slf4j-bom) | 3.4.3 |
| Phoenix | Test and provided scope | Managed only | 1.7.36 | 3.4.2 |
| Spark | **No.** It uses the shaded `hadoop-client-api` and
`hadoop-client-runtime` | No | 2.0.17 | 3.5.0 |
| Iceberg | Declared in its Gradle version catalog | No | 2.0.18 | 3.4.3 |
So following the best practice for dependency management we should declare
the dependency where it is used, while the version stays managed in one place
in hadoop-project, consistent with the 22 other Hadoop modules that already
declare slf4j-api. Exclusions are meant for removing unwanted dependencies
rather than choosing versions, and the clean long-term fix is moving Hadoop to
slf4j 2. Shall I create another JIRA for that?
> Keep slf4j-api at the managed version for consumers once Jetty 12 is in
> -----------------------------------------------------------------------
>
> Key: HADOOP-20049
> URL: https://issues.apache.org/jira/browse/HADOOP-20049
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: build, common
> Reporter: Jose Luis López
> Assignee: Jose Luis López
> Priority: Major
> Labels: pull-request-available
>
> Jetty 12.1.14 declares {{org.slf4j:slf4j-api:2.0.17}} at compile scope; in
> Jetty 9.4 it is provided and optional. It runs correctly on Hadoop's 1.7.36,
> but the declaration reaches Maven consumers of the non-shaded artifacts.
> {{hadoop-yarn-client}}, {{hadoop-hdfs}} and {{hadoop-hdfs-nfs}} depend on
> Jetty directly, do not declare {{slf4j-api}}, and keep {{hadoop-common}} at
> provided. A consumer declaring one of them before {{hadoop-client}} then
> resolves {{slf4j-api:2.0.17}} with {{slf4j-reload4j:1.7.36}}. slf4j 2 ignores
> that binding, so all logging goes to NOP, with only a warning on stderr.
> Shaded clients are not affected.
> This must land before HADOOP-19972 or together with it. Hadoop stays on slf4j
> 1.7.36. Moving to slf4j 2 is out of scope.
> *Verification:* with the fix, no published artifact resolves {{slf4j-api}}
> 2.0.17 through Jetty, alone or declared before {{hadoop-client}}.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
