[
https://issues.apache.org/jira/browse/HADOOP-19974?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18109432#comment-18109432
]
ASF GitHub Bot commented on HADOOP-19974:
-----------------------------------------
ppkarwasz opened a new pull request, #8707:
URL: https://github.com/apache/hadoop/pull/8707
### Description of PR
Every JAR module now attaches a CycloneDX 1.7 XML SBOM under the `dist`
profile (the profile used to deploy artifacts to Maven Central, see
HADOOP-18590).
CycloneDX 1.7 introduces an `isExternal` attribute that makes it possible to
distinguish dependencies shaded inside a JAR from those merely referenced on
the classpath — a distinction essential for vulnerability management. Since
`cyclonedx-maven-plugin` does not yet support `maven-shade-plugin` (see
CycloneDX/cyclonedx-maven-plugin#472), this PR post-processes the generated BOM
with a Groovy script (`dev-support/sbom/sbom.groovy`, run by
`gmavenplus-plugin` in the `verify` phase):
- Ordinary dependencies get an explicit `isExternal="true"`; dependencies
embedded by `maven-shade-plugin` rely on the schema default of `false`.
- Relocated dependencies are technically derivative libraries, so they
receive a `pedigree` ancestor carrying the original artifact's purl and SHA-256
hash. Both the relocated copy and its ancestor deliberately share the
original's PURL, because vulnerability scanners cannot match a newly minted
PURL against vulnerability databases and do not yet consult the pedigree.
- Dependencies embedded verbatim (modules shading without relocations) keep
the original's hash and get a notes-only pedigree documenting the embedding.
- The shaded artifact set is derived from the effective `maven-shade-plugin`
configuration, so it cannot drift from the shade `artifactSet`; `-DskipShade`
builds classify everything as external, matching the jar actually produced.
- Hash lists are trimmed to SHA-256 and the result is re-validated against
the CycloneDX 1.7 schema, failing the build on violations.
Out of scope (follow-ups): SBOMs for the binary distribution tarball,
correcting license metadata, and a possible switch to JSON output (or
publishing both formats).
### How was this patch tested?
`mvn clean install -Pdist -DskipTests` on representative modules, asserting
the generated `target/bom.xml`:
- `hadoop-annotations` (no shading): all components `isExternal="true"`, no
pedigree.
- `hadoop-bos` (include-style `artifactSet` with relocations): only the 3
compile-scope includes embedded, with pedigree ancestors carrying the SHA-256;
provided-scope includes correctly reported external — verified against the
actual jar contents.
- `hadoop-tos` (no `artifactSet`, no relocations): all 5 compile-scope
dependencies embedded verbatim, hashes kept, notes-only pedigree.
- `hadoop-client-runtime` / `hadoop-client-api` (exclude-style with
wildcards / include-style): all 63 externals match the shade excludes (incl.
`io.netty:*` and `org.glassfish.*` wildcard families); 70 embedded components
carry relocation pedigrees.
- `-DskipShade` and `-Dcyclonedx.skip` paths verified as no-ops.
- Schema validity is enforced by the script itself on every module.
### For code changes:
- [x] Does the title of this PR start with the corresponding JIRA issue id
(e.g. 'HADOOP-17799. Your PR title ...')?
- [ ] Object storage: Have the integration tests been executed and the
endpoint
declared according to the connector-specific documentation? *Note:
Automated CI
testing doesn't cover all cases so manual testing with cloud storage
is still
required.*
- [x] If adding new dependencies to the code, are these dependencies
licensed in a way that is compatible for inclusion under [ASF
2.0](http://www.apache.org/legal/resolved.html#category-a)? *(build-time only:
`gmavenplus-plugin`, `org.apache.groovy:groovy` and
`org.cyclonedx:cyclonedx-core-java` are all Apache-2.0; nothing is bundled in
released artifacts, so `LICENSE-binary`/`NOTICE-binary` are unaffected)*
- [ ] If applicable, have you updated the `LICENSE`, `LICENSE-binary`,
`NOTICE-binary` files?
### AI Tooling
If an AI tool was used:
- [x] The PR includes the phrase "Contains content generated by Claude Code"
- [x] My use of AI contributions follows the ASF legal policy
https://www.apache.org/legal/generative-tooling.html
> Publish CycloneDX 1.7 SBOMs that distinguish shaded dependencies
> ----------------------------------------------------------------
>
> Key: HADOOP-19974
> URL: https://issues.apache.org/jira/browse/HADOOP-19974
> Project: Hadoop Common
> Issue Type: Improvement
> Components: build
> Reporter: Piotr Karwasz
> Priority: Minor
>
> Hadoop currently generates CycloneDX SBOMs only under the {{dist}} profile
> (see HADOOP-18590), in XML, at spec version 1.6.
> CycloneDX 1.7 introduces an {{isExternal}} attribute that makes it possible
> to distinguish dependencies shaded inside a JAR from those merely referenced
> on the classpath. This distinction is essential for vulnerability management:
> only the embedded copies actually ship with our artifacts.
> Relocated dependencies are technically derivative libraries, so they
> additionally receive a {{pedigree}} element pointing to the original
> artifact, including its SHA-256 checksum. Vulnerability management systems
> would not be able to match a new PURL minted for the relocated copy against
> vulnerability databases, and they do not yet consult the pedigree, so both
> the relocated copy and its ancestor deliberately share the PURL of the
> original. Dependencies embedded without relocation are verbatim copies of the
> original, so they keep the original's checksum and need no pedigree.
> Since {{cyclonedx-maven-plugin}} does not yet support {{maven-shade-plugin}}
> (see
> [CycloneDX/cyclonedx-maven-plugin#472|https://github.com/CycloneDX/cyclonedx-maven-plugin/issues/472]),
> this issue post-processes the generated BOM with a Groovy script
> ({{dev-support/sbom/sbom.groovy}}) in the same {{dist}} profile. The set of
> shaded dependencies is derived from the effective {{maven-shade-plugin}}
> configuration, so it cannot drift from the shade {{artifactSet}}, and the
> result is validated against the CycloneDX 1.7 schema.
> Out of scope for this issue (follow-ups): SBOMs for the binary distribution
> tarball, and correcting license metadata in the generated documents.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
