This is an automated email from the ASF dual-hosted git repository.
pjfanning pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/pekko-grpc.git
The following commit(s) were added to refs/heads/main by this push:
new 1d2a2f6f Ship a complete LICENSE and NOTICE in the assembly jars (#865)
1d2a2f6f is described below
commit 1d2a2f6f00d39de1a1f71976d2d8f66628d81d74
Author: PJ Fanning <[email protected]>
AuthorDate: Mon Aug 31 12:52:13 2026 +0100
Ship a complete LICENSE and NOTICE in the assembly jars (#865)
* Ship a complete LICENSE and NOTICE in the assembly jars
Motivation:
pekko-grpc-codegen and pekko-grpc-scalapb-protoc-plugin publish assembly
jars
and bat archives that bundle classes from 3rd party projects, so their
META-INF/LICENSE and META-INF/NOTICE have to account for those projects.
legal/AssemblyLicense.txt and legal/AssemblyNotice.txt exist for that, but:
- The codegen assembly jar and bat shipped no META-INF/LICENSE at all. The
merge strategy discarded META-INF/LICENSE to stop guava/failureaccess
colliding, and that also discarded the copy the ApacheSonatypePlugin
resource generator puts in the jar.
- legal/AssemblyLicense.txt had drifted from what is actually bundled. It
listed org.checkerframework:checker-qual, which guava dropped in favour of
org.jspecify:jspecify, and jspecify was not listed anywhere.
- The scalapb-protoc-plugin assembly bundled protobuf-java 4.35.0, pulled in
transitively by compilerplugin, rather than the pinned 4.36.0 that codegen
and runtime bundle.
Nothing failed the build when the legal files fell behind a dependency
change.
Modification:
Merge META-INF/LICENSE and META-INF/NOTICE with MergeStrategy.preferProject
in
both assemblies, so our own files win over the ones the dependencies bring
in.
Swap checker-qual for jspecify in legal/AssemblyLicense.txt and drop the MIT
text that was only there for checker-qual.
Pin protobuf-java in scalapb-protoc-plugin via a new Dependencies group, the
same way codegen and runtime already do.
Add project/AssemblyLicenseCheck.scala with two tasks, wired into both
assembly projects and run in CI as the checkAssemblyLicenses alias:
- assemblyLicenseCheck fails when a module on the assembly classpath is not
named in the assembly LICENSE file, and warns about listed modules that
are
not bundled for the Scala version being built.
- assemblyMetaInfCheck opens the published assembly jar and bat archive and
fails unless each carries META-INF/LICENSE and META-INF/NOTICE matching
the
files in legal/.
Result:
Both assembly jars and both bat archives ship a LICENSE and NOTICE covering
the 3rd party classes they bundle, both bundle protobuf-java 4.36.0, and CI
fails if a dependency change leaves the legal files behind.
Tests:
- sbt checkAssemblyLicenses - passed. assemblyMetaInfCheck fails on the
codegen assembly before the merge strategy fix (META-INF/LICENSE is
missing), and assemblyLicenseCheck fails before the AssemblyLicense.txt
fix
(org.jspecify:jspecify is not accounted for).
- sbt scalafmtSbtCheck - passed.
References:
None - assembly jar license completeness
* Update AssemblyLicenseCheck.scala
* Name the archive when it cannot be read as a zip
Motivation:
`assemblyMetaInfCheck` opened each archive with `new ZipFile`, which throws
`ZipException: error in opening zip file` without naming the file, so a
corrupt
or truncated archive in CI gave no clue which one it was.
Modification:
Open the archive and its entries through `sbt.io.Using`, and report
`IOException`
with the archive name.
Result:
A failure to read an archive names it, like the license problems reported
below it.
Tests:
- sbt checkAssemblyLicenses - passed for both assembly projects (jar and
bat)
References:
Refs #865
* Assert the shipped LICENSE and NOTICE still say something
Motivation:
`assemblyMetaInfCheck` only compared the archived files with the curated
ones in
`legal/`, which says the merge strategy picked our copy but not that the
copy is
still a real LICENSE or NOTICE. An emptied or truncated curated file would
ship
and pass.
Modification:
Check the shipped bytes for the markers each file has to carry: the Apache
License 2.0 heading and terms plus the 3rd party section in LICENSE, and the
product name, the ASF attribution and the 3rd party section in NOTICE.
Missing
markers are reported alongside the existing problems for that archive.
Result:
An assembly that ships a gutted LICENSE or NOTICE fails the check.
Tests:
- sbt checkAssemblyLicenses - passed for both assembly projects (jar and
bat)
- With legal/AssemblyLicense.txt cut down to its first two lines,
codegen/assemblyMetaInfCheck fails: "META-INF/LICENSE does not mention
'TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION',
'contain classes from 3rd party projects', 'Apache License Version 2.0:'"
References:
Refs #865
---
.github/workflows/build-test.yml | 5 ++
build.sbt | 19 +++--
legal/AssemblyLicense.txt | 27 +------
project/AssemblyLicenseCheck.scala | 142 +++++++++++++++++++++++++++++++++++++
project/Dependencies.scala | 4 ++
5 files changed, 167 insertions(+), 30 deletions(-)
diff --git a/.github/workflows/build-test.yml b/.github/workflows/build-test.yml
index a0b73d21..4b03e635 100644
--- a/.github/workflows/build-test.yml
+++ b/.github/workflows/build-test.yml
@@ -52,6 +52,11 @@ jobs:
cp .jvmopts-ci .jvmopts
sbt grpcVersionSyncCheck googleProtobufVersionSyncCheck
mimaReportBinaryIssues
+ - name: Assembly license check
+ run: |-
+ cp .jvmopts-ci .jvmopts
+ sbt checkAssemblyLicenses
+
compile-benchmarks:
name: Compile Benchmarks
runs-on: ubuntu-latest
diff --git a/build.sbt b/build.sbt
index a0ecbfb5..8cf424ee 100644
--- a/build.sbt
+++ b/build.sbt
@@ -49,6 +49,10 @@ ThisBuild / javafmtFormatterCompatibleJavaVersion := 17
addCommandAlias("checkCodeStyle", "scalafmtCheckAll; scalafmtSbtCheck;
javafmtCheckAll; +headerCheckAll")
addCommandAlias("applyCodeStyle", "+headerCreateAll; scalafmtAll; scalafmtSbt;
javafmtAll")
+addCommandAlias(
+ "checkAssemblyLicenses",
+ "codegen/assemblyLicenseCheck; codegen/assemblyMetaInfCheck; " +
+ "scalapb-protoc-plugin/assemblyLicenseCheck;
scalapb-protoc-plugin/assemblyMetaInfCheck")
val pekkoGrpcCodegenId = s"$pekkoPrefix-codegen"
lazy val codegen = Project(id = "codegen", base = file("codegen"))
@@ -58,12 +62,14 @@ lazy val codegen = Project(id = "codegen", base =
file("codegen"))
.settings(Dependencies.codegen)
.settings(resolvers += Resolver.sbtPluginRepo("releases"))
.settings(MetaInfLicenseNoticeCopy.assemblySettings)
+ .settings(AssemblyLicenseCheck.settings)
.settings(
name := s"$pekkoPrefix-codegen",
mkBatAssemblyTask := {
val file = assembly.value
Assemblies.mkBatAssembly(file)
},
+ AssemblyLicenseCheck.assemblyMetaInfArchives += mkBatAssemblyTask.value,
buildInfoKeys ++= Seq[BuildInfoKey](organization, name, version,
scalaVersion, sbtVersion),
buildInfoKeys += "runtimeArtifactName" -> pekkoGrpcRuntimeName,
buildInfoKeys += "pekkoVersion" -> Dependencies.Versions.pekko,
@@ -80,7 +86,9 @@ lazy val codegen = Project(id = "codegen", base =
file("codegen"))
(assembly / assemblyOption) := (assembly /
assemblyOption).value.withPrependShellScript(
Some(sbtassembly.AssemblyPlugin.defaultUniversalScript(shebang = true))),
(assembly / assemblyMergeStrategy) := {
- case PathList("META-INF", "LICENSE") =>
MergeStrategy.discard
+ // our own META-INF/LICENSE and META-INF/NOTICE cover the bundled 3rd
party classes,
+ // so they win over the ones the dependencies bring in
+ case PathList("META-INF", "LICENSE" | "NOTICE") =>
MergeStrategy.preferProject
case PathList("META-INF", "MANIFEST.MF") =>
MergeStrategy.discard
case PathList("META-INF", "versions", _, "module-info.class") =>
MergeStrategy.discard
case "LICENSE" | "LICENSE.txt" | "NOTICE" =>
MergeStrategy.discard
@@ -130,15 +138,15 @@ val pekkoGrpcProtocPluginId =
s"$pekkoPrefix-scalapb-protoc-plugin"
lazy val scalapbProtocPlugin = Project(id = "scalapb-protoc-plugin", base =
file("scalapb-protoc-plugin"))
.disablePlugins(MimaPlugin)
.settings(MetaInfLicenseNoticeCopy.assemblySettings)
+ .settings(AssemblyLicenseCheck.settings)
+ .settings(Dependencies.scalapbProtocPlugin)
.settings(
name := s"$pekkoPrefix-scalapb-protoc-plugin",
- libraryDependencies += {
- Dependencies.Compile.scalapbCompilerPlugin
- },
mkBatAssemblyTask := {
val file = assembly.value
Assemblies.mkBatAssembly(file)
},
+ AssemblyLicenseCheck.assemblyMetaInfArchives += mkBatAssemblyTask.value,
(Compile / assembly / artifact) := {
val art = (Compile / assembly / artifact).value
art.withClassifier(Some("assembly"))
@@ -147,6 +155,9 @@ lazy val scalapbProtocPlugin = Project(id =
"scalapb-protoc-plugin", base = file
(assembly / assemblyOption) := (assembly /
assemblyOption).value.withPrependShellScript(
Some(sbtassembly.AssemblyPlugin.defaultUniversalScript(shebang = true))),
(assembly / assemblyMergeStrategy) := {
+ // our own META-INF/LICENSE and META-INF/NOTICE cover the bundled 3rd
party classes,
+ // so they win over the ones the dependencies bring in
+ case PathList("META-INF", "LICENSE" | "NOTICE") =>
MergeStrategy.preferProject
case PathList("META-INF", "MANIFEST.MF") =>
MergeStrategy.discard
case PathList("META-INF", "versions", _, "module-info.class") =>
MergeStrategy.discard
case "LICENSE" | "LICENSE.txt" | "NOTICE" =>
MergeStrategy.discard
diff --git a/legal/AssemblyLicense.txt b/legal/AssemblyLicense.txt
index 685edcc5..a8316b4b 100644
--- a/legal/AssemblyLicense.txt
+++ b/legal/AssemblyLicense.txt
@@ -220,7 +220,7 @@ com.google.j2objc:j2objc-annotations
io.grpc:grpc-api
io.grpc:grpc-protobuf
io.grpc:grpc-protobuf-lite
-org.checkerframework:checker-qual
+org.jspecify:jspecify
org.playframework.twirl:twirl-api
org.scala-lang:scala-library
org.scala-lang:scala3-library
@@ -229,31 +229,6 @@ org.scala-lang.modules:scala-xml
----
-Checker Framework qualifiers
-Copyright 2004-present by the Checker Framework developers
-
-MIT License:
-
-Permission is hereby granted, free of charge, to any person obtaining a copy
-of this software and associated documentation files (the "Software"), to deal
-in the Software without restriction, including without limitation the rights
-to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
-copies of the Software, and to permit persons to whom the Software is
-furnished to do so, subject to the following conditions:
-
-The above copyright notice and this permission notice shall be included in
-all copies or substantial portions of the Software.
-
-THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
-IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
-FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
-AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
-LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
-OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
-THE SOFTWARE.
-
-----
-
com.google.protobuf:protobuf-java
Copyright 2008 Google Inc. All rights reserved.
diff --git a/project/AssemblyLicenseCheck.scala
b/project/AssemblyLicenseCheck.scala
new file mode 100644
index 00000000..092da9f6
--- /dev/null
+++ b/project/AssemblyLicenseCheck.scala
@@ -0,0 +1,142 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+import java.io.IOException
+import java.nio.charset.StandardCharsets
+
+import sbt._
+import sbt.io.Using
+import sbt.Keys._
+import sbtassembly.AssemblyKeys._
+import org.mdedetrich.apache.sonatype.ApacheSonatypePlugin.autoImport.{
+ apacheSonatypeLicenseFile,
+ apacheSonatypeNoticeFile
+}
+
+/**
+ * Guards the LICENSE and NOTICE files that ship in the assembly jars.
+ *
+ * The assembly jars bundle classes from 3rd party projects, so their
`META-INF/LICENSE` and
+ * `META-INF/NOTICE` have to account for those projects. Those files are
curated by hand in
+ * `legal/`, because most of the bundled jars carry no NOTICE of their own and
the text has to be
+ * taken from the upstream projects. These tasks stop the curated files from
drifting away from
+ * what the jars actually bundle.
+ */
+object AssemblyLicenseCheck {
+
+ val assemblyBundledModules =
+ taskKey[Seq[String]]("The `organization:name` ids of the 3rd party modules
bundled into the assembly jar")
+ val assemblyLicenseCheck =
+ taskKey[Unit]("Check that every module bundled into the assembly jar is
named in the assembly LICENSE file")
+ val assemblyMetaInfArchives =
+ taskKey[Seq[File]]("The published archives that should carry the assembly
LICENSE and NOTICE files")
+ val assemblyMetaInfCheck =
+ taskKey[Unit]("Check that the published archives ship the assembly LICENSE
and NOTICE files in META-INF")
+
+ /**
+ * Text that has to appear in the shipped files. Matching the curated file
byte for byte only says
+ * the merge strategy picked our copy; these markers say that copy is still
the file it should be,
+ * and not one that has been emptied or truncated.
+ */
+ private val requiredContent = Map(
+ "META-INF/LICENSE" -> Seq(
+ "Apache License",
+ "Version 2.0, January 2004",
+ "TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION",
+ // the 3rd party section this file exists for, see
legal/AssemblyLicense.txt
+ "contain classes from 3rd party projects",
+ "Apache License Version 2.0:"),
+ "META-INF/NOTICE" -> Seq(
+ "Apache Pekko gRPC",
+ "The Apache Software Foundation",
+ // the 3rd party section this file exists for, see
legal/AssemblyNotice.txt
+ "contain classes from 3rd party projects"))
+
+ /** A module id as listed in the assembly LICENSE file: `organization:name`,
optionally `:version`. */
+ private val ModuleId =
"""^([a-zA-Z0-9][\w.\-]*):([a-zA-Z0-9][\w.\-]*)(?::.*)?$""".r
+
+ lazy val settings: Seq[Setting[?]] = Seq(
+ assemblyBundledModules := {
+ val suffix = "_" + CrossVersion.binaryScalaVersion(scalaVersion.value)
+ val modules = (assembly /
fullClasspath).value.flatMap(_.get(moduleID.key))
+ modules
+ .filterNot(_.organization == "org.apache.pekko") // our own modules,
covered by the ASF header
+ .map { module =>
+ val name = if (module.name.endsWith(suffix))
module.name.dropRight(suffix.length) else module.name
+ s"${module.organization}:$name"
+ }
+ .distinct
+ .sorted
+ },
+ assemblyLicenseCheck := {
+ val log = streams.value.log
+ val licenseFile = apacheSonatypeLicenseFile.value
+ val listed = IO.readLines(licenseFile).map(_.trim).collect { case
ModuleId(org, module) => s"$org:$module" }.toSet
+ val bundled = assemblyBundledModules.value
+
+ val extra = (listed -- bundled).toSeq.sorted
+ if (extra.nonEmpty)
+ // the assembly jars are cross built, so a module can be listed for a
Scala version other than this one
+ log.warn(
+ s"${licenseFile.getName} lists modules that ${name.value} does not
bundle for Scala ${scalaVersion.value}, " +
+ s"check whether they are still needed: ${extra.mkString(", ")}")
+
+ val missing = bundled.filterNot(listed)
+ if (missing.nonEmpty)
+ sys.error(
+ s"${name.value} bundles modules that ${licenseFile.getName} does not
account for: ${missing.mkString(", ")}")
+
+ log.info(s"${licenseFile.getName} accounts for all ${bundled.size}
modules bundled by ${name.value}")
+ },
+ assemblyMetaInfArchives := Seq(assembly.value),
+ assemblyMetaInfCheck := {
+ val log = streams.value.log
+ val expected =
+ Seq("META-INF/LICENSE" -> apacheSonatypeLicenseFile.value,
"META-INF/NOTICE" -> apacheSonatypeNoticeFile.value)
+
+ assemblyMetaInfArchives.value.foreach { archive =>
+ val problems =
+ try Using.zipFile(archive) { zip =>
+ expected.flatMap { case (path, file) =>
+ Option(zip.getEntry(path)) match {
+ case None =>
+ Some(s"$path is missing, expected a copy of $file")
+ case Some(entry) =>
+ val actual = Using.zipEntry(zip)(entry)(_.readAllBytes())
+ if (!actual.sameElements(IO.readBytes(file))) Some(s"$path
does not match $file")
+ else {
+ val text = new String(actual, StandardCharsets.UTF_8)
+ val absent = requiredContent.getOrElse(path,
Nil).filterNot(text.contains)
+ if (absent.isEmpty) None
+ else Some(s"$path does not mention ${absent.map(marker
=> s"'$marker'").mkString(", ")}")
+ }
+ }
+ }
+ }
+ catch {
+ // the archive name is not part of the messages the zip classes
throw
+ case e: IOException => sys.error(s"${archive.getName} could not be
read as a zip archive: $e")
+ }
+
+ if (problems.nonEmpty)
+ sys.error(s"${archive.getName} has license problems:
${problems.mkString("; ")}")
+
+ log.info(s"${archive.getName} ships ${expected.map(_._1).mkString("
and ")}")
+ }
+ })
+
+}
diff --git a/project/Dependencies.scala b/project/Dependencies.scala
index d6f57793..999d0be8 100644
--- a/project/Dependencies.scala
+++ b/project/Dependencies.scala
@@ -110,6 +110,10 @@ object Dependencies {
Compile.grpcProtobuf,
Test.scalaTest)
+ lazy val scalapbProtocPlugin = l ++= Seq(
+ Compile.scalapbCompilerPlugin,
+ Protobuf.protobufJava) // or else scalapb pulls older version in
transitively
+
lazy val runtime = l ++= Seq(
Compile.scalapbRuntime,
Protobuf.protobufJava, // or else scalapb pulls older version in
transitively
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
