Re: [OE-core] [PATCH 2/2] package.bbclass: Emit HOMEPAGE pkgdata.
Hi Richard, Thanks for your comments. Sorry I missed the questions when I previously sent it. Resent due to the fact I missed the "Signed off by" in the patch last time around. I agree with you in general about the scope creep, however the license.bbclass code is heavily coupled with this class, as it just greps the runtime dir for all the 'lic_data' it deems appropriate. All that data appears to be emitted here. Might be nice if there was another bbclass which actually just emitted the 'data for licenses' rather than the 'lic_data' being a subset of this package data. I'd be happy to propose that class for review. However, just looking at this function it already seems we've been "heading down a path" here for a while anyway? The content of one extra variable doesn't seem like the tipping point to me. ________ From: Richard Purdie Sent: 30 May 2019 14:18:36 To: Rob Walton; openembedded-core@lists.openembedded.org Subject: Re: [OE-core] [PATCH 2/2] package.bbclass: Emit HOMEPAGE pkgdata. On Wed, 2019-05-22 at 12:44 +, Rob Walton wrote: > Emit the contents of HOMEPAGE in the extra pkgdata. This is useful > information to have when creating license manifests. > > Signed-off-by: Rob Walton > --- > meta/classes/package.bbclass | 1 + > 1 file changed, 1 insertion(+) > > diff --git a/meta/classes/package.bbclass > b/meta/classes/package.bbclass > index 4c0a859..a08aefe 100644 > --- a/meta/classes/package.bbclass > +++ b/meta/classes/package.bbclass > @@ -1458,6 +1458,7 @@ fi > write_if_exists(sf, pkg, 'PKGR') > write_if_exists(sf, pkg, 'LICENSE') > write_if_exists(sf, pkg, 'DESCRIPTION') > +write_if_exists(sf, pkg, 'HOMEPAGE') > write_if_exists(sf, pkg, 'SUMMARY') > write_if_exists(sf, pkg, 'RDEPENDS') > rprov = write_if_exists(sf, pkg, 'RPROVIDES') This isn't the first time this has been requested, back in 14/11/2017 a similar patch had the reply: "This was already proposed once and rejected as its using the interface for something it really wasn't designed for" You've also sent this patch once already and questions were raised then. I have to admit I don't like the license.bbclass code, its starting to look like anything every legal department could ask for, meaning it does no one thing well, just many things badly. I don't like that mess starting to creep into the packaging code which does have a clearly defined purpose and use. Put another way, which variables do we draw the line at? I worry about the path accepting this patch leads us down :( Cheers, Richard IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you. -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] [PATCH 1/2] image_license.bbclass: Write JSON files containing license data.
Requesting review. Thanks.
From: Rob Walton
Sent: 22 May 2019 13:44:33
To: openembedded-core@lists.openembedded.org
Cc: Rob Walton
Subject: [PATCH 1/2] image_license.bbclass: Write JSON files containing license
data.
For easier machine readability of license manifests, write a
manifest.json containing license data and package info.
Signed-off-by: Rob Walton
---
meta/classes/license_image.bbclass | 23 ++-
1 file changed, 22 insertions(+), 1 deletion(-)
diff --git a/meta/classes/license_image.bbclass
b/meta/classes/license_image.bbclass
index 6750038..6952417 100644
--- a/meta/classes/license_image.bbclass
+++ b/meta/classes/license_image.bbclass
@@ -36,6 +36,7 @@ python license_create_manifest() {
}
def write_license_files(d, license_manifest, pkg_dic, rootfs=True):
+import json
import re
import stat
@@ -43,6 +44,7 @@ def write_license_files(d, license_manifest, pkg_dic,
rootfs=True):
bad_licenses = map(lambda l: canonical_license(d, l), bad_licenses)
bad_licenses = expand_wildcard_licenses(d, bad_licenses)
+json_data = dict()
with open(license_manifest, "w") as license_file:
for pkg in sorted(pkg_dic):
if bad_licenses:
@@ -63,7 +65,14 @@ def write_license_files(d, license_manifest, pkg_dic,
rootfs=True):
license_file.write("PACKAGE VERSION: %s\n" %
pkg_dic[pkg]["PV"])
license_file.write("RECIPE NAME: %s\n" % pkg_dic[pkg]["PN"])
license_file.write("LICENSE: %s\n\n" % pkg_dic[pkg]["LICENSE"])
-
+json_data[pkg] = {
+"RECIPE NAME": pkg_dic[pkg]["PN"],
+"PACKAGE VERSION": pkg_dic[pkg]["PV"],
+"PACKAGE NAME": pkg,
+"LICENSE": pkg_dic[pkg]["LICENSE"],
+"HOMEPAGE": pkg_dic[pkg].get("HOMEPAGE", ""),
+"SUMMARY": pkg_dic[pkg].get("SUMMARY", ""),
+}
# If the package doesn't contain any file, that is, its size
is 0, the license
# isn't relevant as far as the final image is concerned. So
doing license check
# doesn't make much sense, skip it.
@@ -75,6 +84,14 @@ def write_license_files(d, license_manifest, pkg_dic,
rootfs=True):
license_file.write("VERSION: %s\n" % pkg_dic[pkg]["PV"])
license_file.write("LICENSE: %s\n" % pkg_dic[pkg]["LICENSE"])
license_file.write("FILES: %s\n\n" % pkg_dic[pkg]["FILES"])
+json_data[pkg] = {
+"RECIPE NAME": pkg_dic[pkg]["PN"],
+"VERSION": pkg_dic[pkg]["PV"],
+"FILES": pkg_dic[pkg]["FILES"],
+"LICENSE": pkg_dic[pkg]["LICENSE"],
+"HOMEPAGE": pkg_dic[pkg].get("HOMEPAGE", ""),
+"SUMMARY": pkg_dic[pkg].get("SUMMARY", ""),
+}
for lic in pkg_dic[pkg]["LICENSES"]:
lic_file = os.path.join(d.getVar('LICENSE_DIRECTORY'),
@@ -89,6 +106,10 @@ def write_license_files(d, license_manifest, pkg_dic,
rootfs=True):
"licenses collected for recipe %s"
% (lic, pkg_dic[pkg]["PN"]))
+# Write a json file containing the license data.
+with open("{}.json".format(license_manifest), "w") as license_file_json:
+license_file_json.write(json.dumps(json_data))
+
# Two options here:
# - Just copy the manifest
# - Copy the manifest and the license directories
--
2.7.4
IMPORTANT NOTICE: The contents of this email and any attachments are
confidential and may also be privileged. If you are not the intended recipient,
please notify the sender immediately and do not disclose the contents to any
other person, use it for any purpose, or store or copy the information in any
medium. Thank you.
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] [PATCH 2/2] package.bbclass: Emit HOMEPAGE pkgdata.
Requesting review. Thanks. From: Rob Walton Sent: 22 May 2019 13:44:37 To: openembedded-core@lists.openembedded.org Cc: Rob Walton Subject: [PATCH 2/2] package.bbclass: Emit HOMEPAGE pkgdata. Emit the contents of HOMEPAGE in the extra pkgdata. This is useful information to have when creating license manifests. Signed-off-by: Rob Walton --- meta/classes/package.bbclass | 1 + 1 file changed, 1 insertion(+) diff --git a/meta/classes/package.bbclass b/meta/classes/package.bbclass index 4c0a859..a08aefe 100644 --- a/meta/classes/package.bbclass +++ b/meta/classes/package.bbclass @@ -1458,6 +1458,7 @@ fi write_if_exists(sf, pkg, 'PKGR') write_if_exists(sf, pkg, 'LICENSE') write_if_exists(sf, pkg, 'DESCRIPTION') +write_if_exists(sf, pkg, 'HOMEPAGE') write_if_exists(sf, pkg, 'SUMMARY') write_if_exists(sf, pkg, 'RDEPENDS') rprov = write_if_exists(sf, pkg, 'RPROVIDES') -- 2.7.4 IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you. -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH 2/2] package.bbclass: Emit HOMEPAGE pkgdata.
Emit the contents of HOMEPAGE in the extra pkgdata. This is useful information to have when creating license manifests. Signed-off-by: Rob Walton --- meta/classes/package.bbclass | 1 + 1 file changed, 1 insertion(+) diff --git a/meta/classes/package.bbclass b/meta/classes/package.bbclass index 4c0a859..a08aefe 100644 --- a/meta/classes/package.bbclass +++ b/meta/classes/package.bbclass @@ -1458,6 +1458,7 @@ fi write_if_exists(sf, pkg, 'PKGR') write_if_exists(sf, pkg, 'LICENSE') write_if_exists(sf, pkg, 'DESCRIPTION') +write_if_exists(sf, pkg, 'HOMEPAGE') write_if_exists(sf, pkg, 'SUMMARY') write_if_exists(sf, pkg, 'RDEPENDS') rprov = write_if_exists(sf, pkg, 'RPROVIDES') -- 2.7.4 IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you. -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH 1/2] image_license.bbclass: Write JSON files containing license data.
For easier machine readability of license manifests, write a
manifest.json containing license data and package info.
Signed-off-by: Rob Walton
---
meta/classes/license_image.bbclass | 23 ++-
1 file changed, 22 insertions(+), 1 deletion(-)
diff --git a/meta/classes/license_image.bbclass
b/meta/classes/license_image.bbclass
index 6750038..6952417 100644
--- a/meta/classes/license_image.bbclass
+++ b/meta/classes/license_image.bbclass
@@ -36,6 +36,7 @@ python license_create_manifest() {
}
def write_license_files(d, license_manifest, pkg_dic, rootfs=True):
+import json
import re
import stat
@@ -43,6 +44,7 @@ def write_license_files(d, license_manifest, pkg_dic,
rootfs=True):
bad_licenses = map(lambda l: canonical_license(d, l), bad_licenses)
bad_licenses = expand_wildcard_licenses(d, bad_licenses)
+json_data = dict()
with open(license_manifest, "w") as license_file:
for pkg in sorted(pkg_dic):
if bad_licenses:
@@ -63,7 +65,14 @@ def write_license_files(d, license_manifest, pkg_dic,
rootfs=True):
license_file.write("PACKAGE VERSION: %s\n" %
pkg_dic[pkg]["PV"])
license_file.write("RECIPE NAME: %s\n" % pkg_dic[pkg]["PN"])
license_file.write("LICENSE: %s\n\n" % pkg_dic[pkg]["LICENSE"])
-
+json_data[pkg] = {
+"RECIPE NAME": pkg_dic[pkg]["PN"],
+"PACKAGE VERSION": pkg_dic[pkg]["PV"],
+"PACKAGE NAME": pkg,
+"LICENSE": pkg_dic[pkg]["LICENSE"],
+"HOMEPAGE": pkg_dic[pkg].get("HOMEPAGE", ""),
+"SUMMARY": pkg_dic[pkg].get("SUMMARY", ""),
+}
# If the package doesn't contain any file, that is, its size
is 0, the license
# isn't relevant as far as the final image is concerned. So
doing license check
# doesn't make much sense, skip it.
@@ -75,6 +84,14 @@ def write_license_files(d, license_manifest, pkg_dic,
rootfs=True):
license_file.write("VERSION: %s\n" % pkg_dic[pkg]["PV"])
license_file.write("LICENSE: %s\n" % pkg_dic[pkg]["LICENSE"])
license_file.write("FILES: %s\n\n" % pkg_dic[pkg]["FILES"])
+json_data[pkg] = {
+"RECIPE NAME": pkg_dic[pkg]["PN"],
+"VERSION": pkg_dic[pkg]["PV"],
+"FILES": pkg_dic[pkg]["FILES"],
+"LICENSE": pkg_dic[pkg]["LICENSE"],
+"HOMEPAGE": pkg_dic[pkg].get("HOMEPAGE", ""),
+"SUMMARY": pkg_dic[pkg].get("SUMMARY", ""),
+}
for lic in pkg_dic[pkg]["LICENSES"]:
lic_file = os.path.join(d.getVar('LICENSE_DIRECTORY'),
@@ -89,6 +106,10 @@ def write_license_files(d, license_manifest, pkg_dic,
rootfs=True):
"licenses collected for recipe %s"
% (lic, pkg_dic[pkg]["PN"]))
+# Write a json file containing the license data.
+with open("{}.json".format(license_manifest), "w") as license_file_json:
+license_file_json.write(json.dumps(json_data))
+
# Two options here:
# - Just copy the manifest
# - Copy the manifest and the license directories
--
2.7.4
IMPORTANT NOTICE: The contents of this email and any attachments are
confidential and may also be privileged. If you are not the intended recipient,
please notify the sender immediately and do not disclose the contents to any
other person, use it for any purpose, or store or copy the information in any
medium. Thank you.
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH 2/2] package.bbclass: Emit HOMEPAGE pkgdata.
Emit the contents of HOMEPAGE in the extra pkgdata. This is useful information to have when creating license manifests. --- meta/classes/package.bbclass | 1 + 1 file changed, 1 insertion(+) diff --git a/meta/classes/package.bbclass b/meta/classes/package.bbclass index 4c0a859..a08aefe 100644 --- a/meta/classes/package.bbclass +++ b/meta/classes/package.bbclass @@ -1458,6 +1458,7 @@ fi write_if_exists(sf, pkg, 'PKGR') write_if_exists(sf, pkg, 'LICENSE') write_if_exists(sf, pkg, 'DESCRIPTION') +write_if_exists(sf, pkg, 'HOMEPAGE') write_if_exists(sf, pkg, 'SUMMARY') write_if_exists(sf, pkg, 'RDEPENDS') rprov = write_if_exists(sf, pkg, 'RPROVIDES') -- 2.7.4 IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you. -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] automated license reporting
Hi,
We're working with OpenEmbedded/yocto in a project; we're trying to create an
HTML license report for our users, leveraging the license.manifest files output
from license_image.bbclass.
However some relatively crucial information relevant to the report isn't
provided in the manifests. The extra information I'm interested in is the
contents of the HOMEPAGE and SUMMARY variables for each package/blob recorded
by the manifest.
I'm proposing to submit a patch to add the following to `write_license_files`:
-license_file.write("LICENSE: %s\n\n" % pkg_dic[pkg]["LICENSE"])
-
+license_file.write("LICENSE: %s\n" % pkg_dic[pkg]["LICENSE"])
+license_file.write("SUMMARY: %s\n" %
pkg_dic[pkg].get("SUMMARY", ""))
+license_file.write("HOMEPAGE: %s\n\n" %
pkg_dic[pkg].get("HOMEPAGE", ""))
At the moment it isn't a problem for us if this extra information is missing
for /some/ packages, as long as it is included at least once per recipe (which
is the case).
The above patch would rely on another patch to add extra output in
package.bbclass::emit_pkg_data
+write_if_exists(sf, pkg, 'HOMEPAGE')
We also have a bbclass to convert the license.manifest(s) to JSON format for
easier machine readability, so we can automate this process in our build
system. This bbclass is also potentially upstreamable.
My questions are: Would members of the community object to these patches? Are
there any other suggestions for grabbing this extra info without submitting the
patches? Would anyone else in the community find this useful?
Thanks,
Rob
IMPORTANT NOTICE: The contents of this email and any attachments are
confidential and may also be privileged. If you are not the intended recipient,
please notify the sender immediately and do not disclose the contents to any
other person, use it for any purpose, or store or copy the information in any
medium. Thank you.
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
