Script 'mail_helper' called by obssrc
Hello community,
here is the log from the commit of package obs-service-cargo_audit for
openSUSE:Factory checked in at 2021-07-05 22:22:51
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Comparing /work/SRC/openSUSE:Factory/obs-service-cargo_audit (Old)
and /work/SRC/openSUSE:Factory/.obs-service-cargo_audit.new.2625 (New)
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Package is "obs-service-cargo_audit"
Mon Jul 5 22:22:51 2021 rev:5 rq:904002 version:0.1.5~git0.01fea16
Changes:
--------
---
/work/SRC/openSUSE:Factory/obs-service-cargo_audit/obs-service-cargo_audit.changes
2021-06-09 21:53:15.322576691 +0200
+++
/work/SRC/openSUSE:Factory/.obs-service-cargo_audit.new.2625/obs-service-cargo_audit.changes
2021-07-05 22:23:17.393578056 +0200
@@ -1,0 +2,6 @@
+Mon Jul 05 03:33:51 UTC 2021 - wbr...@suse.de
+
+- Update to version 0.1.5~git0.01fea16:
+ * Allows non-scm packages in obs to be scanned for vulnerabilities
+
+-------------------------------------------------------------------
Old:
----
obs-service-cargo_audit-0.1.4~git0.a3c7623.tar.gz
New:
----
obs-service-cargo_audit-0.1.5~git0.01fea16.tar.gz
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Other differences:
------------------
++++++ obs-service-cargo_audit.spec ++++++
--- /var/tmp/diff_new_pack.UWv1j1/_old 2021-07-05 22:23:17.905574094 +0200
+++ /var/tmp/diff_new_pack.UWv1j1/_new 2021-07-05 22:23:17.905574094 +0200
@@ -1,5 +1,5 @@
#
-# spec file for package obs-service-cargo_audit
+# spec file
#
# Copyright (c) 2021 SUSE LLC
#
@@ -22,7 +22,7 @@
License: MPL-2.0
Group: Development/Tools/Building
URL: https://github.com/openSUSE/obs-service-%{service}
-Version: 0.1.4~git0.a3c7623
+Version: 0.1.5~git0.01fea16
Release: 0
Source: %{name}-%{version}.tar.gz
BuildRequires: python3
++++++ _service ++++++
--- /var/tmp/diff_new_pack.UWv1j1/_old 2021-07-05 22:23:17.953573723 +0200
+++ /var/tmp/diff_new_pack.UWv1j1/_new 2021-07-05 22:23:17.953573723 +0200
@@ -3,7 +3,7 @@
<param
name="url">https://github.com/openSUSE/obs-service-cargo_audit.git</param>
<param name="versionformat">@PARENT_TAG@~git@TAG_OFFSET@.%h</param>
<param name="scm">git</param>
- <param name="revision">v0.1.4</param>
+ <param name="revision">v0.1.5</param>
<param name="match-tag">v*</param>
<param name="versionrewrite-pattern">v(\d+\.\d+\.\d+)</param>
<param name="versionrewrite-replacement">\1</param>
++++++ obs-service-cargo_audit-0.1.4~git0.a3c7623.tar.gz ->
obs-service-cargo_audit-0.1.5~git0.01fea16.tar.gz ++++++
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore'
old/obs-service-cargo_audit-0.1.4~git0.a3c7623/.gitignore
new/obs-service-cargo_audit-0.1.5~git0.01fea16/.gitignore
--- old/obs-service-cargo_audit-0.1.4~git0.a3c7623/.gitignore 1970-01-01
01:00:00.000000000 +0100
+++ new/obs-service-cargo_audit-0.1.5~git0.01fea16/.gitignore 2021-07-05
05:26:03.000000000 +0200
@@ -0,0 +1,5 @@
+.DS_Store
+*.pyc
+*.patch
+openSUSE:Factory/
+target/
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/obs-service-cargo_audit-0.1.4~git0.a3c7623/README.md
new/obs-service-cargo_audit-0.1.5~git0.01fea16/README.md
--- old/obs-service-cargo_audit-0.1.4~git0.a3c7623/README.md 2021-06-08
05:53:44.000000000 +0200
+++ new/obs-service-cargo_audit-0.1.5~git0.01fea16/README.md 2021-07-05
05:26:03.000000000 +0200
@@ -6,7 +6,8 @@
a Rust application have known security vulnerabilities. If vulnerabilities are
found, the source
service will alert allowing you to update and help upstream update their
sources.
-This relies on having unpacked sources available, so you should use the SCM
module.
+It is prefered that you use an SCM service for srcdir to exist, but
alternately this can
+unpack your sources for scanning.
## Usage for packagers (with SCM service)
@@ -29,40 +30,78 @@
$ osc service ra
```
-## Manual (without SCM service)
+## Usage for packagers (with OUT SCM service)
-If you are not using SCM, you can use the cargo audit service manually.
+1. Add to the `_service` file this snippet:
+
+```
+<services>
+ <service name="cargo_audit" mode="disabled">
+ </service>
+</services>
+```
+
+2. Run `osc` command locally:
+
+```
+$ osc service ra
+```
+
+## Manual (without obs services configured)
-1. Extract your source archive into your working directory.
+1. Run `osc` command locally in your checked out obs package.
```
-$ tar -xv archive.tar.xz
+$ osc service lr cargo_audit
```
-2. Examine the folder name it extracted, IE archive-v1.0.0
+## To use a specific lockfile
-3. Set srcdir to match
+Some projects ship multiple Cargo.lock files, in some cases with intentionally
vulnerable content.
+Rustsec does this for example to test that they can find these vulnerabilities.
+
+In these cases you may wish to set a lockfile to scan. This is relative to
your unpacked sources
+or srcdir.
+
+### SCM
```
<services>
+ <service name="obs_scm" mode="disabled">
+ ...
+ </service>
<service name="cargo_audit" mode="disabled">
- <param name="srcdir">archive-v1.0.0</param>
+ <param name="srcdir">projectname</param>
+ <param name="lockfile">Cargo.lock</param>
</service>
</services>
```
-4. Run `osc` command locally:
+### Source Unpacking
```
-$ osc service ra
+<services>
+ <service name="cargo_audit" mode="disabled">
+ <param name="lockfile">projectname-0.7.2/Cargo.lock</param>
+ </service>
+</services>
```
-## Testinc
+## Testing
To test this module directly by hand:
```
-python3 path/to/cargo_audit --srcdir /path/to/unpacked/sources
+python3 path/to/cargo_audit
+python3 path/to/cargo_audit --srcdir=/path/to/unpacked/sources
+python3 path/to/cargo_audit --srcdir=/path/to/unpacked/sources
--lockfile=path/to/Cargo.lock
+python3 path/to/cargo_audit --lockfile=path/to/Cargo.lock
+```
+
+To get further debugging info, run with:
+
+```
+DEBUG=1 python3 path/to/cargo_audit
```
## License
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore'
old/obs-service-cargo_audit-0.1.4~git0.a3c7623/cargo_audit
new/obs-service-cargo_audit-0.1.5~git0.01fea16/cargo_audit
--- old/obs-service-cargo_audit-0.1.4~git0.a3c7623/cargo_audit 2021-06-08
05:53:44.000000000 +0200
+++ new/obs-service-cargo_audit-0.1.5~git0.01fea16/cargo_audit 2021-07-05
05:26:03.000000000 +0200
@@ -22,6 +22,8 @@
See README.md for additional documentation.
"""
+import tarfile
+import tempfile
import logging
import argparse
import os
@@ -35,16 +37,20 @@
service_name = "obs-service-cargo_audit"
description = __doc__
-logging.basicConfig(level=logging.INFO)
+if os.getenv('DEBUG') is not None:
+ logging.basicConfig(level=logging.DEBUG)
+else:
+ logging.basicConfig(level=logging.INFO)
+
log = logging.getLogger(service_name)
parser = argparse.ArgumentParser(
description=description,
formatter_class=argparse.RawDescriptionHelpFormatter
)
-parser.add_argument("--srcdir", default=os.getcwd())
+parser.add_argument("--srcdir", default=None)
parser.add_argument("--lockfile", default=None)
-# We always ignore this parameter.
-parser.add_argument("--outdir")
+# We always ignore this parameter, but it has to exist.
+parser.add_argument("--outdir", default=None)
args = parser.parse_args()
srcdir = args.srcdir
@@ -94,43 +100,70 @@
# Issue may have been found!
vuln_count = details["vulnerabilities"]["count"]
if vuln_count > 0:
- log.error(f" possible vulnerabilties: {vuln_count}")
+ log.error(f" ???? possible vulnerabilties: {vuln_count}")
+ log.error(f" {lock_file}")
+ log.error(f" For more information you SHOULD inspect the output of
cargo audit manually")
vulns = details["vulnerabilities"]["list"]
for vuln in vulns:
affects = vuln["advisory"]["package"]
cvss = vuln["advisory"]["cvss"]
vid = vuln["advisory"]["id"]
categories = vuln["advisory"]["categories"]
- log.error(f" ???? {vid} -> crate: {affects}, cvss: {cvss},
class: {categories}")
- log.error(f" For more information you SHOULD inspect the output of
cargo-audit manually for {lock_file}.")
+ log.error(f" * {vid} -> crate: {affects}, cvss: {cvss}, class:
{categories}")
return True
- log.info(f" ??? No known issues detected in {lock_file}")
+ log.info(f" ??? No known issues detected in {lock_file}")
return False
+def do_extract(tgt_dir):
+ cwd = os.getcwd()
+ content = os.listdir(cwd)
+ maybe_src = [
+ x for x in content
+ if '.tar' in x and 'vendor' not in x and not x.endswith('.asc')
+ ]
+ for src in maybe_src:
+ log.debug(f"Unpacking assumed source tar {src} to {tgt_dir}")
+ with tarfile.open(f"{cwd}/{src}", "r:*") as tar:
+ tar.extractall(path=tgt_dir)
+
def main():
log.info(f" Running OBS Source Service ????: {service_name}")
log.debug(f" Current working dir: {os.getcwd()}")
- cargo_lock_paths = []
- if lockfile:
- cargo_lock_paths = [lockfile]
- log.info(f" _service configured lock file: {lockfile}")
- else:
- log.info(f" Searching for Cargo.lock in: {srcdir}")
- cargo_lock_paths = find_file(srcdir, "Cargo.lock")
-
- if not cargo_lock_paths:
- log.info(f" No Rust Cargo.lock found under {srcdir}")
- log.info(f" Searching for Cargo.toml in: {srcdir}")
- if find_file(srcdir, "Cargo.toml"):
- generate_lock(srcdir)
- else:
- log.error(f" No Rust Cargo.toml found under {srcdir}")
- exit(1)
+ lsrcdir = srcdir
+ status = False
+ # Setup our temp dir, in case we need it.
+ with tempfile.TemporaryDirectory() as tmpdirname:
+ if lsrcdir is None:
+ # We likely need to unpack sources.
+ log.debug(f" setting up sources into {tmpdirname}")
+ lsrcdir = tmpdirname
+ do_extract(tmpdirname)
+
+ cargo_lock_paths = []
+ if lockfile:
+ log.info(f" _service configured lock file: {lockfile}")
+ cargo_lock_paths = [os.path.join(lsrcdir, lockfile)]
+ log.debug(f" {cargo_lock_paths}")
else:
- log.debug(f" Detected Rust lock files: {cargo_lock_paths}")
+ log.debug(f" Searching for Cargo.lock in: {lsrcdir}")
+ cargo_lock_paths = find_file(lsrcdir, "Cargo.lock")
+
+ if not cargo_lock_paths:
+ log.info(f" No Rust Cargo.lock found under {lsrcdir}")
+ log.info(f" Searching for Cargo.toml in: {lsrcdir}")
+ if find_file(lsrcdir, "Cargo.toml"):
+ generate_lock(lsrcdir)
+ else:
+ log.error(f" No Rust Cargo.toml found under {lsrcdir}")
+ exit(1)
+ else:
+ log.debug(f" Detected Rust lock files: {cargo_lock_paths}")
+
+ # Do the scan.
+ status = any([cargo_audit(cargo_lock_path) for cargo_lock_path in
cargo_lock_paths])
- status = any([cargo_audit(cargo_lock_path) for cargo_lock_path in
cargo_lock_paths])
+ # We exit the context manager, so tempdirname is now removed.
if status:
log.error(" ?????? Vulnerabilities may have been found. You must
review these.")
exit(1)
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore'
old/obs-service-cargo_audit-0.1.4~git0.a3c7623/do_scan.py
new/obs-service-cargo_audit-0.1.5~git0.01fea16/do_scan.py
--- old/obs-service-cargo_audit-0.1.4~git0.a3c7623/do_scan.py 2021-06-08
05:53:44.000000000 +0200
+++ new/obs-service-cargo_audit-0.1.5~git0.01fea16/do_scan.py 2021-07-05
05:26:03.000000000 +0200
@@ -66,18 +66,7 @@
print(e.stdout)
def do_unpack_scan(pkgname):
- tgt_dir = f"openSUSE:Factory/{pkgname}"
- # List everything in the folder.
- content = os.listdir(tgt_dir)
- # Find anything that contains .tar
- # Exclude vendor.tar.*
- maybe_src = [x for x in content if '.tar' in x and 'vendor' not in x]
- # Attempt to unpack it into the directory.
- for src in maybe_src:
- print(f"Unpacking assumed source tar {src} to {tgt_dir}")
- with tarfile.open(f"{tgt_dir}/{src}", "r:*") as tar:
- tar.extractall(path=tgt_dir)
- # Now do osc service lr cargo_audit to run manually.
+ # This will automatically do the unpack for use due to how cargo_audit as
a service works :)
try:
out = subprocess.check_output(["osc", "service", "lr", "cargo_audit"],
cwd=f"openSUSE:Factory/{pkgname}", encoding='UTF-8', stderr=subprocess.STDOUT)
print(f"??? -- passed")
@@ -85,11 +74,31 @@
print(f"???? -- services failed")
print(e.stdout)
+# def do_unpack_scan_old(pkgname):
+# tgt_dir = f"openSUSE:Factory/{pkgname}"
+# # List everything in the folder.
+# content = os.listdir(tgt_dir)
+# # Find anything that contains .tar
+# # Exclude vendor.tar.*
+# maybe_src = [x for x in content if '.tar' in x and 'vendor' not in x and
not x.endswith('.asc')]
+# # Attempt to unpack it into the directory.
+# for src in maybe_src:
+# print(f"Unpacking assumed source tar {src} to {tgt_dir}")
+# with tarfile.open(f"{tgt_dir}/{src}", "r:*") as tar:
+# tar.extractall(path=tgt_dir)
+# # Now do osc service lr cargo_audit to run manually.
+# try:
+# out = subprocess.check_output(["osc", "service", "lr",
"cargo_audit"], cwd=f"openSUSE:Factory/{pkgname}", encoding='UTF-8',
stderr=subprocess.STDOUT)
+# print(f"??? -- passed")
+# except subprocess.CalledProcessError as e:
+# print(f"???? -- services failed")
+# print(e.stdout)
+
if __name__ == '__main__':
depends = list_whatdepends()
# For testing, we hardcode the list for dev.
- depends = ['kanidm', 'librsvg', 'rust-cbindgen']
+ # depends = ['kanidm', 'librsvg', 'rust-cbindgen']
# Check them out, or update if they exist.
auditable_depends = []
@@ -97,11 +106,11 @@
for pkgname in depends:
print("---")
checkout_or_update(pkgname)
- # do they have cargo_audit as a service?
+ # do they have cargo_audit as a service? Could we consider adding it?
has_audit = does_have_cargo_audit(pkgname)
if not has_audit:
print(f"??????
https://build.opensuse.org/package/show/openSUSE:Factory/{pkgname} missing
cargo_audit service")
- print(f"??????
https://build.opensuse.org/package/users/openSUSE:Factory/{pkgname}";)
+ # print(f"??????
https://build.opensuse.org/package/users/openSUSE:Factory/{pkgname}";)
# If not, we should contact the developers to add this. We can
attempt to unpack
# and run a scan still though.
unpack_depends.append(pkgname)
