Hi there,
I've got a set of changes here for code review, if anyone has time:
https://cr.opensolaris.org/action/browse/pkg/timf/pkgrepo-verify/pkgrepo-verify-webrev/
This adds two new subcommands to pkgrepo(1M): 'verify' and 'fix', both
designed for file-based repositories only.
I've attached some output showing 'verify' and 'fix' running on a
particularly gruesome repository, and some output from 'verify' showing
the warning we print if parts of the repository aren't readable by
'pkg5srv' (a common case in the wild, causing grief for zones
administrators, since the system-repository and pkg.depot both run as
'pkg5srv')
The quarantine area gets created as needed, and contains subdirectories
per-pkgrepo session. I haven't added any support for listing the
quarantined contents, deciding that a user who has just run 'pkgrepo
fix' already gets a lot of information as to how to rebuild their
repository from that point onwards.
I'll inline the pkgrepo.1 additions here. They're in the webrev too, but
harder to read (I haven't filed a docs bug for them yet)
Comments welcome,
cheers,
tim
----
pkgrepo verify [ -p publisher ... ] -s repo_uri_or_path
Verify the contents of a repository checking the follow-
ing:
o file checksums
o file permissions
o package manifest permissions
o package manifest content
o package signatures
Errors are emitted to stdout and pkgrepo exits with a
non-zero return code if any errors are emitted. The
path leading to the repository is also checked to ensure
that the 'pkg5srv' user can read the repository con-
tents, which can be necessary for repositories that are
made available over svc:/application/pkg/server or
svc:/application/pkg/system-repository (when the system
has non-global zones)
Package manifest signatures are calculated based on the
repository/signature-required-names, repository/trust-
anchor-directory and repository/check-certificate-
revocation properties.
This subcommand can only be used with file system repo-
sitories that use version 4.
-p publisher
Perform the operation only for the given publisher.
If not provided, or if the special value all is
specified, the operation is performed for all pub-
lishers. This option can be specified multiple
times.
-s repo_uri_or_path
Operate on the repository located at the given URI
or file system path.
pkgrepo fix [ -p publisher ... ] [ -v ] -s repo_uri_or_path
Fix the contents of a repository by first verifying the
repository, then moving any invalid repository contents
into a quarantine directory within the repository.
If repository errors are found, a repository rebuild is
automatically performed. If any errors were found, a
message is emitted to stdout showing which packages must
be re-imported using pkgsend(1) or pkgrecv(1) in order
to restore the repository contents.
This subcommand can only be used with file system repo-
sitories that use version 4.
-p publisher
Perform the operation only for the given publisher.
If not provided, or if the special value all is
specified, the operation is performed for all
publishers. This option can be specified multiple
times.
-v
Report information about the errors found.
-s repo_uri_or_path
Operate on the repository located at the given URI
or file system path.
$ pkgrepo fix -s /space/broken\-repo/
Initiating repository fix.
PHASE ITEMS
Fixing repository content 6/6
PHASE ITEMS
Fixing repository content 7/7
Use pkgsend(1) or pkgrecv(1) to republish the
following packages which were quarantined:
/space/broken-repo/publisher/test/pkg/sample/2.0%2C5.11%3A20120302T021808Z
pkg://test/[email protected],5.11:20120302T021808Z
/space/broken-repo/publisher/test/pkg/sample/@@@@
pkg://solaris/driver/graphics/[email protected],5.11-0.175.1.0.0.14.0:20120416T143307Z
/space/broken-repo/publisher/solaris/pkg/driver%2Fgraphics%2Fnvidia/foobar
pkg://solaris/developer/[email protected],5.11-0.175.1.0.0.15.799:20120501T164946Z
pkg://solaris/driver/network/[email protected],5.11-0.173.0.0.0.1.0:20110826T152405Z
/space/broken-repo/publisher/solaris/pkg/driver%2Fgraphics%2Fnvidia/0.295.20.0%2C5.11-0.175.1.0.0.14.0%3A20120416T143309Z
Repository fix completed.
$
$ pkgrepo verify -s /space/broken\-repo/
Initiating repository verify.
pkg://pkg5-repo-verify/unknown-package 2/4 /
ERROR: Bad manifest found.
Error: Illegal FMRI 'sample@@@@@': Version cannot be empty
Repository path:
/space/broken-repo/publisher/test/pkg/sample/@@@@
pkg://test/sample 3/4 /
ERROR: Invalid file hash: 98f2c1c68fbc1d1f0c14664b32708fa28079e966
Package: pkg://test/[email protected],5.11:20120302T021808Z
Repository path:
/space/broken-repo/publisher/test/file/f7/f769b09ae92edf86dc54bfadb7131ab41d91a4dc.
Path: space/validgz
ERROR: Corrupted gzip file
Package: pkg://test/[email protected],5.11:20120302T021808Z
Repository path:
/space/broken-repo/publisher/test/file/31/31c02215f3b6e336609f9868dca1104ac2713b6b
Path: space/largefile.dat
ERROR: Missing file: 31c02215f3b6e336609f9868dca1104ac2713b6c
Package: pkg://test/[email protected],5.11:20120302T021808Z
Repository path:
/space/broken-repo/publisher/test/file/31/31c02215f3b6e336609f9868dca1104ac2713b6c
Path: space/broken.dat
pkg://pkg5-repo-verify/unknown-package 4/4 /
ERROR: Corrupt manifest found.
Use pkglint(1) for more details.
Repository path:
/space/broken-repo/publisher/test/pkg/sample/2.0%2C5.11%3A20120302T021808Z
pkg://solaris/developer/meld 18/248 |
ERROR: Invalid file hash: f3faa9b646db8c26da0eda44b627f632f6e99de1
Package:
pkg://solaris/developer/[email protected],5.11-0.175.1.0.0.15.799:20120501T164946Z
Repository path:
/space/broken-repo/publisher/solaris/file/f3/f3c446fce9eb1920b94a8c6280fe9109b6543e85.
Path: usr/lib/meld/meld/merge.py
pkg://solaris/driver/graphics/nvidia 51/248 -
ERROR: Corrupted gzip file
Package:
pkg://solaris/driver/graphics/[email protected],5.11-0.175.1.0.0.14.0:20120416T143307Z
Repository path:
/space/broken-repo/publisher/solaris/file/f3/f3d0b7b8e5f3c4cde70d387c4bba717ceff40b01
Path: usr/share/man/man1/nvidia-xconfig.1
pkg://pkg5-repo-verify/unknown-package 52/248 -
ERROR: Bad manifest found.
Error: Illegal FMRI 'driver/graphics/nvidia@foobar': Bad
Version: foobar
Repository path:
/space/broken-repo/publisher/solaris/pkg/driver%2Fgraphics%2Fnvidia/foobar
pkg://pkg5-repo-verify/unknown-package 53/248 -
ERROR: Corrupt manifest found.
Use pkglint(1) for more details.
Repository path:
/space/broken-repo/publisher/solaris/pkg/driver%2Fgraphics%2Fnvidia/0.295.20.0%2C5.11-0.175.1.0.0.14.0%3A20120416T143309Z
pkg://solaris/driver/network/iwi 72/248 \
ERROR: Bad signature found: The signature with this signature value:
2aae07e33fc540bc06069668b5a0dc71da85e8dd72f5e9c42d5bb72dd75b9e017490b3a026fe056f17e9d6f5e95847cd23732eea0745f42d1efd5af5f132132c488f78e611e7cbbf33b220d9af4b14efdf518e938b722be6282ac78057fae2ec787b291538916614785fd26a2df34aafa1462e2777d9a59bcde668c2502774565cb37bcc902a1620e9802a29bc6d2b68d0a3d147bfa10117faa67003a4375098c6542a78e8425952fe05c3fff533013d4e7853ca189fe0bd8fa0b69280d4763150443662c005a863da4968b3d4d048403a792256306b98066a3f8e5f5e5987fe93b9037c1a85a447a5c2924a21a677626db61cee60734a7eeae43a1f9e10912a
could not be verified for this reason:
The signature value did not match the expected value. Res: 0
Package:
pkg://solaris/driver/network/[email protected],5.11-0.173.0.0.0.1.0:20110826T152405Z
Repository path:
/space/broken-repo/publisher/solaris/pkg/driver%2Fnetwork%2Fiwi/0.5.11%2C5.11-0.173.0.0.0.1.0%3A20110826T152405Z
$
# pkgrepo verify -s /space/throwaway-repo
Initiating repository verify.
pkg://pkg5-repo-verify/permissions-check 1/3 /
WARNING: Some repository content for publisher 'test',
or paths leading to the repository were not world-readable
or were not readable by 'pkg5srv:pkg5srv', which can cause
access errors if the repository contents are served by the
following services:
svc:/application/pkg/server
svc:/application/pkg/system-repository.
Only the first path found with unacceptable permissions is
shown below.
Path: /space
pkg://pkg5-repo-verify/permissions-check 1/4359 /
WARNING: Some repository content for publisher 'solaris',
or paths leading to the repository were not world-readable
or were not readable by 'pkg5srv:pkg5srv', which can cause
access errors if the repository contents are served by the
following services:
svc:/application/pkg/server
svc:/application/pkg/system-repository.
Only the first path found with unacceptable permissions is
shown below.
Path: /space
_______________________________________________
pkg-discuss mailing list
[email protected]
http://mail.opensolaris.org/mailman/listinfo/pkg-discuss
