On 12/18/12 12:42 PM, Shawn Walker wrote:
https://cr.opensolaris.org/action/browse/pkg/timf/pkgrepo-verify-1/
Thanks for taking a look. I've made the few code changes you suggest,
but they're trivial - if you'd like to see a webrev, let me know.
src/pkgrepo.py:
lines 1231-1236: The formatting here is a bit wonky.
I've moved the tuple up a line.
line 1246: Why wouldn't this be an "else:" case at 1263 instead?
No reason - that's better, thanks.
line 1269: You sure the default width of 70 is the right wrapping
width? How well did textwrap appear to work?
It seems ok. textwrap is fine, except when we're dealing with wrapped
pkgsign errors, in which case we merrily chop signature hashes up. I
think I'm fine with that though, because I can't imagine users ever
wanting to copy/grep for stuff like:
--
pkg://solaris/driver/network/iwi 74/248 |
ERROR: Bad signature.
Package:
pkg://solaris/driver/network/[email protected],5.11-0.173.0.0.0.1.0:20110826T152405Z
Repository path:
/space/still-broken-repo/publisher/solaris/pkg/driver%2Fnetwork%2Fiwi/0.5.11%2C5.11-0.173.0.0.0.1.0%3A20110826T152405Z
Detail:
The signature with this signature value: 2aae07e33fc540bc06069668b5a0d
c71da85e8dd72f5e9c42d5bb72dd75b9e017490b3a026fe056f17e9d6f5e95847cd237
32eea0745f42d1efd5af5f132132c488f78e611e7cbbf33b220d9af4b14efdf518e938
b722be6282ac78057fae2ec787b291538916614785fd26a2df34aafa1462e2777d9a59
bcde668c2502774565cb37bcc902a1620e9802a29bc6d2b68d0a3d147bfa10117faa67
003a4375098c6542a78e8425952fe05c3fff533013d4e7853ca189fe0bd8fa0b69280d
4763150443662c005a863da4968b3d4d048403a792256306b98066a3f8e5f5e5987fe9
3b9037c1a85a447a5c2924a21a677626db61cee60734a7eeae43a1f9e10912a could
not be verified for this reason: The signature value did not match the
expected value. Res: 0
---
(in case thunderbird thinks it knows better, the text below "Detail:"
gets wrapped, but "Repository path:" and "Package:" values aren't
wrapped, because users are more likely to want to copy/grep for those)
line 1293: drop the "else:" and de-indent following line; makes
it obvious no code after will execute
Good point, fixed.
line 1314-1315: if we decide to support specifying specific packages
later, how much would that affect everything? trivially?
We'd need to be careful to error-out if a fix being applied to one of
our list of packages ended up affecting a package that wasn't in that
list, which would require a bit more work, similar to 'pkgrepo remove
<package>', walking the repository to see whether no other manifest
referenced that broken file.
line 1438: I would consider making the fact that repairs happened
more obvious by changing the wording, perhaps: "Repository repairs
completed."
Ok, fixed.
general: I noticed you chose to let the progress tracker do the
output, instead of having the client do the output like pkg(1) does;
that isn't wrong necessarily, but it is an interesting choice. What was
the reasoning? Should we also change pkg(1) to do the same?
I think we should. We have a nice API that does progress tracking now,
and using it might make it easier to decouple CLI tools that print
messages to the terminal from non-CLI tools that might choose to output
status elsewhere, say to a message-bus, for example..
src/tests/pkg5unittest.py"
line 2130: this should move to the top of the file with the rest of
the imports; I didn't mean the placement to be literal :-)
D'oh :-)
How close is the output of pkgrepo verify/fix to pkg(1)? Can you
example output for some various cases?
Sure, the attached shows us running pkg verify/fix and then pkgrepo
verify/fix.
pkgrepo doesn't take the columnar approach that pkg(1) does - partly
because the ERROR column doesn't actually contain anything for pkg(1),
so I don't think it adds much to the output, but also because pkgrepo
can emit WARNING messages too, though there's only one in the code so
far, so it seems like a waste of a column.
pkgrepo(1) also doesn't indent details of the error, but instead
line-separates groups of errors. When we have lots of detail to print
(for example, the pkgrepo error where we've found an invalid signature)
I think this results in more clarity.
pkglint(1) differs from both, in that it always prints the whole message
on a single line, letting the terminal deal with the wrapping. This
makes pkglint(1) easier to script with, but I wouldn't expect either
'pkg verify/fix' or 'pkgrepo verify/fix' to be used that way.
cheers,
tim
timf@kura[423] pkg verify '*pkg*'
PACKAGE STATUS
pkg://pkg5-nightly/package/pkg ERROR
user: pkg5srv
Skipping: Permission denied
pkg://pkg5-nightly/package/pkg/depot ERROR
file: etc/pkg/depot/depot_httpd.conf.mako
Size: 16372 bytes should be 15587
Hash: 1493b68ca78018b60b7c8241a6b1ba9cb2feac7b should be
ac63dfb3a277b491527b911395a28f94d7d502e9
file: lib/svc/manifest/application/pkg/pkg-depot.xml
Size: 5507 bytes should be 4345
Hash: 1a5f3537526208ee0aad3e546ce2be523034a135 should be
9a4cff3a39f58e5e25a6da7a6f506090b346c351
file: lib/svc/method/svc-pkg-depot
Size: 7658 bytes should be 5614
Hash: e9cf6531be276ca1f5a23ff006ae134017aff40c should be
75e53c61d760de0332364fa701a73a36ed0ce930
dir: var/cache/pkg/depot
Group: 'pkg5srv (97)' should be 'bin (2)'
pkg://pkg5-nightly/package/pkg/system-repository ERROR
file: etc/pkg/sysrepo/sysrepo_httpd.conf.mako
Size: 19226 bytes should be 18785
Hash: 17f1e6e3bc2c765b0c9d5ce7382479fe9a07445b should be
e6d6147e8e6997873080af7e788f5aa31d5d1d9a
file: lib/svc/method/svc-pkg-sysrepo
Size: 6374 bytes should be 5215
Hash: 1da523d4a96eeb3926afaf8b989ac166def9715a should be
20834127925f3c19654a4df37de0ebf4022a0471
file: usr/lib/pkg.sysrepo
Size: 42306 bytes should be 39109
Hash: d210d6a704df6e1cc2358e5dba47b7cddecf5b3d should be
a996588ca8cdfd9af7a18590b49196e459735f9e
# chmod 644 /usr/bin/ls
# pkg fix core-os
Verifying: pkg://solaris/system/core-os ERROR
file: etc/nsswitch.ldap
Size: 1452 bytes should be 1221
Hash: 3a16534fde306d965eff9be375b9a31f22a7e210 should be
c1723a047b1607b7227898b1f2573197395d3603
file: usr/bin/ls
Mode: 0644 should be 0555
Created ZFS snapshot: 2012-12-19-20:22:43
Repairing: pkg://solaris/system/core-os
Creating Plan (Evaluating mediators): |
DOWNLOAD PKGS FILES XFER (MB) SPEED
Completed 1/1 2/2 0.0/0.0 4.6k/s
PHASE ITEMS
Updating modified actions 2/2
Updating image state Done
Creating fast lookup database Done
#
timf@kura[424] pkgrepo -s /space/still\-broken\-repo verify
Initiating repository verification.
pkg://test/sample 1/4 /
ERROR: Invalid file hash: f769b09ae92edf86dc54bfadb7131ab41d91a4dc
Package: pkg://test/[email protected],5.11:20120302T021808Z
Repository path:
/space/still-broken-repo/publisher/test/file/f7/f769b09ae92edf86dc54bfadb7131ab41d91a4dc
Computed hash: 98f2c1c68fbc1d1f0c14664b32708fa28079e966
Path: space/validgz
ERROR: Corrupted gzip file.
Package: pkg://test/[email protected],5.11:20120302T021808Z
Repository path:
/space/still-broken-repo/publisher/test/file/31/31c02215f3b6e336609f9868dca1104ac2713b6b
Path: space/largefile.dat
ERROR: Missing file: 31c02215f3b6e336609f9868dca1104ac2713b6c
Package: pkg://test/[email protected],5.11:20120302T021808Z
Repository path:
/space/still-broken-repo/publisher/test/file/31/31c02215f3b6e336609f9868dca1104ac2713b6c
Path: space/broken.dat
Unknown package 1/4 /
ERROR: Bad manifest.
Repository path: /space/still-broken-repo/publisher/test/pkg/sample/@@@@
Detail: Illegal FMRI 'sample@@@@@': Version cannot be empty
pkg://test/sample 2/4 /
ERROR: Corrupt manifest.
Repository path:
/space/still-broken-repo/publisher/test/pkg/sample/2.0%2C5.11%3A20120302T021808Z
Detail: Use pkglint(1) for more details.
pkg://solaris/driver/network/iwi 74/248 \
ERROR: Bad signature.
Package:
pkg://solaris/driver/network/[email protected],5.11-0.173.0.0.0.1.0:20110826T152405Z
Repository path:
/space/still-broken-repo/publisher/solaris/pkg/driver%2Fnetwork%2Fiwi/0.5.11%2C5.11-0.173.0.0.0.1.0%3A20110826T152405Z
Detail:
The signature with this signature value: 2aae07e33fc540bc06069668b5a0d
c71da85e8dd72f5e9c42d5bb72dd75b9e017490b3a026fe056f17e9d6f5e95847cd237
32eea0745f42d1efd5af5f132132c488f78e611e7cbbf33b220d9af4b14efdf518e938
b722be6282ac78057fae2ec787b291538916614785fd26a2df34aafa1462e2777d9a59
bcde668c2502774565cb37bcc902a1620e9802a29bc6d2b68d0a3d147bfa10117faa67
003a4375098c6542a78e8425952fe05c3fff533013d4e7853ca189fe0bd8fa0b69280d
4763150443662c005a863da4968b3d4d048403a792256306b98066a3f8e5f5e5987fe9
3b9037c1a85a447a5c2924a21a677626db61cee60734a7eeae43a1f9e10912a could
not be verified for this reason: The signature value did not match the
expected value. Res: 0
pkg://solaris/driver/graphics/nvidia 76/248 |
ERROR: Corrupt manifest.
Repository path:
/space/still-broken-repo/publisher/solaris/pkg/driver%2Fgraphics%2Fnvidia/0.295.20.0%2C5.11-0.175.1.0.0.14.0%3A20120416T143309Z
Detail: Use pkglint(1) for more details.
Unknown package 76/248 |
ERROR: Bad manifest.
Repository path:
/space/still-broken-repo/publisher/solaris/pkg/driver%2Fgraphics%2Fnvidia/foobar
Detail: Illegal FMRI 'driver/graphics/nvidia@foobar': Bad Version:
foobar
pkg://solaris/driver/graphics/nvidia 77/248 |
ERROR: Corrupted gzip file.
Package:
pkg://solaris/driver/graphics/[email protected],5.11-0.175.1.0.0.14.0:20120416T143307Z
Repository path:
/space/still-broken-repo/publisher/solaris/file/f3/f3d0b7b8e5f3c4cde70d387c4bba717ceff40b01
Path: usr/share/man/man1/nvidia-xconfig.1
pkg://solaris/developer/meld 145/248 /
ERROR: Invalid file hash: f3c446fce9eb1920b94a8c6280fe9109b6543e85
Package:
pkg://solaris/developer/[email protected],5.11-0.175.1.0.0.15.799:20120501T164946Z
Repository path:
/space/still-broken-repo/publisher/solaris/file/f3/f3c446fce9eb1920b94a8c6280fe9109b6543e85
Computed hash: f3faa9b646db8c26da0eda44b627f632f6e99de1
Path: usr/lib/meld/meld/merge.py
timf@kura[426] pkgrepo -s /space/broken\-repo/ fix
Initiating repository fix.
PHASE ITEMS
Fixing repository content 6/6
PHASE ITEMS
Fixing repository content 7/7
Use pkgsend(1) or pkgrecv(1) to republish the
following packages or paths which were quarantined:
pkg://solaris/driver/network/[email protected],5.11-0.173.0.0.0.1.0:20110826T152405Z
/space/broken-repo/publisher/test/pkg/sample/2.0%2C5.11%3A20120302T021808Z
/space/broken-repo/publisher/test/pkg/sample/@@@@
pkg://test/[email protected],5.11:20120302T021808Z
pkg://solaris/driver/graphics/[email protected],5.11-0.175.1.0.0.14.0:20120416T143307Z
/space/broken-repo/publisher/solaris/pkg/driver%2Fgraphics%2Fnvidia/foobar
/space/broken-repo/publisher/solaris/pkg/driver%2Fgraphics%2Fnvidia/0.295.20.0%2C5.11-0.175.1.0.0.14.0%3A20120416T143309Z
pkg://solaris/developer/[email protected],5.11-0.175.1.0.0.15.799:20120501T164946Z
Repository fix completed.
_______________________________________________
pkg-discuss mailing list
[email protected]
http://mail.opensolaris.org/mailman/listinfo/pkg-discuss
