The pulp_deb plugin currently makes use of md5, sha1, sha256, and sha512. Using ALLOWED_CONTENT_CHECKSUMS to "prohibit" one or more of these checksum types currently simply breaks the plugin. This is one (of several) reasons why the pulp_deb CI tests are currently broken against pulpcore master (they use the default configuration for pulpcore 3.11, prohibiting md5 and sha1).
I could adjust the sync and publish, to only store and retrive checksums permitted by ALLOWED_CONTENT_CHECKSUMS. I am pretty sure current APT implementations can get by with (and indeed prefer) sha256. However, I do worry that this new default behaviour violates the basic user expectation that deb repositories published by pulp will include the same metadata fields as the upstream repository that was synced. (In fact I have an open issue to ensure this is the case for non-checksum fields). Publishing md5 and sha1 checksums (in addition to sha256 and possibly others) is widespread practice. The official Debian and Ubuntu repos all do so. I have no idea to what extent real world APT usage depends on these fields. I could of course punt to users, and simply tell them in documentation to re-add md5 and sha1 to ALLOWED_CONTENT_CHECKSUMS if they want the corresponding metadata fields. I don't feel particularly happy about this since this ammounts to an indefinite recommendation against the default pulpcore configuration for all users who want their pulp_deb repositories to resemble official Debian/Ubuntu repositories. (And for a security relevant configuration at that.) Alternatively, I could store the unloved md5 and sha1 hashes in the content models themselves instead of the artifacts, but this is ugly for a whole host of reasons: 1) It is a lot more work 2) For artifacts pulpcore automatically does the work of actually checking the checksums against the artifacts. 3) It duplicates the source of truth for checksums (the horror!) for users that do not go along with the new default pulpcore configuration. So that approach is probably a non-starter. >From the point of view of pulp_deb it would be better if pulpcore did not so >much refuse to handle md5 and sha1, but rather would guarantee that at least >one strong checksum is also present and used for integrity checking. Which I believe is the case anyway since we absolutely require sha256 to be present, no? To summarize: I am uncertain how best to proceed, but perhaps I am overthinking this and simply respecting ALLOWED_CONTENT_CHECKSUMS and letting users decide is best. regards, Quirin ________________________________ From: pulp-dev-boun...@redhat.com <pulp-dev-boun...@redhat.com> on behalf of Brian Bouterse <bmbou...@redhat.com> Sent: 12 February 2021 21:13 To: Pulp-dev <pulp-dev@redhat.com>; pulp-list <pulp-l...@redhat.com> Subject: [Pulp-dev] Removing MD5 and SHA-1 as default available checksums in 3.11 tl;dr With pulpcore 3.11, the plan is to remove MD5 and SHA-1 from the list of default available checksums. RPM and Migration plugin users will need to add this back in at 3.11 upgrade time for your systems to continue working. Please give on-list feedback on this change. ## Background Pulp has the ALLOWED_CONTENT_CHECKSUMS setting [0] which, by default, currently includes md5, sha-1, sha-224, sha-256, sha-384, and sha-512. Pulp code is restricted to only using hashers from this list. This feature gives admins the ability to prohibit hashers they do not trust. Pulp uses these checksums for package integrity verification purposes when syncing and publishing content. ## Motivation We need to make Pulp secure by default. MD5 is known to be insecure, and therefore it is unsafe for Pulp to allow its use for calculating package integrity by default. SHA-1 is widely believed to be insecure, or will be soon, and should not be allowed by default for the same reason. ## Proposal Pulpcore 3.11 would remove md5 and sha-1 from the default list of allowed checksums, leaving sha-224..sha-512. Specifically this change is occuring in the `ALLOWED_CONTENT_CHECKSUMS` setting [0]. This is only a change to the default settings; any specific system can be configured as desired. Nothing is "being taken away". ## Required User Action with 3.11 We believe both RPM plugin users and Migration plugin users will be impacted by this and mostly from the SHA-1 removal. SHA-1 is still used on a variety of CDNs including Red Hat's. Also as data is migrated from Pulp2 systems, this also likely uses SHA-1 and MD5 as the migration plugin runs. If users are using the defaults for `ALLOWED_CONTENT_CHECKSUMS` and want to continue using SHA-1, they will need to update `ALLOWED_CONTENT_CHECKSUMS` in their settings file. Alternatively, users will need to run `pulpcore-manager handle-artifact-checksums` after upgrade to update any existing artifacts after upgrading. ## Why not automate this? We do not take manual user action at upgrade time lightly. However, this is a security change, and we believe we need each Pulp system to opt-in for themselves. [0]: https://docs.pulpproject.org/pulpcore/settings.html#allowed-content-checksums Thanks! The Pulpcore Team
_______________________________________________ Pulp-dev mailing list Pulp-dev@redhat.com https://listman.redhat.com/mailman/listinfo/pulp-dev
