The pulp_deb plugin currently makes use of md5, sha1, sha256, and sha512.
Using ALLOWED_CONTENT_CHECKSUMS to "prohibit" one or more of these checksum 
types currently simply breaks the plugin.
This is one (of several) reasons why the pulp_deb CI tests are currently broken 
against pulpcore master (they use the default configuration for pulpcore 3.11, 
prohibiting md5 and sha1).

I could adjust the sync and publish, to only store and retrive checksums 
permitted by ALLOWED_CONTENT_CHECKSUMS.
I am pretty sure current APT implementations can get by with (and indeed 
prefer) sha256.

However, I do worry that this new default behaviour violates the basic user 
expectation that deb repositories published by pulp will include the same 
metadata fields as the upstream repository that was synced.
(In fact I have an open issue to ensure this is the case for non-checksum 
fields).
Publishing md5 and sha1 checksums (in addition to sha256 and possibly others) 
is widespread practice. The official Debian and Ubuntu repos all do so.
I have no idea to what extent real world APT usage depends on these fields.

I could of course punt to users, and simply tell them in documentation to 
re-add md5 and sha1 to ALLOWED_CONTENT_CHECKSUMS if they want the corresponding 
metadata fields.
I don't feel particularly happy about this since this ammounts to an indefinite 
recommendation against the default pulpcore configuration for all users who 
want their pulp_deb repositories to resemble official Debian/Ubuntu 
repositories.
(And for a security relevant configuration at that.)

Alternatively, I could store the unloved md5 and sha1 hashes in the content 
models themselves instead of the artifacts, but this is ugly for a whole host 
of reasons:

1) It is a lot more work
2) For artifacts pulpcore automatically does the work of actually checking the 
checksums against the artifacts.
3) It duplicates the source of truth for checksums (the horror!) for users that 
do not go along with the new default pulpcore configuration.

So that approach is probably a non-starter.

>From the point of view of pulp_deb it would be better if pulpcore did not so 
>much refuse to handle md5 and sha1, but rather would guarantee that at least 
>one strong checksum is also present and used for integrity checking.
Which I believe is the case anyway since we absolutely require sha256 to be 
present, no?

To summarize: I am uncertain how best to proceed, but perhaps I am overthinking 
this and simply respecting ALLOWED_CONTENT_CHECKSUMS and letting users decide 
is best.

regards,
Quirin
________________________________
From: pulp-dev-boun...@redhat.com <pulp-dev-boun...@redhat.com> on behalf of 
Brian Bouterse <bmbou...@redhat.com>
Sent: 12 February 2021 21:13
To: Pulp-dev <pulp-dev@redhat.com>; pulp-list <pulp-l...@redhat.com>
Subject: [Pulp-dev] Removing MD5 and SHA-1 as default available checksums in 
3.11

tl;dr With pulpcore 3.11, the plan is to remove MD5 and SHA-1 from the list of 
default available checksums.  RPM and Migration plugin users will need to add 
this back in at 3.11 upgrade time for your systems to continue working. Please 
give on-list feedback on this change.

## Background

Pulp has the ALLOWED_CONTENT_CHECKSUMS setting [0] which, by default, currently 
includes md5, sha-1, sha-224, sha-256, sha-384, and sha-512. Pulp code is 
restricted to only using hashers from this list. This feature gives admins the 
ability to prohibit hashers they do not trust. Pulp uses these checksums for 
package integrity verification purposes when syncing and publishing content.

## Motivation

We need to make Pulp secure by default. MD5 is known to be insecure, and 
therefore it is unsafe for Pulp to allow its use for calculating package 
integrity by default. SHA-1 is widely believed to be insecure, or will be soon, 
and should not be allowed by default for the same reason.

## Proposal

Pulpcore 3.11 would remove md5 and sha-1 from the default list of allowed 
checksums, leaving sha-224..sha-512. Specifically this change is occuring in 
the `ALLOWED_CONTENT_CHECKSUMS` setting [0]. This is only a change to the 
default settings; any specific system can be configured as desired. Nothing is 
"being taken away".

## Required User Action with 3.11

We believe both RPM plugin users and Migration plugin users will be impacted by 
this and mostly from the SHA-1 removal. SHA-1 is still used on a variety of 
CDNs including Red Hat's. Also as data is migrated from Pulp2 systems, this 
also likely uses SHA-1 and MD5 as the migration plugin runs.

If users are using the defaults for `ALLOWED_CONTENT_CHECKSUMS` and want to 
continue using SHA-1, they will need to update `ALLOWED_CONTENT_CHECKSUMS` in 
their settings file. Alternatively, users will need to run `pulpcore-manager 
handle-artifact-checksums` after upgrade to update any existing artifacts after 
upgrading.

## Why not automate this?

We do not take manual user action at upgrade time lightly. However, this is a 
security change, and we believe we need each Pulp system to opt-in for 
themselves.

[0]: 
https://docs.pulpproject.org/pulpcore/settings.html#allowed-content-checksums

Thanks!
The Pulpcore Team
_______________________________________________
Pulp-dev mailing list
Pulp-dev@redhat.com
https://listman.redhat.com/mailman/listinfo/pulp-dev

Reply via email to