Distributions definitely do not need to be migrated (i see them more as
configuration similar to importers, than i do as something that can be
exported/imported).
Publications i think would be a nice to have, but can be re-generated on
the other side based on the repository version. While the publication
won't be 100% exact (due to timestamps and ordering differences), i'm
not sure that it matters. The biggest drawback will just be that the
importing side needs to be smarter, and it will take a little longer.
I could see import/export of publications being an optimization or
extension later on, but not necessary for a first pass. If i'm missing
some issue or use case, please do speak up!
Justin
On 2/17/20 1:05 PM, David Davis wrote:
The user stories that Katello gave us (which I've entered into redmine
here[0]) don't mention publications or distributions. I will follow up
with Katello though.
[0] https://pulp.plan.io/issues/6134
David
On Mon, Feb 17, 2020 at 12:49 PM Dennis Kliban <dkli...@redhat.com
<mailto:dkli...@redhat.com>> wrote:
On Fri, Feb 14, 2020 at 1:11 PM David Davis <davidda...@redhat.com
<mailto:davidda...@redhat.com>> wrote:
Grant and I met today to discuss importers and exporters[0]
and we'd like some feedback before we proceed with the design.
To sum up this feature briefly: users can export a repository
version from one Pulp instance and import it to another.
Exporting just repository versions is not sufficient for
reproducing a Pulp instance in an air gapped environment. Users
need to be able to use the "export" to populate a Pulp instance
without needing to create any publications and/or distributions
afterwards. What about letting users specify a repository to
export and then have pulpcore figure out which repository
versions, publications, distributions, content, metadata, and
artifacts need to be exported?
# Master/Detail vs Core
So one fundamental question is whether we should use a
Master/Detail approach or just have core control the flow but
call out to plugins to get export formats.
To give some background: we currently define Exporters (ie
FileSystemExporter) in core as Master models. Plugins extend
this model which allows them to configure or customize the
Exporter. This was necessary because some plugins need to
export Publications (along with repository metadata) while
other plugins who don't have Publications or metadata export
RepositoryVersions.
The other option is to have core handle the workflow. The user
would call a core endpoint and provide a RepositoryVersion.
This would work because for importing/exporting, you wouldn't
ever use Publications because metadata won't be used for
importing back into Pulp. If needed, core could provide a way
for plugin writers to write custom handlers/exporters for
content types.
If we go with the second option, the question then becomes
whether we should divorce the concept of Exporters and
import/export. Or do we also switch Exporters from
Master/Detail to core only?
# Foreign Keys
Content can be distributed across multiple tables (eg
UpdateRecord has UpdateCollection, etc). In our export, we
could either use primary keys (UUIDs) or natural keys to
relate records. The former assumes that UUIDs are unique
across Pulp instances. The safer but more complex alternative
is to use natural keys. This would involve storing a set of
fields on a record that would be used to identify a related
record.
# Incremental Exports
There are two big pieces of data contained in an export: the
dataset of Content from the database and the artifact files.
An incremental export cuts down on the size of an export by
only exporting the differences. However, when performing an
incremental export, we could still export the complete dataset
instead of just a set of differences
(additions/removals/updates). This approach would be simpler
and it would allow us to ensure that the new repo version
matches the exported repo version exactly. It would however
increase the export size but not by much I think--probably
some number of megabytes at most.
[0] https://pulp.plan.io/issues/6134
David
_______________________________________________
Pulp-dev mailing list
Pulp-dev@redhat.com <mailto:Pulp-dev@redhat.com>
https://www.redhat.com/mailman/listinfo/pulp-dev
_______________________________________________
Pulp-dev mailing list
Pulp-dev@redhat.com
https://www.redhat.com/mailman/listinfo/pulp-dev
_______________________________________________
Pulp-dev mailing list
Pulp-dev@redhat.com
https://www.redhat.com/mailman/listinfo/pulp-dev
