Re: [Pulp-dev] Pulp3 CLI Proof of Concept

[Dennis Kliban]

Please try out the POC and send all feedback about the implementation or
any additional use cases before July 2nd.

There are a lot of details on the issue[0] in pulp.plan.io.

[0] https://pulp.plan.io/issues/3756


On Mon, Jun 11, 2018 at 1:20 PM, Bihan Zhang  wrote:

> After gathering some initial user stories[0] and feedback from Pulp
> users[1], the Pulp CLI mini-team[2] has been exploring implementation
> options. In doing so, we've spent a few days creating a prototype that we'd
> like the Pulp community to review and test. This proof of concept is
> available at the following repository:
>
> https://github.com/werwty/pulpcli-POC/
>
> This PoC provides an autogenerated set of commands for each endpoint
> available by the REST API schema. Having an autogenerated CLI means it
> automatically supports all plugins and provides complete feature parity
> with our REST API.
>
> Another highlight of the PoC's design is that it's easy to extend. One of
> the feedback items we got from pulp users is that they use scripts to
> automate their usage of the CLI, while other teams like RHUI have their own
> CLI. With this PoC design, users will be able to write their own custom
> commands and functionality in Python. This will also allow us (the Pulp
> team) to provide more complex commands and workflows in the future too.
>
>
> Installation/Usage
> ==
>
> To see information about how to install and use the PoC CLI, check out the
> README:
>
> https://github.com/werwty/pulpcli-POC/#installation
>
>
> Features
> ===
>
> - Auto-generated CLI commands with one command for each API endpoint
> - Support for plugins
> - Pagination support
> - Lookup via resource UUIDs or names
> - Help screens
> - Task polling
> - Autocompletion for commands and arguments
> - Extensibility
>   - Support in the future to create our own complex commands/workflows
>   - Users/plugin writers can create their own commands and ship their own
> cli plugins
>
>
> Implementation
> ===
>
> We first began by exploring pre-existing solutions like openapi-cli-client
> and coreapi-cli. The former, openapli-cli-client, was unmaintained and
> didn't seem to work with our API. The coreapi-cli package seemed promising
> but we found it hard to extend and it lacked some of the features we wanted
> to support in our initial set of user stories (e.g. pagination, task
> polling, etc). We realized that it was a rather small codebase that merely
> integrated the coreapi python package[3] with the cli package, click[4].
>
> So next we began investigating writing our own CLI using coreapi and
> click. Doing so was easy and it only required a few lines of code. We were
> then able to add on more features like task polling with little effort. We
> also explored using an alternative to click called argparse[5] but found it
> very hard to work with.
>
>
> Next Steps
> 
>
> We first want to get feedback on this PoC. If we decide to move forward
> with it, there are a few things we'd like to do to prepare this CLI for an
> MVP:
>
> - Provide docs for how to extend the CLI and write custom commands
> - Better error handling
> - Fix resource lookup for master/detail routes
> - Progress reporting when polling tasks
> - Support for file uploads
> - Authentication and configuration
> - Format of responses
> - Swich out coreapi for openapi
>
> All CLI-related tasks will be tracked through this redmine Epic:
> https://pulp.plan.io/issues/3756
>
> Before we proceed though, we'd like to get feedback. So please review and
> test out our PoC. Let us know if you have any comments and suggestions.
>
> [0] https://www.redhat.com/archives/pulp-dev/2018-May/msg4.html
> [1] https://www.redhat.com/archives/pulp-list/2018-May/msg7.html
> [2] asmacdo, bizhang, daviddavis, dawalker, and dkliban
> [3] https://github.com/core-api/python-client
> [4] http://click.pocoo.org/5/
> [5] https://docs.python.org/3/library/argparse.html
>
>
> ___
> Pulp-dev mailing list
> Pulp-dev@redhat.com
> https://www.redhat.com/mailman/listinfo/pulp-dev
>
>
___
Pulp-dev mailing list
Pulp-dev@redhat.com
https://www.redhat.com/mailman/listinfo/pulp-dev

Re: [Pulp-dev] Ideas for the plugin template

[Dennis Kliban]

+1

On Mon, Jun 18, 2018 at 3:14 PM, Dana Walker  wrote:

> +1
>
> (Already said it during review, but figured I'd make it official here.)
>
> Dana Walker
>
> Associate Software Engineer
>
> Red Hat
>
> 
> 
>
> On Mon, Jun 18, 2018 at 2:01 PM, Daniel Alley  wrote:
>
>> +1
>>
>> On Mon, Jun 18, 2018 at 12:16 PM, Brian Bouterse 
>> wrote:
>>
>>> +1
>>>
>>> On Mon, Jun 18, 2018 at 8:47 AM, Ina Panova  wrote:
>>>
 +1



 
 Regards,

 Ina Panova
 Software Engineer| Pulp| Red Hat Inc.

 "Do not go where the path may lead,
  go instead where there is no path and leave a trail."

 On Thu, Jun 14, 2018 at 10:19 PM, Bihan Zhang 
 wrote:

> +1
> I think the plugin_template is very valuable for bootstrapping plugin
> development, but we have had issues with keeping it up to date. Creating a
> smash test that will enforce this on new PRs make perfect sense to me.
>
>
>
> On Thu, Jun 14, 2018 at 11:29 AM, Austin Macdonald 
> wrote:
>
>> I've recently updated the plugin_template to work with the latest
>> master (3.0). [0] The template handles almost all of the bootstrapping
>> work necessary to write a new plugin, so it is valuable to keep it up
>> to
>> date. Given human nature, it's likely that the plugin_template will
>> tend
>> to fall behind as it did recently. I have some ideas to save time
>> while
>> keeping the template current and useful.
>>
>> 1) Move the plugin writer docs [1] into the plugin_template repository
>> - Leave a (very) high level overview in the core docs with a
>> link to
>>   the template docs.
>> - Plugin writer docs PRs would only go to one place, and it
>> would
>>   be easier keep the docs in line with the code.
>> - Narrative docs in the template would explain what needs to
>> be
>>   done generally, linking to the modules.
>> - Specific instructions would live in the code modules
>> alongside
>>   basic working code, and additional commented out code
>>   to demonstrate and explain more complex behaviors.
>>
>>  2) Add pulp_smash tests for basic functionality of a bootstrapped
>> plugin.
>>  - Run these tests as a check on pulpcore and template PRs
>>  - Ensure that discoverability works
>>  - Fail with breaking Plugin API changes
>>  - If the test uses pulp_smash, it would include a base set of
>>integration tests for every new plugin.
>>
>> My reasoning is that no matter what changes we make to pulpcore,
>> we need to keep the plugin writer docs updated. Doing this in the
>> template will provide value for plugin writers, and will inform
>> pulpcore
>> developers when it needs to be updated.
>>
>> [0]: https://github.com/pulp/plugin_template/pull/9
>> [1]: https://github.com/pulp/pulp/tree/master/docs/plugins/plugin
>> -writer
>>
>>
>>
>> ___
>> Pulp-dev mailing list
>> Pulp-dev@redhat.com
>> https://www.redhat.com/mailman/listinfo/pulp-dev
>>
>>
>
> ___
> Pulp-dev mailing list
> Pulp-dev@redhat.com
> https://www.redhat.com/mailman/listinfo/pulp-dev
>
>

 ___
 Pulp-dev mailing list
 Pulp-dev@redhat.com
 https://www.redhat.com/mailman/listinfo/pulp-dev


>>>
>>> ___
>>> Pulp-dev mailing list
>>> Pulp-dev@redhat.com
>>> https://www.redhat.com/mailman/listinfo/pulp-dev
>>>
>>>
>>
>> ___
>> Pulp-dev mailing list
>> Pulp-dev@redhat.com
>> https://www.redhat.com/mailman/listinfo/pulp-dev
>>
>>
>
> ___
> Pulp-dev mailing list
> Pulp-dev@redhat.com
> https://www.redhat.com/mailman/listinfo/pulp-dev
>
>
___
Pulp-dev mailing list
Pulp-dev@redhat.com
https://www.redhat.com/mailman/listinfo/pulp-dev

Re: [Pulp-dev] 'id' versus 'pulp_id' on Content

[Brian Bouterse]

That is a great reason to not use the underscore prefix also for that.
We've never formally called this out (that I know of) so that is useful.

A good prefix is tough to come up with. Part of the issue is that changing
MasterModel affects all models so a prefix like 'pulp_id' would affect all
models even though the collision concern is only for subclasses of Content.
A name like pulp_id would look very strange on a Distributor, Repository,
Remote, etc.

In terms of Content though, is anyone else concerned that 'created_at' and
'updated_at' will be very ambiguous to end users? Does it mean when the rpm
was made or when it was added to Pulp? I think this question will come up a
lot. I believe all this stems from us loading up Content with fields that
a) plugin writer's have to deal with and b) users have to figure out.

There is another option to consider. What if 'Content' didn't inherit the
fields from MasterModel and instead just defined pulp_id as the primary key
only for Cotnent? Content would still have the functionality of
MasterModel, just not the inherited fields. That would resolve the issues
without needing a convention or the need to 'separate' the fields. We (I
think) have to keep 'artifacts' and/or call them 'pulp_artifacts' because
Content units need a foreign key so we have to name it something. All code
could refer to the pk as 'pk', which is why Django has this feature to
handle differing pk field names. End users would clearly see the
pulp_artifacts and pulp_id on serialized Content units. Other pulp objects
would serialize with 'id', 'created_at', etc like we have today.

What do you all think about ^?

As an aside, calling the field 'pk' is not an option either. A little test
I ran throws this django error on the model definition: hello.my_model.pk:
(fields.E003) 'pk' is a reserved word that cannot be used as a field name.



On Mon, Jun 18, 2018 at 2:15 PM, Daniel Alley  wrote:

> I'm -1 on going the underscore idea, partly because of the aforementioned
> confusion issue, but also partly because but I've noticed that in our API,
> the "underscore" basically has a semantic meeting of "href, [which is]
> generated on the fly, not stored in the db".
>
> Specifically:
>
>- '_href'
>- '_added_href'
>- '_removed_href'
>- '_content_href'
>
> So I think if we use a prefix, we should avoid using one that already has
> a semantic meaning (I don't know whether we actually planned for that to be
> the case, but I think it's a useful pattern / distinction and I don't think
> we should mess with it).
>
> On Mon, Jun 18, 2018 at 2:01 PM, Brian Bouterse 
> wrote:
>
>> Having a user focus made me realize that it would be useful if a user
>> could easily tell which attributes were common to all content units versus
>> just that one content unit. When scripting for instance that is really
>> useful to know. We could document the 5 attributes that platform provides,
>> but when there are 20+ attributes on a subclassed content unit the
>> underscores would provide an easy, consistent answer to this question. This
>> is an additional reason separate from the the issue that our content
>> attribute names are colliding (id at least for now). The underscore prefix
>> would make collisions highly unlikely also. This problem is only scoped to
>> the Content unit since that is the place where we expect a large number of
>> subclassed attributes.
>>
>> For this reason I believe using the _ as the prefix will provide 2
>> benefits. I wrote them here on this ticket:
>> https://pulp.plan.io/issues/3704
>>
>> I am still +1 on adopting those changes for those reasons. More feedback
>> is welcome given the additional problem statements and discussion.
>>
>> On Mon, Jun 18, 2018 at 8:43 AM, Ina Panova  wrote:
>>
>>> uuid sounds like good compromise.
>>>
>>>
>>>
>>> 
>>> Regards,
>>>
>>> Ina Panova
>>> Software Engineer| Pulp| Red Hat Inc.
>>>
>>> "Do not go where the path may lead,
>>>  go instead where there is no path and leave a trail."
>>>
>>> On Thu, Jun 14, 2018 at 9:38 PM, Jeff Ortel  wrote:
>>>


 On 06/14/2018 12:19 PM, Jeff Ortel wrote:

>
>
> On 06/14/2018 10:37 AM, Daniel Alley wrote:
>
>> I will make one more suggestion.  What about naming "id" -> "uuid"?
>> This carries the clear connotation that it is a unique identifier so it 
>> is
>> less likely to be confusing a la "id and _id", and is still less likely 
>> to
>> have a namespace conflict.
>>
>
> Appreciate the suggestion but this would only be marginally less
> confusing.
>

 Reconsidering this suggestion for the reasons you outlined.


 ___
 Pulp-dev mailing list
 Pulp-dev@redhat.com
 https://www.redhat.com/mailman/listinfo/pulp-dev

>>>
>>>
>>> ___
>>> Pulp-dev mailing list
>>> Pulp-dev@redhat.com
>>> 

Re: [Pulp-dev] Ideas for the plugin template

[Dana Walker]

+1

(Already said it during review, but figured I'd make it official here.)

Dana Walker

Associate Software Engineer

Red Hat




On Mon, Jun 18, 2018 at 2:01 PM, Daniel Alley  wrote:

> +1
>
> On Mon, Jun 18, 2018 at 12:16 PM, Brian Bouterse 
> wrote:
>
>> +1
>>
>> On Mon, Jun 18, 2018 at 8:47 AM, Ina Panova  wrote:
>>
>>> +1
>>>
>>>
>>>
>>> 
>>> Regards,
>>>
>>> Ina Panova
>>> Software Engineer| Pulp| Red Hat Inc.
>>>
>>> "Do not go where the path may lead,
>>>  go instead where there is no path and leave a trail."
>>>
>>> On Thu, Jun 14, 2018 at 10:19 PM, Bihan Zhang 
>>> wrote:
>>>
 +1
 I think the plugin_template is very valuable for bootstrapping plugin
 development, but we have had issues with keeping it up to date. Creating a
 smash test that will enforce this on new PRs make perfect sense to me.



 On Thu, Jun 14, 2018 at 11:29 AM, Austin Macdonald 
 wrote:

> I've recently updated the plugin_template to work with the latest
> master (3.0). [0] The template handles almost all of the bootstrapping
> work necessary to write a new plugin, so it is valuable to keep it up
> to
> date. Given human nature, it's likely that the plugin_template will
> tend
> to fall behind as it did recently. I have some ideas to save time
> while
> keeping the template current and useful.
>
> 1) Move the plugin writer docs [1] into the plugin_template repository
> - Leave a (very) high level overview in the core docs with a
> link to
>   the template docs.
> - Plugin writer docs PRs would only go to one place, and it
> would
>   be easier keep the docs in line with the code.
> - Narrative docs in the template would explain what needs to
> be
>   done generally, linking to the modules.
> - Specific instructions would live in the code modules
> alongside
>   basic working code, and additional commented out code
>   to demonstrate and explain more complex behaviors.
>
>  2) Add pulp_smash tests for basic functionality of a bootstrapped
> plugin.
>  - Run these tests as a check on pulpcore and template PRs
>  - Ensure that discoverability works
>  - Fail with breaking Plugin API changes
>  - If the test uses pulp_smash, it would include a base set of
>integration tests for every new plugin.
>
> My reasoning is that no matter what changes we make to pulpcore,
> we need to keep the plugin writer docs updated. Doing this in the
> template will provide value for plugin writers, and will inform
> pulpcore
> developers when it needs to be updated.
>
> [0]: https://github.com/pulp/plugin_template/pull/9
> [1]: https://github.com/pulp/pulp/tree/master/docs/plugins/plugin
> -writer
>
>
>
> ___
> Pulp-dev mailing list
> Pulp-dev@redhat.com
> https://www.redhat.com/mailman/listinfo/pulp-dev
>
>

 ___
 Pulp-dev mailing list
 Pulp-dev@redhat.com
 https://www.redhat.com/mailman/listinfo/pulp-dev


>>>
>>> ___
>>> Pulp-dev mailing list
>>> Pulp-dev@redhat.com
>>> https://www.redhat.com/mailman/listinfo/pulp-dev
>>>
>>>
>>
>> ___
>> Pulp-dev mailing list
>> Pulp-dev@redhat.com
>> https://www.redhat.com/mailman/listinfo/pulp-dev
>>
>>
>
> ___
> Pulp-dev mailing list
> Pulp-dev@redhat.com
> https://www.redhat.com/mailman/listinfo/pulp-dev
>
>
___
Pulp-dev mailing list
Pulp-dev@redhat.com
https://www.redhat.com/mailman/listinfo/pulp-dev

[Pulp-dev] Pulp 2 plugin release plan

[Dennis Kliban]

Earlier today a few of us met to discuss how we can release new Y releases
of plugins without a Y release of the platform accompanying them.

The initial proposal was to publish a new Y release of a plugin at the same
time as a Z release of platform and other plugins. More concretely, we were
discussing putting pulp-docker-* 3.2.0 packages into the 2.16
repository[0]. This repository currently contains 3.1.3 packages.
Publishing 3.2.0 packages to this repository would completely remove the
3.1.3 pulp-docker packages. Since 3.1.3 pulp-docker-* packages were only
published to the 2.16 repository, the only 3.1.z package available after a
publish of 3.2.0 would be 3.1.2 in the 2.15 repository[1]. After
identifying this problem, we decided to NOT release pulp-docker-* 3.2.0
with the 2.16.2 z-stream release.

In order to eliminate this problem in the future, we would like to
investigate if it will be possible to compose repositories with new Y
releases of plugins while retaining the previous versions of packages that
were already published to the repository before. If this is possible, we
would try to start composing our Z stream repositories in such a way
starting with 2.17.0 release.

Questions? Thoughts? Ideas?


[0] https://repos.fedorapeople.org/pulp/pulp/stable/2.16/7Server/x86_64/
[1] https://repos.fedorapeople.org/pulp/pulp/stable/2.15/7Server/x86_64/
___
Pulp-dev mailing list
Pulp-dev@redhat.com
https://www.redhat.com/mailman/listinfo/pulp-dev

Re: [Pulp-dev] 'id' versus 'pulp_id' on Content

[Daniel Alley]

I'm -1 on going the underscore idea, partly because of the aforementioned
confusion issue, but also partly because but I've noticed that in our API,
the "underscore" basically has a semantic meeting of "href, [which is]
generated on the fly, not stored in the db".

Specifically:

   - '_href'
   - '_added_href'
   - '_removed_href'
   - '_content_href'

So I think if we use a prefix, we should avoid using one that already has a
semantic meaning (I don't know whether we actually planned for that to be
the case, but I think it's a useful pattern / distinction and I don't think
we should mess with it).

On Mon, Jun 18, 2018 at 2:01 PM, Brian Bouterse  wrote:

> Having a user focus made me realize that it would be useful if a user
> could easily tell which attributes were common to all content units versus
> just that one content unit. When scripting for instance that is really
> useful to know. We could document the 5 attributes that platform provides,
> but when there are 20+ attributes on a subclassed content unit the
> underscores would provide an easy, consistent answer to this question. This
> is an additional reason separate from the the issue that our content
> attribute names are colliding (id at least for now). The underscore prefix
> would make collisions highly unlikely also. This problem is only scoped to
> the Content unit since that is the place where we expect a large number of
> subclassed attributes.
>
> For this reason I believe using the _ as the prefix will provide 2
> benefits. I wrote them here on this ticket:  https://pulp.plan.io/issues/
> 3704
>
> I am still +1 on adopting those changes for those reasons. More feedback
> is welcome given the additional problem statements and discussion.
>
> On Mon, Jun 18, 2018 at 8:43 AM, Ina Panova  wrote:
>
>> uuid sounds like good compromise.
>>
>>
>>
>> 
>> Regards,
>>
>> Ina Panova
>> Software Engineer| Pulp| Red Hat Inc.
>>
>> "Do not go where the path may lead,
>>  go instead where there is no path and leave a trail."
>>
>> On Thu, Jun 14, 2018 at 9:38 PM, Jeff Ortel  wrote:
>>
>>>
>>>
>>> On 06/14/2018 12:19 PM, Jeff Ortel wrote:
>>>


 On 06/14/2018 10:37 AM, Daniel Alley wrote:

> I will make one more suggestion.  What about naming "id" -> "uuid"?
> This carries the clear connotation that it is a unique identifier so it is
> less likely to be confusing a la "id and _id", and is still less likely to
> have a namespace conflict.
>

 Appreciate the suggestion but this would only be marginally less
 confusing.

>>>
>>> Reconsidering this suggestion for the reasons you outlined.
>>>
>>>
>>> ___
>>> Pulp-dev mailing list
>>> Pulp-dev@redhat.com
>>> https://www.redhat.com/mailman/listinfo/pulp-dev
>>>
>>
>>
>> ___
>> Pulp-dev mailing list
>> Pulp-dev@redhat.com
>> https://www.redhat.com/mailman/listinfo/pulp-dev
>>
>>
>
> ___
> Pulp-dev mailing list
> Pulp-dev@redhat.com
> https://www.redhat.com/mailman/listinfo/pulp-dev
>
>
___
Pulp-dev mailing list
Pulp-dev@redhat.com
https://www.redhat.com/mailman/listinfo/pulp-dev

Re: [Pulp-dev] 'id' versus 'pulp_id' on Content

[Brian Bouterse]

Having a user focus made me realize that it would be useful if a user could
easily tell which attributes were common to all content units versus just
that one content unit. When scripting for instance that is really useful to
know. We could document the 5 attributes that platform provides, but when
there are 20+ attributes on a subclassed content unit the underscores would
provide an easy, consistent answer to this question. This is an additional
reason separate from the the issue that our content attribute names are
colliding (id at least for now). The underscore prefix would make
collisions highly unlikely also. This problem is only scoped to the Content
unit since that is the place where we expect a large number of subclassed
attributes.

For this reason I believe using the _ as the prefix will provide 2
benefits. I wrote them here on this ticket:
https://pulp.plan.io/issues/3704

I am still +1 on adopting those changes for those reasons. More feedback is
welcome given the additional problem statements and discussion.

On Mon, Jun 18, 2018 at 8:43 AM, Ina Panova  wrote:

> uuid sounds like good compromise.
>
>
>
> 
> Regards,
>
> Ina Panova
> Software Engineer| Pulp| Red Hat Inc.
>
> "Do not go where the path may lead,
>  go instead where there is no path and leave a trail."
>
> On Thu, Jun 14, 2018 at 9:38 PM, Jeff Ortel  wrote:
>
>>
>>
>> On 06/14/2018 12:19 PM, Jeff Ortel wrote:
>>
>>>
>>>
>>> On 06/14/2018 10:37 AM, Daniel Alley wrote:
>>>
 I will make one more suggestion.  What about naming "id" -> "uuid"?
 This carries the clear connotation that it is a unique identifier so it is
 less likely to be confusing a la "id and _id", and is still less likely to
 have a namespace conflict.

>>>
>>> Appreciate the suggestion but this would only be marginally less
>>> confusing.
>>>
>>
>> Reconsidering this suggestion for the reasons you outlined.
>>
>>
>> ___
>> Pulp-dev mailing list
>> Pulp-dev@redhat.com
>> https://www.redhat.com/mailman/listinfo/pulp-dev
>>
>
>
> ___
> Pulp-dev mailing list
> Pulp-dev@redhat.com
> https://www.redhat.com/mailman/listinfo/pulp-dev
>
>
___
Pulp-dev mailing list
Pulp-dev@redhat.com
https://www.redhat.com/mailman/listinfo/pulp-dev

Re: [Pulp-dev] Ideas for the plugin template

[Daniel Alley]

+1

On Mon, Jun 18, 2018 at 12:16 PM, Brian Bouterse 
wrote:

> +1
>
> On Mon, Jun 18, 2018 at 8:47 AM, Ina Panova  wrote:
>
>> +1
>>
>>
>>
>> 
>> Regards,
>>
>> Ina Panova
>> Software Engineer| Pulp| Red Hat Inc.
>>
>> "Do not go where the path may lead,
>>  go instead where there is no path and leave a trail."
>>
>> On Thu, Jun 14, 2018 at 10:19 PM, Bihan Zhang  wrote:
>>
>>> +1
>>> I think the plugin_template is very valuable for bootstrapping plugin
>>> development, but we have had issues with keeping it up to date. Creating a
>>> smash test that will enforce this on new PRs make perfect sense to me.
>>>
>>>
>>>
>>> On Thu, Jun 14, 2018 at 11:29 AM, Austin Macdonald 
>>> wrote:
>>>
 I've recently updated the plugin_template to work with the latest
 master (3.0). [0] The template handles almost all of the bootstrapping
 work necessary to write a new plugin, so it is valuable to keep it up to
 date. Given human nature, it's likely that the plugin_template will
 tend
 to fall behind as it did recently. I have some ideas to save time while
 keeping the template current and useful.

 1) Move the plugin writer docs [1] into the plugin_template repository
 - Leave a (very) high level overview in the core docs with a
 link to
   the template docs.
 - Plugin writer docs PRs would only go to one place, and it
 would
   be easier keep the docs in line with the code.
 - Narrative docs in the template would explain what needs to be
   done generally, linking to the modules.
 - Specific instructions would live in the code modules
 alongside
   basic working code, and additional commented out code
   to demonstrate and explain more complex behaviors.

  2) Add pulp_smash tests for basic functionality of a bootstrapped
 plugin.
  - Run these tests as a check on pulpcore and template PRs
  - Ensure that discoverability works
  - Fail with breaking Plugin API changes
  - If the test uses pulp_smash, it would include a base set of
integration tests for every new plugin.

 My reasoning is that no matter what changes we make to pulpcore,
 we need to keep the plugin writer docs updated. Doing this in the
 template will provide value for plugin writers, and will inform pulpcore
 developers when it needs to be updated.

 [0]: https://github.com/pulp/plugin_template/pull/9
 [1]: https://github.com/pulp/pulp/tree/master/docs/plugins/plugin
 -writer



 ___
 Pulp-dev mailing list
 Pulp-dev@redhat.com
 https://www.redhat.com/mailman/listinfo/pulp-dev


>>>
>>> ___
>>> Pulp-dev mailing list
>>> Pulp-dev@redhat.com
>>> https://www.redhat.com/mailman/listinfo/pulp-dev
>>>
>>>
>>
>> ___
>> Pulp-dev mailing list
>> Pulp-dev@redhat.com
>> https://www.redhat.com/mailman/listinfo/pulp-dev
>>
>>
>
> ___
> Pulp-dev mailing list
> Pulp-dev@redhat.com
> https://www.redhat.com/mailman/listinfo/pulp-dev
>
>
___
Pulp-dev mailing list
Pulp-dev@redhat.com
https://www.redhat.com/mailman/listinfo/pulp-dev

Re: [Pulp-dev] Integrating Katello with Pulp3 Recap

[Brian Bouterse]

Some of the Pulp devs involved in the gap analysis put together this
roundup of the gap problem statements [0]. I've scheduled two calls so that
we can go over the problem statements to ensure a shared understanding, and
think of some possible resolutions. My intention for the calls is to
generate ideas and not decide on them. Final decisions and written recaps
will be sent to the list for additional input.

June 22nd: noon - 1pm UTC < https://bluejeans.com/663034124/
June 29th: 12:30pm - 1:30pm UTC   <
https://bluejeans.com/748695092/

[0]:
http://projects.theforeman.org/projects/katello/wiki/PulpV3GapAnalysis#Core-Problem-Statements

We'll send a round-up of the best options that come out of the call to the
list for discussion in roughly two weeks. Thank you to everyone who has
collaborated throughout this process so far!

-Brian

On Fri, Jun 15, 2018 at 3:37 PM, Brian Bouterse  wrote:

> # Update
>
> The Katello/Pulp gap analysis mini-group met for the final time today.
> Here is the full gap analysis notes documenting how Katello interacts with
> Pulp[0]. There are sections labeled GAP which are specific gaps.
>
> # Next steps
>
> Early next week, the pulp devs involved in the gap analysis will produce a
> list of problem statements and send that to pulp-dev. After that hopefully
> all core developers can join one or two calls to solidify our understanding
> of the problem and discuss options to resolve them.
>
> [0]: http://projects.theforeman.org/projects/katello/wiki/PulpV3G
> apAnalysis
>
> -Brian
>
>
> On Tue, May 22, 2018 at 6:19 PM, Brian Bouterse 
> wrote:
>
>> Today 4 Pulp devs and 2 Katello devs* met, and we produced this page in
>> the katello wiki:   http://projects.theforeman.org
>> /projects/katello/wiki/PulpV3GapAnalysis
>>
>> It includes a section for each part of the Katello UI that involves Pulp
>> in any way. Some sections will be large, some small, which makes makes
>> putting dates on topics challenging. We are going top-to-bottom through
>> this doc, so you can see the order at least. Next time (thursday) we're
>> trying to wrap up the 'Products' page of Katello. I'm told it's the largest
>> of them all. Based on what I've seen so far and the total number of pages
>> I've seen, we'll have 6 -8 meetings total. With two meetings / week that
>> would wrap this up around June 7 or 14th.
>>
>> If anyone does specifically want to join, let jsherrill (cc'd) know; it's
>> his call bridge. We are meeting Tuesdays and Thursdays from 12pm - 1pm EDT
>> (4pm - 5pm UTC). All content updates will post to the wiki. Any input or
>> questions via the list are welcome also.
>>
>> Specifically where we need help from other Pulp contributors is in
>> answering how Pulp3 could resolve gaps discovered through this process.
>> These gaps are cases where Pulp3 currently can't support Katello's existing
>> workflows. We'll bring back a list of these gap areas to this list once we
>> get a bit further through this process.
>>
>> *: jsherrill, paji, daviddavis, ttereshc, dkliban, bmbouter
>>
>> -Brian
>>
>>
>>
>> On Thu, May 17, 2018 at 3:43 PM, Robin Chan  wrote:
>>
>>> +1 to the schedule - we have folks who will likely head up different
>>> efforts and we will want them to participate.
>>>
>>> On Thu, May 17, 2018 at 3:32 PM, Brian Bouterse 
>>> wrote:
>>>
 Today @dkliban, @daviddavis, and I met with two Katello developers,
 @jsherrill and @paji, to talk through some of the integration of Katello
 and Pulp3. Here is a recap of what we discussed, a goal we identified, and
 next steps.

 # What we discussed

 We went through Katello's UI and learned about:

 - the various filtering needs of Katello. Pulp is pretty well setup to
 do this already. I say filtering because the value is being matched,
 compared, substring matched, etc for a specific field. They need to filter
 by a group of repositories. For example, show me all RPM packages that are
 in repositories X, Y, and Z as one filter. Similarly show me all RPM
 packages that are in repository versions W, Q, T.

 - full text search. Katello needs to be able to search for a term,
 full-text style across some set of tables. Pulp does not have this
 currently. It would be limited to some tables; knowing which tables need to
 be searched would always be known.

 - "groups of repos". Katello subscribes a machine to a "set of repos"
 all of which Katello treats as one thing. This allows the client to stay
 subscribed to one set of content which makes repo protection easier because
 you don't have to change certs constantly. Then the content in those repos
 is what changes. This means when Katello does something it typically
 involves that same operation for a group of repos.

 - applicability. There are calculations that Katello has done above and
 beyond Pulp even in Pulp2. Pulp should be able to run all calculations 

Re: [Pulp-dev] Ideas for the plugin template

[Brian Bouterse]

+1

On Mon, Jun 18, 2018 at 8:47 AM, Ina Panova  wrote:

> +1
>
>
>
> 
> Regards,
>
> Ina Panova
> Software Engineer| Pulp| Red Hat Inc.
>
> "Do not go where the path may lead,
>  go instead where there is no path and leave a trail."
>
> On Thu, Jun 14, 2018 at 10:19 PM, Bihan Zhang  wrote:
>
>> +1
>> I think the plugin_template is very valuable for bootstrapping plugin
>> development, but we have had issues with keeping it up to date. Creating a
>> smash test that will enforce this on new PRs make perfect sense to me.
>>
>>
>>
>> On Thu, Jun 14, 2018 at 11:29 AM, Austin Macdonald 
>> wrote:
>>
>>> I've recently updated the plugin_template to work with the latest
>>> master (3.0). [0] The template handles almost all of the bootstrapping
>>> work necessary to write a new plugin, so it is valuable to keep it up to
>>> date. Given human nature, it's likely that the plugin_template will tend
>>> to fall behind as it did recently. I have some ideas to save time while
>>> keeping the template current and useful.
>>>
>>> 1) Move the plugin writer docs [1] into the plugin_template repository
>>> - Leave a (very) high level overview in the core docs with a
>>> link to
>>>   the template docs.
>>> - Plugin writer docs PRs would only go to one place, and it would
>>>   be easier keep the docs in line with the code.
>>> - Narrative docs in the template would explain what needs to be
>>>   done generally, linking to the modules.
>>> - Specific instructions would live in the code modules alongside
>>>   basic working code, and additional commented out code
>>>   to demonstrate and explain more complex behaviors.
>>>
>>>  2) Add pulp_smash tests for basic functionality of a bootstrapped
>>> plugin.
>>>  - Run these tests as a check on pulpcore and template PRs
>>>  - Ensure that discoverability works
>>>  - Fail with breaking Plugin API changes
>>>  - If the test uses pulp_smash, it would include a base set of
>>>integration tests for every new plugin.
>>>
>>> My reasoning is that no matter what changes we make to pulpcore,
>>> we need to keep the plugin writer docs updated. Doing this in the
>>> template will provide value for plugin writers, and will inform pulpcore
>>> developers when it needs to be updated.
>>>
>>> [0]: https://github.com/pulp/plugin_template/pull/9
>>> [1]: https://github.com/pulp/pulp/tree/master/docs/plugins/plugin-writer
>>>
>>>
>>>
>>> ___
>>> Pulp-dev mailing list
>>> Pulp-dev@redhat.com
>>> https://www.redhat.com/mailman/listinfo/pulp-dev
>>>
>>>
>>
>> ___
>> Pulp-dev mailing list
>> Pulp-dev@redhat.com
>> https://www.redhat.com/mailman/listinfo/pulp-dev
>>
>>
>
> ___
> Pulp-dev mailing list
> Pulp-dev@redhat.com
> https://www.redhat.com/mailman/listinfo/pulp-dev
>
>
___
Pulp-dev mailing list
Pulp-dev@redhat.com
https://www.redhat.com/mailman/listinfo/pulp-dev

Re: [Pulp-dev] 'id' versus 'pulp_id' on Content

[Ina Panova]

uuid sounds like good compromise.




Regards,

Ina Panova
Software Engineer| Pulp| Red Hat Inc.

"Do not go where the path may lead,
 go instead where there is no path and leave a trail."

On Thu, Jun 14, 2018 at 9:38 PM, Jeff Ortel  wrote:

>
>
> On 06/14/2018 12:19 PM, Jeff Ortel wrote:
>
>>
>>
>> On 06/14/2018 10:37 AM, Daniel Alley wrote:
>>
>>> I will make one more suggestion.  What about naming "id" -> "uuid"?
>>> This carries the clear connotation that it is a unique identifier so it is
>>> less likely to be confusing a la "id and _id", and is still less likely to
>>> have a namespace conflict.
>>>
>>
>> Appreciate the suggestion but this would only be marginally less
>> confusing.
>>
>
> Reconsidering this suggestion for the reasons you outlined.
>
>
> ___
> Pulp-dev mailing list
> Pulp-dev@redhat.com
> https://www.redhat.com/mailman/listinfo/pulp-dev
>
___
Pulp-dev mailing list
Pulp-dev@redhat.com
https://www.redhat.com/mailman/listinfo/pulp-dev