Re: [Pulp-dev] Proposal to remove 'notes' fields from the Pulp 3 RC

[Dana Walker]

+1

Dana Walker

Associate Software Engineer

Red Hat





On Tue, Dec 4, 2018 at 1:52 PM Tatiana Tereshchenko 
wrote:

> +1
>
> On Tue, Dec 4, 2018 at 4:49 PM Brian Bouterse  wrote:
>
>> +1. Ty for bringing this up @dalley
>>
>> On Tue, Dec 4, 2018 at 10:47 AM Jeff Ortel  wrote:
>>
>>> no objection
>>>
>>> On 12/3/18 10:32 PM, Daniel Alley wrote:
>>>
>>> *Background:*
>>>
>>> "Notes" are a generic key value store where data can be attached to
>>> repositories and content and publications and so forth.  The eventual plan
>>> is to use this to enable adding tags to those sorts of objects, which is
>>> important for Katello.
>>>
>>> Most of the code for this is located in pulp/app/models/generic.py
>>>
>>> *Motivation:*
>>>
>>> "Notes" have been in Pulp 3 for a very very long time and are completely
>>> unchanged for the last 12 months (the git history doesn't go back further
>>> because the file was moved).  The data model behind it is extremely complex
>>> and while we have a few unit tests around it, we have no functional tests
>>> for it whatsoever, and (to my knowledge) we haven't been using/exercising
>>> this functionality manually in a meaningful way (if at all).  I could be
>>> wrong here, but I haven't seen it discussed  or any issues related to it
>>> filed in quite some time.
>>>
>>> *Proposal:*
>>>
>>> We should pull out all of the "notes" code (models/generic.py + the
>>> fields on the aforementioned models) until we've had a chance to properly
>>> evaluate our needs and whether the current design fits them.
>>>
>>> ___
>>> Pulp-dev mailing 
>>> listPulp-dev@redhat.comhttps://www.redhat.com/mailman/listinfo/pulp-dev
>>>
>>>
>>> ___
>>> Pulp-dev mailing list
>>> Pulp-dev@redhat.com
>>> https://www.redhat.com/mailman/listinfo/pulp-dev
>>>
>> ___
>> Pulp-dev mailing list
>> Pulp-dev@redhat.com
>> https://www.redhat.com/mailman/listinfo/pulp-dev
>>
> ___
> Pulp-dev mailing list
> Pulp-dev@redhat.com
> https://www.redhat.com/mailman/listinfo/pulp-dev
>
___
Pulp-dev mailing list
Pulp-dev@redhat.com
https://www.redhat.com/mailman/listinfo/pulp-dev

[Pulp-dev] Changes to pulp’s directory structure

[David Davis]

As part of https://pulp.plan.io/issues/4101, the pulp repository[0] will be
restructured so that the source code will be in the pulpcore/ subdirectory
instead of in pulpcore/pulpcore/. Unless anyone objects, I hope to do this
at the end of this week.

If you have any changes, open PRs, etc, I would recommend getting them
merged by  Friday, December 7, 2018 to avoid conflicts.

[0] https://github.com/pulp/pulp

David
___
Pulp-dev mailing list
Pulp-dev@redhat.com
https://www.redhat.com/mailman/listinfo/pulp-dev

Re: [Pulp-dev] Proposal: merge the content-app & streamer

[Tatiana Tereshchenko]

+1 to merge
+1 to have clear docs for plugin writers how to create their own content app

On Mon, Dec 3, 2018 at 11:25 PM Dennis Kliban  wrote:

> It was pointed out on IRC that plugins that have to supply their own
> content app (such as docker) currently need to supply 2 implementations of
> it in order to support on-demand use cases. One using django and another
> using aiohttp.
>
> We should not burden plugin writers in such a way. We really have to make
> the proposed change.
>
> On Mon, Dec 3, 2018 at 3:24 PM Dana Walker  wrote:
>
>> In light of the efficiency gains and lack of significant drawbacks, I'm
>> +1 on this proposal.
>>
>> --Dana
>>
>>
>> Dana Walker
>>
>> Associate Software Engineer
>>
>> Red Hat
>>
>> 
>> 
>>
>>
>> On Mon, Dec 3, 2018 at 2:40 PM Dennis Kliban  wrote:
>>
>>> I like the idea of combining the two applications for all the reasons
>>> already outlined on this thread. The user experience is going to be
>>> simplified by this change. However, I want to point out that it will also
>>> alter the plugin writer experience. Plugin writers that want to have their
>>> own content app will now need to provide it as a plugin for the content app
>>> (which is not a Django project). We should be able to clearly document this
>>> for plugin writers. pulp_docker plugin will need to adopt this change. For
>>> that reason I'd like us to make a decision on this soon.
>>>
>>> On Fri, Nov 30, 2018 at 4:59 PM Jeff Ortel  wrote:
>>>
 *BACKGROUND*

 The pulp3 content app and the streamer (in-progress) currently have a
 lot of duplicate code and functionality.  At the very least, I think there
 is a opportunity to refactor both and share code.  But, this would leave us
 with two components with significant overlap in functionality.

 The functionality exclusive to the content-app:
   - Optionally delegate file serving to a web server. (Eg:
 mod_xsendfile).
   - Optional redirect to the streamer.

 The functionality exclusive to the streamer:
   - Using the Remote & RemoteArtifact to download the file and stream
 on demand.

 Not much difference which raises the question: "Why do we have both?"
 I think the answer may be that we don't.

 *PROPOSAL*

 Let's pull the content-app out and merge it with the streamer.  The new
 content (app) would have *streamer* architecture & functionality.
 When a requested artifact has not been downloaded, it would
 download/streamed instead of REDIRECT.  This does mean that deployments and
 development environments would need to run an additional service to serve
 content.  The /pulp/content endpoint would be on a different port than the
 API.  I see this separation as a healthy thing.  There is significant
 efficiency to be gained as well.  Let's start with eliminating the
 REDIRECTs.  Cutting the GET requests in half is a win for both the client,
 the network and the Pulp web stack.  Next is database queries.  Since both
 applications needed to perform many of the same queries, combining the
 applications will roughly cut them in half as well.  Since the streamer is
 based on asyncio and so would the merged app.

 There are probably lots of other pros/cons I have not considered but it
 seems relatively straight forward.

 I'm thinking the new content app/service would be named: *pulp-content*
 .

 Thoughts?

 ___
 Pulp-dev mailing list
 Pulp-dev@redhat.com
 https://www.redhat.com/mailman/listinfo/pulp-dev

>>> ___
>>> Pulp-dev mailing list
>>> Pulp-dev@redhat.com
>>> https://www.redhat.com/mailman/listinfo/pulp-dev
>>>
>> ___
> Pulp-dev mailing list
> Pulp-dev@redhat.com
> https://www.redhat.com/mailman/listinfo/pulp-dev
>
___
Pulp-dev mailing list
Pulp-dev@redhat.com
https://www.redhat.com/mailman/listinfo/pulp-dev

Re: [Pulp-dev] Proposal to remove 'notes' fields from the Pulp 3 RC

[Tatiana Tereshchenko]

+1

On Tue, Dec 4, 2018 at 4:49 PM Brian Bouterse  wrote:

> +1. Ty for bringing this up @dalley
>
> On Tue, Dec 4, 2018 at 10:47 AM Jeff Ortel  wrote:
>
>> no objection
>>
>> On 12/3/18 10:32 PM, Daniel Alley wrote:
>>
>> *Background:*
>>
>> "Notes" are a generic key value store where data can be attached to
>> repositories and content and publications and so forth.  The eventual plan
>> is to use this to enable adding tags to those sorts of objects, which is
>> important for Katello.
>>
>> Most of the code for this is located in pulp/app/models/generic.py
>>
>> *Motivation:*
>>
>> "Notes" have been in Pulp 3 for a very very long time and are completely
>> unchanged for the last 12 months (the git history doesn't go back further
>> because the file was moved).  The data model behind it is extremely complex
>> and while we have a few unit tests around it, we have no functional tests
>> for it whatsoever, and (to my knowledge) we haven't been using/exercising
>> this functionality manually in a meaningful way (if at all).  I could be
>> wrong here, but I haven't seen it discussed  or any issues related to it
>> filed in quite some time.
>>
>> *Proposal:*
>>
>> We should pull out all of the "notes" code (models/generic.py + the
>> fields on the aforementioned models) until we've had a chance to properly
>> evaluate our needs and whether the current design fits them.
>>
>> ___
>> Pulp-dev mailing 
>> listPulp-dev@redhat.comhttps://www.redhat.com/mailman/listinfo/pulp-dev
>>
>>
>> ___
>> Pulp-dev mailing list
>> Pulp-dev@redhat.com
>> https://www.redhat.com/mailman/listinfo/pulp-dev
>>
> ___
> Pulp-dev mailing list
> Pulp-dev@redhat.com
> https://www.redhat.com/mailman/listinfo/pulp-dev
>
___
Pulp-dev mailing list
Pulp-dev@redhat.com
https://www.redhat.com/mailman/listinfo/pulp-dev

Re: [Pulp-dev] Distributing Pulp3 Plans

[Tatiana Tereshchenko]

+1

On Mon, Dec 3, 2018 at 5:54 PM Daniel Alley  wrote:

> No objection
>
> On Mon, Dec 3, 2018 at 11:52 AM Jeff Ortel  wrote:
>
>> +1
>>
>> On 12/1/18 6:01 AM, David Davis wrote:
>>
>> +1 from me.
>>
>> David
>>
>>
>> On Fri, Nov 30, 2018 at 10:26 AM Dennis Kliban 
>> wrote:
>>
>>> No objections from me.
>>>
>>> On Thu, Nov 29, 2018 at 7:50 AM Brian Bouterse 
>>> wrote:
>>>
 The plan about 12-24 months ago was to distribute Pulp3 with Pulp
 itself on a machine hosted in the osci.io community environment. We
 have this ticket tracking that work [0] (still at NEW).

 I commented [1] that I think our distribution plans now involve mainly
 PyPI releases, and we probably won't self-host our release infrastructure.
 Is that what others think?

 If we aren't self-hosting with Pulp, can we close this ticket [0],
 clean up the infra wiki [2], and ask OSCI to deprovision their machine?

 [0]: https://pulp.plan.io/issues/2325
 [1]: https://pulp.plan.io/issues/2325#note-32
 [2]:
 https://pulp.plan.io/projects/pulp/wiki/Infrastructure_&_Hosting#Distribute-Pulp-with-Pulp

 Thanks!
 Brian
 ___
 Pulp-dev mailing list
 Pulp-dev@redhat.com
 https://www.redhat.com/mailman/listinfo/pulp-dev

>>> ___
>>> Pulp-dev mailing list
>>> Pulp-dev@redhat.com
>>> https://www.redhat.com/mailman/listinfo/pulp-dev
>>>
>>
>> ___
>> Pulp-dev mailing 
>> listPulp-dev@redhat.comhttps://www.redhat.com/mailman/listinfo/pulp-dev
>>
>>
>> ___
>> Pulp-dev mailing list
>> Pulp-dev@redhat.com
>> https://www.redhat.com/mailman/listinfo/pulp-dev
>>
> ___
> Pulp-dev mailing list
> Pulp-dev@redhat.com
> https://www.redhat.com/mailman/listinfo/pulp-dev
>
___
Pulp-dev mailing list
Pulp-dev@redhat.com
https://www.redhat.com/mailman/listinfo/pulp-dev

Re: [Pulp-dev] pulp_gem reached beta status

[Brian Bouterse]

That is great! I know several users who have asked us about gem support in
the past. Thank you for contributing this.

To get it on that list, could you send a PR adding it to this table [2]? We
can merge it once it's opened.

[2]: https://github.com/pulp/pulpproject.org/blob/gh-pages/pulp-3-plugins.md


On Tue, Dec 4, 2018 at 10:51 AM Matthias Dellweg  wrote:

> Hi all,
> i think it is time to call it a beta now, so i cordially ask you to
> include pulp_gem on the plugin list [0].
> I uploaded version 0.0.1b1 to pypi a minute ago [1], so it should be
> installable via pip. Let me know if you think there's anything
> wrong/missing.
> Thank you all!
>
> Matthias
>
> [0] https://pulpproject.org/pulp-3-plugins/
> [1] https://pypi.org/project/pulp-gem/
> ___
> Pulp-dev mailing list
> Pulp-dev@redhat.com
> https://www.redhat.com/mailman/listinfo/pulp-dev
>
___
Pulp-dev mailing list
Pulp-dev@redhat.com
https://www.redhat.com/mailman/listinfo/pulp-dev

[Pulp-dev] pulp_gem reached beta status

[Matthias Dellweg]

Hi all,
i think it is time to call it a beta now, so i cordially ask you to
include pulp_gem on the plugin list [0].
I uploaded version 0.0.1b1 to pypi a minute ago [1], so it should be
installable via pip. Let me know if you think there's anything
wrong/missing.
Thank you all!

Matthias

[0] https://pulpproject.org/pulp-3-plugins/
[1] https://pypi.org/project/pulp-gem/


pgp56Mkrbkk2R.pgp
Description: OpenPGP digital signature
___
Pulp-dev mailing list
Pulp-dev@redhat.com
https://www.redhat.com/mailman/listinfo/pulp-dev

Re: [Pulp-dev] Proposal to remove 'notes' fields from the Pulp 3 RC

[Brian Bouterse]

+1. Ty for bringing this up @dalley

On Tue, Dec 4, 2018 at 10:47 AM Jeff Ortel  wrote:

> no objection
>
> On 12/3/18 10:32 PM, Daniel Alley wrote:
>
> *Background:*
>
> "Notes" are a generic key value store where data can be attached to
> repositories and content and publications and so forth.  The eventual plan
> is to use this to enable adding tags to those sorts of objects, which is
> important for Katello.
>
> Most of the code for this is located in pulp/app/models/generic.py
>
> *Motivation:*
>
> "Notes" have been in Pulp 3 for a very very long time and are completely
> unchanged for the last 12 months (the git history doesn't go back further
> because the file was moved).  The data model behind it is extremely complex
> and while we have a few unit tests around it, we have no functional tests
> for it whatsoever, and (to my knowledge) we haven't been using/exercising
> this functionality manually in a meaningful way (if at all).  I could be
> wrong here, but I haven't seen it discussed  or any issues related to it
> filed in quite some time.
>
> *Proposal:*
>
> We should pull out all of the "notes" code (models/generic.py + the fields
> on the aforementioned models) until we've had a chance to properly evaluate
> our needs and whether the current design fits them.
>
> ___
> Pulp-dev mailing 
> listPulp-dev@redhat.comhttps://www.redhat.com/mailman/listinfo/pulp-dev
>
>
> ___
> Pulp-dev mailing list
> Pulp-dev@redhat.com
> https://www.redhat.com/mailman/listinfo/pulp-dev
>
___
Pulp-dev mailing list
Pulp-dev@redhat.com
https://www.redhat.com/mailman/listinfo/pulp-dev

Re: [Pulp-dev] Proposal to remove 'notes' fields from the Pulp 3 RC

[Daniel Alley]

Issue filed and added to sprint https://pulp.plan.io/issues/4222

On Tue, Dec 4, 2018 at 9:14 AM Dennis Kliban  wrote:

> +1 to removing it
>
> On Tue, Dec 4, 2018 at 8:34 AM Austin Macdonald 
> wrote:
>
>> +1, nice catch Daniel.
>>
>> On Tue, Dec 4, 2018, 7:14 AM David Davis >
>>> Big +1. This seems like something we could add later on at any time when
>>> we need it. Hopefully we can get Katello and our users involved in sussing
>>> out the requirements for how this field should work too.
>>>
>>> David
>>>
>>>
>>> On Mon, Dec 3, 2018 at 11:34 PM Daniel Alley  wrote:
>>>
 *Background:*

 "Notes" are a generic key value store where data can be attached to
 repositories and content and publications and so forth.  The eventual plan
 is to use this to enable adding tags to those sorts of objects, which is
 important for Katello.

 Most of the code for this is located in pulp/app/models/generic.py

 *Motivation:*

 "Notes" have been in Pulp 3 for a very very long time and are
 completely unchanged for the last 12 months (the git history doesn't go
 back further because the file was moved).  The data model behind it is
 extremely complex and while we have a few unit tests around it, we have no
 functional tests for it whatsoever, and (to my knowledge) we haven't been
 using/exercising this functionality manually in a meaningful way (if at
 all).  I could be wrong here, but I haven't seen it discussed  or any
 issues related to it filed in quite some time.

 *Proposal:*

 We should pull out all of the "notes" code (models/generic.py + the
 fields on the aforementioned models) until we've had a chance to properly
 evaluate our needs and whether the current design fits them.
 ___
 Pulp-dev mailing list
 Pulp-dev@redhat.com
 https://www.redhat.com/mailman/listinfo/pulp-dev

>>> ___
>>> Pulp-dev mailing list
>>> Pulp-dev@redhat.com
>>> https://www.redhat.com/mailman/listinfo/pulp-dev
>>>
>> ___
>> Pulp-dev mailing list
>> Pulp-dev@redhat.com
>> https://www.redhat.com/mailman/listinfo/pulp-dev
>>
> ___
> Pulp-dev mailing list
> Pulp-dev@redhat.com
> https://www.redhat.com/mailman/listinfo/pulp-dev
>
___
Pulp-dev mailing list
Pulp-dev@redhat.com
https://www.redhat.com/mailman/listinfo/pulp-dev

Re: [Pulp-dev] Proposal to remove 'notes' fields from the Pulp 3 RC

[Jeff Ortel]

no objection

On 12/3/18 10:32 PM, Daniel Alley wrote:

*Background:*

"Notes" are a generic key value store where data can be attached to 
repositories and content and publications and so forth.  The eventual 
plan is to use this to enable adding tags to those sorts of objects, 
which is important for Katello.


Most of the code for this is located in pulp/app/models/generic.py

*Motivation:*

"Notes" have been in Pulp 3 for a very very long time and are 
completely unchanged for the last 12 months (the git history doesn't 
go back further because the file was moved). The data model behind it 
is extremely complex and while we have a few unit tests around it, we 
have no functional tests for it whatsoever, and (to my knowledge) we 
haven't been using/exercising this functionality manually in a 
meaningful way (if at all).  I could be wrong here, but I haven't seen 
it discussed  or any issues related to it filed in quite some time.


*Proposal:*

We should pull out all of the "notes" code (models/generic.py + the 
fields on the aforementioned models) until we've had a chance to 
properly evaluate our needs and whether the current design fits them.


___
Pulp-dev mailing list
Pulp-dev@redhat.com
https://www.redhat.com/mailman/listinfo/pulp-dev


___
Pulp-dev mailing list
Pulp-dev@redhat.com
https://www.redhat.com/mailman/listinfo/pulp-dev

Re: [Pulp-dev] Proposal to remove 'notes' fields from the Pulp 3 RC

[Dennis Kliban]

+1 to removing it

On Tue, Dec 4, 2018 at 8:34 AM Austin Macdonald  wrote:

> +1, nice catch Daniel.
>
> On Tue, Dec 4, 2018, 7:14 AM David Davis 
>> Big +1. This seems like something we could add later on at any time when
>> we need it. Hopefully we can get Katello and our users involved in sussing
>> out the requirements for how this field should work too.
>>
>> David
>>
>>
>> On Mon, Dec 3, 2018 at 11:34 PM Daniel Alley  wrote:
>>
>>> *Background:*
>>>
>>> "Notes" are a generic key value store where data can be attached to
>>> repositories and content and publications and so forth.  The eventual plan
>>> is to use this to enable adding tags to those sorts of objects, which is
>>> important for Katello.
>>>
>>> Most of the code for this is located in pulp/app/models/generic.py
>>>
>>> *Motivation:*
>>>
>>> "Notes" have been in Pulp 3 for a very very long time and are completely
>>> unchanged for the last 12 months (the git history doesn't go back further
>>> because the file was moved).  The data model behind it is extremely complex
>>> and while we have a few unit tests around it, we have no functional tests
>>> for it whatsoever, and (to my knowledge) we haven't been using/exercising
>>> this functionality manually in a meaningful way (if at all).  I could be
>>> wrong here, but I haven't seen it discussed  or any issues related to it
>>> filed in quite some time.
>>>
>>> *Proposal:*
>>>
>>> We should pull out all of the "notes" code (models/generic.py + the
>>> fields on the aforementioned models) until we've had a chance to properly
>>> evaluate our needs and whether the current design fits them.
>>> ___
>>> Pulp-dev mailing list
>>> Pulp-dev@redhat.com
>>> https://www.redhat.com/mailman/listinfo/pulp-dev
>>>
>> ___
>> Pulp-dev mailing list
>> Pulp-dev@redhat.com
>> https://www.redhat.com/mailman/listinfo/pulp-dev
>>
> ___
> Pulp-dev mailing list
> Pulp-dev@redhat.com
> https://www.redhat.com/mailman/listinfo/pulp-dev
>
___
Pulp-dev mailing list
Pulp-dev@redhat.com
https://www.redhat.com/mailman/listinfo/pulp-dev

Re: [Pulp-dev] Proposal to remove 'notes' fields from the Pulp 3 RC

[Austin Macdonald]

+1, nice catch Daniel.

On Tue, Dec 4, 2018, 7:14 AM David Davis  Big +1. This seems like something we could add later on at any time when
> we need it. Hopefully we can get Katello and our users involved in sussing
> out the requirements for how this field should work too.
>
> David
>
>
> On Mon, Dec 3, 2018 at 11:34 PM Daniel Alley  wrote:
>
>> *Background:*
>>
>> "Notes" are a generic key value store where data can be attached to
>> repositories and content and publications and so forth.  The eventual plan
>> is to use this to enable adding tags to those sorts of objects, which is
>> important for Katello.
>>
>> Most of the code for this is located in pulp/app/models/generic.py
>>
>> *Motivation:*
>>
>> "Notes" have been in Pulp 3 for a very very long time and are completely
>> unchanged for the last 12 months (the git history doesn't go back further
>> because the file was moved).  The data model behind it is extremely complex
>> and while we have a few unit tests around it, we have no functional tests
>> for it whatsoever, and (to my knowledge) we haven't been using/exercising
>> this functionality manually in a meaningful way (if at all).  I could be
>> wrong here, but I haven't seen it discussed  or any issues related to it
>> filed in quite some time.
>>
>> *Proposal:*
>>
>> We should pull out all of the "notes" code (models/generic.py + the
>> fields on the aforementioned models) until we've had a chance to properly
>> evaluate our needs and whether the current design fits them.
>> ___
>> Pulp-dev mailing list
>> Pulp-dev@redhat.com
>> https://www.redhat.com/mailman/listinfo/pulp-dev
>>
> ___
> Pulp-dev mailing list
> Pulp-dev@redhat.com
> https://www.redhat.com/mailman/listinfo/pulp-dev
>
___
Pulp-dev mailing list
Pulp-dev@redhat.com
https://www.redhat.com/mailman/listinfo/pulp-dev

Re: [Pulp-dev] Proposal to remove 'notes' fields from the Pulp 3 RC

[David Davis]

Big +1. This seems like something we could add later on at any time when we
need it. Hopefully we can get Katello and our users involved in sussing out
the requirements for how this field should work too.

David


On Mon, Dec 3, 2018 at 11:34 PM Daniel Alley  wrote:

> *Background:*
>
> "Notes" are a generic key value store where data can be attached to
> repositories and content and publications and so forth.  The eventual plan
> is to use this to enable adding tags to those sorts of objects, which is
> important for Katello.
>
> Most of the code for this is located in pulp/app/models/generic.py
>
> *Motivation:*
>
> "Notes" have been in Pulp 3 for a very very long time and are completely
> unchanged for the last 12 months (the git history doesn't go back further
> because the file was moved).  The data model behind it is extremely complex
> and while we have a few unit tests around it, we have no functional tests
> for it whatsoever, and (to my knowledge) we haven't been using/exercising
> this functionality manually in a meaningful way (if at all).  I could be
> wrong here, but I haven't seen it discussed  or any issues related to it
> filed in quite some time.
>
> *Proposal:*
>
> We should pull out all of the "notes" code (models/generic.py + the fields
> on the aforementioned models) until we've had a chance to properly evaluate
> our needs and whether the current design fits them.
> ___
> Pulp-dev mailing list
> Pulp-dev@redhat.com
> https://www.redhat.com/mailman/listinfo/pulp-dev
>
___
Pulp-dev mailing list
Pulp-dev@redhat.com
https://www.redhat.com/mailman/listinfo/pulp-dev