[Pulp-dev] Setting naming conventions

[David Davis]

Background
=

We discovered that some of our settings in server.yaml all uppercase
(SERVER_KEY, MEDIA_ROOT, etc) while others are lowercase
(working_directory, databases, etc)[0]. We originally decided to go with
lowercase to match the yaml convention (although yaml itself doesn’t
prescribe all lowercase).

The problem we’ve now run into is that we’ve run into some technical
obstacles in dealing with stuff like nested dictionaries.


Solutions
===

1. Just use UPPER_SNAKE_CASE. This is probably the easiest solution.
2. Drop yaml for another config language like Python (which Django uses) or
toml[1]
3. Fix the problem of nested dictionaries so we can keep using
lower_snake_case.

Does anyone have a strong preference on which we go with?

[0] https://pulp.plan.io/issues/3589
[1] https://en.wikipedia.org/wiki/TOML

David
___
Pulp-dev mailing list
Pulp-dev@redhat.com
https://www.redhat.com/mailman/listinfo/pulp-dev

Re: [Pulp-dev] Pulp CLI MVP User Stories

[Dana Walker]

>
>
> The pulp-admin CLI provided value not because it combined things but
> because it managed auth and reduced syntax complexity.
>

Here's a (perhaps naive, please humor me) question--does it reduce syntax
complexity enough over httpie to bother with at all?  Now that we have a
Django app, would most users either a) be comfortable enough with the REST
API or b) prefer to see an actual Django web UI instead of the CLI?

Just curious about the typical users.

Dana Walker

Associate Software Engineer

Red Hat




On Fri, May 18, 2018 at 1:30 PM, Jeff Ortel  wrote:

> The main goal of the CLI is to make it easier (than using the REST API
> and http) for admins to perform routine tasks.  It seems likely that A
> CLI *could* provide an improvement by reducing the REST syntax complexity
> without combining the steps to complete a task.  Also, I think we should
> consider the frequency of tasks & steps.  For example: I would anticipate
> that work flows involving creating and deleting of repositories, remotes,
> publishers and distributions are performed with relatively low frequency.
> And, each of those will likely happen with different frequency.  Work flows
> involving sync & publish would be performed with relatively high
> frequency.  Updating resources (perhaps except Distribution) don't seem
> likely to be a regular thing.
>
> My point is: I think that providing high level, task oriented, combination
> CLI commands won't provide enough value to justify the effort.  An admin
> running (1) complex command (think pulp-admin) rather than 3-4 simple
> commands does not seem like an improvement.  Especially since the CLI would
> probably need to provide the simple commands a well for work flows not
> accounted for.  For example: Edit a remote.  Or, would the admin need to
> use the REST API for that?
>
> The pulp-admin CLI provided value not because it combined things but
> because it managed auth and reduced syntax complexity.
>
>
> On 05/17/2018 10:52 AM, Dennis Kliban wrote:
>
> The use cases we outlined earlier provide very little value over using
> httpie to interact with the REST API directly. I'd like to propose 5 new
> use cases:
>
>- As a CLI user, I can create a repository, a remote, a publisher, and
>a distribution with a single command.
>- As a CLI user, I can create a repository version, a publication, and
>update the distribution with a single command.
>- As a CLI user, I can list remote types available on the Pulp server.
>- As a CLI user, I can list publisher types available on the Pulp
>server.
>- As a CLI user, I can list all repositories available on the Pulp
>server.
>
>
> The use cases proposed at the beginning on this thread require the user to
> perform 4 steps before any content can be synced:
>
> 1) Create repository
> 2) Create remote
> 3) Create publisher
> 4) Create distribution
>
> The goal for the CLI should be to reduce this to a single step. The CLI
> will need to make some assumptions for the user: publisher name,
> distribution name, auto publish, auto distribute, and maybe others.
> However, this will allow the user to use a single command to create a
> repository that's ready for sync/publish.
>
> Sync/Publish/Distribute workflow can also be 3 steps:
>
> 1) Create a new repository version
> 2) Create a new publication
> 3) Update distribution
>
> The goal here is to also reduce this to a single step.
>
> The other use cases are auxiliary.
>
> Questions? Thoughts? Ideas?
>
> -Dennis
>
>
>
>
>
>
> On Mon, May 14, 2018 at 11:58 AM, Dana Walker  wrote:
>
>> +1
>>
>> Dana Walker
>>
>> Associate Software Engineer
>>
>> Red Hat
>>
>> 
>> 
>>
>> On Tue, May 8, 2018 at 10:31 AM, Jeremy Audet  wrote:
>>
>>> A configuration file in the user's home dir, right?
>

 Yes, exactly.

>>>
>>> Can we make sure to avoid placing configuration files directly in users
>>> home directories, and instead place them into directories like ~/.config?
>>> This is in line with the XDG Base Directory Specification
>>> .
>>> The spec is pretty straightforward, but Pulp Smash uses pyxdg
>>>  to avoid mistakes.
>>> There's two big benefits to doing this:
>>>
>>>- Less clutter in home directories.
>>>- Guidance on what to do with other types of files, such as cached
>>>files and runtime files.
>>>
>>> Projects such as git, htop, lftp, mpd, neovim, tmuxinator, boybo, and
>>> more do this.
>>>
>>> ___
>>> Pulp-dev mailing list
>>> Pulp-dev@redhat.com
>>> https://www.redhat.com/mailman/listinfo/pulp-dev
>>>
>>>
>>
>> ___
>> Pulp-dev mailing list
>> Pulp-dev@redhat.com
>> 

Re: [Pulp-dev] Pulp CLI MVP User Stories

[Jeff Ortel]

The main goal of the CLI is to make it easier (than using the REST API 
and http) for admins to perform routine tasks.  It seems likely that A 
CLI /could/ provide an improvement by reducing the REST syntax 
complexity without combining the steps to complete a task.  Also, I 
think we should consider the frequency of tasks & steps.  For example: I 
would anticipate that work flows involving creating and deleting of 
repositories, remotes, publishers and distributions are performed with 
relatively low frequency.  And, each of those will likely happen with 
different frequency.  Work flows involving sync & publish would be 
performed with relatively high frequency. Updating resources (perhaps 
except Distribution) don't seem likely to be a regular thing.


My point is: I think that providing high level, task oriented, 
combination CLI commands won't provide enough value to justify the 
effort.  An admin running (1) complex command (think pulp-admin) rather 
than 3-4 simple commands does not seem like an improvement.  Especially 
since the CLI would probably need to provide the simple commands a well 
for work flows not accounted for.  For example: Edit a remote.  Or, 
would the admin need to use the REST API for that?


The pulp-admin CLI provided value not because it combined things but 
because it managed auth and reduced syntax complexity.



On 05/17/2018 10:52 AM, Dennis Kliban wrote:
The use cases we outlined earlier provide very little value over using 
httpie to interact with the REST API directly. I'd like to propose 5 
new use cases:


  * As a CLI user, I can create a repository, a remote, a publisher,
and a distribution with a single command.
  * As a CLI user, I can create a repository version, a publication,
and update the distribution with a single command.
  * As a CLI user, I can list remote types available on the Pulp server.
  * As a CLI user, I can list publisher types available on the Pulp
server.
  * As a CLI user, I can list all repositories available on the Pulp
server.


The use cases proposed at the beginning on this thread require the 
user to perform 4 steps before any content can be synced:


1) Create repository
2) Create remote
3) Create publisher
4) Create distribution

The goal for the CLI should be to reduce this to a single step. The 
CLI will need to make some assumptions for the user: publisher name, 
distribution name, auto publish, auto distribute, and maybe others. 
However, this will allow the user to use a single command to create a 
repository that's ready for sync/publish.


Sync/Publish/Distribute workflow can also be 3 steps:

1) Create a new repository version
2) Create a new publication
3) Update distribution

The goal here is to also reduce this to a single step.

The other use cases are auxiliary.

Questions? Thoughts? Ideas?

-Dennis






On Mon, May 14, 2018 at 11:58 AM, Dana Walker > wrote:


+1

Dana Walker

Associate Software Engineer

Red Hat






On Tue, May 8, 2018 at 10:31 AM, Jeremy Audet > wrote:

A configuration file in the user's home dir, right?


Yes, exactly.


Can we make sure to avoid placing configuration files directly
in users home directories, and instead place them into
directories like ~/.config? This is in line with the XDG Base
Directory Specification

.
The spec is pretty straightforward, but Pulp Smash uses pyxdg
 to avoid
mistakes. There's two big benefits to doing this:

  * Less clutter in home directories.
  * Guidance on what to do with other types of files, such as
cached files and runtime files.

Projects such as git, htop, lftp, mpd, neovim, tmuxinator,
boybo, and more do this.


___
Pulp-dev mailing list
Pulp-dev@redhat.com 
https://www.redhat.com/mailman/listinfo/pulp-dev




___
Pulp-dev mailing list
Pulp-dev@redhat.com 
https://www.redhat.com/mailman/listinfo/pulp-dev





___
Pulp-dev mailing list
Pulp-dev@redhat.com
https://www.redhat.com/mailman/listinfo/pulp-dev


___
Pulp-dev mailing list
Pulp-dev@redhat.com
https://www.redhat.com/mailman/listinfo/pulp-dev

Re: [Pulp-dev] Pulp CLI MVP User Stories

[Milan Kovacik]

On Thu, May 17, 2018 at 8:14 PM, David Davis  wrote:

> Overall, I am hesitant to start with complex CLI commands. Instead I had
> hoped to start with basic, auto-generated CLI commands which we could then
> build off to add more complex commands in the future.
>

+1 ideally the CLI would provide some "mutable, high-level combinations" of
the basic CLI commands later on e.g by giving the user macros they could
reassemble on the CLI; considering the lisp-like Hylang[1][2][3] as an
example macro language:

  $ pulp-admin --eval '(create (repo-by-url  foo http://zoo.repo/ ))'
  {'repo': {'remote': {'name': 'zoo-remote', 'id': 1, 'url': '
http://zoo.repo'}, 'name': 'foo', 'id': 1}

could as well be specified by:

  $ pulp-admin --eval '(create (remote [[name hy_zoo] [url
http://zoo.repo/]]))'
  {'remote': {'name': 'Hy_zoo', 'url': 'http://zoo.repo', 'id': 2}}

  $ pulp-admin --eval '(create (repo [[name foo] [remote_id 2]]))'
  {'repo': {'remote': {'name': 'Hy_zoo', 'url': 'http://zoo.repo', 'id':
2}, 'name': 'foo', 'id': 2}}

or equally by:

  $ pulp-admin --eval '(create (repo [[name foo]] {'remote' {'name'
'Hy_zoo' 'url' 'http://zoo.repo' 'id' 3}}))'
   {'repo': {'remote': {'name': 'Hy_zoo', 'url': 'http://zoo.repo', 'id':
3}, 'name': 'foo', 'id': 3}}


or as well by:

  pulp-admin --eval '((->) (create (remote [[name Hy_zoo] [url
http://zoo.repo]]) (create (repo [name foo] ))'

so that the user is in control of the process rather than the converse ;)

--
milan

[1] http://docs.hylang.org/en/stable/index.html
[2] http://docs.hylang.org/en/stable/tutorial.html#macros
[3] http://docs.hylang.org/en/stable/tutorial.html#protips

PS: I'm not totally Hy right now ;)


> Questions/responses inline though.
>
> On Thu, May 17, 2018 at 11:52 AM, Dennis Kliban  wr
> ote:
>
>> The use cases we outlined earlier provide very little value over using
>> httpie to interact with the REST API directly.
>>
>
> I think this may be true for the first iteration of the CLI, but
> successive iterations will handle more complex use cases.
>

>
>> I'd like to propose 5 new use cases:
>>
>>- As a CLI user, I can create a repository, a remote, a publisher,
>>and a distribution with a single command.
>>- As a CLI user, I can create a repository version, a publication,
>>and update the distribution with a single command.
>>- As a CLI user, I can list remote types available on the Pulp server.
>>- As a CLI user, I can list publisher types available on the Pulp
>>server.
>>- As a CLI user, I can list all repositories available on the Pulp
>>server.
>>
>> How would users do things like update objects? Like what if I want to
> edit the URL I am syncing a repo from?
>
>
>>
>> The use cases proposed at the beginning on this thread require the user
>> to perform 4 steps before any content can be synced:
>>
>> 1) Create repository
>> 2) Create remote
>> 3) Create publisher
>> 4) Create distribution
>>
>> The goal for the CLI should be to reduce this to a single step. The CLI
>> will need to make some assumptions for the user: publisher name,
>> distribution name, auto publish, auto distribute, and maybe others.
>> However, this will allow the user to use a single command to create a
>> repository that's ready for sync/publish.
>>
>
> I think this is very brittle given the mutability of object names. Users
> couldn’t use both the CLI and REST API. They’d have to pick or choose or
> else run the risk of disconnecting objects.
>

  +1; we'd have to assume certain object name "shapes" such as
'repository-{name}'.format(name='foo')


>
> Also, how do we support things in the future like multiple remotes for a
> repo or shared remotes between repos?
>
>
 +1


>
>> Sync/Publish/Distribute workflow can also be 3 steps:
>>
>> 1) Create a new repository version
>> 2) Create a new publication
>> 3) Update distribution
>>
>> The goal here is to also reduce this to a single step.
>>
>> The other use cases are auxiliary.
>>
>> Questions? Thoughts? Ideas?
>>
>> -Dennis
>>
>>
>>
>>
>>
> A couple more questions:
>
> How would we handle things in the future that are plugin-specific like
> content upload/creation?
>
> Also, would this CLI use auto-generation at all? Or would it rely entirely
> on custom code? I assume the latter which means this would be similar to
> the Pulp 2 CLI in which every new API change required a change to the CLI
> code.
>
>
> David
>
> On Thu, May 17, 2018 at 11:52 AM, Dennis Kliban 
> wrote:
>
>> The use cases we outlined earlier provide very little value over using
>> httpie to interact with the REST API directly. I'd like to propose 5 new
>> use cases:
>>
>>- As a CLI user, I can create a repository, a remote, a publisher,
>>and a distribution with a single command.
>>- As a CLI user, I can create a repository version, a publication,
>>and update the distribution with a single command.
>>- As a CLI user, I can list