Re: Python package providing both modules and an app

[Thomas Goirand]

On 07/05/2016 04:47 PM, Hugo Lefeuvre wrote:
> Hi Thomas,
> 
>> I did lots of that in OpenStack. You can have a look at one of our tool
>> to handle the dh_auto_install for Python:
>>
>> http://anonscm.debian.org/cgit/openstack/openstack-pkg-tools.git/tree/build-tools/pkgos-dh_auto_install
>>
>> The way to use it in your debian/rules file is to do:
>> pkgos-alternative slugify slugify
>>
>> [ this tool is available at:
>> http://anonscm.debian.org/cgit/openstack/openstack-pkg-tools.git/tree/misc/pkgos-alternative-bin
>> ]
>>
>> then simply do:
>>
>> override_dh_auto_install:
>>  pkgos-dh_auto_install
>>
>> and that's it, it will handle the /usr/bin/python{3,}-slugify
>> alternative implementations for you.
> 
> That looks very interesting ! 
> 
> However, I have to say I'm a bit reluctant to make the package depend on a 
> supplementary Build-Dependency only in order to prettify debian/rules.

You don't have to do that. You can open pkgos-dh_auto_install, see what
it does, and pickup ideas there. It's only 10 lines of shell scripts,
and you probably don't even need everything that's inside.

Cheers,

Thomas Goirand (zigo)

Re: Python package providing both modules and an app

[Thomas Goirand]

On 06/22/2016 09:51 PM, Hugo Lefeuvre wrote:
> Hi Ben,
> 
>>> The package is originally requested as a Python module[2] and it seems 
>>> clear to me that the whole thing is only useful as a library
>>
>> I don't understand this statement. If it is *only* useful as a library,
>> why install the command-line entry point?
> 
> Indeed, I though about that, but python-slugify has been developed as an 
> application[0] so I think it's rather a bad idea to completely ignore the 
> entry point, even if it's not really useful. I think upstream would disagree, 
> too. In my opinion, better build a third package, even if it won't be used
> a lot.

The 3rd package is mandatory if your package needs to store config files.

Cheers,

Thomas

Re: Python package providing both modules and an app

[Thomas Goirand]

On 06/21/2016 11:13 PM, Hugo Lefeuvre wrote:
> Hi all !
> 
> I'm currently packaging python-slugify[0], a Python application for
> slugifying unicode strings. Like here[1] upstream decided to write 
> everything as a python module with an entry script placed in /usr/bin
> that calls main(). So we can consider python-slugify as both an application
> and a library.
> 
> The package is originally requested as a Python module[2] and it seems 
> clear to me that the whole thing is only useful as a library, so I've 
> named the source package 'python-slugify'. It provides following packages: 
>  * 'python-slugify' (Python 2 lib), 
>  * 'python3-slugify' (Python 3 lib),
>  * 'slugify' (ships the entry script[3], depends on python3-slugify). 
> 
> Everything is working well. However, I'm not really satisfied by the way I 
> solved the problem in debian/rules[4]. Does anybody know a better way 
> of doing that ?
> 
> Cheers,
>  Hugo

Hugo,

I did lots of that in OpenStack. You can have a look at one of our tool
to handle the dh_auto_install for Python:

http://anonscm.debian.org/cgit/openstack/openstack-pkg-tools.git/tree/build-tools/pkgos-dh_auto_install

The way to use it in your debian/rules file is to do:
pkgos-alternative slugify slugify

[ this tool is available at:
http://anonscm.debian.org/cgit/openstack/openstack-pkg-tools.git/tree/misc/pkgos-alternative-bin
]

then simply do:

override_dh_auto_install:
pkgos-dh_auto_install

and that's it, it will handle the /usr/bin/python{3,}-slugify
alternative implementations for you.

I hope that helps,
Cheers,

Thomas Goirand (zigo)

Re: Python package providing both modules and an app

[Hugo Lefeuvre]

> So the question would rather be: how can I properly ask pybuild (setup.py ?) 
> to install the entry point in the application package (slugify) instead of 
> installing it in the library packages (python{3}-slugify) ?

So, if nobody knows, I think I'll upload the package as is.

Thanks anyway ! :-)
 Hugo

-- 
 Hugo Lefeuvre (hle)|www.owl.eu.com
4096/ ACB7 B67F 197F 9B32 1533 431C AC90 AC3E C524 065E


signature.asc
Description: PGP signature

Re: Python program as a command-line program (was: Python package providing both modules and an app)

[Barry Warsaw]

On Jun 23, 2016, at 11:17 AM, Ben Finney wrote:

>There isn't, AFAIK, anything portable that I can write in the shebang to
>turn a command invocation of ‘./foo/bar.py’ into ‘python3 -m foo.bar’.

Why not just have a file that contains only?

exec python3 -m foo.bar

Cheers,
-Barry

Re: Python package providing both modules and an app

[Hugo Lefeuvre]

> > The entry point is generated by 'setup.py install_scripts', which is
> > executed by pybuild after files specified in *.install have been
> > installed (correct me if I'm wrong).
> 
> I haven't demonstrated for this case, but I would be surprised if that's
> how it behaves. To my mind the ‘setup.py install’ command (and hence the
> ‘… install_scripts’ command) should be part of the ‘dh build’ sequence.
> 
> Again, someone should correct me if I'm wrong.

Hum, the build logs suggest that ‘setup.py install’ is ran during the 
‘dh binary’/‘dh_auto_install’ sequence. Concerning the order of execution 
in dh_auto_install, you're true: In fact, the files specified in 
debian/*.install are installed after ‘setup.py install’ have been executed. 
However, it seems that setup.py is directly installing the entry point without 
storing a copy of it in the current directory, so anyway I don't see how I 
could install the entry point in the application package via a debian/install 
file[0].

So the question would rather be: how can I properly ask pybuild (setup.py ?) 
to install the entry point in the application package (slugify) instead of 
installing it in the library packages (python{3}-slugify) ?

[0] and, even if I could do it, I think I would still have to override
dh_auto_install to remove the installed entry points from the library packages,
which I originally wanted to avoid. :)

-- 
 Hugo Lefeuvre (hle)|www.owl.eu.com
4096/ ACB7 B67F 197F 9B32 1533 431C AC90 AC3E C524 065E


signature.asc
Description: PGP signature

Re: Python package providing both modules and an app

[Ian Cordasco]

On Jun 22, 2016 7:54 PM, "Barry Warsaw"  wrote:
>
> On Jun 22, 2016, at 11:25 AM, Ben Finney wrote:
>
> >This seems to be more common now that command-line invocation is
> >becoming even more discouraged. When the upstream documentation
> >recommends ‘python3 -m foo.bar’ as the primary means of invoking the
> >command line functionality, that really blurs the line between
> >command-line application versus library.
>
> Indeed.  It's true that -m invocation isn't as pretty as a /usr/bin
script,
> but it does have the advantage of unambiguously choosing the Python
version
> you want to run the script with.  How important that is depends on the
> application of course.
>
> >There is a compounding tendency to disparage ‘python3 ./foo/bar.py’,
> >which is subject to weird hacks and incomplete workarounds like
> >. I wish this trend could
> >be effectively reversed, because IMO this is a serious impediment to
> >considering Python a good choice for command-line scripting. But this is
> >all a digression, my apologies.
>
> Sorry, I don't quite understand the above.  Do you mean that you would
rather
> use `python3 ./foo/bar.py` more often and `python3 -m foo.bar` less often?
>
> In any case, thanks Hugo for choosing Python 3 as the version to use for
the
> /usr/bin script. :)

Speaking with my application developer hat on, the latter invocation
ideally helps the user be more aware of what version of Python they're
using.

This is beneficial in a couple cases:

- pip installing something: it's important for the user to know which
version of Python they're using

- running flake8 against a codebase. Flake8 relies on the version of
Python's AST for certain things. If the user is writing code with async
keywords but running Flake8 on Python 2.7, then they'll be in for a world
of unpleasantness.

By extension the same logic applies to any application that supports
multiple versions of Python but depends on things like the stdlib AST
module.

It's certainly a hideous invocation, but it has it's benefits.

Cheers,
Ian

Python program as a command-line program (was: Python package providing both modules and an app)

[Ben Finney]

Barry Warsaw  writes:

> On Jun 22, 2016, at 11:25 AM, Ben Finney wrote:
>
> >There is a compounding tendency to disparage ‘python3 ./foo/bar.py’,
> >[…]. I wish this trend could be effectively reversed, because IMO
> >this is a serious impediment to considering Python a good choice for
> >command-line scripting.
>
> Sorry, I don't quite understand the above. Do you mean that you would
> rather use `python3 ./foo/bar.py` more often and `python3 -m foo.bar`
> less often?

When ‘./foo/bar.py’ has a shebang of ‘#! /usr/bin/python3’, and I invoke
that path as a command, the Unix shebang semantics turn that into
‘/usr/bin/python3 ./foo/bar.py’. Python will run the file as the main
module.

But invoking it that way runs into the nastiness and constraints
described in PEP 366 (primarily, relative imports break).

There isn't, AFAIK, anything portable that I can write in the shebang to
turn a command invocation of ‘./foo/bar.py’ into ‘python3 -m foo.bar’.

So yes, I expect Unix shebang semantics to work, and the fact that
design decisions in Python's import mechanism defeat that mode is an
impediment to using Python for writing command-line programs.

-- 
 \  “Dvorak users of the world flgkd!” —Kirsten Chevalier, |
  `\rec.humor.oracle.d |
_o__)  |
Ben Finney

Re: Python package providing both modules and an app

[Ben Finney]

Hugo Lefeuvre  writes:

> The entry point is generated by 'setup.py install_scripts', which is
> executed by pybuild after files specified in *.install have been
> installed (correct me if I'm wrong).

I haven't demonstrated for this case, but I would be surprised if that's
how it behaves. To my mind the ‘setup.py install’ command (and hence the
‘… install_scripts’ command) should be part of the ‘dh build’ sequence.

Again, someone should correct me if I'm wrong.

-- 
 \  “Holy polar ice sheet, Batman!” —Robin |
  `\   |
_o__)  |
Ben Finney

Re: Python package providing both modules and an app

[Barry Warsaw]

On Jun 22, 2016, at 11:25 AM, Ben Finney wrote:

>This seems to be more common now that command-line invocation is
>becoming even more discouraged. When the upstream documentation
>recommends ‘python3 -m foo.bar’ as the primary means of invoking the
>command line functionality, that really blurs the line between
>command-line application versus library.

Indeed.  It's true that -m invocation isn't as pretty as a /usr/bin script,
but it does have the advantage of unambiguously choosing the Python version
you want to run the script with.  How important that is depends on the
application of course.

>There is a compounding tendency to disparage ‘python3 ./foo/bar.py’,
>which is subject to weird hacks and incomplete workarounds like
>. I wish this trend could
>be effectively reversed, because IMO this is a serious impediment to
>considering Python a good choice for command-line scripting. But this is
>all a digression, my apologies.

Sorry, I don't quite understand the above.  Do you mean that you would rather
use `python3 ./foo/bar.py` more often and `python3 -m foo.bar` less often?

In any case, thanks Hugo for choosing Python 3 as the version to use for the
/usr/bin script. :)

Cheers,
-Barry

Re: Python package providing both modules and an app

[Hugo Lefeuvre]

Hi Ben,

> > The package is originally requested as a Python module[2] and it seems 
> > clear to me that the whole thing is only useful as a library
> 
> I don't understand this statement. If it is *only* useful as a library,
> why install the command-line entry point?

Indeed, I though about that, but python-slugify has been developed as an 
application[0] so I think it's rather a bad idea to completely ignore the 
entry point, even if it's not really useful. I think upstream would disagree, 
too. In my opinion, better build a third package, even if it won't be used
a lot.

> Why not install the command by using the configuration files for
> ‘dh_install(1)’; that is, specify in ‘debian/slugify.install’ the files
> to install.

The entry point is generated by 'setup.py install_scripts', which is executed
by pybuild after files specified in *.install have been installed (correct me 
if I'm wrong). So, I think that what you suggest won't work unless I write my 
own entry point, install it via debian/slugify.install and ignore those 
generated 
by setup.py. I think this would be even dirtier than what I already done. ;-)

[0] https://github.com/un33k/python-slugify

-- 
 Hugo Lefeuvre (hle)|www.owl.eu.com
4096/ ACB7 B67F 197F 9B32 1533 431C AC90 AC3E C524 065E


signature.asc
Description: PGP signature

Re: Python package providing both modules and an app

[Ben Finney]

Hugo Lefeuvre  writes:

> I'm currently packaging python-slugify[0], a Python application for
> slugifying unicode strings.

Thank you for considering the issues and asking for advice.

> Like here[1] upstream decided to write everything as a python module
> with an entry script placed in /usr/bin that calls main(). So we can
> consider python-slugify as both an application and a library.

This seems to be more common now that command-line invocation is
becoming even more discouraged. When the upstream documentation
recommends ‘python3 -m foo.bar’ as the primary means of invoking the
command line functionality, that really blurs the line between
command-line application versus library.

There is a compounding tendency to disparage ‘python3 ./foo/bar.py’,
which is subject to weird hacks and incomplete workarounds like
. I wish this trend could
be effectively reversed, because IMO this is a serious impediment to
considering Python a good choice for command-line scripting. But this is
all a digression, my apologies.

> The package is originally requested as a Python module[2] and it seems 
> clear to me that the whole thing is only useful as a library

I don't understand this statement. If it is *only* useful as a library,
why install the command-line entry point?

> Everything is working well. However, I'm not really satisfied by the way I 
> solved the problem in debian/rules[4]. Does anybody know a better way 
> of doing that ?

Why not install the command by using the configuration files for
‘dh_install(1)’; that is, specify in ‘debian/slugify.install’ the files
to install.

-- 
 \ “If you can do no good, at least do no harm.” —_Slapstick_, |
  `\ Kurt Vonnegut |
_o__)  |
Ben Finney