Re: Separate lib and executable packages?
On Sat, Feb 08, 2020 at 08:54:27PM +0100, Ondrej Novy wrote:
> Hi,
>
> so 8. 2. 2020 v 20:51 odesílatel Gordon Ball napsal:
>
> > Perhaps this is worth making an explicit recommendation for new packages
> > of this type, given that anything new _should_ be python3-only and we
> >
>
> and what about pypy- prefix?
That's a good point.
Probably not many packages are likely to provide pypy{,3} invoking binaries
(ipython is probably actually a good candidate here) and so it probably
counts as an exceptional case which can reasonably bypass a recommendation.
I suppose you could:
* Not ship any executables which invoke pypy. For pypy3, that appears
to be the case today (nothing in /usr/bin using pypy3 except the
interpreter itself).
* Ship the library and python3 executable together (possibly with a
Provides: for the executable), and the pypy3 executable in a
separate package (since the library itself presumably
doesn't want to depend on pypy3, and might need to depend on
pypy-specific library packages). This saves 1 binary package but
seems inconsistent. The actual implementation could be picked with
update-alternatives.
* Ship library, python3 executable and pypy3 executable separately.
This seems more consistent but generates an extra binary package.
This is probably a good case to consider in this thread because ipython
and ipykernel are probably reasonable cases where a pypy-specialised
executable might make sense, and ipython at least depends only on
pure-python modules which work out-of-the-box with pypy.
>
> --
> Best regards
> Ondřej Nový
Re: Separate lib and executable packages?
On Saturday, February 8, 2020 2:50:54 PM EST Gordon Ball wrote: > On Sat, Feb 08, 2020 at 02:42:11AM +, Scott Kitterman wrote: > > On February 7, 2020 3:59:46 PM UTC, Mattia Rizzolo wrote: > > >On Fri, Feb 07, 2020 at 11:36:00AM +, Gordon Ball wrote: > > >> I wonder if this split really makes sense; it feels like adding the > > >> overhead of an extra binary package to avoid not having a very small > > >> file in /usr/bin if you're only planning to use the library. > > >> > > >> Does it seem reasonable to drop the executable package and just make > > > > > >it > > > > > >> a Provides: of the library package? (and is there any potential > > > > > >breakage > > > > > >> here that I'm overlooking) > > > > > >Maybe not for ipython3, since that's very much tied to > > >python3-ipython3. > > > > > >BUT, as a user (even forgetting I'm also a DD) I was hurt many times by > > >executables in python-foo but wanting to use python3-foo instead, or by > > >executables that moved from python-foo to python3-foo and I had to fix > > >my own deployments, and whatnot. > > > > > >We are not going to have a python4 anytime soon (hopefully never), but > > >I > > >think that keeping a separate binary package makes sense for almost all > > >cases I can think of packages shipping executables under /usr/bin/. > > > > Except: Every time we add a binary to Debian it impacts everyone. The > > packages file gets bigger and so updates get slower. Although there's no > > firm rule, the FTP Masters discourage addition of trivially small > > packages and so your package might be rejected. > Perhaps this is worth making an explicit recommendation for new packages > of this type, given that anything new _should_ be python3-only and we > hope for at least some time before another python major version becomes > an issue. "Packages which are primarily libraries but include an > executable should avoid multiple binary packages without good reason"? - > At least in the above linked style guide, since it's not normative > policy in any case. Sounds reasonable to me. It's a wiki, so I'd say go for it. > For packages already split (eg, ipython3, python3-ipython), having > thought a bit more I suspect it's probably more trouble than it is worth > to drop a handful of binary packages - I'm wondering about eg, what > happens if the executable package is manually installed and the library > hence flagged as automatically installed; if the former than becomes a > Provides: of the latter, is it immediately a candidate for autoremoval? Yes. Removing the existing cases where it might not really be needed can be problematic. I think it generally makes sense to leave them be. Scott K signature.asc Description: This is a digitally signed message part.
Re: Separate lib and executable packages?
Hi, so 8. 2. 2020 v 20:51 odesílatel Gordon Ball napsal: > Perhaps this is worth making an explicit recommendation for new packages > of this type, given that anything new _should_ be python3-only and we > and what about pypy- prefix? -- Best regards Ondřej Nový
Re: Separate lib and executable packages?
On Sat, Feb 08, 2020 at 02:42:11AM +, Scott Kitterman wrote: > > > On February 7, 2020 3:59:46 PM UTC, Mattia Rizzolo wrote: > >On Fri, Feb 07, 2020 at 11:36:00AM +, Gordon Ball wrote: > >> I wonder if this split really makes sense; it feels like adding the > >> overhead of an extra binary package to avoid not having a very small > >> file in /usr/bin if you're only planning to use the library. > >> > >> Does it seem reasonable to drop the executable package and just make > >it > >> a Provides: of the library package? (and is there any potential > >breakage > >> here that I'm overlooking) > > > > > >Maybe not for ipython3, since that's very much tied to > >python3-ipython3. > > > >BUT, as a user (even forgetting I'm also a DD) I was hurt many times by > >executables in python-foo but wanting to use python3-foo instead, or by > >executables that moved from python-foo to python3-foo and I had to fix > >my own deployments, and whatnot. > > > >We are not going to have a python4 anytime soon (hopefully never), but > >I > >think that keeping a separate binary package makes sense for almost all > >cases I can think of packages shipping executables under /usr/bin/. > > Except: Every time we add a binary to Debian it impacts everyone. The > packages file gets bigger and so updates get slower. Although there's no > firm rule, the FTP Masters discourage addition of trivially small packages > and so your package might be rejected. > Perhaps this is worth making an explicit recommendation for new packages of this type, given that anything new _should_ be python3-only and we hope for at least some time before another python major version becomes an issue. "Packages which are primarily libraries but include an executable should avoid multiple binary packages without good reason"? - At least in the above linked style guide, since it's not normative policy in any case. For packages already split (eg, ipython3, python3-ipython), having thought a bit more I suspect it's probably more trouble than it is worth to drop a handful of binary packages - I'm wondering about eg, what happens if the executable package is manually installed and the library hence flagged as automatically installed; if the former than becomes a Provides: of the latter, is it immediately a candidate for autoremoval? > As an example, the dkimpy module that I maintain provides some scripts. > Originally they were provided in python-dkim. Now they are provided in > python3-dkim (and since python-dkim is gone in unstable/testing, there's no > more chance of users being confused about which one to install to get the > scripts). > > I recall a few questions by people that didn't know where to find the > scripts. I could have had both packages provide it using > update-alternatives, but the problem never seemed severe enough to be worth > the work. > > While splitting things out as suggested makes sense from the perspective of a > single package, it has negative implications for the distro as a whole. > Maintainers: please be mindful of the cumulative impact of the addition many > small packages that aren't strictly needed. > > Scott K >
Re: Separate lib and executable packages?
On February 8, 2020 12:38:54 PM UTC, Ondrej Novy wrote: >Hi, > >so 8. 2. 2020 v 4:11 odesílatel Scott Kitterman >napsal: > >> Except: Every time we add a binary to Debian it impacts everyone. >The >> packages file gets bigger and so updates get slower. Although >there's no >> firm rule, the FTP Masters discourage addition of trivially small >packages >> and so your package might be rejected. >> > >so let's fix infrastructure? That's what everyone always suggests, but no one ever does. AFAIK, there's no agreed design for secure package updates that doesn't require a local copy of the packages file. If Debian could quit worrying about adding lots of little binaries, it would help in many areas. Scott K
Re: Separate lib and executable packages?
Hi, so 8. 2. 2020 v 4:11 odesílatel Scott Kitterman napsal: > Except: Every time we add a binary to Debian it impacts everyone. The > packages file gets bigger and so updates get slower. Although there's no > firm rule, the FTP Masters discourage addition of trivially small packages > and so your package might be rejected. > so let's fix infrastructure? -- Best regards Ondřej Nový
Re: Separate lib and executable packages?
On February 7, 2020 3:59:46 PM UTC, Mattia Rizzolo wrote: >On Fri, Feb 07, 2020 at 11:36:00AM +, Gordon Ball wrote: >> I wonder if this split really makes sense; it feels like adding the >> overhead of an extra binary package to avoid not having a very small >> file in /usr/bin if you're only planning to use the library. >> >> Does it seem reasonable to drop the executable package and just make >it >> a Provides: of the library package? (and is there any potential >breakage >> here that I'm overlooking) > > >Maybe not for ipython3, since that's very much tied to >python3-ipython3. > >BUT, as a user (even forgetting I'm also a DD) I was hurt many times by >executables in python-foo but wanting to use python3-foo instead, or by >executables that moved from python-foo to python3-foo and I had to fix >my own deployments, and whatnot. > >We are not going to have a python4 anytime soon (hopefully never), but >I >think that keeping a separate binary package makes sense for almost all >cases I can think of packages shipping executables under /usr/bin/. Except: Every time we add a binary to Debian it impacts everyone. The packages file gets bigger and so updates get slower. Although there's no firm rule, the FTP Masters discourage addition of trivially small packages and so your package might be rejected. As an example, the dkimpy module that I maintain provides some scripts. Originally they were provided in python-dkim. Now they are provided in python3-dkim (and since python-dkim is gone in unstable/testing, there's no more chance of users being confused about which one to install to get the scripts). I recall a few questions by people that didn't know where to find the scripts. I could have had both packages provide it using update-alternatives, but the problem never seemed severe enough to be worth the work. While splitting things out as suggested makes sense from the perspective of a single package, it has negative implications for the distro as a whole. Maintainers: please be mindful of the cumulative impact of the addition many small packages that aren't strictly needed. Scott K
Re: Separate lib and executable packages?
On Fri, Feb 07, 2020 at 11:36:00AM +, Gordon Ball wrote: > I wonder if this split really makes sense; it feels like adding the > overhead of an extra binary package to avoid not having a very small > file in /usr/bin if you're only planning to use the library. > > Does it seem reasonable to drop the executable package and just make it > a Provides: of the library package? (and is there any potential breakage > here that I'm overlooking) Maybe not for ipython3, since that's very much tied to python3-ipython3. BUT, as a user (even forgetting I'm also a DD) I was hurt many times by executables in python-foo but wanting to use python3-foo instead, or by executables that moved from python-foo to python3-foo and I had to fix my own deployments, and whatnot. We are not going to have a python4 anytime soon (hopefully never), but I think that keeping a separate binary package makes sense for almost all cases I can think of packages shipping executables under /usr/bin/. -- regards, Mattia Rizzolo GPG Key: 66AE 2B4A FCCF 3F52 DA18 4D18 4B04 3FCD B944 4540 .''`. More about me: https://mapreri.org : :' : Launchpad user: https://launchpad.net/~mapreri `. `'` Debian QA page: https://qa.debian.org/developer.php?login=mattia `- signature.asc Description: PGP signature
Re: Separate lib and executable packages?
Hi, it depends :). My two cents: If you are packaging application (PAPT team): * user is using it in CLI / Xorg * user don't need to know it's written in Python * Python modules are typically private, not usable by other applications/libraries -> one binary package: foo If you are packaging library (DPMT team): * your package is typically not usable by user directly * your package is typically dependency of other packages (Python applications/libraries) -> as many binary packages as "language branches" we have, e.g. python-foo, python3-foo, pypy-foo If you are packaging application with public library part (PAPT team): * both of above * user is using it in CLI / Xorg * BUT package have library part, which can be used in other Python applications/libraries -> as many binary packages as "language branches" we have, e.g. python-foo, python3-foo, pypy-foo + foo for application. "foo" package can contain more than entrypoints, but icons, menu items, ... -- Best regards Ondřej Nový
Separate lib and executable packages?
The library style guide [1] says under _Executables and library packages_: > Here are some recommendations. We do not have a standard (though maybe > we should) regarding whether a library with an associated executable should be split across `foo` and `python3-foo` or just bundled into `python3-foo`. Does anyone consider that there is now a standard either way? In eg, ipython this means shipping `ipython3` (containing a ~300 byte entrypoints script and a similarly sized man page) as a separate package to the implementation in `python3-ipython`. I wonder if this split really makes sense; it feels like adding the overhead of an extra binary package to avoid not having a very small file in /usr/bin if you're only planning to use the library. Does it seem reasonable to drop the executable package and just make it a Provides: of the library package? (and is there any potential breakage here that I'm overlooking) Gordon [1]: https://wiki.debian.org/Python/LibraryStyleGuide
