Re: [aur-general] Python packaging: to build or not to build (Re: Review request for 3 related PKGBUILDs)

[Eli Schwartz via aur-general]

On 02/14/2017 07:44 PM, Bruno Pagani wrote:
> So, if I understand things well, python{,2} setup.py build should do the
> same thing for most package, and any one of the two could be used to
> then package the two versions? So this would work:
> build() {
> python setup.py build
> }
> package_python-lib() {
> python setup.py install --root="$pkgdir" --optimize=1 --skip-build
> }
> package_python2-lib(){
> python2 setup.py install --root="$pkgdir" --optimize=1 --skip-build
> }
> 
> Then I would be more in favour of splitting build and package.
> 
> Thanks for your input,
> Bruno

Basically, yeah. Run the setuptools PKGBUILD and diff the srcdir --
you'll see they are practically byte-identical. There is a small handful
of 2/3 references, but setuptools rewrote those in the first place, and
other than that there's just __pycache__ vs side-by-side bytecode.

https://paste.xinu.at/aY2BmFZryz/

-- 
Eli Schwartz



signature.asc
Description: OpenPGP digital signature

Re: [aur-general] Python packaging: to build or not to build (Re: Review request for 3 related PKGBUILDs)

[Bruno Pagani via aur-general]

Le 15/02/2017 à 01:20, Eli Schwartz a écrit :

> On 02/14/2017 06:11 PM, Bruno Pagani wrote:
>> Le 03/01/2017 à 22:36, Eli Schwartz via aur-general a écrit :
>>
>>> On 01/03/2017 04:12 PM, Leonid Bloch wrote:
 Thanks! That was very helpful!

 All applied, except... "--skip-build" - indeed it makes sense, but I
 have never seen it with other Python packages. So I wonder if indeed it
 is a good practice, or is there some reason not to include it?
>>> Well, python-setuptools does it, but it doesn't seem to be very popular.
>>> Really, for Make-powered builds the dependencies for "install" are going
>>> to run anyway (but they were built during build() and usually do
>>> nothing, silently).
>>> Then again, a lot of python PKGBUILDs don't have a build() function at
>>> all, which means the package() function will invoke "build" itself.
>>> Apparently, there is an arcane difference between building a python
>>> module and compiling an ELF binary, but no one has told me what that
>>> difference may be... I don't usually pay attention to what other people
>>> do. :)
>>>
>>> It makes no difference whether you look at the repos or the AUR, both
>>> have people who do all three styles.
>>>
>>> The only practical difference would be if someone, say, ran `makepkg
>>> --nobuild && makepkg --repackage` on a VCS package, which they shouldn't.
>> I’m sort of reviving this thread, because I’ve stumbled upon discussions
>> around this recently, and in fact I see one practical reason of doing
>> the build in the package() function rather than in the build() one:
>>
>> – split python2-lib/python-lib package without build() function (only
>> the relevant parts):
>> package_python-lib() {
>> python setup.py install --prefix=/usr --root="$pkgdir" --optimize=1
>> }
>> package_python2-lib(){
>> python2 setup.py install --prefix=/usr --root="$pkgdir" --optimize=1
>> }
>>
>> – vs with build() function:
>> prepare() {
>> cp -a libname{,-py2}
>> }
>> build() {
>> cd libname
>> python setup.py build
>>
>> cd libname-py2
>> python2 setup.py build
>> }
>> package_python-lib() {
>> python setup.py install --prefix=/usr --root="$pkgdir" --optimize=1
>> --skip-build
>> }
>> package_python2-lib(){
>> python2 setup.py install --prefix=/usr --root="$pkgdir" --optimize=1
>> --skip-build
>> }
>>
>> Now, whether this is a good practice or not and what may be the
>> implications regarding makepkg, that’s a different thing. But it would
>> be good to agree on one way or another, and keep it the same everywhere
>> for consistency. If they are good reason to do it the second way, I’d
>> like to know them. Else, I have a tendency to find the shortest one to
>> be the best. ;)
> As I explained before, `python* setup.py install` "depends" on `python*
> setup.py build`, in much the same way `make install` generally depends
> on some target like `make all`.
>
> So, running `python setup.py install` in package_*() is explicitly
> identical in effect to running `python setup.py build && python setup.py
> install`, just like `make install` would, for the average Makefile, be
> identical to `make all && make install`.

Yes, but I know no split packages other than python ones for which
running make install alone serves any purpose.

> Therefore, the *only* question is whether you wish to run `python
> setup.py build` separately, in the build() function. But if you felt the
> need to make a python2 copy in prepare() before using build(), then you
> should also feel the need to make a python2 copy in prepare() when you
> *aren't* using a build() function.

OK, I admit having borrowed my example from setuptools since that’s the
only one I know (and that’s only thanks to you), I had no idea the copy
wasn’t necessary generally.

> The only difference it could make, and this is a difference that would
> apply equally to standard Makefile-powered packages as well, would be if
> you wanted to build only part of a split package, and therefore skip
> building it as well. And makepkg no longer supports building only part
> of a split package, so that concern would only be relevant a long time ago.
>
> ...
>
> As for making a python2 copy in the first place, that is not always
> necessary... any python package which includes a binary extension will
> get built in e.g. "build/lib.linux-i686-2.7/" vs.
> "build/lib.linux-i686-3.6/", whereas pure *.py packages will get built
> in "build/lib/".
>
> The latter don't actually have any python2/python3-specific differences
> anyway, usually. The recommended way to support both is, after all, to
> write polyglot code that doesn't care which version of python you use.
> Although setuptools *can* run 2to3 if the package requests it.

So, if I understand things well, python{,2} setup.py build should do the
same thing for most package, and any one of the two could be used to
then package the two versions? So this would work:
build() {
python setup.py build
}

Re: [aur-general] Python packaging: to build or not to build (Re: Review request for 3 related PKGBUILDs)

[Eli Schwartz via aur-general]

On 02/14/2017 06:11 PM, Bruno Pagani wrote:
> Le 03/01/2017 à 22:36, Eli Schwartz via aur-general a écrit :
> 
>> On 01/03/2017 04:12 PM, Leonid Bloch wrote:
>>> Thanks! That was very helpful!
>>>
>>> All applied, except... "--skip-build" - indeed it makes sense, but I
>>> have never seen it with other Python packages. So I wonder if indeed it
>>> is a good practice, or is there some reason not to include it?
>> Well, python-setuptools does it, but it doesn't seem to be very popular.
>> Really, for Make-powered builds the dependencies for "install" are going
>> to run anyway (but they were built during build() and usually do
>> nothing, silently).
>> Then again, a lot of python PKGBUILDs don't have a build() function at
>> all, which means the package() function will invoke "build" itself.
>> Apparently, there is an arcane difference between building a python
>> module and compiling an ELF binary, but no one has told me what that
>> difference may be... I don't usually pay attention to what other people
>> do. :)
>>
>> It makes no difference whether you look at the repos or the AUR, both
>> have people who do all three styles.
>>
>> The only practical difference would be if someone, say, ran `makepkg
>> --nobuild && makepkg --repackage` on a VCS package, which they shouldn't.
> 
> I’m sort of reviving this thread, because I’ve stumbled upon discussions
> around this recently, and in fact I see one practical reason of doing
> the build in the package() function rather than in the build() one:
> 
> – split python2-lib/python-lib package without build() function (only
> the relevant parts):
> package_python-lib() {
> python setup.py install --prefix=/usr --root="$pkgdir" --optimize=1
> }
> package_python2-lib(){
> python2 setup.py install --prefix=/usr --root="$pkgdir" --optimize=1
> }
> 
> – vs with build() function:
> prepare() {
> cp -a libname{,-py2}
> }
> build() {
> cd libname
> python setup.py build
> 
> cd libname-py2
> python2 setup.py build
> }
> package_python-lib() {
> python setup.py install --prefix=/usr --root="$pkgdir" --optimize=1
> --skip-build
> }
> package_python2-lib(){
> python2 setup.py install --prefix=/usr --root="$pkgdir" --optimize=1
> --skip-build
> }
> 
> Now, whether this is a good practice or not and what may be the
> implications regarding makepkg, that’s a different thing. But it would
> be good to agree on one way or another, and keep it the same everywhere
> for consistency. If they are good reason to do it the second way, I’d
> like to know them. Else, I have a tendency to find the shortest one to
> be the best. ;)

As I explained before, `python* setup.py install` "depends" on `python*
setup.py build`, in much the same way `make install` generally depends
on some target like `make all`.

So, running `python setup.py install` in package_*() is explicitly
identical in effect to running `python setup.py build && python setup.py
install`, just like `make install` would, for the average Makefile, be
identical to `make all && make install`.

Therefore, the *only* question is whether you wish to run `python
setup.py build` separately, in the build() function. But if you felt the
need to make a python2 copy in prepare() before using build(), then you
should also feel the need to make a python2 copy in prepare() when you
*aren't* using a build() function.

The only difference it could make, and this is a difference that would
apply equally to standard Makefile-powered packages as well, would be if
you wanted to build only part of a split package, and therefore skip
building it as well. And makepkg no longer supports building only part
of a split package, so that concern would only be relevant a long time ago.

...

As for making a python2 copy in the first place, that is not always
necessary... any python package which includes a binary extension will
get built in e.g. "build/lib.linux-i686-2.7/" vs.
"build/lib.linux-i686-3.6/", whereas pure *.py packages will get built
in "build/lib/".

The latter don't actually have any python2/python3-specific differences
anyway, usually. The recommended way to support both is, after all, to
write polyglot code that doesn't care which version of python you use.
Although setuptools *can* run 2to3 if the package requests it.

...

tl;dr
Nothing has changed since the last email, and that prepare function is
just as necessary (or possibly unnecessary) with a separate build() as
it is without a separate build().

-- 
Eli Schwartz



signature.asc
Description: OpenPGP digital signature

[aur-general] Python packaging: to build or not to build (Re: Review request for 3 related PKGBUILDs)

[Bruno Pagani via aur-general]

Le 03/01/2017 à 22:36, Eli Schwartz via aur-general a écrit :

> On 01/03/2017 04:12 PM, Leonid Bloch wrote:
>> Thanks! That was very helpful!
>>
>> All applied, except... "--skip-build" - indeed it makes sense, but I
>> have never seen it with other Python packages. So I wonder if indeed it
>> is a good practice, or is there some reason not to include it?
> Well, python-setuptools does it, but it doesn't seem to be very popular.
> Really, for Make-powered builds the dependencies for "install" are going
> to run anyway (but they were built during build() and usually do
> nothing, silently).
> Then again, a lot of python PKGBUILDs don't have a build() function at
> all, which means the package() function will invoke "build" itself.
> Apparently, there is an arcane difference between building a python
> module and compiling an ELF binary, but no one has told me what that
> difference may be... I don't usually pay attention to what other people
> do. :)
>
> It makes no difference whether you look at the repos or the AUR, both
> have people who do all three styles.
>
> The only practical difference would be if someone, say, ran `makepkg
> --nobuild && makepkg --repackage` on a VCS package, which they shouldn't.

I’m sort of reviving this thread, because I’ve stumbled upon discussions
around this recently, and in fact I see one practical reason of doing
the build in the package() function rather than in the build() one:

– split python2-lib/python-lib package without build() function (only
the relevant parts):
package_python-lib() {
python setup.py install --prefix=/usr --root="$pkgdir" --optimize=1
}
package_python2-lib(){
python2 setup.py install --prefix=/usr --root="$pkgdir" --optimize=1
}

– vs with build() function:
prepare() {
cp -a libname{,-py2}
}
build() {
cd libname
python setup.py build

cd libname-py2
python2 setup.py build
}
package_python-lib() {
python setup.py install --prefix=/usr --root="$pkgdir" --optimize=1
--skip-build
}
package_python2-lib(){
python2 setup.py install --prefix=/usr --root="$pkgdir" --optimize=1
--skip-build
}

Now, whether this is a good practice or not and what may be the
implications regarding makepkg, that’s a different thing. But it would
be good to agree on one way or another, and keep it the same everywhere
for consistency. If they are good reason to do it the second way, I’d
like to know them. Else, I have a tendency to find the shortest one to
be the best. ;)

Regards,
Bruno



signature.asc
Description: OpenPGP digital signature

Re: [aur-general] pending merge request for python-git/python-gitpython, please cancel

[Bruno Pagani via aur-general]

Le 14/02/2017 à 14:56, Daniel Milde a écrit :

> Dne 14.2.2017 v 14:50 Frederik “Freso” S. Olesen napsal(a):
>> Den 14-02-2017 kl. 12:19 skrev Daniel Milde:
>>> Yes, python-git should be probably renamed to something like
>>> python-gitlib and python-hg[1] should be renamed to python-git.
>> If the sources are in a Mercurial repository, the package should be -hg,
>> not -git. But speaking of python-hg... it points to a cpython
>> repository, so maybe it should really be cpython-hg?
>>
>> Oh boy! What a mess. :)
> Cpython sources are moved to github now.
>
> Yes, if python and python2 packages are renamed to cpython, then
> python-hg/git should be also renamed :)
>
> Daniel

Just to be clear for Freso and other people that might be tricked into
that: cpython is upstream name for the standard reference python
interpreter itself (absolutely not an expert in this area, but I would
bet that’s because this is a C implementation of python).

The repo was indeed in mercurial, but as you can see here
https://docs.python.org/devguide/ or here
https://docs.python.org/devguide/setup.html#checkout, it’s now
https://github.com/python/cpython.

So yes, there’s a bit of cleaning to be done regarding this in AUR.

Regards,
Bruno



signature.asc
Description: OpenPGP digital signature

Re: [aur-general] pending merge request for python-git/python-gitpython, please cancel

[Eli Schwartz via aur-general]

On 02/14/2017 08:50 AM, Frederik “Freso” S. Olesen wrote:
> Den 14-02-2017 kl. 12:19 skrev Daniel Milde:
>> Yes, python-git should be probably renamed to something like
>> python-gitlib and python-hg[1] should be renamed to python-git.
> 
> If the sources are in a Mercurial repository, the package should be -hg,
> not -git. But speaking of python-hg... it points to a cpython
> repository, so maybe it should really be cpython-hg?
> 
> Oh boy! What a mess. :)

Just like [extra]/python and [extra]/python2, this is the package for
the default C implementation of Python. What is your point?

-- 
Eli Schwartz



signature.asc
Description: OpenPGP digital signature

Re: [aur-general] pending merge request for python-git/python-gitpython, please cancel

[Daniel Milde]

Dne 14.2.2017 v 14:50 Frederik “Freso” S. Olesen napsal(a):
> Den 14-02-2017 kl. 12:19 skrev Daniel Milde:
>> Yes, python-git should be probably renamed to something like
>> python-gitlib and python-hg[1] should be renamed to python-git.
> If the sources are in a Mercurial repository, the package should be -hg,
> not -git. But speaking of python-hg... it points to a cpython
> repository, so maybe it should really be cpython-hg?
>
> Oh boy! What a mess. :)
>

Cpython sources are moved to github now.

Yes, if python and python2 packages are renamed to cpython, then
python-hg/git should be also renamed :)

Daniel



signature.asc
Description: OpenPGP digital signature

Re: [aur-general] pending merge request for python-git/python-gitpython, please cancel

[Frederik “Freso” S . Olesen]

Den 14-02-2017 kl. 12:19 skrev Daniel Milde:
> Yes, python-git should be probably renamed to something like
> python-gitlib and python-hg[1] should be renamed to python-git.

If the sources are in a Mercurial repository, the package should be -hg,
not -git. But speaking of python-hg... it points to a cpython
repository, so maybe it should really be cpython-hg?

Oh boy! What a mess. :)

-- 
Namasté,
Frederik “Freso” S. Olesen 
AUR: https://aur.archlinux.org/account/Freso
Wiki: https://wiki.archlinux.org/index.php/User:Freso



signature.asc
Description: OpenPGP digital signature

Re: [aur-general] pending merge request for python-git/python-gitpython, please cancel

[Daniel Milde]

Dne 14.2.2017 v 12:14 Bruno Pagani via aur-general napsal(a):
> Le 14/02/2017 à 12:11, Bruno Pagani a écrit :
>
>> Le 13/02/2017 à 17:03, brent timothy saner via aur-general a écrit :
>>
>>> i requested a merge of python-gitpython into python-git
>>>
>>> THIS SHOULD BE CANCELED.
>>>
>>> They need to be separate packages. they are for different things.
>>>
>>> PRQ # is 7484
>>>
>>> Sorry for the confusion
>> Please note that PRQs have they own list at
>> https://lists.archlinux.org/listinfo/aur-requests, where they should be
>> discussed when needed.
>>
>> That being said, I’ve tried to understand the differences between both
>> packages, and if indeed python-gitpython shouldn’t be merged into
>> python-git, this last one is currently building the GitPython 2.1.1
>> module instead of python from git (3.7.xxx), and that should be changed.
>> You should either take care of or orphan it.
>>
>> Cheers,
>> Bruno
> Also, you should make python-gitpython a split PKGBUILD to also provide
> python2-gitpython as the current python-git package provides.
>
Yes, python-git should be probably renamed to something like
python-gitlib and python-hg[1] should be renamed to python-git.

Daniel

[1] https://aur.archlinux.org/packages/python-hg/



signature.asc
Description: OpenPGP digital signature

Re: [aur-general] pending merge request for python-git/python-gitpython, please cancel

[Bruno Pagani via aur-general]

Le 14/02/2017 à 12:11, Bruno Pagani a écrit :

> Le 13/02/2017 à 17:03, brent timothy saner via aur-general a écrit :
>
>> i requested a merge of python-gitpython into python-git
>>
>> THIS SHOULD BE CANCELED.
>>
>> They need to be separate packages. they are for different things.
>>
>> PRQ # is 7484
>>
>> Sorry for the confusion
> Please note that PRQs have they own list at
> https://lists.archlinux.org/listinfo/aur-requests, where they should be
> discussed when needed.
>
> That being said, I’ve tried to understand the differences between both
> packages, and if indeed python-gitpython shouldn’t be merged into
> python-git, this last one is currently building the GitPython 2.1.1
> module instead of python from git (3.7.xxx), and that should be changed.
> You should either take care of or orphan it.
>
> Cheers,
> Bruno

Also, you should make python-gitpython a split PKGBUILD to also provide
python2-gitpython as the current python-git package provides.



signature.asc
Description: OpenPGP digital signature

Re: [aur-general] pending merge request for python-git/python-gitpython, please cancel

[Bruno Pagani via aur-general]

Le 13/02/2017 à 17:03, brent timothy saner via aur-general a écrit :

> i requested a merge of python-gitpython into python-git
>
> THIS SHOULD BE CANCELED.
>
> They need to be separate packages. they are for different things.
>
> PRQ # is 7484
>
> Sorry for the confusion

Please note that PRQs have they own list at
https://lists.archlinux.org/listinfo/aur-requests, where they should be
discussed when needed.

That being said, I’ve tried to understand the differences between both
packages, and if indeed python-gitpython shouldn’t be merged into
python-git, this last one is currently building the GitPython 2.1.1
module instead of python from git (3.7.xxx), and that should be changed.
You should either take care of or orphan it.

Cheers,
Bruno



signature.asc
Description: OpenPGP digital signature