[issue25825] AIX shared library extension modules installation broken

[Michael Felt]

Michael Felt  added the comment:

On 18/09/2018 16:46, Michael Felt wrote:
> Michael Felt  added the comment:

Ignore this. If only I could remove stuff!

Good day all.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25825] AIX shared library extension modules installation broken

[Michael Felt]

Michael Felt  added the comment:

On 10/08/2016 17:22, David Edelsohn wrote:
> David Edelsohn added the comment:
Hi, Just thought I would mention that I have a branch with all the test
PR fixes I have made in the last two months.

a) would appreciate your testig them with gcc - to be sure all is good
there as well.
b) If all test also pass for you - maybe if you make a bit of noise
python may be a bit quicker to actually merge them.

Hope this helps (AIX :) )

Michael

Links:
https://github.com/python/cpython/pulls?q=is%3Apr+is%3Aopen+AIX+author%3Aaixtools
https://github.com/aixtools/cpython/tree/aix-pr

--
title: AIX shared library extension modules installation broken: wrong dir 
names -> AIX shared library extension modules installation broken

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25825] AIX shared library extension modules installation broken: wrong dir names

[Martin Panter]

Changes by Martin Panter :


--
title: AIX shared library extension modules installation broken -> AIX shared 
library extension modules installation broken: wrong dir names

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25825] AIX shared library extension modules installation broken

[Martin Panter]

Changes by Martin Panter :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed
versions:  -Python 2.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25825] AIX shared library extension modules installation broken

[David Edelsohn]

David Edelsohn added the comment:

I believe that everything is functioning correctly.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25825] AIX shared library extension modules installation broken

[Martin Panter]

Martin Panter added the comment:

Okay, the second patch is committed to 3.5+. Is everything working now (on 2.7, 
3.5, 3.6), or is there more to do?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25825] AIX shared library extension modules installation broken

[Roundup Robot]

Roundup Robot added the comment:

New changeset ca1ddd365f5f by Martin Panter in branch '3.5':
Issue #25825: Fix references to $(LIBPL) installation path on AIX
https://hg.python.org/cpython/rev/ca1ddd365f5f

New changeset 5a05c0eeefc3 by Martin Panter in branch 'default':
Issue #25825: Merge $(LIBPL) fix from 3.5
https://hg.python.org/cpython/rev/5a05c0eeefc3

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25825] AIX shared library extension modules installation broken

[David Edelsohn]

David Edelsohn added the comment:

$(BINLIBDEST)/config is equivalent to $(LIBPL) in Python 2.7, so Python 2.7 
should be okay.

Patch2 will not make test_distutils results worse, but the test results may not 
represent the true status of distutils on AIX if the matching Python version is 
not installed so that the test can find the files in the installed location.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25825] AIX shared library extension modules installation broken

[Martin Panter]

Martin Panter added the comment:

Patch 1 does not apply to 2.7. The offending commit 88a532a31eb3 was not made 
in that branch.

Patch 2 may apply to 2.7, but I am not sure if it is needed or worthwhile. 
Again, the offending commit r86731 not made to 2.7. There is also a 
$(BINLIBDEST)/config reference for Be OS; surely if one is updated the other 
should too.

David: I don’t understand “Patch2 makes test_distutils worse because LDSHARED 
refers to a file that only exists when Python is installed.” At the moment, 
doesn’t LDSHARED refer to a file that never exists? So at least the test will 
be improved when Python is installed, right?

I looked into the test_distutils failures and here’s what I found:

* Revision c958678720fd assigns BLDSHARED → LDSHARED in distutils if the source 
tree is detected. This survived in the 3.5 branch, so may explain why the 3.5 
buildbot failure mentions “./Modules/ld_so_aix”.

* In 2.7, revision fa69e891edf4 moved the assignment into sysconfig.py.

* Then be3b4aa2ad28 loads LDSHARED from _sysconfigdata.py, which is created by 
sysconfig.py with the alternative BLDSHARED → LDSHARED value. This explains 
“./Modules/ld_so_aix” in 2.7.

* In 3.6, c554194240fc (Issue 18235) reversed the assignment in sysconfig.py, 
which now assigns LDSHARED → BLDSHARED, and creates _sysconfigdata.py with this 
alternative value.

* Then 3fa8aebed636 loads LDSHARED directly from _sysconfigdata.py. In this 
case it is the original LDSHARED value, so the 3.6 buildbot failure refers to 
/usr/local/lib/python3.6/config/ld_so_aix.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25825] AIX shared library extension modules installation broken

[David Edelsohn]

David Edelsohn added the comment:

> This bug was marked for 2.7 as well. Is there anything that needs to be done 
> for 2.7?

It would be great if both patches were applied to 2.7 also.

> How does patch 2 make the test_distutils situation worse? Is there anything 
> that should be done to improve things first, or should that just be handled 
> independently? I see that the 2.7 and 3.5 buildbots are currently failing 
> with errors like

distutils.errors.LinkError: command './Modules/ld_so_aix' failed with exit 
status 1

while 3.6 is failing with

distutils.errors.LinkError: command '/usr/local/lib/python3.6/config/ld_so_aix' 
failed with exit status 1

Presumably the 3.6 failure may be helped by Patch 2, since it is complaining 
about the plain “config” path.

Patch2 makes test_distutils worse because LDSHARED refers to a file that only 
exists when Python is installed.  If one tests in tree, the files don't exist.  
This may be related to the existence of BLDSHARED, used to build modules in the 
source tree, which can be overridden separately -- otherwise it defaults to 
LDSHARED.  If the rest of the testsuite ever is cleaned up for AIX, one either 
could skip the parts of test_distutils that require the LDSHARED files or add 
special hooks in the test for AIX.  As you point out, the change doesn't make 
the current testsuite results situation any worse.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25825] AIX shared library extension modules installation broken

[Martin Panter]

Martin Panter added the comment:

It looks like the change from plain config to config- was made in Issue 
9807 (r86731, 3.2). So the patch seems reasonable to me; it is just catching up 
with that change.

This bug was marked for 2.7 as well. Is there anything that needs to be done 
for 2.7?

How does patch 2 make the test_distutils situation worse? Is there anything 
that should be done to improve things first, or should that just be handled 
independently? I see that the 2.7 and 3.5 buildbots are currently failing with 
errors like

distutils.errors.LinkError: command './Modules/ld_so_aix' failed with exit 
status 1

while 3.6 is failing with

distutils.errors.LinkError: command '/usr/local/lib/python3.6/config/ld_so_aix' 
failed with exit status 1

Presumably the 3.6 failure may be helped by Patch 2, since it is complaining 
about the plain “config” path.

--
stage: needs patch -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25825] AIX shared library extension modules installation broken

[Eric N. Vander Weele]

Eric N. Vander Weele added the comment:

> Patch1 looks fine to me, though I will have to rely on you people to verify 
> that it does what it’s supposed to. Do you want me to commit it straight 
> away, or wait for your follow-up patch?

Independently, I have created a similar patch as well and can also verify that 
it does what it's suppose to do.

> The second of two patches. This patch changes the definition of LDSHARED for 
> AIX in configure to reference the matching installed location as defined in 
> Makefile.pre.in by Patch1. [...]

I can also confirm that Patch2 is necessary as well and does what it's suppose 
to do, having creating this patch independently as well.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25825] AIX shared library extension modules installation broken

[Eric N. Vander Weele]

Changes by Eric N. Vander Weele :


--
nosy: +ericvw

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25825] AIX shared library extension modules installation broken

[David Edelsohn]

David Edelsohn added the comment:

The second of two patches. This patch changes the definition of LDSHARED for 
AIX in configure to reference the matching installed location as defined in 
Makefile.pre.in by Patch1.  The definition from configure propagates into 
_sysconfigdata.py.

This change will (further) break test_distutils, but the testcase is wrong on 
AIX. And, for AIX, a correct test result clearly does not correspond to correct 
behavior.

The patch affects both configure.ac and configure because the corresponding 
change to configure is obvious. The person who installs the patch may 
regenerate configure, as appropriate.

--
Added file: http://bugs.python.org/file44096/Issue25825-patch2.txt

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25825] AIX shared library extension modules installation broken

[Roundup Robot]

Roundup Robot added the comment:

New changeset 58c8cae6c61a by Martin Panter in branch '3.5':
Issue #25825: Fix references to Modules/python.exp
https://hg.python.org/cpython/rev/58c8cae6c61a

New changeset 4d4b5b978b7e by Martin Panter in branch 'default':
Issue #25825: Merge AIX fix from 3.5
https://hg.python.org/cpython/rev/4d4b5b978b7e

--
nosy: +python-dev

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25825] AIX shared library extension modules installation broken

[David Edelsohn]

David Edelsohn added the comment:

Yes, please apply Patch 1 that reverts the mistaken change of revision 
88a532a31eb3 .  I want to work through this incrementally so that it's clear to 
reviewers.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25825] AIX shared library extension modules installation broken

[Martin Panter]

Martin Panter added the comment:

Patch1 looks fine to me, though I will have to rely on you people to verify 
that it does what it’s supposed to. Do you want me to commit it straight away, 
or wait for your follow-up patch?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25825] AIX shared library extension modules installation broken

[David Edelsohn]

David Edelsohn added the comment:

Let's start with this patch to revert the change mentioned in msg256136 to look 
for python.exp in Modules -- the test, the source for the file, and the 
location to delete the file.

A follow-up patch will fix the data in _sysconfigdata.py.

--
Added file: http://bugs.python.org/file44068/Issue25825-patch1.txt

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25825] AIX shared library extension modules installation broken

[Michael Felt]

Michael Felt added the comment:

hmm. needs patch. For what?

One comment says a change will be undone.

If a patch is expected for src/builddir issue:

a) I sinned - that is a new issue, and should be posted seperately.
b) i do not know autotools well enough to solve the srcdir and builddir problem.

rather than a patch i can on that is see if the same issue occurs with linux 
and post a new issue.

So, my question would be - was a rollback done - if yes, perhaps a comment to 
verify and then close this issue.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25825] AIX shared library extension modules installation broken

[Martin Panter]

Changes by Martin Panter :


--
stage:  -> needs patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25825] AIX shared library extension modules installation broken

[Michael Felt]

Michael Felt added the comment:

FYI: when build and src are the same directory, there is no error message. 
However, when src and build are in seperate directories (e.g.,

build = '.',
src = '../src/python-2.7.11'

the following message occurs MANY times:

unable to execute '../src/python-2.7.11/Modules/ld_so_aix': No such file or 
directory

during configure the file
./Modules/ld_so_aix is 'created' in the build area.

root@x064:[/data/prj/aixtools/python-2.7.11]ls -l Modules/ld*aix*
-rwxr-xr-x   1 root system 6205 Mar 02 16:24 Modules/ld_so_aix

Just guessing, but a line such as:
BLDSHARED="\$(srcdir)/Modules/ld_so_aix \$(CC) 
-bI:\$(srcdir)/Modules/python.exp"

using $(srcdir) is not correct.

Further, a find in $(srcdir) for ld_so_aix returns:
root@x064:[/data/prj/aixtools/python-2.7.11]ls -dl ../src/py*
drwxr-xr-x  17 199  1954   4096 May 23 2015  ../src/python-2.7.10
drwxr-xr-x  17 199  1954   4096 Dec 05 19:47 ../src/python-2.7.11

So, $(srcdir) seems to be wrong, by definition.

For now, I am copying $(srcdir) to $(builddir) to work-around this.

--
nosy: +Michael.Felt

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25825] AIX shared library extension modules installation broken

[David Edelsohn]

David Edelsohn added the comment:

$(prefix) and $(exec_prefix) result in the same path on AIX, so it does not 
matter in practice, although the semantics are different.

# Install prefix for architecture-dependent files
exec_prefix=${prefix}

python.exp is not architecture dependent, although it only is useful on AIX 
target.  It is essentially equivalent to a list of symbols with ELF global, 
non-hidden visibility.  It is less confusing if the list is co-located with the 
scripts that use it.

LIBPL is fine with me.  configure.ac and Makefile.pre.in must match.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25825] AIX shared library extension modules installation broken

[David Edelsohn]

New submission from David Edelsohn:

AIX requires helper scripts to build Python shared extension modules.  The 
definitions and Makefile installation rules have bitrotted.

Makefile.pre.in:

  @if [ -s Programs/python.exp -a \

except python.exp is created in Modules/python.exp, not Programs.exp

   $(INSTALL_DATA) Programs/python.exp \
   $(DESTDIR)$(LIBPL)/python.exp;

Post-substitution Makefile.pre:

BINLIBDEST= $(LIBDIR)/python$(VERSION)
LIBDEST=$(SCRIPTDIR)/python$(VERSION)
LIBPL=  $(LIBDEST)/config-$(LDVERSION)

configure.ac:

AIX*)
BLDSHARED="\$(srcdir)/Modules/ld_so_aix \$(CC) 
-bI:\$(srcdir)/Modules/python.exp"
LDSHARED="\$(BINLIBDEST)/config/ld_so_aix \$(CC) 
-bI:\$(BINLIBDEST)/config/python.exp"

which is forever enshrined in _sysconfigdata.py

In other words, configure sets LDSHARED to $(BINLIBDEST)/config, which does not 
exist in current installations.  And Makefile installs the files in LIBPL, 
which is based on LIBDEST (prefix and exec_prefix could be different).

And the files are not installed, because the installation tests 
Programs/python.exp instead of Modules/python.exp.

Changing Makefile.pre.in to test Modules/python.exp is easy enough as a partial 
fix.

What is the preferred location to install the files so that configure.ac and 
Makefile.pre.in can be adjusted to match each other?

--
components: Installation
messages: 256123
nosy: David.Edelsohn
priority: normal
severity: normal
status: open
title: AIX shared library extension modules installation broken
type: behavior
versions: Python 2.7, Python 3.2, Python 3.3, Python 3.4, Python 3.5, Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25825] AIX shared library extension modules installation broken

[David Edelsohn]

Changes by David Edelsohn :


--
nosy: +pitrou

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25825] AIX shared library extension modules installation broken

[Martin Panter]

Martin Panter added the comment:

It looks like the Programs/python.exp change was done accidentally in revision 
88a532a31eb3 (Issue 18093), so we should be safe reverting the relevant lines 
back to Modules/python.exp.

The configure script currently sets

LIBPL='$(prefix)'"/lib/python${VERSION}/config-${LDVERSION}"
AC_SUBST(LIBPL)

However there is a contradictory comment in the makefile:

# This goes into $(exec_prefix)
LIBPL=  @LIBPL@

It looks like the comment came first. $(LIBPL) was changed from $(LIBDIR) 
[presumably using $(exec_prefix)] to use $(prefix) in r86731 (Issue 9807).

This isn’t my area of expertise, but my understanding is $(exec_prefix) is for 
architecture-specific files only. If python.exp is not architecture-specific, 
perhaps it should go in $(prefix), and the configure script (and makefile 
comment) needs fixing.

--
nosy: +martin.panter
versions:  -Python 3.2, Python 3.3, Python 3.4

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com