from:"Sébastien Sablé"


Sébastien Sablé sa...@users.sourceforge.net added the comment:

I don't have commit access on hg.python.org, so I also created a clone on 
bitbucket at:
https://bitbucket.org/sablefr/py27vs2010/overview

I work with a patch queue for the moment since everything is not completely 
settled yet.

The patch are against python 2.7 for the moment, I will do the same for python 
trunk soon.
I can run all the tests with builbot and I am working on reducing the errors.

Currently I am at:
337 tests OK.
5 tests failed:
test_anydbm test_pep277 test_shelve test_subprocess test_trace
1 test altered the execution environment:
test_distutils
45 tests skipped:
test_aepack test_al test_applesingle test_bsddb185 test_cd test_cl
test_commands test_crypt test_curses test_dbm test_dl test_epoll
test_fcntl test_fork1 test_gdb test_gdbm test_gl test_grp
test_imgfile test_ioctl test_kqueue test_linuxaudiodev test_macos
test_macostools test_mhlib test_nis test_openpty test_ossaudiodev
test_pipes test_poll test_posix test_pty test_pwd test_readline
test_resource test_scriptpackages test_sunaudiodev test_tcl
test_threadsignals test_tk test_ttk_guionly test_ttk_textonly
test_wait3 test_wait4 test_zipfile64
6 skips unexpected on win32:
test_gdb test_readline test_tcl test_tk test_ttk_guionly
test_ttk_textonly

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13210
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13210] Support Visual Studio 2010


Sébastien Sablé sa...@users.sourceforge.net added the comment:

Yes I know, but this is my primary target as this is the version that I use in 
my product for the moment.

I will test python trunk soon now that Python 2.7 with VS2010 is in a rather 
good shape.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13210
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13461] Error on test_issue_1395_5 with Python 2.7 and VS2010


Sébastien Sablé sa...@users.sourceforge.net added the comment:

Thanks Antoine! It solved the issue.

I will check soon with Python trunk to see if the same thing applies.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13461
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue10752] build_ssl.py is relying on unreliable behaviour of os.popen


Sébastien Sablé sa...@users.sourceforge.net added the comment:

I had the same issue while compiling Python 2.7 with ActivePerl on windows, and 
I can confirm that the proposed patch solves the issue.

--
nosy: +sable
versions: +Python 2.7

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue10752
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1395] py3k: duplicated line endings when using read(1)


Sébastien Sablé sa...@users.sourceforge.net added the comment:

I am trying to get Python working when compiled with Visual Studio 2010 (cf 
issue 13210).

When running the tests with the python 2.7 branch compiled with VS2010, the 
test_issue_1395_5 in test_io.py will cause Python to eat the whole memory 
within a few seconds and make the server completely unresponsive.

--
nosy: +sable

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue1395
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13461] Error on test_issue_1395_5 with Python 2.7 and VS2010


New submission from Sébastien Sablé sa...@users.sourceforge.net:

I am trying to get Python working when compiled with Visual Studio 2010 (cf 
issue 13210).

When running the tests with the python 2.7 branch compiled with VS2010, the 
test_issue_1395_5 in test_io.py will cause Python to eat the whole memory 
within a few seconds and make the server completely unresponsive.

--
components: IO
messages: 148184
nosy: sable
priority: normal
severity: normal
status: open
title: Error on test_issue_1395_5 with Python 2.7 and VS2010
versions: Python 2.7

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13461
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1395] py3k: duplicated line endings when using read(1)


Sébastien Sablé sa...@users.sourceforge.net added the comment:

OK, sorry. Done in issue 13461.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue1395
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13461] Error on test_issue_1395_5 with Python 2.7 and VS2010


Sébastien Sablé sa...@users.sourceforge.net added the comment:

The problem is in CTextIOWrapperTest.test_issue1395_5
Here is the backtrace:

msvcr100d.dll!memset()  Line 145Asm
   msvcr100d.dll!_heap_alloc_dbg_impl(unsigned __int64 nSize, int 
 nBlockUse, const char * szFileName, int nLine, int * errno_tmp)  Line 498 
C++
msvcr100d.dll!_nh_malloc_dbg_impl(unsigned __int64 nSize, int nhFlag, 
int nBlockUse, const char * szFileName, int nLine, int * errno_tmp)  Line 239 + 
0x22 bytesC++
msvcr100d.dll!_nh_malloc_dbg(unsigned __int64 nSize, int nhFlag, int 
nBlockUse, const char * szFileName, int nLine)  Line 302 + 0x2a bytes  C++
msvcr100d.dll!malloc(unsigned __int64 nSize)  Line 56 + 0x21 bytes  
C++
python27_d.dll!PyObject_Malloc(unsigned __int64 nbytes)  Line 944   
C
python27_d.dll!_PyObject_DebugMallocApi(char id, unsigned __int64 
nbytes)  Line 1445 + 0xa bytesC
python27_d.dll!_PyObject_DebugMalloc(unsigned __int64 nbytes)  Line 
1413C
python27_d.dll!PyString_FromStringAndSize(const char * str, __int64 
size)  Line 88 + 0x11 bytes C
python27_d.dll!do_mkvalue(const char * * p_format, char * * p_va, int 
flags)  Line 427 + 0xf bytes  C
python27_d.dll!va_build_value(const char * format, char * va, int 
flags)  Line 537 + 0x14 bytes C
python27_d.dll!_Py_VaBuildValue_SizeT(const char * format, char * va)  
Line 511 C
python27_d.dll!_PyObject_CallMethod_SizeT(_object * o, char * name, 
char * format, ...)  Line 2671 + 0xf bytes  C
python27_d.dll!textiowrapper_tell(textio * self, _object * args)  Line 
 + 0x2c bytesC



So the problem happens when calling in textio.c:
{{{
PyObject *decoded = PyObject_CallMethod(
self-decoder, decode, s#, input, 1);
}}}

self-decoder is of type _io.IncrementalNewlineDecoder and input is BBB.

This will result in PyString_FromStringAndSize being called with size = 
4294967297, which will cause the server to fall.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13461
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue10469] test_socket fails using Visual Studio 2010

2011-11-18 Thread Sébastien Sablé


Sébastien Sablé sa...@users.sourceforge.net added the comment:

I can also confirm that this patch corrects the problem with test_asyncore for 
Python 2.7 built with VS2010.

--
nosy: +sable

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue10469
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue2286] Stack overflow exception caused by test_marshal on Windows x64

2011-11-18 Thread Sébastien Sablé


Sébastien Sablé sa...@users.sourceforge.net added the comment:

I had the exact same problem when compiling the Python 2.7 branch with Visual 
Studio 2010.

In debug mode and 64 bits, python will crash on test_loads_recursion in 
test_marshal and on the code provided in msg63536.

Could you please reopen this issue?

--
nosy: +sable
versions: +Python 2.7

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue2286
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue2286] Stack overflow exception caused by test_marshal on Windows x64

2011-11-18 Thread Sébastien Sablé


Sébastien Sablé sa...@users.sourceforge.net added the comment:

I don't have the issue anymore when bumping the stack size to 230.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue2286
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13210] Support Visual Studio 2010

2011-10-19 Thread Sébastien Sablé


Sébastien Sablé sa...@users.sourceforge.net added the comment:

Martin, what I want to achieve is to ensure that someone can download Python 
sources and compile them without any modification using a standard install of 
Windows + Visual Studio 2010 SP1.

I don't really care what is the default compiler used to generate the official 
binaries, this is a decision that should be taken according to what most people 
use. But a least the compilation should work easily with the recent and popular 
VS2010 compiler.

As I said the patch is for the moment a quick and dirty draft: it breaks 
compilation with VS2008 and assume VS2010. Of course this is not my goal: the 
final patch should work with existing compilers as well as VS2010.

Which part of the changes do you consider unnecessary?

Concerning the target version: I need to use Python 2.7 internally because our 
application has not been migrated to Python 3.
I think other people may be in the same case.

I understand there is a feature freeze on this branch (event though the changes 
in this case are well localized and not intrusive), so I will target primarily 
Python trunk for the inclusion of patches, but I will also maintain internally 
my own set of patches for Python 2.7 (and in the wiki for those who are 
interested).

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13210
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13210] Support Visual Studio 2010

2011-10-19 Thread Sébastien Sablé


Sébastien Sablé sa...@users.sourceforge.net added the comment:

Packaging makes it easier to distribute Python among my colleagues and 
customers, so I think it is a nice addition.

OK for the hg clone, I will set up that (I actually already use mercurial 
internally to handle those modifications).

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13210
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4431] Distutils MSVC doesn't create manifest file


Sébastien Sablé sa...@users.sourceforge.net added the comment:

I need Python compiled with Visual Studio 2010 because I use it embedded in an 
application compiled with Visual Studio 2010.

I am pretty sure there are quite a lot of people who would like to compile 
Python with this compiler.

I have compiled Python 2.7.2 with this version of the compiler as well as many 
extensions, but this required to patch quite a few things. I also intend to 
setup a buildbot running Python tests using this compiler.

If I can provide the patch to support Visual Studio 2010 and setup a builtbot 
that will pass most of the tests, could we get Python supported on this 
platform?

I intended to announce all of that on python-dev soon, but the buildbot is not 
quite ready yet.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue4431
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13210] Support Visual Studio 2010


New submission from Sébastien Sablé sa...@users.sourceforge.net:

Python for Windows is currently compiled with Visual Studio 2008.
It is not possible to use Visual Studio 2010.

It would be great to be able to compile Python with Visual Studio 2010:

* some people may only have access to this compiler which is more recent

* when embedding Python in an application compiled with Visual Studio 2010, you 
mix 2 C Runtime Libraries (msvcr90.dll and msvcr100.dll) which will generally 
result in a crash

This is the parent issue. I have various patches to achieve that and I will try 
to open one issue by patch. I may also run a builtbot using that compiler.

--
assignee: tarek
components: Build, Distutils, Distutils2, Installation, Windows
messages: 145803
nosy: alexis, eric.araujo, mhammond, sable, tarek
priority: normal
severity: normal
status: open
title: Support Visual Studio 2010
type: feature request
versions: Python 2.7, Python 3.2

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13210
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4431] Distutils MSVC doesn't create manifest file


Sébastien Sablé sa...@users.sourceforge.net added the comment:

Amaury:
Great! I have opened issue 13210 to achieve that.

Mark:
Mixing the 2 CRTs results in plenty of crashes in our application. The 
application also uses a lot of Qt. It is not an option to use a prebuilt Python 
binary generated with VS2008.

I added you to issue 13210 and I will post the patches there soon.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue4431
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13210] Support Visual Studio 2010


Sébastien Sablé sa...@users.sourceforge.net added the comment:

Here is a quick and dirty draft of the modifications I had to do in order to 
get Python 2.7.2 to compile with VS2010 on the wiki:

http://wiki.python.org/moin/VS2010

I will improve/complete it as I progress.
I will also do a 64 bits version.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13210
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4431] Distutils MSVC doesn't create manifest file (with fix)

2011-10-12 Thread Sébastien Sablé


Sébastien Sablé sa...@users.sourceforge.net added the comment:

I got this problem again this morning while compiling pywin32.

This problem is not specific to me, anyone using Visual Studio 2010 to compile 
Python is experiencing the same issue:

http://nukeit.org/compile-python-2-7-packages-with-visual-studio-2010-express/
https://bitbucket.org/jaraco/jaraco.develop/src/f6e937d98e7f/jaraco/develop/msvc.py

Could you please reopen the issue or should I open a new one?
Thanks in advance

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue4431
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue6807] No such file or directory: 'msisupport.dll' in msi.py

2011-10-12 Thread Sébastien Sablé


Sébastien Sablé sa...@users.sourceforge.net added the comment:

I had the same issue today. The patch solved the problem.
Thanks

--
nosy: +sable

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue6807
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4431] Distutils MSVC doesn't create manifest file (with fix)

2011-09-08 Thread Sébastien Sablé


Sébastien Sablé sa...@users.sourceforge.net added the comment:

I encountered the same problem while compiling the psutil package.

I used Python 3.2.2 that I compiled myself using Visual Studio 2010 SP1 in 
Debug mode by following the PCBuild/readme.txt documentation.

I could not compile psutil until I added in setup.py:
extra_link_args=['/MANIFEST']

I have not tried to compile any other C extension yet, but this is not 
mysql-python and I am not using setuptools either.

--
nosy: +sable
versions: +Python 2.6, Python 3.2

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue4431
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12372] semaphore errors on AIX 7.1


Sébastien Sablé sa...@users.sourceforge.net added the comment:

Hi,

sorry for the late reply.

  STINNER Victor added the comment:
 
 At least, I would like to know if Sébastien Sablé (the author of the original 
 patch) changed his opinion about this issue since 2007 ;-)

I haven't changed my mind concerning this patch and we still use it for 
the Python that we deploy on AIX 5.3 and AIX 6.1 systems (I don't use 
AIX 7 yet).

Not using this patch will result in performance degradation and 
sem_post: Permission denied error messages from the system popping on 
the console.

  Charles-François Natali added the comment:
 
  Well, it looks like a libc or kernel bug, and we probably can't do
  much about it. Just falling back to mutex + condition variable instead
  of POSIX semaphores does the trick, so I'll say just avoid POSIX
  semaphores altogether on AIX and we should be fine. People could
  report this to their AIX vendor, but they should definitely consider
  dropping this terminally broken platform and switch to, let's says,
  Linux or BSD...

I agree with Charles-François that it is an AIX bug that should be 
reported to IBM so that they could fix it in a future release.
I already submitted some tickets to IBM concerning other problems and 
it is not exactly an agile company so you have to be very patient.

I also agree that people should consider dropping this broken platform, 
but unfortunately IBM is aggressively and successfully (AFAIK) pushing 
this platform to many customers (in the finance industry in particular) 
so I don't expect it to die soon.

regards

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue12372
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue3526] Customized malloc implementation on SunOS and AIX


Sébastien Sablé sa...@users.sourceforge.net added the comment:

Sorry for the very late reply; I have been quite busy recently with the 
birth of my second daughter, a new job, a new home town and soon a new home.

...
 But with your patch, such code wouldn't be thread-safe anymore. This
 patch implies that a thread can't call malloc directly or indirectly
 (printf, opendir, and many others) while it doesn't hold the GIL. This
 is going to break a lot of existing code.

I didn't have this problem since the threads in my application are 
handled by Python and so hold the GIL. But you are right it is a concern.

Fortunately, it is easy to solve by defining the following in dlmalloc:
#define HAVE_MORECORE 0

That way, all the memory allocations handled by Python will go in a 
dedicated mmaped memory segment controlled by dlmalloc, while all the 
calls to the system malloc will work as before (probably going into a 
segment handled by sbrk).

 It will also be slower, and consume more memory.

It should be noted that sbrk is deprecated on some platforms where mmap 
is suggested as a better replacement (Mac OS X, FreeBSD...).
sbrk is generally considered quite archaic.

I attach a new patch that can be applied to Python 2.7.1. It includes 
the dlmalloc modification and uses only mmap in this case (no sbrk).

We have delivered it in production with the new version of our software 
that works on AIX 6.1 and it works fine.

I also did some benchmarks and did not notice any slow down compared to 
a pristine Python 2.7.1 (actually it was slightly faster YMMV).
It also consumes a lot less memory, but that is the reason for this 
patch in the first place.

Since I am changing of job, I won't be working on AIX anymore (yeah!); I 
also don't expect this patch to be integrated spontaneously without 
someone interested in AIX pushing for it. So I leave this patch more as 
a reference for someone who would be impacted by this problem and would 
like to integrate it in his own Python. I hope it helps.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue3526
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue3526] Customized malloc implementation on SunOS and AIX


Changes by Sébastien Sablé sa...@users.sourceforge.net:


Added file: http://bugs.python.org/file22698/patch_dlmalloc_Python_2_7_1.diff

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue3526
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue3526] Customized malloc implementation on SunOS and AIX


Sébastien Sablé sa...@users.sourceforge.net added the comment:

 Does it mean the project to have an AIX buildbot is abandoned?

We have a buildbot running internally on AIX. I could not get the necessary 
modifications integrated upstream in the official Python buildbot so that we 
could plug directly on it.

cf this thread:
http://mail.python.org/pipermail/python-dev/2010-October/thread.html#104714

I will try to get someone at my company to keep this buildbot running and 
report any outstanding bug, but I can't guarantee anything. 

 Indeed, thanks for your contributions.

Thanks! And thank you for your help in most of the issues related to AIX.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue3526
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue3526] Customized malloc implementation on SunOS and AIX

2011-05-04 Thread Sébastien Sablé

Sébastien Sablé sa...@users.sourceforge.net added the comment:

Another reason why you should not force dlmalloc for all applications linked
with libpython is because dlmalloc is (by default) not thread safe, while the
system malloc is (generally) thread-safe. It is possible to define a constant
in dlmalloc to make it thread-safe (using locks) but it will be slower and it
is not needed in Python since the GIL must be held when using PyMem_ functions.

If a thread-safe implementation was needed, it would be better to switch to
ptmalloc2.

Also that addresses the issue of two threads inside different malloc
implementations at the same time: it is currently not allowed with
PyMem_Malloc.

Most of the allocations come from the heap - through sbrk

Most python objects will be allocated in pymalloc arenas (if they are smaller
than 256 bytes) which (if compiled with --with-pymalloc-mmap) will be directly
allocated by calling mmap, or (without --with-pymalloc-mmap) will be allocated
in dlmalloc by calling mmap (because arenas are 256KB).
So most of the python objects will end up in mmap segments separate from the
heap.

The only allocations that will end up in the heap are for the medium python
objects (256 bytes and 256KB) or for allocations directly by calling
PyMem_Malloc (and for a size 256KB). Also dlmalloc will not call sbrk for each
of those allocations: dlmalloc allocates some large memory pools and manage the
smaller allocations within those pools in a very efficient way. So the heap
fragmentation should be indeed reduced by using dlmalloc.

Most modern malloc implementations are also using pools/arenas anyway, so the
heap will mostly contain a mix of native malloc arenas and dlmalloc pools. So
the fragmentation should not be too much of a concern if you mix 2 malloc
implementations.
Here is OpenSolaris malloc implementation for example:
http://src.opensolaris.org/source/xref/onnv/onnv-gate/usr/src/lib/libmalloc/common/malloc.c#514

Concerning trimming: the reason why I am proposing to use dlmalloc on AIX and
Solaris is that the native malloc/free do not correctly trim the heap in the
first place on those platforms! If malloc/free correctly worked on those
platforms and the heap was trimmed when possible, I would not have taken the
trouble of proposing this patch and using dlmalloc, I would happily use the
native malloc/free.

So mixing 2 malloc implementations should not be a problem as long as you keep
track of the right 'free' implementation to use for each pointer (which should
already be the case when you call PyMem_Malloc/PyMem_Free instead of
malloc/free).

If you are really concerned about mixing 2 malloc implementations in the heap,
you can define HAVE_MORECORE 0 in dlmalloc and that way dlmalloc will always
use mmap and not use the heap at all.

My application uses the provided patch so that dlmalloc is used for Python
objects and the native malloc for all the rest (much less consuming than the
Python part) on AIX and SunOS. It has been in production for years and we never
experienced any crash related to memory problems.

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue3526
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue3526] Customized malloc implementation on SunOS and AIX

2011-04-29 Thread Sébastien Sablé


Sébastien Sablé sa...@users.sourceforge.net added the comment:

 I'm just proposing an alternative that I find cleaner, simpler and easier to 
 maintain.

I understand how LD_PRELOAD works but I find it neither clean nor simple to 
maintain.

Also by using a wrapper to call Python you still impact all the applications 
that may be executed from Python since the environment variables are 
propagated. You also need to configure the path to the alternative malloc 
library at runtime.

And as I said above, I am afraid most AIX and SunOS users will never hear about 
that and will just use the default behavior, with their Python application 
taking much more memory than necessary.

As mentioned by Floris, AIX is being pushed by IBM quite a lot, and in some 
markets it is very common (if not predominant in finance for example - 50/50 
with SunOS for my clients I would say).

 I mean, if you start embedding malloc in python, why stop there, and
not embed the whole glibc ;-)

Concerning AIX, that would not be such a bad idea given the number of bugs in 
the native C library (cf some of my other issues reported in python bug 
tracker) - just kidding ;-)

Concerning the fact that dlmalloc or ptmalloc evolve quickly:
* dlmalloc V2.8.3 Thu Sep 22 11:16:32 2005
* ptmalloc2 release Jun 5th, 2006
* ptmalloc3 release May 31st, 2006
* dlmalloc V2.8.4 Wed May 27 09:56:23 2009
I think we can cope with that kind of fast evolution ;-)

Also an old dlmalloc is better than no dlmalloc at all.
And as you noticed, an old dlmalloc is already provided in libffi.

 So how about a --with-dlmalloc=path/to/dlmalloc.c?

That looks like a good alternative. I can implement that if that can help to 
get the patch in Python.

 Can't you just add dlmalloc to LDFLAGS or something? Or would the
default malloc still be selected?

There is a USE_DL_PREFIX in malloc.c. If this flag is defined, all functions 
will be prefixed by dl (dlmalloc, dlfree, dlrealloc...).
If it is not set, the functions will be named as usual (malloc, free...).

In my patch, I preferred to set USE_DL_PREFIX and call dlmalloc/dlfree 
explicitly where needed.

Since I want PyMem_MALLOC to call dlmalloc, I would need to export the malloc 
symbol from libpython so that Python extensions could use it when calling 
PyMem_MALLOC, but that would impact all malloc calls in applications which 
embed Python for example.

So I think it is probably better to explicitly distinguish when you want to 
call dlmalloc and leave the native malloc for the host application.

Also this only addresses the --with-dlmalloc part of my patch.
The other part concerning --with-pymalloc-mmap ensures that pymalloc uses mmap 
to allocate arenas rather than malloc.

I perfectly understand that people are reluctant to make the memory allocation 
system more complex than it is already in Python in order to bypass some 
limitations of systems which are not very widespread among Python users.

But Python eating a lot of memory on SunOS and AIX does not look very good 
either.

I have some strong requirements as far as memory is concerned for my 
application so I have maintained this patch internally and distributed it as 
part of my application.

I will probably change of job soon and will not have access to AIX systems 
anymore. I don't really expect this patch to be accepted soon as few people 
have expressed some interest and I don't have much time/interest to push it on 
python-dev, but I will update the patch for Python 2.7 and 3.2 before leaving 
so that people impacted by this problem could a least manually patch their 
Python if they find this issue.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue3526
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue3526] Customized malloc implementation on SunOS and AIX

2011-04-29 Thread Sébastien Sablé


Sébastien Sablé sa...@users.sourceforge.net added the comment:

Yes, I was probably not clear:
When --with-dlmalloc is activated, PyMem_MALLOC/PyMem_Malloc will call 
dlmalloc, PyMem_REALLOC/PyMem_Realloc will call dlrealloc and 
PyMem_FREE/PyMem_Free will call dlfree.

While calls to malloc/free/realloc will use the platform implementation.

So I think there should not be any mix, since as it is mentioned in pymem.h, 
people should not mix PyMem_MALLOC/PyMem_FREE with malloc/free:

/* BEWARE:

   Each interface exports both functions and macros.  Extension modules should
   use the functions, to ensure binary compatibility across Python versions.
   Because the Python implementation is free to change internal details, and
   the macros may (or may not) expose details for speed, if you do use the
   macros you must recompile your extensions with each Python release.

   Never mix calls to PyMem_ with calls to the platform malloc/realloc/
   calloc/free.  For example, on Windows different DLLs may end up using
   different heaps, and if you use PyMem_Malloc you'll get the memory from the
   heap used by the Python DLL; it could be a disaster if you free()'ed that
   directly in your own extension.  Using PyMem_Free instead ensures Python
   can return the memory to the proper heap.  As another example, in
   PYMALLOC_DEBUG mode, Python wraps all calls to all PyMem_ and PyObject_
   memory functions in special debugging wrappers that add additional
   debugging info to dynamic memory blocks.  The system routines have no idea
   what to do with that stuff, and the Python wrappers have no idea what to do
   with raw blocks obtained directly by the system routines then.

   The GIL must be held when using these APIs.
*/

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue3526
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue3526] Customized malloc implementation on SunOS and AIX

2011-04-29 Thread Sébastien Sablé


Sébastien Sablé sa...@users.sourceforge.net added the comment:

I share the opinion of Floris on this: just because you link your application 
with python does not mean you want it to handle all memory management.

If you want the memory to be handled by Python, you should call PyMem_Malloc.

Otherwise people may want to use different malloc implementations in different 
parts of their application/libraries for different reasons (dmalloc for 
debugging http://dmalloc.com/ for example - we have seen that libffi bundles 
its own dlmalloc - someone may prefer a derivative of ptmalloc for performance 
reasons with threads...).

My application is linked with various libraries including libpython, glib and 
gmp, and I sometimes like to be able to distinguish how much memory is 
allocated by which library for profiling/debugging purpose for example.

I don't understand the point concerning trimming/fragmentation/threading by 
Charles-Francois: dlmalloc will allocate its own memory segment using mmap and 
handle memory inside that segment when you do a dlmalloc/dlfree/dlrealloc. 
Other malloc implementations will work in their own separate space and so won't 
impact or be impacted by what happens in dlmalloc segments.

dlmalloc is not that much different from pymalloc in that regard: it handles 
its own memory pool on top of the system memory implementations.
Yet you can have an application that uses the ordinary malloc while calling 
some Python code which uses pymalloc without any 
trimming/fragmentation/threading issues.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue3526
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue3526] Customized malloc implementation on SunOS and AIX

2011-04-26 Thread Sébastien Sablé


Sébastien Sablé sa...@users.sourceforge.net added the comment:

Hi Charles-François,

it is possible to impact the memory allocation system on AIX using some 
environment variables (MALLOCOPTIONS and others), but it is not very elegant 
(it will impact all applications running with this environment and it is 
difficult to ensure that those environment variables will be correctly set when 
distributing an application to a customer) and I am afraid most users will 
never hear about that and will just use the default behavior.

Concerning mmap performances, dlmalloc has a pool mechanism and Python has its 
own pool mechanism on top of that.
As a result, system calls to allocate memory do not happen frequently since the 
memory allocation is usually handled internally in those pools and dlmalloc is 
often faster than the native malloc.

I have been distributing a version of Python which integrates this patch with 
the application on which I work to various customers for the last few years and 
the benchmarks have not shown any significant performance degradation. On the 
other hand, the decrease in memory consumption has been clearly noticed and 
appreciated.

Also note that dlmalloc (or a derivative - ptmalloc) is part of GNU glibc which 
is used by most Linux systems, and is what you get when you call malloc.
http://en.wikipedia.org/wiki/Malloc#dlmalloc_and_its_derivatives

So by using dlmalloc on SunOS and AIX you would get the same level of 
performance for memory operations that you already probably can appreciate on 
Linux systems.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue3526
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11185] test_wait4 error on AIX

2011-03-08 Thread Sébastien Sablé


Sébastien Sablé sa...@users.sourceforge.net added the comment:

Yes, for the test, as I put in msg128727, it works fine by removing WNOHANG.

However I should put a note in the AIX-NOTES file to explain that wait4 is 
broken with WNOHANG on AIX and suggest the 2 workarounds.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue11185
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11185] test_wait4 error on AIX

2011-03-07 Thread Sébastien Sablé


Sébastien Sablé sa...@users.sourceforge.net added the comment:

I had seen that post you mentioned and already tested the -lbsd without success.

wait4 is not even present in libbsd.

phenix:~$ nm /usr/lib/libbsd.a  | grep wait
phenix:~$ 

Maybe it was present on older versions of the system. But I couldn't find any 
documentation mentioning wait4 and -lbsd anywhere.

Actually wait4 is never mentioned in the IBM documentation concerning AIX.

wait4 without WNOHANG works fine. waitpid works fine even with WNOHANG.
I don't know which workaround is the better.
I will also try to report this bug to IBM so that a future version of AIX could 
work correctly.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue11185
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue10709] Misc/AIX-NOTES needs updating

2011-02-19 Thread Sébastien Sablé


Sébastien Sablé sa...@users.sourceforge.net added the comment:

Great!

I intend to check that branch 2.7 works fine on AIX once I will have finished 
with 3.x (which should be soon).
I will then update this file for Python 2.7 (there shouldn't be much changes 
hopefully).

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue10709
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue10709] Misc/AIX-NOTES needs updating

2011-02-18 Thread Sébastien Sablé


Sébastien Sablé sa...@users.sourceforge.net added the comment:

Here is a new version of the file:
It integrates the corrections by R. David, and I also modified the memory 
limitations section in order to use ldedit rather than some flags for the 
linker.

--
Added file: http://bugs.python.org/file20782/AIX-NOTES

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue10709
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11212] Python memory limit on AIX

2011-02-18 Thread Sébastien Sablé


Sébastien Sablé sa...@users.sourceforge.net added the comment:

I updated the documentation in issue 10709 so that it mentions ldedit rather 
than the linker flags (more convenient to activate support for more memory).

Once Python 3.2 is released, we may add LDR_CNTRL or ldedit in the Makefile 
when running buildbottest.

I will ask IBM if there is any reason not to activate support for 2GB in an 
application by default (impact on performances or something).
If there is none, we may systematically call ldedit -b maxdata:0x8000 
${srcdir}/python when compiling python on AIX...

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue11212
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11223] interruption of locks by signals not guaranteed when the semaphore implementation is not used


Sébastien Sablé sa...@users.sourceforge.net added the comment:

Here is what faulthandler reports when I trigger it as Python is locked in 
test_socket:


[108/349] test_socket
Traceback (most recent call first):
  File 
/san_cis/home/cis/.buildbot/python-aix6/3.x.phenix.xlc/build/Lib/test/fork_wait.py,
 line 30 in f
 (sic)

It is coherent with what truss tells:
_getpid()   = 1474634
_select(0, 0x, 0x, 0x, 0x317535F0) = 0
_getpid()   = 1474634
_select(0, 0x, 0x, 0x, 0x315E35F0) = 0


And we can find this comment in the file:
# waitpid() shouldn't hang, but some of the buildbots seem to hang
# in the forking tests.  This is an attempt to fix the problem.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue11223
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11223] interruption of locks by signals not guaranteed when the semaphore implementation is not used


Sébastien Sablé sa...@users.sourceforge.net added the comment:

faulthandler is great by the way! I will use that in my application.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue11223
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11223] interruption of locks by signals not guaranteed when the semaphore implementation is not used


Sébastien Sablé sa...@users.sourceforge.net added the comment:

At the time of the lock, there is only one python process running. So I guess 
the other processes have already left but this is not correctly detected.

I will try again with all_threads=True.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue11223
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11223] interruption of locks by signals not guaranteed when the semaphore implementation is not used


Sébastien Sablé sa...@users.sourceforge.net added the comment:

I kinda feel this is related to issue 11185 which I reported earlier.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue11223
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue10709] Misc/AIX-NOTES needs updating


Sébastien Sablé sa...@users.sourceforge.net added the comment:

Could this doc be added before the 3.2 release?
I think it is convenient for the (few) AIX users and harmless.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue10709
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11185] test_wait4 error on AIX


Sébastien Sablé sa...@users.sourceforge.net added the comment:

This issue already existed on Python 2.5.2 with AIX 5.2:

http://www.mail-archive.com/python-list@python.org/msg192219.html

The documentation for WNOHANG says:
http://docs.python.org/library/os.html#os.WNOHANG

The option for waitpid() to return immediately if no child process status is 
available immediately. The function returns (0, 0) in this case.


It seems wait4 always returns 0 on AIX when WNOHANG is specified.

Removing WNOHANG will make the test succeed.

waitpid does not have the same limitation.

I suppose this is a bug of AIX, though there is not even a man page to describe 
wait4 on this platform.

Here is a proposition for a patch that will workaround this bug...

Index: Lib/test/test_wait4.py
===
--- Lib/test/test_wait4.py  (revision 88430)
+++ Lib/test/test_wait4.py  (working copy)
@@ -3,6 +3,7 @@
 
 import os
 import time
+import sys
 from test.fork_wait import ForkWait
 from test.support import run_unittest, reap_children, get_attribute
 
@@ -13,10 +14,14 @@
 
 class Wait4Test(ForkWait):
 def wait_impl(self, cpid):
+option = os.WNOHANG
+if sys.platform.startswith('aix'):
+# wait4 is broken on AIX and will always return 0 with WNOHANG
+option = 0
 for i in range(10):
 # wait4() shouldn't hang, but some of the buildbots seem to hang
 # in the forking tests.  This is an attempt to fix the problem.
-spid, status, rusage = os.wait4(cpid, os.WNOHANG)
+spid, status, rusage = os.wait4(cpid, option)
 if spid == cpid:
 break
 time.sleep(1.0)

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue11185
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11212] Python memory limit on AIX

Sébastien Sablé sa...@users.sourceforge.net added the comment:

The AIX documentation says:
http://publib.boulder.ibm.com/infocenter/tivihelp/v2r1/index.jsp?topic=/com.ibm.itame3.doc_5.1/am51_perftune113.htm
It is a good idea to unset the LDR_CNTRL environment variable, so that it
does not unintentionally affect other processes.

I think people may be more interested in setting permanently the maximum memory
that can be allocated by python rather than using an environment variable that
may be lost and which impacts all the applications.

Another way to increase the memory available for an application is to use the
ldedit command on the python executable:

ldedit -b maxdata:0x2000 ./python

That may be easier than passing it directly to the linker.

Concerning large pages, I found that in an IBM documentation:

Users are advised to be cautious in their use of the environment
variable to specify large page usage. Performance tests have shown there
can be a significant performance loss in environments where a number of
shell scripts or small, short running applications are invoked. One
example saw a shell script's execution time increase over 10 times when
the large page environment variable was specified. Customers are advised
to only set the large page environment variable around specific
applications which can benefit from large page usage.

Also large pages must be activated on the system by root. This does not seem to
be available by default according to the doc.

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue11212
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11212] Python memory limit on AIX


Sébastien Sablé sa...@users.sourceforge.net added the comment:

The doc would become:


To allocate more segment registers to Python, you must use the linker option 
-bmaxdata or ldedit to specify the number of bytes you need in the data
 segment.
 
For example, if you want to allow 512MB of memory for Python (this
is enough for the test suite to run without MemroyErrors), you should
run the following command at the end of compilation:

 ldedit -b maxdata:0x2000 ./python


--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue11212
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue10709] Misc/AIX-NOTES needs updating


Sébastien Sablé sa...@users.sourceforge.net added the comment:

Yes it is correct. And thank you for proof reading it too.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue10709
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11188] test_time error on AIX


Sébastien Sablé sa...@users.sourceforge.net added the comment:

gmtime(-1) worked fine :/

 import time
 time.gmtime(-1)
time.struct_time(tm_year=1969, tm_mon=12, tm_mday=31, tm_hour=23, tm_min=59, 
tm_sec=59, tm_wday=2, tm_yday=365, tm_isdst=0)

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue11188
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue941346] AIX shared library fix


Sébastien Sablé sa...@users.sourceforge.net added the comment:

Are you using --enable-shared when compiling python?
That is when you should see the error (it could not find python.exp at some 
stage).

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue941346
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue941346] AIX shared library fix


Sébastien Sablé sa...@users.sourceforge.net added the comment:

By the way... thanks Georg!

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue941346
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue941346] AIX shared library fix


Sébastien Sablé sa...@users.sourceforge.net added the comment:

Also Michael, I am working on the 3.2 branch, it may be different with Python 
2.7.1 but I haven't tested yet.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue941346
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11184] Broken large file support on AIX


Sébastien Sablé sa...@users.sourceforge.net added the comment:

Here is the patch.
It only impacts AIX systems and is minimalist, so I think it should be safe for 
Python 3.2.

--
keywords: +patch
Added file: http://bugs.python.org/file20767/patch_aix_largefile.diff

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue11184
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11172] Avoid '.' as runpath on AIX


Sébastien Sablé sa...@users.sourceforge.net added the comment:

This has been corrected for python 3.2 in issue 941346.
Need to be tested on python 2.7.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue11172
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue10709] Misc/AIX-NOTES needs updating


Sébastien Sablé sa...@users.sourceforge.net added the comment:

Here is a first version of this doc for Python 3.2.
It would be great if someone could proof read it since I am not a native 
english speaker (though it is not high level literature...).

--
Added file: http://bugs.python.org/file20770/AIX-NOTES

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue10709
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11223] test_threadsignals.py hanging on AIX


New submission from Sébastien Sablé sa...@users.sourceforge.net:

The test test_lock_acquire_interruption in test_threadsignals.py will lock 
forever on AIX 5.3.
It works fine on 6.1.

truss shows it is stuck in thread_tsleep:
$ truss -p 1404986
thread_tsleep(0, 0x, 0x, 0x) (sleeping...)

--
messages: 128668
nosy: sable
priority: normal
severity: normal
status: open
title: test_threadsignals.py hanging on AIX
versions: Python 3.2

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue11223
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11223] test_threadsignals.py hanging on AIX


Sébastien Sablé sa...@users.sourceforge.net added the comment:

$ grep SEM pyconfig.h
#define HAVE_BROKEN_POSIX_SEMAPHORES 1
/* #undef HAVE_BROKEN_SEM_GETVALUE */
#define HAVE_SEM_GETVALUE 1
#define HAVE_SEM_OPEN 1
#define HAVE_SEM_TIMEDWAIT 1
#define HAVE_SEM_UNLINK 1
/* #undef POSIX_SEMAPHORES_NOT_ENABLED */
#ifndef _POSIX_PTHREAD_SEMANTICS
# define _POSIX_PTHREAD_SEMANTICS 1

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue11223
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11192] test_socket error on AIX


Sébastien Sablé sa...@users.sourceforge.net added the comment:

I am not sure this is directly related, but test_socket.py will hang forever on 
AIX 6.1 when this script is run as part of make buildbottest.

It won't hang when running the script alone.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue11192
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11223] interruption of locks by signals not guaranteed when the semaphore implementation is not used


Sébastien Sablé sa...@users.sourceforge.net added the comment:

Thanks Antoine, the patch solved the problem.

I have other tests locking semi-randomly on AIX 5.3 and/or AIX 6.1, like 
test_socket and test_io. That may be related.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue11223
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11223] interruption of locks by signals not guaranteed when the semaphore implementation is not used


Sébastien Sablé sa...@users.sourceforge.net added the comment:

Well I don't know precisely:

* when running make buildbottest, test_socket will systematically hang on AIX 
5.3 and 6.1.

* but when running ./python -Wd -E -bb Lib/test/test_socket.py, the tests 
complete rather quickly (with one failure cf 11192).

Ran 158 tests in 16.867s

FAILED (failures=1, skipped=4)

I will try with pdb or something.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue11223
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11223] interruption of locks by signals not guaranteed when the semaphore implementation is not used


Sébastien Sablé sa...@users.sourceforge.net added the comment:

test_socket didnot lock this time (I will retry next).

But test_subprocess.ProcessTestCasePOSIXPurePython.test_leaking_fds_on_error 
did.

Looping forever on the following syscall:

pipe(0x2FF1FD88)= 0
kfcntl(15, F_GETFD, 0x) = 0
kfcntl(15, F_SETFD, 0x0001) = 0
kfcntl(17, F_GETFD, 0x) = 0
kfcntl(17, F_SETFD, 0x0001) = 0
pipe(0x2FF1FD88)= 0
kfcntl(18, F_GETFD, 0x) = 0
kfcntl(18, F_SETFD, 0x0001) = 0
kfcntl(19, F_GETFD, 0x) = 0
kfcntl(19, F_SETFD, 0x0001) = 0
fstatx(15, 0x2FF1FAEC, 76, 0)   = 0
kioctl(15, 22528, 0x, 0x)   Err#25 ENOTTY
fstatx(15, 0x2FF1FD7C, 76, 0)   = 0
lseek(15, 0, 1) Err#29 ESPIPE
fstatx(18, 0x2FF1FAEC, 76, 0)   = 0
kioctl(18, 22528, 0x, 0x)   Err#25 ENOTTY
fstatx(18, 0x2FF1FD7C, 76, 0)   = 0
lseek(18, 0, 1) Err#29 ESPIPE
pipe(0x2FF1FD08)= 0
kfcntl(20, F_GETFD, 0x) = 0
kfcntl(20, F_SETFD, 0x0001) = 0
kfcntl(21, F_GETFD, 0x) = 0
kfcntl(21, F_SETFD, 0x0001) = 0
sigprocmask(0, 0xF029C148, 0xF029C150)  = 0
kfork() = 2011158
thread_setmymask_fast(0x, 0x, 0x, 0xD0504100, 
0x, 0x1216F00B, 0x1216F00B, 0x) = 0x
close(21)   = 0
close(17)   = 0
close(19)   = 0
kread(20,  O S E r r o r : 2 :\0 i.., 5)  = 10
kread(20, 0x33041C38, 5)= 0
close(20)   = 0
kwaitpid(0x2FF1FB50, 2011158, 4, 0x, 0x) = 2011158
close(15)   = 0
close(18)   = 0
close(15)   Err#9  EBADF
close(18)   Err#9  EBADF
pipe(0x2FF1FD88)= 0

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue11223
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue877121] configure detects incorrect compiler optimization


Sébastien Sablé sa...@users.sourceforge.net added the comment:

If have the same issue with Oracle Studio 12.2 on Solaris 10 with Python 2.7.1:


cc -G -L/livraison/test/sparc-sun-solaris2.10/support/support-2.6.x-py27/lib 
-L/livraison/test/sparc-sun-solaris2.10/support/support-internal-2.6.x-py27/lib 
-L/livraison/test/sparc-sun-solaris2.10/support/support-2.6.x-py27/lib 
-L/livraison/test/sparc-sun-solaris2.10/support/support-internal-2.6.x-py27/lib 
-OPT:Olimit=0 -g -DNDEBUG -xO4 -I. -IInclude -I./Include 
-I/livraison/test/sparc-sun-solaris2.10/support/support-2.6.x-py27/include 
-I/livraison/test/sparc-sun-solaris2.10/support/support-2.6.x-py27/include/ncurses
 
-I/livraison/test/sparc-sun-solaris2.10/support/support-internal-2.6.x-py27/include
 
build/temp.solaris-2.10-sun4u-2.7/san_u10/home/recette/ssa/support-2.6.x/Python-2.7.1/Modules/_struct.o
 -L/livraison/test/sparc-sun-solaris2.10/support/support-2.6.x-py27/lib 
-L/usr/local/lib -L. -lpython2.7 -o build/lib.solaris-2.10-sun4u-2.7/_struct.so
cc: Warning: Option -OPT:Olimit=0 passed to ld, if ld is invoked, ignored 
otherwise
/usr/ccs/bin/ld: illegal option -- O
usage: ld [-6:abc:d:e:f:h:il:mo:p:rstu:z:B:CD:F:GI:L:M:N:P:Q:R:S:VY:?] file(s)

--
nosy: +sable

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue877121
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue941346] AIX shared library fix


Sébastien Sablé sa...@users.sourceforge.net added the comment:

Hum, I was incorrect in previous note:
You can remove -L\$(srcdir) on this line, but then you need to specify the full 
path to Modules/python.exp. See patch below:

Index: configure.in
===
--- configure.in(revision 88422)
+++ configure.in(working copy)
@@ -1642,7 +1655,7 @@
 then
case $ac_sys_system/$ac_sys_release in
AIX*)
-   BLDSHARED=\$(srcdir)/Modules/ld_so_aix \$(CC) 
-bI:Modules/python.exp -L\$(srcdir)
+   BLDSHARED=\$(srcdir)/Modules/ld_so_aix \$(CC) 
-bI:\$(srcdir)/Modules/python.exp
LDSHARED=\$(BINLIBDEST)/config/ld_so_aix \$(CC) 
-bI:\$(BINLIBDEST)/config/python.exp
;;
IRIX/5*) LDSHARED=ld -shared;;


Also, I think there are other issues with building Python 3.2 with shared 
libraries. I am currently investigating.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue941346
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue941346] AIX shared library fix


Sébastien Sablé sa...@users.sourceforge.net added the comment:

OK, Python with shared libraries is broken in trunk since the library was 
renamed to libpython3.2m.
Here is a patch to correct that:

Index: Modules/ld_so_aix.in
===
--- Modules/ld_so_aix.in(revision 88422)
+++ Modules/ld_so_aix.in(working copy)
@@ -131,7 +131,7 @@
   shift
 done
 
-if test $objfile = libpython@VERSION@.so; then
+if test $objfile = libpython@VERSION@@ABIFLAGS@.so; then
   ldsocoremode=true 
 fi

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue941346
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue941346] AIX shared library fix


Sébastien Sablé sa...@users.sourceforge.net added the comment:

Here is the full patch.
I also modified Makefile.pre.in to clean Modules/python.exp when doing 
distclean.

--
Added file: http://bugs.python.org/file20764/patch_aix_shared_32.diff

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue941346
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11212] Python memory limit on AIX


Sébastien Sablé sa...@users.sourceforge.net added the comment:

This link flag should only be applied to the python executable.
There is already LINKFORSHARED which is great for that.
LDFLAGS will apply it to all the shared libraries which is not really what we 
want.

I can compile Python with support for 512MB without any modification to 
configure like that:

CC=xlc_r OPT=-O2 -qmaxmem=10 ./configure --without-computed-gotos 
--enable-shared
make LINKFORSHARED=-Wl,-bE:Modules/python.exp -lld -Wl,-bmaxdata:0x2000

It is just not very elegant and will make the builbdot script even more complex 
than it is currently.
Also it will make it more difficult for AIX users to find out about this 
limitation or build python with support for more memory (though I agree this 
represents very few people).

If you don't want it in configure though, I will add a note in AIX-NOTES about 
this limitation and how to build Python with support for more memory.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue11212
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11212] Python memory limit on AIX


Changes by Sébastien Sablé sa...@users.sourceforge.net:


--
versions: +Python 2.6, Python 2.7, Python 3.1

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue11212
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11212] Python memory limit on AIX


Sébastien Sablé sa...@users.sourceforge.net added the comment:

I don't know distutils well enough, but I am afraid the LDFLAGS flag used to 
compile python is reused in some way when linking some python extensions or 
some application that embed python.
(I use an application that embed python on AIX, I should check what happens 
when I compile it).

Except for that, I can't see any problem now, but I would not be feel 
comfortable propagating this option in LDFLAGS.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue11212
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11188] test_time error on AIX


Sébastien Sablé sa...@users.sourceforge.net added the comment:

Here is what I could find in the man mktime:

   Upon successful completion, the mktime subroutine sets the values of the 
tm_wday and tm_yday fields appropriately. Other fields are set to
   represent the specified time since January 1, 1970. However, the values 
are forced to the ranges specified in the /usr/include/time.h file.
   The final value of the tm_mday field is not set until the values of the 
tm_mon and tm_year fields are determined. Note: The mktime
   subroutine cannot convert time values before 00:00:00 UTC, January 1, 
1970 and after 03:14:07 UTC, January 19, 2038.


I tried tm_yday=-1 then tm_isdst=-2 and both gave the same errors as before.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue11188
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue640553] Misuse of /usr/local/in setup.py


Sébastien Sablé sa...@users.sourceforge.net added the comment:

This bug is still present in Python 2.7.1. I have also just been beaten by it 
on solaris.

I have libintl.so.1 in /usr/lib and libintl.so.8 in /usr/local/lib.

I have never specified that I wanted to use /usr/local/lib anywhere, but the 
compilation scripts insist on linking _locale.so with libintl.so.8, which may 
not be available on the platforms where I will deploy my python binaries.

/usr/local/lib should only be used if it is explicitly specified in LDFLAGS in 
my opinion.

--
nosy: +sable
versions: +Python 2.5, Python 2.6, Python 2.7

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue640553
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue640553] Misuse of /usr/local/in setup.py


Sébastien Sablé sa...@users.sourceforge.net added the comment:

I can only bypass this by explicitly specifying -L/usr/lib so that it will take 
precedence over -L/usr/local/lib.

But this behavior is still broken...
Explicit is better than implicit.
I can see no reason why /usr/local should be implicitly added; people that 
really need it can always do export LDFLAGS=$LDFLAGS -L/usr/local/lib

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue640553
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11188] test_time error on AIX


Sébastien Sablé sa...@users.sourceforge.net added the comment:

Python 3.2rc3+ (py3k:88422M, Feb 15 2011, 16:57:29) [C] on aix6
Type help, copyright, credits or license for more information.
 import time
 for t in (-2, -1, 0, 1):
... print(time.localtime(t))
... 
time.struct_time(tm_year=1970, tm_mon=1, tm_mday=1, tm_hour=0, tm_min=59, 
tm_sec=58, tm_wday=3, tm_yday=1, tm_isdst=0)
time.struct_time(tm_year=1970, tm_mon=1, tm_mday=1, tm_hour=0, tm_min=59, 
tm_sec=59, tm_wday=3, tm_yday=1, tm_isdst=0)
time.struct_time(tm_year=1970, tm_mon=1, tm_mday=1, tm_hour=1, tm_min=0, 
tm_sec=0, tm_wday=3, tm_yday=1, tm_isdst=0)
time.struct_time(tm_year=1970, tm_mon=1, tm_mday=1, tm_hour=1, tm_min=0, 
tm_sec=1, tm_wday=3, tm_yday=1, tm_isdst=0)

I am in Paris. On the system, I get:
$ date
Tue Feb 15 18:18:58 NFT 2011
$ env | grep TZ
TZ=NFT-1DFT,M3.5.0,M10.5.0

which is strange since NFT seems to be in Australia.

I will check that tomorrow with the sysadmin.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue11188
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue10709] Misc/AIX-NOTES needs updating


Sébastien Sablé sa...@users.sourceforge.net added the comment:

Antoine,

can you take a look at issue 730467? That's a trivial patch to correct a 
compilation issue on AIX with a C++ compiler; and that means one less entry in 
the AIX-NOTES file to write before the 3.2 release...

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue10709
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11193] test_subprocess error on AIX


Sébastien Sablé sa...@users.sourceforge.net added the comment:

I am not sure this is what you want:

 LC_ALL=C ./python
Python 3.2rc3 (py3k:88417M, Feb 14 2011, 10:37:42) 
[GCC 4.2.0] on aix6
Type help, copyright, credits or license for more information.
 import sys
 sys.getfilesystemencoding()
'iso8859-1'
 

What information can I provide to help solve this bug?

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue11193
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11212] Python memory limit on AIX


New submission from Sébastien Sablé sa...@users.sourceforge.net:

On AIX, executables are compiled by default so that they cannot allocate more 
than 256MB of memory.

This is not enough in some cases; for example this is not enough to get the 
Python test suite to run completely.

As explained in IBM documentation:

By default each program gets one segment register (see 2.24) for its
data segment. As each segment register covers 256 MB, any calls to
malloc more will fail. Also programs that declare large global or static
arrays may fail to load. To allocate more segment registers to your
program, use the linker option -bmaxdata to specify the number of bytes
you need in the data segment as follows:

cc -o myprog -bmaxdata:0x2000 myprog.c

The example above would allocate an additional segment register to allow
for 512MB of data.

export LDR_CNTRL=MAXDATA=0x2000
start_process
unset LDR_CNTRL

Marking An Executable For Large Page Use
The XCOFF header in an executable file contains a new flag to indicate that the 
program wants to use
large pages to back its data and heap segments. This flag can be set when the 
application is linked by
specifying the -blpdata option on the ld command. The flag can also be set or 
cleared using the ldedit
command. The ldedit -blpdata filename command sets the large page data/heap 
flag in the specified
file. The ldedit -bnolpdata filename clears the large page flag. The ldedit 
command may also be used
to set an executable's maxdata value

--
components: Build
messages: 128553
nosy: sable
priority: normal
severity: normal
status: open
title: Python memory limit on AIX
versions: Python 3.2

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue11212
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11212] Python memory limit on AIX


Sébastien Sablé sa...@users.sourceforge.net added the comment:

Here is a patch that would provide an --enable-aix-maxdata option to configure, 
in order to support 512MB of mermory.

I have not tested it yet.

--
keywords: +patch
Added file: http://bugs.python.org/file20761/patch_maxdata.diff

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue11212
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11193] test_subprocess error on AIX


Sébastien Sablé sa...@users.sourceforge.net added the comment:

$ LANG=C ./python -Wd -E -bb -c import sys; print(ascii(sys.argv)) $(echo -ne 
abc\xff)['-c', 'abc\xff']
['-c', 'abc\xff']
$ LANG=C ./python -Wd -E -bb -c import locale; 
print(locale.nl_langinfo(locale.CODESET))
ISO8859-1
$ LANG=C ./python -Wd -E -bb -c import locale; 
print(locale.getpreferredencoding())
ISO8859-1

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue11193
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11188] test_time error on AIX


Sébastien Sablé sa...@users.sourceforge.net added the comment:

I tried the following patch (_AIX is defined on AIX platforms):

Index: Modules/timemodule.c
===
--- Modules/timemodule.c(révision 88420)
+++ Modules/timemodule.c(copie de travail)
@@ -474,7 +474,7 @@
 else if (!gettmarg(tup, buf) || !checktm(buf))
 return NULL;
 
-#if defined(_MSC_VER) || defined(sun)
+#if defined(_MSC_VER) || defined(sun) || defined(_AIX)
 if (buf.tm_year + 1900  1 ||   buf.tm_year + 1900) {
 PyErr_Format(PyExc_ValueError,
  strftime() requires year in [1; ],
@@ -694,11 +694,12 @@
 time_t tt;
 if (!gettmarg(tup, buf))
 return NULL;
-buf.tm_wday = -1;  /* sentinel; original value ignored */
+/* invalid value that will not be changed if there is an error. */
+buf.tm_wday = 42;
 tt = mktime(buf);
 /* Return value of -1 does not necessarily mean an error, but tm_wday
  * cannot remain set to -1 if mktime succedded. */
-if (tt == (time_t)(-1)  buf.tm_wday == -1) {
+if ((tt == (time_t)(-1))  (buf.tm_wday == 42)) {
 PyErr_SetString(PyExc_OverflowError,
 mktime argument out of range);
 return NULL;

This resulted in the following:
==
ERROR: test_mktime (__main__.TestAsctime4dyear)
--
Traceback (most recent call last):
  File Lib/test/test_time.py, line 351, in test_mktime
self.assertEqual(time.mktime(tt), t)
OverflowError: mktime argument out of range

==
ERROR: test_mktime (__main__.TestStrftime4dyear)
--
Traceback (most recent call last):
  File Lib/test/test_time.py, line 351, in test_mktime
self.assertEqual(time.mktime(tt), t)
OverflowError: mktime argument out of range

==
ERROR: test_mktime (__main__.Test4dyearBool)
--
Traceback (most recent call last):
  File Lib/test/test_time.py, line 351, in test_mktime
self.assertEqual(time.mktime(tt), t)
OverflowError: mktime argument out of range

--
Ran 42 tests in 1.395s

FAILED (errors=3)

So test_negative is now OK, but tm_wday = 42 did not solve the problem it seems.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue11188
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue9920] test_cmath on atan fails on AIX


Sébastien Sablé sa...@users.sourceforge.net added the comment:

Here is the result after applying that patch:

==
FAIL: test_specific_values (__main__.CMathTests)
--
Traceback (most recent call last):
  File Lib/test/test_cmath.py, line 380, in test_specific_values
msg=error_message)
  File Lib/test/test_cmath.py, line 128, in rAssertAlmostEqual
'got {!r}'.format(a, b))
AssertionError: atanh0225: atanh(complex(-0.0, 5.6067e-320))
Expected: complex(-0.0, 5.6067e-320)
Received: complex(0.0, 5.6067e-320)
Received value insufficiently close to expected value.

--
Ran 16 tests in 0.081s

FAILED (failures=1, skipped=2)

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue9920
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11214] test_asyncore fails on AIX


New submission from Sébastien Sablé sa...@users.sourceforge.net:

When running test_asyncore on AIX, I get the following error:

[122/337] test_asyncore
test test_asyncore failed -- Traceback (most recent call last):
  File 
/san_cis/home/cis/.buildbot/python-aix6/3.x.phenix.xlc/build/Lib/test/test_asyncore.py,
 line 332, in test_strerror
self.assertIn(unknown error, err.lower())
AssertionError: 'unknown error' not found in 'error -1 occurred.'

I haven't investigated yet.

--
messages: 128563
nosy: sable
priority: normal
severity: normal
status: open
title: test_asyncore fails on AIX
versions: Python 3.2

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue11214
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11215] test_fileio error on AIX


New submission from Sébastien Sablé sa...@users.sourceforge.net:

I have the following error when running test_fileio on AIX:

==
FAIL: testAbles (test.test_fileio.OtherFileTests)
--
Traceback (most recent call last):
  File 
/san_cis/home/cis/.buildbot/python-aix6/3.x.phenix.xlc/build/Lib/test/test_fileio.py,
 line 269, in testAbles
self.assertEqual(f.seekable(), False)
AssertionError: True != False

--
Ran 34 tests in 0.032s

FAILED (failures=1)

I haven't investigated yet.

--
messages: 128564
nosy: sable
priority: normal
severity: normal
status: open
title: test_fileio error on AIX
versions: Python 3.2

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue11215
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11212] Python memory limit on AIX


Sébastien Sablé sa...@users.sourceforge.net added the comment:

Yes, welcome into the strange world of AIX...

You can allow up to 8 segments, which represent 2GB:
http://publib.boulder.ibm.com/infocenter/tivihelp/v2r1/index.jsp?topic=/com.ibm.itame3.doc_5.1/am51_perftune113.htm

We could change the option to become:
--with-aix-maxdata=value
where value could be between 0 and 8, with a default at 0 (nothing passed to 
the compiler).

Would that be OK with you?

The good news is that thanks to this patch I was able to play the full python 
test suite on AIX:

313 tests OK.
8 tests failed:
test_asyncore test_cmath test_distutils test_fileio test_locale
test_resource test_time test_wait4
16 tests skipped:
test_ctypes test_curses test_dbm_gnu test_epoll test_gdb
test_kqueue test_largefile test_ossaudiodev test_pep277
test_startfile test_tk test_ttk_guionly test_unicode_file
test_winreg test_winsound test_zipfile64

Not too bad.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue11212
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11184] test_io error on AIX


New submission from Sébastien Sablé sa...@users.sourceforge.net:

I get 2 errors when running test_io.py with trunk on AIX. 

==
ERROR: test_large_file_ops (__main__.CIOTest)
--
Traceback (most recent call last):
  File ./Lib/test/test_io.py, line 418, in test_large_file_ops
self.large_file_ops(f)
  File ./Lib/test/test_io.py, line 321, in large_file_ops
self.assertEqual(f.seek(self.LARGE), self.LARGE)
OverflowError: Python int too large to convert to C long

==
ERROR: test_large_file_ops (__main__.PyIOTest)
--
Traceback (most recent call last):
  File ./Lib/test/test_io.py, line 418, in test_large_file_ops
self.large_file_ops(f)
  File ./Lib/test/test_io.py, line 321, in large_file_ops
self.assertEqual(f.seek(self.LARGE), self.LARGE)
OverflowError: Python int too large to convert to C long

--
Ran 395 tests in 27.958s

FAILED (errors=2, skipped=8)

thanks in advance

--
components: IO
messages: 128374
nosy: sable
priority: normal
severity: normal
status: open
title: test_io error on AIX
versions: Python 3.2

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue11184
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11185] test_wait4 error on AIX


New submission from Sébastien Sablé sa...@users.sourceforge.net:

I get an error when running test_wait4 with trunk on AIX:

test_wait (__main__.Wait4Test) ... FAIL

==
FAIL: test_wait (__main__.Wait4Test)
--
Traceback (most recent call last):
  File 
/san_cis/home/cis/.buildbot/python-aix6/3.x.phenix.xlc/build/Lib/test/fork_wait.py,
 line 72, in test_wait
self.wait_impl(cpid)
  File ./Lib/test/test_wait4.py, line 23, in wait_impl
self.assertEqual(spid, cpid)
AssertionError: 0 != 1486954

--
Ran 1 test in 12.030s

FAILED (failures=1)
Traceback (most recent call last):
  File ./Lib/test/test_wait4.py, line 32, in module
test_main()
  File ./Lib/test/test_wait4.py, line 28, in test_main
run_unittest(Wait4Test)
  File 
/san_cis/home/cis/.buildbot/python-aix6/3.x.phenix.xlc/build/Lib/test/support.py,
 line 1145, in run_unittest
_run_suite(suite)
  File 
/san_cis/home/cis/.buildbot/python-aix6/3.x.phenix.xlc/build/Lib/test/support.py,
 line 1128, in _run_suite
raise TestFailed(err)
test.support.TestFailed: Traceback (most recent call last):
  File 
/san_cis/home/cis/.buildbot/python-aix6/3.x.phenix.xlc/build/Lib/test/fork_wait.py,
 line 72, in test_wait
self.wait_impl(cpid)
  File ./Lib/test/test_wait4.py, line 23, in wait_impl
self.assertEqual(spid, cpid)
AssertionError: 0 != 1486954

Thanks in advance

--
messages: 128375
nosy: sable
priority: normal
severity: normal
status: open
title: test_wait4 error on AIX
versions: Python 3.2

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue11185
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11184] test_io error on AIX


Sébastien Sablé sa...@users.sourceforge.net added the comment:

OK, so the following patch should help:

Index: configure.in
===
--- configure.in(revision 88393)
+++ configure.in(working copy)
@@ -1375,6 +1375,8 @@
 
 if test $use_lfs = yes; then
 # Two defines needed to enable largefile support on various platforms
+AC_DEFINE(_LARGEFILES, 1, 
+[This must be defined on some systems to enable large file support.])
 # These may affect some typedefs
 AC_DEFINE(_LARGEFILE_SOURCE, 1, 
 [This must be defined on some systems to enable large file support.])

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue11184
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11184] test_io error on AIX


Sébastien Sablé sa...@users.sourceforge.net added the comment:

the error is different now that _LARGEFILES is defined:

==
ERROR: test_large_file_ops (__main__.CIOTest)
--
Traceback (most recent call last):
  File ./Lib/test/test_io.py, line 418, in test_large_file_ops
self.large_file_ops(f)
  File ./Lib/test/test_io.py, line 321, in large_file_ops
self.assertEqual(f.seek(self.LARGE), self.LARGE)
IOError: [Errno 22] Invalid argument

==
ERROR: test_large_file_ops (__main__.PyIOTest)
--
Traceback (most recent call last):
  File ./Lib/test/test_io.py, line 418, in test_large_file_ops
self.large_file_ops(f)
  File ./Lib/test/test_io.py, line 321, in large_file_ops
self.assertEqual(f.seek(self.LARGE), self.LARGE)
IOError: [Errno 22] Invalid argument

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue11184
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11184] test_io error on AIX


Sébastien Sablé sa...@users.sourceforge.net added the comment:

Sorry I made a mistake in my previous patch (_LARGEFILES instead of 
_LARGE_FILES).

Here is a better one:

Index: configure.in
===
--- configure.in(révision 88395)
+++ configure.in(copie de travail)
@@ -1376,6 +1376,14 @@
 if test $use_lfs = yes; then
 # Two defines needed to enable largefile support on various platforms
 # These may affect some typedefs
+case $ac_sys_system/$ac_sys_release in
+AIX*)
+AC_DEFINE(_LARGE_FILES, 1, 
+[This must be defined on AIX systems to enable large file support.])
+   ;;
+*)
+;;
+esac
 AC_DEFINE(_LARGEFILE_SOURCE, 1, 
 [This must be defined on some systems to enable large file support.])
 AC_DEFINE(_FILE_OFFSET_BITS, 64,

The test fails in a different way now:


==
ERROR: test_large_file_ops (__main__.CIOTest)
--
Traceback (most recent call last):
  File ./Lib/test/test_io.py, line 418, in test_large_file_ops
self.large_file_ops(f)
  File ./Lib/test/test_io.py, line 323, in large_file_ops
self.assertEqual(f.write(bxxx), 3)
IOError: [Errno 27] File too large

==
ERROR: test_large_file_ops (__main__.PyIOTest)
--
Traceback (most recent call last):
  File ./Lib/test/test_io.py, line 418, in test_large_file_ops
self.large_file_ops(f)
  File ./Lib/test/test_io.py, line 323, in large_file_ops
self.assertEqual(f.write(bxxx), 3)
IOError: [Errno 27] File too large

--
Ran 395 tests in 27.958s


Here is your trace:
phenix:~/.buildbot/python-aix6/3.x.phenix.xlc/build\ ./python 
Python 3.2rc2+ (py3k:88393M, Feb 11 2011, 14:56:34) [C] on aix6
Type help, copyright, credits or license for more information.
 f = open('foo', 'wb')
[55983 refs]
 f.seek(2**32)
4294967296
[55987 refs]
 f = open('foo', 'wb')
__main__:1: ResourceWarning: unclosed file _io.BufferedWriter name='foo'
[55994 refs]
 f.truncate(2**32)
Traceback (most recent call last):
  File stdin, line 1, in module
IOError: [Errno 27] File too large
[56027 refs]

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue11184
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11184] test_io error on AIX


Sébastien Sablé sa...@users.sourceforge.net added the comment:

test_largefile complains about the filesystem having no largefile support.
It is probably the case, I will ask a sysadmin, and see if he can get me a file 
system with large file support so that I can test this feature.


 ./python -Wd -E -bb ./Lib/test/test_largefile.py 
Traceback (most recent call last):
  File ./Lib/test/test_largefile.py, line 163, in test_main
f.write(b'x')
IOError: [Errno 27] File too large

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File ./Lib/test/test_largefile.py, line 192, in module
test_main()
  File ./Lib/test/test_largefile.py, line 168, in test_main
raise unittest.SkipTest(filesystem does not have largefile support)
unittest.case.SkipTest: filesystem does not have largefile support
[81125 refs]

In the meantime, this test should probably be skipped just like in 
test_largefile.py and the patch with _LARGE_FILES should probably be applied 
too.

Merci pour l'aide

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue11184
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11184] Broken large file support on AIX


Changes by Sébastien Sablé sa...@users.sourceforge.net:


--
title: test_io error on AIX - Broken large file support on AIX

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue11184
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11184] Broken large file support on AIX


Sébastien Sablé sa...@users.sourceforge.net added the comment:

Also:

is it OK if I open a new issue for each broken unit test on AIX even if I have 
not investigated them at the moment?

I have a dozen broken unit tests on AIX that need to be investigated, but I 
don't want to spam the bug tracker followers too much.
So tell me if it is a good thing to report each problem as soon as possible or 
if I should do it step by step.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue11184
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11188] test_time error on AIX


New submission from Sébastien Sablé sa...@users.sourceforge.net:

I have the following errors on test_time on AIX:

==
ERROR: test_mktime (test.test_time.TestAsctime4dyear)
--
Traceback (most recent call last):
  File 
/san_cis/home/cis/.buildbot/python-aix6/3.x.phenix.xlc/build/Lib/test/test_time.py,
 line 351, in test_mktime
self.assertEqual(time.mktime(tt), t)
OverflowError: mktime argument out of range

==
ERROR: test_mktime (test.test_time.TestStrftime4dyear)
--
Traceback (most recent call last):
  File 
/san_cis/home/cis/.buildbot/python-aix6/3.x.phenix.xlc/build/Lib/test/test_time.py,
 line 351, in test_mktime
self.assertEqual(time.mktime(tt), t)
OverflowError: mktime argument out of range

==
ERROR: test_mktime (test.test_time.Test4dyearBool)
--
Traceback (most recent call last):
  File 
/san_cis/home/cis/.buildbot/python-aix6/3.x.phenix.xlc/build/Lib/test/test_time.py,
 line 351, in test_mktime
self.assertEqual(time.mktime(tt), t)
OverflowError: mktime argument out of range

==
FAIL: test_negative (test.test_time.TestStrftime4dyear)
--
Traceback (most recent call last):
  File 
/san_cis/home/cis/.buildbot/python-aix6/3.x.phenix.xlc/build/Lib/test/test_time.py,
 line 337, in test_negative
self.assertIn(text, ('-1', '-001'))
AssertionError: '000/' not found in ('-1', '-001')

--
Ran 42 tests in 1.217s

FAILED (failures=1, errors=3)

Haven't investigated yet.

--
messages: 128396
nosy: sable
priority: normal
severity: normal
status: open
title: test_time error on AIX
versions: Python 3.2

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue11188
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11189] test_resource error on AIX


New submission from Sébastien Sablé sa...@users.sourceforge.net:

I have the following error on test_resource on AIX:

test_resource
test test_resource failed -- Traceback (most recent call last):
  File 
/san_cis/home/cis/.buildbot/python-aix6/3.x.phenix.xlc/build/Lib/test/test_resource.py,
 line 28, in test_fsize_ismax
self.assertEqual(resource.RLIM_INFINITY, max)
AssertionError: 2147483647 != 1073741312

I haven't investigated yet.

--
messages: 128397
nosy: sable
priority: normal
severity: normal
status: open
title: test_resource error on AIX
versions: Python 3.2

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue11189
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11190] test_locale error on AIX


New submission from Sébastien Sablé sa...@users.sourceforge.net:

I have the following errors on test_locale on AIX:

==
FAIL: test_strcoll_with_diacritic (test.test_locale.TestEnUSCollation)
--
Traceback (most recent call last):
  File 
/san_cis/home/cis/.buildbot/python-aix6/3.x.phenix.xlc/build/Lib/test/test_locale.py,
 line 364, in test_strcoll_with_diacritic
self.assertLess(locale.strcoll('à', 'b'), 0)
AssertionError: 1 not less than 0

==
FAIL: test_strxfrm_with_diacritic (test.test_locale.TestEnUSCollation)
--
Traceback (most recent call last):
  File 
/san_cis/home/cis/.buildbot/python-aix6/3.x.phenix.xlc/build/Lib/test/test_locale.py,
 line 367, in test_strxfrm_with_diacritic
self.assertLess(locale.strxfrm('à'), locale.strxfrm('b'))
AssertionError: '\u01e2\u0100' not less than '\u0164\u0100'

--
Ran 35 tests in 0.020s

FAILED (failures=2)

I haven't investigated yet.

--
messages: 128398
nosy: sable
priority: normal
severity: normal
status: open
title: test_locale error on AIX
versions: Python 3.2

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue11190
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11191] test_search_cpp error on AIX (aith xlc)


New submission from Sébastien Sablé sa...@users.sourceforge.net:

I have the following error in distutils on AIX.

==
ERROR: test_search_cpp (distutils.tests.test_config_cmd.ConfigTestCase)
--
Traceback (most recent call last):
  File 
/san_cis/home/cis/.buildbot/python-aix6/3.x.phenix.xlc/build/Lib/distutils/unixccompiler.py,
 line 166, in preprocess
self.spawn(pp_args)
  File 
/san_cis/home/cis/.buildbot/python-aix6/3.x.phenix.xlc/build/Lib/distutils/ccompiler.py,
 line 911, in spawn
spawn(cmd, dry_run=self.dry_run)
  File 
/san_cis/home/cis/.buildbot/python-aix6/3.x.phenix.xlc/build/Lib/distutils/spawn.py,
 line 34, in spawn
_spawn_posix(cmd, search_path, dry_run=dry_run)
  File 
/san_cis/home/cis/.buildbot/python-aix6/3.x.phenix.xlc/build/Lib/distutils/spawn.py,
 line 138, in _spawn_posix
% (cmd[0], exit_status))
distutils.errors.DistutilsExecError: command 'xlc' failed with exit status 40

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File 
/san_cis/home/cis/.buildbot/python-aix6/3.x.phenix.xlc/build/Lib/distutils/tests/test_config_cmd.py,
 line 47, in test_search_cpp
match = cmd.search_cpp(pattern='xxx', body='// xxx')
  File 
/san_cis/home/cis/.buildbot/python-aix6/3.x.phenix.xlc/build/Lib/distutils/command/config.py,
 line 203, in search_cpp
src, out = self._preprocess(body, headers, include_dirs, lang)
  File 
/san_cis/home/cis/.buildbot/python-aix6/3.x.phenix.xlc/build/Lib/distutils/command/config.py,
 line 126, in _preprocess
self.compiler.preprocess(src, out, include_dirs=include_dirs)
  File 
/san_cis/home/cis/.buildbot/python-aix6/3.x.phenix.xlc/build/Lib/distutils/unixccompiler.py,
 line 168, in preprocess
raise CompileError(msg)
distutils.errors.CompileError: command 'xlc' failed with exit status 40

I haven't investigated yet.

--
assignee: tarek
components: Distutils
messages: 128399
nosy: eric.araujo, sable, tarek
priority: normal
severity: normal
status: open
title: test_search_cpp error on AIX (aith xlc)
versions: Python 3.2

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue11191
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11191] test_search_cpp error on AIX (with xlc)


Changes by Sébastien Sablé sa...@users.sourceforge.net:


--
title: test_search_cpp error on AIX (aith xlc) - test_search_cpp error on AIX 
(with xlc)

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue11191
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11192] test_socket error on AIX


New submission from Sébastien Sablé sa...@users.sourceforge.net:

I have the following error in test_socket on AIX:

==
FAIL: testGetaddrinfo (__main__.GeneralModuleTests)
--
Traceback (most recent call last):
  File ./Lib/test/test_socket.py, line 640, in testGetaddrinfo
self.assertEqual(socktype, socket.SOCK_STREAM)
AssertionError: 0 != 1

--
Ran 158 tests in 16.904s

FAILED (failures=1, skipped=4)
Traceback (most recent call last):
  File ./Lib/test/test_socket.py, line 2012, in module
test_main()
  File ./Lib/test/test_socket.py, line 2008, in test_main
support.run_unittest(*tests)
  File 
/san_cis/home/cis/.buildbot/python-aix6/3.x.phenix.xlc/build/Lib/test/support.py,
 line 1145, in run_unittest
_run_suite(suite)
  File 
/san_cis/home/cis/.buildbot/python-aix6/3.x.phenix.xlc/build/Lib/test/support.py,
 line 1128, in _run_suite
raise TestFailed(err)
test.support.TestFailed: Traceback (most recent call last):
  File ./Lib/test/test_socket.py, line 640, in testGetaddrinfo
self.assertEqual(socktype, socket.SOCK_STREAM)
AssertionError: 0 != 1

[119732 refs]

I haven't investigated yet.

--
messages: 128400
nosy: sable
priority: normal
severity: normal
status: open
title: test_socket error on AIX
versions: Python 3.2

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue11192
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11189] test_resource error on AIX


Sébastien Sablé sa...@users.sourceforge.net added the comment:

This is a duplicate of issue 678264.

--
resolution:  - duplicate
status: open - closed

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue11189
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11193] test_subprocess error on AIX