Re: [Python-Dev] test_subprocess and sparc buildbots

2008-12-30 Thread Alexandre Vassalotti
On Tue, Dec 30, 2008 at 10:41 PM, Daniel (ajax) Diniz  wrote:
> A reliable way to get that in a --with-pydebug build seems to be:
>
> ~/py3k$ ./python -c "import locale; locale.format_string(1,1)"
> * ob
> object  : 
> type: tuple
> refcount: 0
> address : 0x825c76c
> * op->_ob_prev->_ob_next
> NULL
> * op->_ob_next->_ob_prev
> object  : 
> type: tuple
> refcount: 0
> address : 0x825c76c
> Fatal Python error: UNREF invalid object
> TypeError: expected string or buffer
> Aborted
>

Nice catch! I reduced your example to: "import _sre;  _sre.compile(0,
0, [])". And, it doesn't seem to be an input validation problem with
_sre. From what I saw, it's actually a bug in Py_TRACE_REFS's code.
Now, it's getting interesting!

It seems something is breaking the refchain. However, I don't know
what is causing the problem exactly.

> Found using Fusil in a very quick run on top of:
> Python 3.1a0 (py3k:68055M, Dec 31 2008, 01:34:52)
> [GCC 4.2.4 (Ubuntu 4.2.4-1ubuntu3)] on linux2
>
> So kudos to Victor again :)
>

Could share the details on how you used Fusil to find another crasher?
It sounds like a useful tool.

Thanks!

-- Alexandre
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


[Python-Dev] I would like an svn account

2008-12-30 Thread Stephen J. Turnbull
Victor Stinner writes:

 > I already asked in September to get an svn account to be able to
 > commit directly patches to trunk (or other branches like py3k). My
 > query was rejected because I didn't know Python core enough (and
 > maybe other reasons that I don't know).

One possible reason is that commit privilege is not about quality of
code, it's about quality of review.  Would you review your own code in
the same way that other committers review their own?  Would you make
the same decisions about which fixes to commit, which changes to wait
for others' review, and which to propose on Python-Dev first?
Remember, to be appropriate for Python, a patch needs not only to be
good code, it must also be "Pythonic".  Does your personal sense of
code quality result in Pythonic patches?  (I can't answer that,
because my own sense of Pythonicity is dubiously reliable at
best.)

Another possible reason is that, while it's not an absolute
requirement, in my projects I'm always a lot more supportive of
candidates who have a track record of helping others get their patches
committed.  Of course if your patches have a history of being accepted
often without substantial change, then implicitly you are doing good
self-review, and that might be enough.  But in my book, that path
*should* take longer and demand higher standards than the "review
others' patches" path.

 > The bigger patch was the bytes filename support for Python3,
 > accepted by Guido (after a long review ;-)).

Would you have committed that patch if nobody else had reviewed it?

 > Just because there are not enough people to review/commit patches
 > on the tracker and

Are you planning to review and commit other people's patches, and help
reduce this backlog?  Or just your own?  Your emphasis on your own
working speed suggests the latter.  Again, I'm more supportive of
people who want commit privileges in part to help improve the
project's process, as well as to remove obstacles to their own work.

 > so there are more and more open issues (and so more and more lost
 > patches) :-(

An open issue is not a lost patch.  It's an open issue.  In my own
projects, I oppose candidates who seem to think that the presumption
is that a patch should be applied quickly unless there's good reason
given not to.  Your phrasing suggests that attitude to me.



You don't have to pay attention to me, since I don't have a vote in
the matter.  And I don't mean to be negatively critical of you,
because I'm not in a position to speak for the Powers That Be in
Python.  Those are my criteria, and other people and projects use
different ones.  But it seems to me that the committers in Python do
mostly conform to my criteria, and thus it's *possible* that those
criteria are somewhat representative of the "maybe other reasons [you]
don't know."

If so, I suppose an explicit explanation may be of use to you (and
others in your position).

Happy New Year to you!
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] A wart which should have been repaired in 3.0?

2008-12-30 Thread Phillip J. Eby

At 08:57 PM 12/30/2008 -0600, s...@pobox.com wrote:


Phillip> At 02:32 PM 12/30/2008 -0800, Scott David Daniels wrote:
>> More trouble with the "just take the dirname":
>>
>> paths = ['/a/b/c', '/a/b/d', '/a/b']
>> os.path.dirname(os.path.commonprefix([
>> os.path.normpath(p) for p in paths]))
>>
>> give '/a', not '/a/b'.

Phillip> ...because that's the correct answer.

I don't understand.  If you search for os.path.commonprefix at
codesearch.google.com you'll find uses like this:

if os.path.commonprefix([basedir, somepath]) != basedir:
...

which leads me to believe that other people using the current function in
the real world would be confused by your interpretation.


It never would've occurred to me to use it for that, versus checking 
for somepath.startswith(basedir+sep).


The only thing I've ever used commonprefix for is to find the 
most-specific directory that contains all the specified paths.  Never 
occurred to me that there was any other use for it, actually.


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] A wart which should have been repaired in 3.0?

2008-12-30 Thread Phillip J. Eby

At 09:30 PM 12/30/2008 -0500, rdmur...@bitdance.com wrote:

On Tue, 30 Dec 2008 at 17:51, Phillip J. Eby wrote:

At 02:32 PM 12/30/2008 -0800, Scott David Daniels wrote:

More trouble with the "just take the dirname":

 paths = ['/a/b/c', '/a/b/d', '/a/b']
 os.path.dirname(os.path.commonprefix([
 os.path.normpath(p) for p in paths]))
give '/a', not '/a/b'.


...because that's the correct answer.


But not the answer that is wanted.

So the challenge now is to write a single expression that will yield
'/a/b' when passed the above paths list, and also produce '/a/b' when
passed the following paths list:

paths = ['/a/b/c', '/a/b/cd']


Change that to [os.path.normpath(p)+'/' for p in paths] and you've 
got yourself a winner.


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] I would like an svn account

2008-12-30 Thread Raymond Hettinger

From: "Victor Stinner" 

Why an svn account instead of just using the amazing bug tracker? Just because 
there are not enough people to review/commit patches on the tracker and so 
there are more and more open issues (and so more and more lost patches) :-( I 
will be able to work faster using the svn.


Based on the work I've seen so far, my preference is that you continue to use 
the
tracker instead of directly committing patches.


Raymond

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] test_subprocess and sparc buildbots

2008-12-30 Thread Daniel (ajax) Diniz
Alexandre Vassalotti wrote:
> The logs of failing test runs all shows the same error message:
>
> [31481 refs]
> * ob
> object  : 
> type: str
> refcount: 0
> address : 0x3a97728
> * op->_ob_prev->_ob_next
> object  : 
> type: str
> refcount: 0
> address : 0x3a97728
> * op->_ob_next->_ob_prev
> object  : [31776 refs]

A reliable way to get that in a --with-pydebug build seems to be:

~/py3k$ ./python -c "import locale; locale.format_string(1,1)"
* ob
object  : 
type: tuple
refcount: 0
address : 0x825c76c
* op->_ob_prev->_ob_next
NULL
* op->_ob_next->_ob_prev
object  : 
type: tuple
refcount: 0
address : 0x825c76c
Fatal Python error: UNREF invalid object
TypeError: expected string or buffer
Aborted

Found using Fusil in a very quick run on top of:
Python 3.1a0 (py3k:68055M, Dec 31 2008, 01:34:52)
[GCC 4.2.4 (Ubuntu 4.2.4-1ubuntu3)] on linux2

So kudos to Victor again :)

HTH,
Daniel
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Missing FAQ about Python3 and unicode

2008-12-30 Thread Benjamin Peterson
On Tue, Dec 30, 2008 at 6:49 PM, Victor Stinner
 wrote:
> Hi,
>
> Slowly, we get recurrent questions about Python3 and unicode. It's maybe time
> to start a FAQ? Here is an ugly draft to start it ;-)

Looks like good stuff! It would probably make a good addition to the
meager porting docs in development on the wiki. [1]

...

[1] http://wiki.python.org/moin/PortingToPy3k



-- 
Regards,
Benjamin Peterson
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] A wart which should have been repaired in 3.0?

2008-12-30 Thread skip

Phillip> At 02:32 PM 12/30/2008 -0800, Scott David Daniels wrote:
>> More trouble with the "just take the dirname":
>> 
>> paths = ['/a/b/c', '/a/b/d', '/a/b']
>> os.path.dirname(os.path.commonprefix([
>> os.path.normpath(p) for p in paths]))
>> 
>> give '/a', not '/a/b'.

Phillip> ...because that's the correct answer.

I don't understand.  If you search for os.path.commonprefix at
codesearch.google.com you'll find uses like this:

if os.path.commonprefix([basedir, somepath]) != basedir:
...

which leads me to believe that other people using the current function in
the real world would be confused by your interpretation.

Skip


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] A wart which should have been repaired in 3.0?

2008-12-30 Thread rdmurray

On Tue, 30 Dec 2008 at 21:30, rdmur...@bitdance.com wrote:

On Tue, 30 Dec 2008 at 17:51, Phillip J. Eby wrote:

 At 02:32 PM 12/30/2008 -0800, Scott David Daniels wrote:
>  More trouble with the "just take the dirname":
> 
>   paths = ['/a/b/c', '/a/b/d', '/a/b']

>   os.path.dirname(os.path.commonprefix([
>   os.path.normpath(p) for p in paths]))
> 
>  give '/a', not '/a/b'.


 ...because that's the correct answer.


But not the answer that is wanted.

So the challenge now is to write a single expression that will yield
'/a/b' when passed the above paths list, and also produce '/a/b' when
passed the following paths list:

   paths = ['/a/b/c', '/a/b/cd']


Sorry, now I see what you are saying: that in '/a/b' the 'b' is the
filename.  Clearly that wasn't what I intuitively expected our
notional 'commonpathprefix' command to produce, for whatever
that is worth :)

--RDM
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] test_subprocess and sparc buildbots

2008-12-30 Thread Alexandre Vassalotti
Here is what I found just by analyzing the logs. It seems the first
failures appeared after this change:

http://svn.python.org/view/python/branches/release30-maint/Objects/object.c?rev=67888&view=diff&r1=67888&r2=67887&p1=python/branches/release30-maint/Objects/object.c&p2=/python/branches/release30-maint/Objects/object.c

The logs of failing test runs all shows the same error message:

[31481 refs]
* ob
object  : 
type: str
refcount: 0
address : 0x3a97728
* op->_ob_prev->_ob_next
object  : 
type: str
refcount: 0
address : 0x3a97728
* op->_ob_next->_ob_prev
object  : [31776 refs]

This is the output of _Py_ForgetReference (which calls _PyObject_Dump)
called either from _PyUnicode_New or unicode_subtype_new. In both
cases, this implies PyObject_MALLOC returned NULL when allocating the
internal array of a str object. However, I have no idea why malloc()
is failing there.

By counting the number of [reftotal] printed in the log, I found that
the failing test could be one of the following: test_invalid_args,
test_invalid_bufsize, test_list2cmdline, test_no_leaking. Looking at
the tests, it seems only test_no_leaking could be problematic:

* test_list2cmdline checks if the subprocess.line2cmdline function
  works correctly, only Python code is involved here;
* test_invalid_args checks if using an option unsupported by a
platform raises an
  exception, only Python code is involved here;
* test_invalid_bufsize only checks whether Popen rejects non-integer
bufsize, only
  Python code is involved here.

And unsurprisingly, that is the failing test:

test test_subprocess failed -- Traceback (most recent call last):
  File 
"/home/pybot/buildarea-sid/3.0.klose-debian-sparc/build/Lib/test/test_subprocess.py",
line 423, in test_no_leaking
data = p.communicate(b"lime")[0]
  File 
"/home/pybot/buildarea-sid/3.0.klose-debian-sparc/build/Lib/subprocess.py",
line 671, in communicate
return self._communicate(input)
  File 
"/home/pybot/buildarea-sid/3.0.klose-debian-sparc/build/Lib/subprocess.py",
line 1171, in _communicate
bytes_written = os.write(self.stdin.fileno(), chunk)
OSError: [Errno 32] Broken pipe

It seems one of the spawned processes goes out of memory while
allocating a new PyUnicode object. I believe we don't see the usual
MemoryError because the parent process catches stderr and stdout of
the children.

Also, only klose-*-sparc buildbots are failing this way; loewis-sun is
failing too but for a different reason. So, how much memory is
available on this machine (or actually, on this virtual machine)?

Now, I wonder why manipulating the GIL caused the bug to appear in
3.0, but not in 2.x. Maybe it is related to the new I/O library in
Python 3.0.

Regards,
-- Alexandre

On Tue, Dec 30, 2008 at 4:20 PM, Nick Coghlan  wrote:
> Does anyone have local access to a sparc machine to try to track down
> the ongoing buildbot failures in test_subprocess?
>
> (I think the problem is specific to 3.x builds on sparc machines, but I
> haven't checked the buildbots all that closely - that assessment is just
> based on what I recall of the buildbot failure emails).
>
> Cheers,
> Nick.
>
> --
> Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
> ---
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> http://mail.python.org/mailman/options/python-dev/alexandre%40peadrop.com
>
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] A wart which should have been repaired in 3.0?

2008-12-30 Thread rdmurray

On Tue, 30 Dec 2008 at 17:51, Phillip J. Eby wrote:

At 02:32 PM 12/30/2008 -0800, Scott David Daniels wrote:

More trouble with the "just take the dirname":

 paths = ['/a/b/c', '/a/b/d', '/a/b']
 os.path.dirname(os.path.commonprefix([
 os.path.normpath(p) for p in paths]))

give '/a', not '/a/b'.


...because that's the correct answer.


But not the answer that is wanted.

So the challenge now is to write a single expression that will yield
'/a/b' when passed the above paths list, and also produce '/a/b' when
passed the following paths list:

paths = ['/a/b/c', '/a/b/cd']

--RDM
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] I would like an svn account

2008-12-30 Thread Jesse Noller



On Dec 30, 2008, at 8:30 PM, Nick Coghlan  wrote:


Victor Stinner wrote:

Hi,

I already asked in September to get an svn account to be able to  
commit

directly patches to trunk (or other branches like py3k). My query was
rejected because I didn't know Python core enough (and maybe other  
reasons

that I don't know).

I helped to fix many issues using the bug tracker. The bigger patch  
was the

bytes filename support for Python3, accepted by Guido (after a long
review ;-)).

Why an svn account instead of just using the amazing bug tracker?  
Just because
there are not enough people to review/commit patches on the tracker  
and so
there are more and more open issues (and so more and more lost  
patches) :-( I

will be able to work faster using the svn.


+1 here

Cheers,
Nick.




Also +1 FWIW

Jesse
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] I would like an svn account

2008-12-30 Thread Nick Coghlan
Victor Stinner wrote:
> Hi,
> 
> I already asked in September to get an svn account to be able to commit 
> directly patches to trunk (or other branches like py3k). My query was 
> rejected because I didn't know Python core enough (and maybe other reasons 
> that I don't know).
> 
> I helped to fix many issues using the bug tracker. The bigger patch was the 
> bytes filename support for Python3, accepted by Guido (after a long 
> review ;-)).
> 
> Why an svn account instead of just using the amazing bug tracker? Just 
> because 
> there are not enough people to review/commit patches on the tracker and so 
> there are more and more open issues (and so more and more lost patches) :-( I 
> will be able to work faster using the svn.

+1 here

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
---
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


[Python-Dev] I would like an svn account

2008-12-30 Thread Victor Stinner
Hi,

I already asked in September to get an svn account to be able to commit 
directly patches to trunk (or other branches like py3k). My query was 
rejected because I didn't know Python core enough (and maybe other reasons 
that I don't know).

I helped to fix many issues using the bug tracker. The bigger patch was the 
bytes filename support for Python3, accepted by Guido (after a long 
review ;-)).

Why an svn account instead of just using the amazing bug tracker? Just because 
there are not enough people to review/commit patches on the tracker and so 
there are more and more open issues (and so more and more lost patches) :-( I 
will be able to work faster using the svn.

-- 
Victor Stinner aka haypo
http://www.haypocalc.com/blog/
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


[Python-Dev] Missing FAQ about Python3 and unicode

2008-12-30 Thread Victor Stinner
Hi,

Slowly, we get recurrent questions about Python3 and unicode. It's maybe time 
to start a FAQ? Here is an ugly draft to start it ;-)


(1) Exit on undecodable command line arguments

   $ LANG=en_GB.UTF-8 python3.0 test.py $'\xff'
   Could not convert argument 2 to string$

Is it an expected behaviour? Yes!

Example of the question: http://bugs.python.org/issue3023


(2) Undecodable filenames

os.listdir(str)->str raises an exception on undecodable filenames.

Solution: use os.listdir(bytes)->bytes. To display the filename to the user, 
use a function like:

   import sys
   def humanFilename(filename):
  encoding = sys.getfilesystemencoding()
  return filename.encode(encoding, "replace")

See also http://bugs.python.org/issue3187


(3) Bytes environment variables

Python 3.0 only supports decodable variables for os.environ. Undecodable 
variables are skipped for the creation of os.environ but original variables 
still exist at the C level.

$ A=$(echo -e "\xff") B=c ./python
Python 3.1a0 (py3k:67973M, Dec 31 2008, 00:51:49)
>>> import os
>>> os.environ.get('A'), os.environ.get('B')
(None, 'c')
>>> retcode=os.system('echo -n $A|hexdump -C')
  ff|.|
0001
>>> retcode=os.system('echo -n $B|hexdump -C')
  63|c|
0001

Discussion to support bytes environment variables:
http://mail.python.org/pipermail/python-dev/2008-December/083856.html

-- 
Victor Stinner aka haypo
http://www.haypocalc.com/blog/
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] extremely slow exit for program having huge (45G) dict (python 2.5.2)

2008-12-30 Thread Antoine Pitrou
Jesus Cea  jcea.es> writes:
> 
> Mike Coleman wrote:
> > I guess if ints are 12 bytes (per Beazley's book, but not sure if that
> > still holds), then that would correspond to a 1GB reduction.
> 
> Python 2.6.1 (r261:67515, Dec 11 2008, 20:28:07)
> [GCC 4.2.3] on sunos5
> Type "help", "copyright", "credits" or "license" for more information.
> >>> import sys
> >>> sys.getsizeof(0)
> 12

On a 32-bit system, sure, but given Mike creates a 45 GB dict, he has a 64-bit
system, where ints are 24 bytes:

>>> sys.getsizeof(0)
24
>>> sys.getsizeof(10)
24

cheers

Antoine.


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] extremely slow exit for program having huge (45G) dict (python 2.5.2)

2008-12-30 Thread Jesus Cea
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Mike Coleman wrote:
> I guess if ints are 12 bytes (per Beazley's book, but not sure if that
> still holds), then that would correspond to a 1GB reduction.

Python 2.6.1 (r261:67515, Dec 11 2008, 20:28:07)
[GCC 4.2.3] on sunos5
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> sys.getsizeof(0)
12

- --
Jesus Cea Avion _/_/  _/_/_/_/_/_/
j...@jcea.es - http://www.jcea.es/ _/_/_/_/  _/_/_/_/  _/_/
jabber / xmpp:j...@jabber.org _/_/_/_/  _/_/_/_/_/
.  _/_/  _/_/_/_/  _/_/  _/_/
"Things are not so easy"  _/_/  _/_/_/_/  _/_/_/_/  _/_/
"My name is Dump, Core Dump"   _/_/_/_/_/_/  _/_/  _/_/
"El amor es poner tu felicidad en la felicidad de otro" - Leibniz
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.8 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iQCVAwUBSVq3+Zlgi5GaxT1NAQLYUAP+Jc0JPYf2GPdNCKypORO+mD887xs81hQ0
MM7QBbRgLflcQ6g2tijpWPhN2/INscbtFn41lptHEYFTv/kka9EICuxgoNP1COYT
Or+1uChnSsx1Z7Xxr8YwLFe6ZW/LDyvPjCMpIT32mGSlc1/mfPZk3WjpqTJPeCwY
vqu9xD0T0iw=
=gXQ5
-END PGP SIGNATURE-
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] A wart which should have been repaired in 3.0?

2008-12-30 Thread Phillip J. Eby

At 02:32 PM 12/30/2008 -0800, Scott David Daniels wrote:

More trouble with the "just take the dirname":

paths = ['/a/b/c', '/a/b/d', '/a/b']
os.path.dirname(os.path.commonprefix([
os.path.normpath(p) for p in paths]))

give '/a', not '/a/b'.


...because that's the correct answer.

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] A wart which should have been repaired in 3.0?

2008-12-30 Thread Scott David Daniels

Paul Moore wrote:

2008/12/30 Phillip J. Eby :

You know, all this path separator and list complication isn't really
necessary, when you can just take the os.path.dirname() of the return from
commonprefix()


Actually, consider: ...

os.path.commonprefix(["foo\\bar\\baz", "foo/bar/boink"])

'foo'

... I'm not sure how to deal with this, except by recommending that all
paths passed to os.path.commonprefix should at the very least be
normalised via os.path.normpath first - which starts to get clumsy
fast. So the "recommended" usage to get the common directory is

paths = [...]
common = os.path.dirname(os.path.commonprefix([

>os.path.normpath(p) for p in paths]))


More trouble with the "just take the dirname":

paths = ['/a/b/c', '/a/b/d', '/a/b']
os.path.dirname(os.path.commonprefix([
os.path.normpath(p) for p in paths]))

give '/a', not '/a/b'.

--Scott David Daniels
scott.dani...@acm.org

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Python 3 - Mac Installer?

2008-12-30 Thread Benjamin Peterson
On Tue, Dec 30, 2008 at 3:45 PM, Barry Scott  wrote:
>
> build-install.py seems to have been half converted to py 3.0.
> Going full 3.0 was not hard but then there is the problem of
> the imports.

Thanks for your help, but just today Ronald Oussoren, the Mac
maintainer, spent some time making the installer work. As a result, we
should be ready to go for 3.0.1!

>
> Python 3.0 does not have MacOS or Carbon modules.
>
> Seems that there are two ways to go.
>
> Put back the Carbon and MacOS modules into 3.0.
> Use Python 2 to build the python 3 package.

I've converted it back to 2.x for the time being. Eventually, I think
some 3.x bindings should be released.



-- 
Regards,
Benjamin Peterson
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Python 3 - Mac Installer?

2008-12-30 Thread Barry Scott


On 26 Dec 2008, at 23:30, Benjamin Peterson wrote:

On Fri, Dec 26, 2008 at 2:55 AM, Mark Summerfield  
 wrote:

Hi,

Just wondered if/when there'd be a Mac installer for Python 3?


I think there should be one eventually. Unfortunately, the 3.x build
process is not ironed out. If somebody wants to make a patch which
makes the build script in Mac/BuildScript/ work, I'd be very happy. :)


Since I've been building 3.0 for a while now I looked at the script.

build-install.py seems to have been half converted to py 3.0.
Going full 3.0 was not hard but then there is the problem of
the imports.

Python 3.0 does not have MacOS or Carbon modules.

Seems that there are two ways to go.

Put back the Carbon and MacOS modules into 3.0.
Use Python 2 to build the python 3 package.

Barry

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


[Python-Dev] test_subprocess and sparc buildbots

2008-12-30 Thread Nick Coghlan
Does anyone have local access to a sparc machine to try to track down
the ongoing buildbot failures in test_subprocess?

(I think the problem is specific to 3.x builds on sparc machines, but I
haven't checked the buildbots all that closely - that assessment is just
based on what I recall of the buildbot failure emails).

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
---
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Commands for correctly merging to the python 3.0 maintenance branch

2008-12-30 Thread Nick Coghlan
Martin v. Löwis wrote:
>> svn revert .
>> svnmerge -M -F 
> 
> [are you sure you don't need a command for svnmerge here?]

D'oh, I thought I fixed that before sending the message. Yes, that line
should indeed be:

svnmerge merge -M -F 

> Instead of these two, I always do
> 
>   svn resolved .

That's what I had been doing before today, and I believe it works
correctly so long as you never get the svn update and svnmerge merge
operations out of sequence (i.e. always update and only then merge).

However, I encountered the case today where I had already merged to the
maintenance branch and did the svn update afterwards. In that situation,
reverting the property changes and reapplying them was the only way for
me to avoid losing the record of the changes everyone else had already
merged.

If I hadn't checked the property diff and noticed that several merged
revisions were no longer listed in the property in my working copy, then
svnmerge may have become very confused. The
revert+redo-merge-bookkeeping approach is definitely slower than just
marking the conflict as resolved, but has a definite advantage in doing
the right thing even if the earlier update+merge operations were
performed out of sequence (or if an extra update becomes necessary due
to checkins after the merge was performed).

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
---
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] A wart which should have been repaired in 3.0?

2008-12-30 Thread skip

Phillip> But it doesn't handle the fact that Windows paths are
Phillip> case-insensitive, or that Posix paths can have symlinks...  or
Phillip> that one path might be relative and another absolute...

Phillip> As soon as you move away from being a string operation, you get
Phillip> an endless series of gotchas...  none of which are currently
Phillip> documented.

Well, then we can document (some of?) the gotchas* and work on a better
implementation of commonpathprefix.  I don't do Windows.  You're lucky I got
as far as I did with the Windows side of things. ;-)

Skip

* I would argue that symlinks should be transparent.  By the very nature of
  the operations and the fact that they might be performed on other
  platforms (import posixpath on Windows for instance) there is not much, if
  anything, you can infer about the paths themselves other than their
  structure.

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


[Python-Dev] WinCE port (issues #4075 #4051)

2008-12-30 Thread Ulrich Eckhardt
Hi!

I'm currently working again on the CE port, and since 2.6 and 3.0 are now out 
of the door, could you apply the patches in #4075 & #4051? Both patches are 
fairly isolated and easy to review and I'm pretty sure they won't cause any 
inconveniences.

Note: this is far from everything that is necessary for Python to rock on CE, 
but these are prerequisites, as explained in both bugs' histories.

thanks

Uli
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] A wart which should have been repaired in 3.0?

2008-12-30 Thread Phillip J. Eby

At 06:14 AM 12/30/2008 -0600, s...@pobox.com wrote:


Paul demonstrates the shortcoming of commonprefix:

>>> os.path.commonprefix(["foo\\bar\\baz", "foo/bar/boink"])
'foo'

With the patch in issue4755:

>>> import ntpath
>>> ntpath.commonpathprefix(["foo\\bar\\baz", "foo/bar/boink"])
'foo\\bar'


But it doesn't handle the fact that Windows paths are 
case-insensitive, or that Posix paths can have symlinks...  or that 
one path might be relative and another absolute...


As soon as you move away from being a string operation, you get an 
endless series of gotchas...  none of which are currently documented.


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] A wart which should have been repaired in 3.0?

2008-12-30 Thread skip

Paul demonstrates the shortcoming of commonprefix:

>>> os.path.commonprefix(["foo\\bar\\baz", "foo/bar/boink"])
'foo'

With the patch in issue4755:

>>> import ntpath
>>> ntpath.commonpathprefix(["foo\\bar\\baz", "foo/bar/boink"])
'foo\\bar'

Ta da ...

Skip
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Commands for correctly merging to the python 3.0 maintenance branch

2008-12-30 Thread Martin v. Löwis
> svn revert .
> svnmerge -M -F 

[are you sure you don't need a command for svnmerge here?]

Instead of these two, I always do

  svn resolved .

Regards,
Martin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] A wart which should have been repaired in 3.0?

2008-12-30 Thread Paul Moore
2008/12/30 Phillip J. Eby :
> You know, all this path separator and list complication isn't really
> necessary, when you can just take the os.path.dirname() of the return from
> commonprefix().
>
> Perhaps we could just add that recommendation to the docs?

Actually, consider the following (on Windows):

>python
Python 2.6.1 (r261:67517, Dec  4 2008, 16:51:00) [MSC v.1500 32 bit
(Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import os
>>> os.path.commonprefix(["foo\\bar\\baz", "foo/bar/boink"])
'foo'
>>>

This very clearly shows that commonprefix is a string operation rather
than a path operation, as it does not respect the equivalence of
os.sep and os.altsep. In path semantics, the common prefix is
"foo/bar" (or equivalently "foo\\bar").

I'm not sure how to deal with this, except by recommending that all
paths passed to os.path.commonprefix should at the very least be
normalised via os.path.normpath first - which starts to get clumsy
fast. So the "recommended" usage to get the common directory is

paths = [...]
common = os.path.dirname(os.path.commonprefix([os.path.normpath(p)
for p in paths]))

Hmm...

Paul.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com