Re: [Python-Dev] fixing tests on windows

2008-04-02 Thread Trent Nelson

> def rename_and_remove (filename):
>  os.rename (filename, filename + ".deleted")
>  os.remove (filename + ".deleted")

> Isn't this still going to run into problems when the rename
> fails because the earlier tests remove still left the .deleted
> file around due to some other running desktop search service
> that now has the .deleted file open?

I haven't looked into all the various places the tests write temp files to, but 
if we could localise everything to a common root directory, i.e. 
%TEMP%\python-regrtest, we could then attempt to blow this away at the start of 
regrtest.py before any tests run, and refuse to run if this fails.  This would 
be in combination with the unlinking/renaming approach discussed.  This 
approach seems like it would cover all bases a bit more effectively.

Trent.
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] No time for svn merge

2008-04-02 Thread Trent Nelson
> > Yes, that's all I meant:  make it the committer's job
> > to merge or block as appropriate.  I just wasn't sure if
> > there was some reason that this would be difficult or
> > undesirable.
>
> Ah, yes. It is indeed difficult or undesirable, or was
> so in the past: Some committers don't care (didn't care)
> at all about 3k. They would have to setup sandboxes,
> learn what the nature of changes is, and invest some
> regular time into forward-porting.

Is this *really* the case still?  Who are these rogue committers? ;-)

I think holding a developer accountable for merging or blocking to py3k when 
they commit to trunk is a great idea.  Who better to pass judgement on such an 
activity than the person closest to it?

Trent.

___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] [Distutils] FW: [issue2513] 64bit cross compilation on windows

2008-04-02 Thread Trent Nelson
> Further, I
> assert that there are a greater number of build tools which do not support
> cross-compilation, but will build natively on x64 and expect 'PCBuild'
> to have libraries they can link with to create an x64 binary.

I'm with Martin on this one as well I think.  If I understand correctly, you're 
proposing:

PCbuild - may have contents of x86 or x64 depending
  on the build machine's architecture
PCbuild/amd64   - always x64
PCbuild/x86 - always x86

And what we've currently got is:

PCbuild/- always x86
PCbuild/amd64   - always x64

I think this is fine; we don't really have a notion of compiling for a native 
platform, nor is the build machine's architecture factored into the equation.  
I believe this is a Good Thing.  If you want a 32-bit build, use the 'Win32' 
target platform in VS; if you want a 64-bit build, use 'x64'.

Trent.


___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] [Python-3000] the release gods are angry at python

2008-04-02 Thread Trent Nelson
> > In the py3k branch I've assigned the audio resource to the winsound
> > tests. Only regrtest.py -uall or -uaudio runs the winsound test.
> Reason:
> > the test sound was freaking out my poor cat. :/
>
> I feel with your cat ;-).
> This would not help on the buildbot since it runs 'rt.bat -d -q -uall -
> rw'.

I feel for the poor NOC engineers at my colo that freak out when some random 
server in a farm of thousands starts making bizarre sounds.

I detest test_winsound.  There are so many corner cases you need to account for 
that makes the test pointless as you end up wrapping everything in except: pass 
blocks.  Does the system have a legacy beep driver?  Is it enabled?  Is it 
disabled?  Is there a sound card?  Is it enabled or disabled?  Pah!

+1 to removing audio out of -uall, if only for the sake of cats, erroneously 
red buildbots, and poor ServerCentral NOC engineers.

Trent.

___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] fixing tests on windows

2008-04-02 Thread Tim Golden
Gregory P. Smith wrote:
> On Tue, Apr 1, 2008 at 9:13 AM, Tim Golden <[EMAIL PROTECTED]> wrote:
> 
>> Tim Golden wrote:

>> 
>> import os, sys
>> import win32file
>>
>> FILENAME = "test"
>>
>> def rename_and_remove (filename):
>>   os.rename (filename, filename + ".deleted")
>>   os.remove (filename + ".deleted")

> Isn't this still going to run into problems when the rename fails because
> the earlier tests remove still left the .deleted file around due to some
> other running desktop search service that now has the .deleted file open?

I admit: this did occur to me on the train this am. While I
try to think of a robust way to handle this, other people have
proposed variations on pid-based / tempdir based filenames instead
of the same name for each test. In principle this sounds good to me,
but I'm not at all well-placed to assess the impact it might have
on the unit tests in general.

TJG
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] xmlrpclib and dates before 1900

2008-04-02 Thread Christian Heimes
Martin v. Löwis schrieb:
> Can you please explain why this is an important problem?
> Dates before 1900 have all passed long ago, so they shouldn't
> occur that often in real applications.

Does xmlrpc support dates for 1900? For historic dates the Julian Day
Number family (MJD or JDN) or Rata Die are more appropriate and much
easier to use. I wish somebody could add both to the datetime module.

Christian

___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] [Python-3000] the release gods are angry at python

2008-04-02 Thread Amaury Forgeot d'Arc
On Wed, Apr 2, 2008 at 9:58 AM, Trent Nelson <[EMAIL PROTECTED]> wrote:
> > > In the py3k branch I've assigned the audio resource to the winsound
>  > > tests. Only regrtest.py -uall or -uaudio runs the winsound test.
>  > Reason:
>  > > the test sound was freaking out my poor cat. :/
>  >
>  > I feel with your cat ;-).
>  > This would not help on the buildbot since it runs 'rt.bat -d -q -uall -
>  > rw'.
>
>  I feel for the poor NOC engineers at my colo that freak out when some random 
> server in a farm of thousands starts making bizarre sounds.
>
>  I detest test_winsound.  There are so many corner cases you need to account 
> for that makes the test pointless as you end up wrapping everything in 
> except: pass blocks.  Does the system have a legacy beep driver?  Is it 
> enabled?  Is it disabled?  Is there a sound card?  Is it enabled or disabled? 
>  Pah!
>
>  +1 to removing audio out of -uall, if only for the sake of cats, erroneously 
> red buildbots, and poor ServerCentral NOC engineers.

And I would not mind removing this module altogether, and provide a
ctypes implementation.

-- 
Amaury Forgeot d'Arc
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] fixing tests on windows

2008-04-02 Thread Tim Golden
Nick Coghlan wrote:
> Tim Golden wrote:
>> I admit: this did occur to me on the train this am. While I
>> try to think of a robust way to handle this, other people have
>> proposed variations on pid-based / tempdir based filenames instead
>> of the same name for each test. In principle this sounds good to me,
>> but I'm not at all well-placed to assess the impact it might have
>> on the unit tests in general.
> 
> Personally, I've never really understood the purpose of 
> test_support.TESTFN. Whenever I've needed a temporary file for a test, I 
> just use the tempfile module (e.g. test_cmd_line_script, test_runpy). 
> Tests using that module don't care if the old files take 'a while' to 
> get deleted on Windows, as tempfile uses a different name each time anyway.
> 
> Is using a fixed TESTFN just an old approach that predates the existence 
> of a robust tempfile module in the standard library?

I'm a neophyte when it comes to core development, so I've simply
cloned existing tests, assumed that there was some kind of (possibly 
unwritten) standard which used test_support.TESTFN. As I look at
it, though it seems a slightly odd choice, although it has variants
for testing unicode filenames specifically which I imagine are
useful in some places.

I'm perfectly happy to run through the test suite, patching it one
way or another. The trouble is that I've little confidence that I
can assess whether or not such a change will have affected the
actual meaning of a test. And, since these are tests,
Quis custodiet...?

TJG
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] fixing tests on windows

2008-04-02 Thread Nick Coghlan
Tim Golden wrote:
> I admit: this did occur to me on the train this am. While I
> try to think of a robust way to handle this, other people have
> proposed variations on pid-based / tempdir based filenames instead
> of the same name for each test. In principle this sounds good to me,
> but I'm not at all well-placed to assess the impact it might have
> on the unit tests in general.

Personally, I've never really understood the purpose of 
test_support.TESTFN. Whenever I've needed a temporary file for a test, I 
just use the tempfile module (e.g. test_cmd_line_script, test_runpy). 
Tests using that module don't care if the old files take 'a while' to 
get deleted on Windows, as tempfile uses a different name each time anyway.

Is using a fixed TESTFN just an old approach that predates the existence 
of a robust tempfile module in the standard library?

Cheers,
Nick.

-- 
Nick Coghlan   |   [EMAIL PROTECTED]   |   Brisbane, Australia
---
 http://www.boredomandlaziness.org
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] No time for svn merge

2008-04-02 Thread Christian Heimes
Trent Nelson schrieb:
> I think holding a developer accountable for merging or blocking to py3k when 
> they commit to trunk is a great idea.  Who better to pass judgement on such 
> an activity than the person closest to it?

Blocking a revision makes my job as The Merger easier.

I'm not so sure about the merging part. It takes some experience with
the Python 3.0 code base to know the subtle differences in the C API.
Most merges are straight forward for me. If you enforce the forward
merging on every developer it may slow down development.
Each regular merge takes me about 45 minutes of computer time but less
than 15 supervisor time. The computer time is mostly compile and test
time in the background. If everybody merges her own code to 3.0 it still
takes *everybody* about 10 minutes of time and 45 minutes of computer time.

Christian
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] fixing tests on windows

2008-04-02 Thread Tim Golden
Tim Golden wrote:
> Gregory P. Smith wrote:
>> On Tue, Apr 1, 2008 at 9:13 AM, Tim Golden <[EMAIL PROTECTED]> wrote:
>>
>>> Tim Golden wrote:
> 
>>> 
>>> import os, sys
>>> import win32file
>>>
>>> FILENAME = "test"
>>>
>>> def rename_and_remove (filename):
>>>   os.rename (filename, filename + ".deleted")
>>>   os.remove (filename + ".deleted")
> 
>> Isn't this still going to run into problems when the rename fails because
>> the earlier tests remove still left the .deleted file around due to some
>> other running desktop search service that now has the .deleted file open?

I've looked around this issue as far as I can, and I can't
come up with a foolproof way of guaranteeing that this won't
happen. I considered, eg, trying to open the test file with
semantics which preclude FILE_SHARE_DELETE from succeeding
(ie stopping the search services / TSVN from getting a handle
at all) but I can't find anything which doesn't cause more
problems than it solves.

Lest this discussion descend into inactivity, I've done a
fresh checkout and I'm producing a patch which will add
open_testfile and close_testfile functions to test_support
(or something like that) and which will update all the tests
to use those. I'm inclined to go with the tempfile approach
but at least having the functions means that that could
change.

This is obviously a wide-reaching change, but unless anyone's
got better ideas, the alternative seems to be intermittently-failing
test runs on the Windows buildbots.

TJG
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Thread-safe file objects, the return

2008-04-02 Thread Antoine Pitrou

Guido van Rossum  python.org> writes:
> Your solution (a counter) seems fine except I think perhaps the
> close() call should not raise IOError -- instead, it should set a flag
> so that the thread that makes the counter go to zero can close the
> thread (after all the file got closed while it was being used).

I agree with Gregory: we should be explicit about what happens. I wonder
what we would gain from that approach - apart from encouraging dangerous
coding practices :)
It also depends how far we want to go: I am merely proposing to fix the 
crashes, do we want to provide a "smarter" close() variation that does what
you suggest for people that want (or need) to take the risk?

> There are of course other concurrency issues besides close -- what if
> two threads both try to do I/O on the file? What will the C stdio
> library do in that case? Are stdio files thread-safe at the C level?

According to the glibc documentation, at
http://www.gnu.org/software/libc/manual/html_node/Streams-and-Threads.html :

« The POSIX standard requires that by default the stream operations are
atomic. I.e., issuing two stream operations for the same stream in two
threads at the same time will cause the operations to be executed as if
they were issued sequentially. The buffer operations performed while
reading or writing are protected from other uses of the same stream. To do
this each stream has an internal lock object which has to be (implicitly)
acquired before any work can be done. »

So according to POSIX rules it should be perfectly safe.
In any case, someone would have to try my patch under Windows and OS X and 
see if test_file.py passes without crashing.

Regards

Antoine.


___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] xmlrpclib and dates before 1900

2008-04-02 Thread Ralf Schmitt
On Wed, Apr 2, 2008 at 11:42 AM, Christian Heimes <[EMAIL PROTECTED]> wrote:

> Martin v. Löwis schrieb:
> > Can you please explain why this is an important problem?
> > Dates before 1900 have all passed long ago, so they shouldn't
> > occur that often in real applications.
>

In the application where I needed it, the customer wanted to send/store
dates for e.g. the date of birth of some people.


>
> Does xmlrpc support dates for 1900? For historic dates the Julian Day


The xmlrpc spec says dates should be sent in the following format:

 19980717T14:08:55
1900 is a rather arbitrary limit with this format.

Note that the unpatched xmlrpclib is able to receive datetime objects with
dates before 1900:
~/ /usr/bin/python2.5   [EMAIL 
PROTECTED]
Python 2.5.2 (r252:60911, Mar  9 2008, 11:14:55)
[GCC 4.2.3 (Debian 4.2.3-2)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import xmlrpclib, datetime
>>>
xmlrpclib.loads('\n\n18500101T00:00:00\n\n\n',
use_datetime=True)
((datetime.datetime(1850, 1, 1),), None)

Dumping however doesn't work:
>>> xmlrpclib.dumps((datetime.datetime(1850, 1, 1),))
Traceback (most recent call last):
  File "", line 1, in 
  File "/usr/lib/python2.5/xmlrpclib.py", line 1080, in dumps
data = m.dumps(params)
  File "/usr/lib/python2.5/xmlrpclib.py", line 623, in dumps
dump(v, write)
  File "/usr/lib/python2.5/xmlrpclib.py", line 635, in __dump
f(self, value, write)
  File "/usr/lib/python2.5/xmlrpclib.py", line 725, in dump_datetime
write(value.strftime("%Y%m%dT%H:%M:%S"))
  File "datetime.py", line 791, in strftime
return _wrap_strftime(self, fmt, self.timetuple())
  File "datetime.py", line 181, in _wrap_strftime
"methods require year >= 1900" % year)
ValueError: year=1850 is before 1900; the datetime strftime() methods
require year >= 1900

This ValueError just shows an implementation detail.

Note that it's also possible to send and receive dates before 1900 using
xmlrpclib.DateTime objects.


> Number family (MJD or JDN) or Rata Die are more appropriate and much
> easier to use. I wish somebody could add both to the datetime module.
>
>
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] No time for svn merge

2008-04-02 Thread Trent Nelson
Christian Heimes [mailto:[EMAIL PROTECTED]:
> Trent Nelson schrieb:
> > I think holding a developer accountable for merging or blocking to
> py3k when they commit to trunk is a great idea.  Who better to pass
> judgement on such an activity than the person closest to it?
>
> Blocking a revision makes my job as The Merger easier.
>
> I'm not so sure about the merging part. It takes some experience with
> the Python 3.0 code base to know the subtle differences in the C API.
> Most merges are straight forward for me. If you enforce the forward
> merging on every developer it may slow down development.
> Each regular merge takes me about 45 minutes of computer time but less
> than 15 supervisor time. The computer time is mostly compile and test
> time in the background. If everybody merges her own code to 3.0 it
> still takes *everybody* about 10 minutes of time and 45 minutes of computer
> time.

Ah, right, I wasn't thinking about the implication of code affecting the C base 
for some reason, but that's entirely reasonable.  Perhaps each developer should 
be accountable for either:

a) blocking
b) merging, if they're able to do so
c) if they're unable to merge, replying to the relevant python-checkins@ e-mail 
indicating that they're unable to handle trunk -> py3k for whatever reason 
(e.g. not familiar with py3k code base)

Other developers could then pitch in and help merge if someone requests it via 
e-mail.  I'd think that would make The Merger's life easier.

Trent.
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


[Python-Dev] Tools\buildbot\kill_python.c can't be helping our cause

2008-04-02 Thread Trent Nelson
Looking into some of the recent Windows buildbot failures, I see things like 
this:

sqlite3 : error PRJ0008 : Could not delete file 
'c:\buildbot\trunk.heller-windows-amd64\build\PCbuild\amd64\sqlite3_d.dll'.

build-amd64.bat doesn't go through the kill_python.c hoopla, so I figure the 
above error is being caused by the fact that an erroneous/stalled python_d.exe 
from a previous run is still open.  I was looking at modifying kill_python.c to 
accept an 'x64' argument if we want to kill amd64\python_d.exe instead of the 
usual 32-bit exe, however, this caught my attention:

if ((strstr(path, "pcbuild\\python_d.exe") != NULL) ||
(strstr(path, "\\build\\python.exe") != NULL)) {
printf("Terminating %s (pid %d)\n", path, pids[i]);
if (!TerminateProcess(hProcess, 1)) {
printf("Termination failed: %d\n", GetLastError());
return 1;
}
return 0;

That'll kill the first python_d.exe instance it finds matching the given path; 
given that our buildbots run trunk/release25-maint/py3k in parallel, it seems 
as though it wouldn't be hard for us to get into a situation where 
kill_python.exe ends up killing the wrong python_d.exe (i.e. trunk checkin, 
trunk builds, starts testing, py3k checkin, calls kill_python.exe, kills 
trunk's python_d.exe that was in the middle of testing).

That can't be helping our cause, unless I'm missing something...  Unless anyone 
advises otherwise, I'll start on a patch.


Trent.



___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Tools\buildbot\kill_python.c can't be helping our cause

2008-04-02 Thread Martin v. Löwis
> That'll kill the first python_d.exe instance it finds matching the
> given path; given that our buildbots run trunk/release25-maint/py3k
> in parallel

That's actually not a given: we currently *don't* run multiple builds
simultaneously on the same slave.

> Unless anyone advises otherwise, I'll start on a patch.

If you can make it less error-prone, sure, go ahead.

Regards,
Martin

___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Thread-safe file objects, the return

2008-04-02 Thread Guido van Rossum
On Wed, Apr 2, 2008 at 3:17 AM, Antoine Pitrou <[EMAIL PROTECTED]> wrote:
>  Guido van Rossum  python.org> writes:
>  > Your solution (a counter) seems fine except I think perhaps the
>  > close() call should not raise IOError -- instead, it should set a flag
>  > so that the thread that makes the counter go to zero can close the
>  > thread (after all the file got closed while it was being used).
>
>  I agree with Gregory: we should be explicit about what happens. I wonder
>  what we would gain from that approach - apart from encouraging dangerous
>  coding practices :)
>  It also depends how far we want to go: I am merely proposing to fix the
>  crashes, do we want to provide a "smarter" close() variation that does what
>  you suggest for people that want (or need) to take the risk?

I also agree with Gregory now -- at least on the issue of fclose().

I think that for other (non-closing) operations we should be fine,
given the Posix requirement that streams have an internal lock. While
multiple threads reading from a file sounds insane, multiple files
writing to a file is pretty common (think of stdout or stderr) and
should be supported.

>  > There are of course other concurrency issues besides close -- what if
>  > two threads both try to do I/O on the file? What will the C stdio
>  > library do in that case? Are stdio files thread-safe at the C level?
>
>  According to the glibc documentation, at
>  http://www.gnu.org/software/libc/manual/html_node/Streams-and-Threads.html :
>
>  « The POSIX standard requires that by default the stream operations are
>  atomic. I.e., issuing two stream operations for the same stream in two
>  threads at the same time will cause the operations to be executed as if
>  they were issued sequentially. The buffer operations performed while
>  reading or writing are protected from other uses of the same stream. To do
>  this each stream has an internal lock object which has to be (implicitly)
>  acquired before any work can be done. »
>
>  So according to POSIX rules it should be perfectly safe.

Cool.

>  In any case, someone would have to try my patch under Windows and OS X and
>  see if test_file.py passes without crashing.

I know Windows has internal locks on stdio. I trust that OSX, being a
BSD descendant, is posix compliant. So I'm not worried about these.

+1 on your patch, assuming some other developer reviews it and submits it.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] fixing tests on windows

2008-04-02 Thread Martin v. Löwis
> Personally, I've never really understood the purpose of 
> test_support.TESTFN. Whenever I've needed a temporary file for a test, I 
> just use the tempfile module (e.g. test_cmd_line_script, test_runpy). 
> Tests using that module don't care if the old files take 'a while' to 
> get deleted on Windows, as tempfile uses a different name each time anyway.
> 
> Is using a fixed TESTFN just an old approach that predates the existence 
> of a robust tempfile module in the standard library?

No. I believe the rationale for TESTFN is to provide a fixed name,
precisely so that the test suite doesn't leave tons of garbage around.

Regards,
Martin
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Tools\buildbot\kill_python.c can't be helping our cause

2008-04-02 Thread Trent Nelson
> > That'll kill the first python_d.exe instance it finds matching the
> > given path; given that our buildbots run trunk/release25-maint/py3k
> > in parallel
>
> That's actually not a given: we currently *don't* run multiple builds
> simultaneously on the same slave.

I thought the slave lock only applied per branch, not per host?

> > Unless anyone advises otherwise, I'll start on a patch.
>
> If you can make it less error-prone, sure, go ahead.

Spent a bit of time on it this evening; as it happens, in order to enumerate 
64-bit processes, you need to be a 64-bit process yourself.  As it's much 
easier managing 32-bit vs. x64 binaries when they're a .vcproj part of 
pcbuild.sln, I'm looking into adding kill_python as a .vcproj and configure the 
solution such that it builds and runs this before any other project.  That'll 
automatically take care of choosing the right version to run depending on 
whether 'Win32' or 'x64' is selected as the platform.  It'll also simplify the 
verification logic that checks if it's the right python_d.exe -- the path of 
the .exe needs to match the path of the running kill_python.exe.

Trent.
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Tools\buildbot\kill_python.c can't be helping our cause

2008-04-02 Thread Martin v. Löwis
Trent Nelson wrote:
>>> That'll kill the first python_d.exe instance it finds matching the
>>> given path; given that our buildbots run trunk/release25-maint/py3k
>>> in parallel
>> That's actually not a given: we currently *don't* run multiple builds
>> simultaneously on the same slave.
> 
> I thought the slave lock only applied per branch, not per host?

As the name suggests, it's per slave :-) a slave being the name/password
pair, along with the buildbot.tac file.

The sequencing of builds per builder is not a locking mechanism,
actually; a single builder just won't schedule a new build as long
as a build is already running.

So we have currently three builders per slave, and these are serialized
with the slave lock (hence the occasional long sequences of yellow
"Build nnn": the build starts, tries to acquire the slave lock, which
then a different builder still holds).

Now, on your machine, I understand you also have two slaves running.
They might, in principle, kill each other's python_d.exe processes,
except that the paths will be different there (amd64 vs. not).

Regards,
Martin
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


[Python-Dev] Python on ThreadX

2008-04-02 Thread Pree Raj
Hi,
I have an application that used python on linux.
I want to port it to run on ThreadX.
Can you tell me if this can be possible.
If yes, how can I get started.

Thanks.
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Python on ThreadX

2008-04-02 Thread Alexander Neundorf
On Wed, Apr 2, 2008 at 10:12 PM, Pree Raj <[EMAIL PROTECTED]> wrote:
> Hi,
> I have an application that used python on linux.
> I want to port it to run on ThreadX.
> Can you tell me if this can be possible.
> If yes, how can I get started.

I managed to get Python to work under eCos, which is also a small RTOS
(one binary firmware image, no shared libraries, one process space).
So I think it should be possible for ThreadX too.
I did this with the cmake-based build.

Alex
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] No time for svn merge

2008-04-02 Thread Benjamin Peterson
On Wed, Apr 2, 2008 at 7:16 AM, Trent Nelson <[EMAIL PROTECTED]> wrote:

> Christian Heimes [mailto:[EMAIL PROTECTED]:
> > Trent Nelson schrieb:
> > > I think holding a developer accountable for merging or blocking to
> > py3k when they commit to trunk is a great idea.  Who better to pass
> > judgement on such an activity than the person closest to it?
> >
> > Blocking a revision makes my job as The Merger easier.
> >
> > I'm not so sure about the merging part. It takes some experience with
> > the Python 3.0 code base to know the subtle differences in the C API.
> > Most merges are straight forward for me. If you enforce the forward
> > merging on every developer it may slow down development.
> > Each regular merge takes me about 45 minutes of computer time but less
> > than 15 supervisor time. The computer time is mostly compile and test
> > time in the background. If everybody merges her own code to 3.0 it
> > still takes *everybody* about 10 minutes of time and 45 minutes of
> computer
> > time.
>
> Ah, right, I wasn't thinking about the implication of code affecting the C
> base for some reason, but that's entirely reasonable.  Perhaps each
> developer should be accountable for either:
>
> a) blocking
> b) merging, if they're able to do so
> c) if they're unable to merge, replying to the relevant python-checkins@
> e-mail indicating that they're unable to handle trunk -> py3k for whatever
> reason (e.g. not familiar with py3k code base)
>
> Other developers could then pitch in and help merge if someone requests it
> via e-mail.  I'd think that would make The Merger's life easier.

I think we should let The Merger decide what makes his life easier. :)

>
>
>Trent.
> ___
> Python-Dev mailing list
> [email protected]
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/musiccomposition%40gmail.com
>



-- 
Cheers,
Benjamin Peterson
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] fixing tests on windows

2008-04-02 Thread Scott Dial
Martin v. Löwis wrote:
>> Is using a fixed TESTFN just an old approach that predates the existence 
>> of a robust tempfile module in the standard library?
> 
> No. I believe the rationale for TESTFN is to provide a fixed name,
> precisely so that the test suite doesn't leave tons of garbage around.

I figured this was going to be the answer, and I wonder if it would be 
better to use the tempfile module to generate unique filenames that are 
all contained in a special directory. It would seem functionally 
equivalent in the sense that removing the garbage is easy 
"rmtree(TESTDIR)", and it would also solve the windows problem of files 
that can't be unlinked. I suppose one would have to periodically 
clean-up their temp folder of the files left behind, but that's still an 
improvement over what is currently required if TESTFN is not unlinked.

The only reason I could see the switch being undesirable is if you 
wanted to go read the test file by hand, but I'm not sure anyone is 
doing that.

-Scott

-- 
Scott Dial
[EMAIL PROTECTED]
[EMAIL PROTECTED]
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


[Python-Dev] Building next alphas

2008-04-02 Thread Barry Warsaw
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

This is a reminder that I am going to start building the next alpha  
releases for Python 2.6 and 3.0 now.  Please, no checkins unless you  
get approval from me, and until you hear that the freeze is lifted.

I am now on freenode #python-dev, IM, and Jabber if you need to  
contact me.

- -Barry

-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.8 (Darwin)

iQCVAwUBR/QB6XEjvBPtnXfVAQLn8QQArynMsNeeb6gqjUCSYuupM2XXAbwP5XOX
LbTeGN+vM13uNK32fI47rDaPEfudfGnrd3Ttc1pg6/S/MOo5T41zs/TX2jdMEQ4g
6zCtk6xJiexGbExKioiTVdYgiqA8C6u+XY8aU2ogklD1h7kfEOWKw5urXkValFhG
Iymq6mrEyJQ=
=d/L3
-END PGP SIGNATURE-
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] [Distutils] FW: [issue2513] 64bit cross compilation on windows

2008-04-02 Thread Mark Hammond
> I think this is fine; we don't really have a notion of compiling for a
> native platform, nor is the build machine's architecture factored into
> the equation.

Actually, I think it is slightly.  IIUC, the AMD64 build currently assumes
it can execute x86 executables in various places.  To fix this, the build
process for each of the platforms would be slightly different.  My proposal
would allow (eg) 'PCBuild\python.exe' or 'PCBuild\make_versioninfo.exe' to
specified in build processes, with the knowledge it will work, regardless of
what platform it is being run on.

For example, with my idea, you could write code such as:

  exe = os.path.join(os.getenv("PYTHON_HOME"), "PCBuild", "python.exe"))
  popen(exe, ...)

and expect it to work on all platforms.  If I understand correctly, I now
need to write something like:

  if "AMD64" in sys.platform:
exe = os.path.join(os.getenv("PYTHON_HOME"), "PCBuild", "amd64",
"python.exe"))
  else if "32 bit" in sys.platform:
exe = os.path.join(os.getenv("PYTHON_HOME"), "PCBuild", "python.exe"))
  else:
# hrm - I obviously don’t know about new platforms yet...
  popen(exe, ...)

In other words, I think there is real advantage to simple scripts knowing
that './python' or './PCBuild/python.exe' will always be executable (and
always give the "native" version if the platform does emulation)

But I accept I'm waving appendages in the wind, and without a concrete
use-case I will take Martin's advice and let it go.

Cheers,

Mark


___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] xmlrpclib and dates before 1900

2008-04-02 Thread skip
Ralf> anyone care to take a look at:
Ralf> http://bugs.python.org/issue2014
Ralf> It's about xmlrpclib not being able to send datetime objects with
Ralf> dates before 1900.

It's actually not xmlrpclib which has the limitation, but
datetime.strftime().  That's a known limitation.  Here's the comment in the
datetime code:

/* Give up if the year is before 1900.
 * Python strftime() plays games with the year, and different
 * games depending on whether envar PYTHON2K is set.  This makes
 * years before 1900 a nightmare, even if the platform strftime
 * supports them (and not all do).
 * We could get a lot farther here by avoiding Python's strftime
 * wrapper and calling the C strftime() directly, but that isn't
 * an option in the Python implementation of this module.
 */

That, in turn, calls time.strftime() which calls the underlying system's
strftime() library function.

Personally, I don't think patching xmlrpclib is the right place to "fix"
this problem.  It's possible that the datetime comment is no longer correct
and that limitation should be reconsidered.  I see no other mention of
PYTHON2K in any .c, .h or .py files in the trunk.

Skip

___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] xmlrpclib and dates before 1900

2008-04-02 Thread Guido van Rossum
On Wed, Apr 2, 2008 at 8:36 PM,  <[EMAIL PROTECTED]> wrote:
> Ralf> anyone care to take a look at:
> Ralf> http://bugs.python.org/issue2014
> Ralf> It's about xmlrpclib not being able to send datetime objects with
> Ralf> dates before 1900.
>
>  It's actually not xmlrpclib which has the limitation, but
>  datetime.strftime().  That's a known limitation.  Here's the comment in the
>  datetime code:
>
> /* Give up if the year is before 1900.
>  * Python strftime() plays games with the year, and different
>  * games depending on whether envar PYTHON2K is set.  This makes
>  * years before 1900 a nightmare, even if the platform strftime
>  * supports them (and not all do).
>  * We could get a lot farther here by avoiding Python's strftime
>  * wrapper and calling the C strftime() directly, but that isn't
>  * an option in the Python implementation of this module.
>  */
>
>  That, in turn, calls time.strftime() which calls the underlying system's
>  strftime() library function.
>
>  Personally, I don't think patching xmlrpclib is the right place to "fix"
>  this problem.  It's possible that the datetime comment is no longer correct
>  and that limitation should be reconsidered.  I see no other mention of
>  PYTHON2K in any .c, .h or .py files in the trunk.

I'd like to see this fixed if possible, but I'm not sure how -- the C
level 'struct tm' has (year - 1900) in the tm_year member, and I'm not
sure that implementations are required to do anything with negative
values there. We'd have to reimplement all of strftime ourselves. I'm
not against that, but there are higher priorities.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Building next alphas

2008-04-02 Thread Barry Warsaw
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On Apr 2, 2008, at 6:00 PM, Barry Warsaw wrote:
>
> This is a reminder that I am going to start building the next alpha  
> releases for Python 2.6 and 3.0 now.  Please, no checkins unless you  
> get approval from me, and until you hear that the freeze is lifted.
>
> I am now on freenode #python-dev, IM, and Jabber if you need to  
> contact me.

I've been battling the flu and got distracted for a few hours tonight,  
so I'm not quite done with the releases.  However, I've tagged and  
tar'd 'em so it should just be a matter of uploading the files and  
updating the site.  I should finish tomorrow.

I'm thawing the trees, so you can go ahead and start committing things  
again, but /please/ be especially conservative over the next 24-48  
hours.  Make sure your changes don't break anything, just in case my  
virus-addled brain screwed something up and I need to cut another  
release.

- -Barry

P.S. Huge thanks to Benjamin Peterson, both for a quick last minute  
fix to the 3.0 NEWS file via IRC, and his wonderful release.py  
script.  I've hacked it up a bit, but it was exactly what I was  
looking for, and it made things go much smoother this time.

-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.8 (Darwin)

iQCVAwUBR/RbOHEjvBPtnXfVAQKCOwQArrYXC9X3lOvqRTwWQ9SYPH+6n1VN4MrT
WNm+jhsbiwZq8EuNslCBW3/52HP/wM7jlYizKZCL+cbcFaevNhWjjbPtwSTkJjVy
/uKG/NcDYQsPH3n4mET3/XlF5JrfS51avLSD7YebucTph9+otzI8LkK0Unvdbtq+
/86m3lEZAlY=
=q005
-END PGP SIGNATURE-
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] [Distutils] FW: [issue2513] 64bit cross compilation on windows

2008-04-02 Thread Martin v. Löwis

> Actually, I think it is slightly.  IIUC, the AMD64 build currently assumes
> it can execute x86 executables in various places.  To fix this, the build
> process for each of the platforms would be slightly different.

Why does that need fixing? The AMD64 build *can* execute x86 binaries,
whether it is performed on Win32 or Win64.

A problem only arises if the AMD64 build assumes that it can execute
AMD64 binaries, and it is a cross-compilation. As a consequence, we
must never execute the python.exe that we just built during the build
process.

>  My proposal
> would allow (eg) 'PCBuild\python.exe' or 'PCBuild\make_versioninfo.exe' to
> specified in build processes, with the knowledge it will work, regardless of
> what platform it is being run on.

No to python.exe, yes to make_versioninfo.exe. make_versioninfo.exe is
*always* an x86 binary, so it is always safe to execute on Windows.
If you tried to execute PCBuild\python.exe, it would break my builds:
I don't *have* x86 binaries in PCBuild when I cross-compile for AMD64.

Instead, I've changed the build process to run HOST_PYTHON if that is
set, and only fall back to PCBuild\python.exe if it isn't.

Regards,
Martin
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com