Re: [Python-Dev] Still looking for volunteer to run Windows buildbot
Gregory P. Smith wrote: FWIW, thats an old BerkeleyDB (yes it'll still work). Python 2.5 should ship built with BerkeleyDB 4.4.20 so thats what buildbot should use if it builds the module. The buildbots now fetch bsddb automatically from http://svn.python.org/projects/external/db-4.4.20/ so the buildbot slave admin doesn't need to do anything here. This also has some 4.4.20 patches already applied. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Still looking for volunteer to run Windows buildbot
On Sat, Mar 11, 2006 at 07:08:14PM +0100, Thomas Heller wrote: Martin v. L?wis wrote: Josiah Carlson told me had has given up getting a Windows buildbot running, because every time he installed VS.NET on his machine, the installation would immediately crash. So if anybody wants to contribute both a machine and time to operate it (including the likely very tedious phase to get any results out of this at all), please contact me. This is an ideal job for VMWare on an existing linux build machine if someone can ante up a win xp and msvc++ license to the cause. A vmware license for the build host probably isn't even necessary assuming vmware player works and one of us with a license can setup the vmware image (http://www.vmware.com/products/player/). (theres also a chance we could get vmware inc. to donate a license to python if one was needed) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Still looking for volunteer to run Windows buildbot
On Sun, Mar 12, 2006 at 06:48:13PM -0500, Tim Peters wrote: [Trent] :) Did you apply the Berkeley DB patches to your db-4.2.52 sources? Ah, _which_ patches? As with my buildbot Wiki page, I write down everything I do if there's a good chance I may need to do it again. So, e.g., these are my words in PCbuild\readme.txt: As of 11-Apr-2004, you also need to download and manually apply two patches before proceeding (and the sleepycat download page tells you about this). Cygwin patch worked for me. cd to dist\db-4.2.52 and use patch -p0 patchfile once for each downloaded patchfile. It's possible that there are more patches needed since then, but if so I wouldn't know about that (last time I built the externals in the _bsddb part was indeed 11-Apr-2004). FWIW, thats an old BerkeleyDB (yes it'll still work). Python 2.5 should ship built with BerkeleyDB 4.4.20 so thats what buildbot should use if it builds the module. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Still looking for volunteer to run Windows buildbot
Gregory P. Smith wrote: On Sat, Mar 11, 2006 at 07:08:14PM +0100, Thomas Heller wrote: Martin v. L?wis wrote: Josiah Carlson told me had has given up getting a Windows buildbot running, because every time he installed VS.NET on his machine, the installation would immediately crash. So if anybody wants to contribute both a machine and time to operate it (including the likely very tedious phase to get any results out of this at all), please contact me. This is an ideal job for VMWare on an existing linux build machine if someone can ante up a win xp and msvc++ license to the cause. A vmware license for the build host probably isn't even necessary assuming vmware player works and one of us with a license can setup the vmware image (http://www.vmware.com/products/player/). (theres also a chance we could get vmware inc. to donate a license to python if one was needed) Right. I have setup 64-bit Ubuntu on a new Athon64 machine, created WinXP and WinXP64 empty virtual machines with VMWare workstation, then installed XP and XP64 into these images. The images run on the Ubuntu machine in VMWare player. I don't think we need more x86 or x86-64 buildbots, so I made no attempt to install the buildbots. Thomas ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Still looking for volunteer to run Windows buildbot
On 3/29/06, Gregory P. Smith [EMAIL PROTECTED] wrote: This is an ideal job for VMWare on an existing linux build machine if someone can ante up a win xp and msvc++ license to the cause. It probably isn't great from a practical point of view if you wanted to run buildbot for both the server and host platforms. The problem is that jobs for both platforms will be scheduled concurrently. If checkins occur in both 2.5 and 2.4 as is somewhat common, you would be running 4 test suites simultaneously. The full suite already takes quite a bit of time (over 20 minutes depending on the box). The machine could be crushed if we run 4. If you just wanted to run Windows in VMware, that would be fine. We already have 3 windows buildbots of the XP and 2k variety. Other varieties might be desirable. n ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Still looking for volunteer to run Windows buildbot
[Trent Mick wrote] [Martin v. Loewis wrote] Tim Peters wrote: I'd say instead that they should never be skipped: the real difference on your box is the expected _outcome_ in the third category. That is indeed more reasonable than what I proposed. I'll do this tonight or tomorrow. Done now (finally). Trent -- Trent Mick [EMAIL PROTECTED] ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Still looking for volunteer to run Windows buildbot
[Trent Mick, on changing test_winsound to expect RuntimeError on a box without a sound card] Done now (finally). Cool -- thanks! I merged that to the 2.4 branch, and (of course) your buildbot slave passes the tests on that too now. Green is a lovely color :-) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Still looking for volunteer to run Windows buildbot
So PythonWin needs to be installed on a Windows buildbot slave, right? FWIW, we are having reasonable success with buildbot service packaged as a py2exe application - just unzip into a directory and go! This has the primary advantage (to me!) of not using the Python installed on the box, thereby avoiding inconvenient file is in use errors when you do need to update that Python or similar. As I've been chatting with Trent about recently, I intend updating my buildbot patch to use the win32 Job related functions to manage the build subtasks more effectively and potentially guard against the buildbot using too many resources. Sadly though, my buildbot patch has been languishing in their collector without comment, so my motivation is fairly low. Cheers, Mark ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Still looking for volunteer to run Windows buildbot
On Wed, 15 Mar 2006 00:00:06 -0500, Tim Peters [EMAIL PROTECTED] wrote: [Trent Mick] Yes I've noticed it too. I've had to kill python_d.exe a few times. I haven't yet had the chance to look into it. I am NOT getting this error on another Windows Python build slave that I am running in-house for play. The last run on your Win2K slave that got beyond the compile step: http://www.python.org/dev/buildbot/all/x86%20W2k%20trunk/builds/16/step-test/0 Looks like it was running test_bsddb at the time, and the test framework gave up after waiting 20 minutes for more output. I had one of those recently that waited 20 minutes for output after starting test_shelve, but it's scrolled off the page. Berkeley DB is fishy. Looks like the buildbot doesn't know how to kill a process on Windows either (SIGKILL sure ain't gonna do it ;-)). It should actually be using TerminateProcess (depending on the Twisted version being used, the relevant code is either in twisted/internet/_dumbwin32proc.py or twisted/internet/win32eventreactor.py, in the signalProcess method in either case), but even this doesn't seem to be a completely reliable way to end a process. Twisted's buildbot has run into this problem as well, but we haven't figure out how to fix it yet. Suggestions welcome - patches even more so :) Jean-Paul ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Still looking for volunteer to run Windows buildbot
Jean-Paul Calderone wrote: It should actually be using TerminateProcess (depending on the Twisted version being used, the relevant code is either in twisted/internet/_dumbwin32proc.py or twisted/internet/win32eventreactor.py, in the signalProcess method in either case) So PythonWin needs to be installed on a Windows buildbot slave, right? Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Still looking for volunteer to run Windows buildbot
Martin v. Löwis wrote: Jean-Paul Calderone wrote: It should actually be using TerminateProcess (depending on the Twisted version being used, the relevant code is either in twisted/internet/_dumbwin32proc.py or twisted/internet/win32eventreactor.py, in the signalProcess method in either case) So PythonWin needs to be installed on a Windows buildbot slave, right? unless someone hacks Twisted to use _subprocess.TerminateProcess instead of the win32all version... /F ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Still looking for volunteer to run Windows buildbot
Martin v. L?wis wrote: So PythonWin needs to be installed on a Windows buildbot slave, right? PyWin32 you mean. PythonWin is the IDE-thing that is part of PyWin32. [Fredrik Lundh wrote] unless someone hacks Twisted to use _subprocess.TerminateProcess instead of the win32all version... +1 Trent -- Trent Mick [EMAIL PROTECTED] ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Still looking for volunteer to run Windows buildbot
On Wed, 15 Mar 2006 20:18:28 +0100, Fredrik Lundh [EMAIL PROTECTED] wrote: Martin v. L� wrote: Jean-Paul Calderone wrote: It should actually be using TerminateProcess (depending on the Twisted version being used, the relevant code is either in twisted/internet/_dumbwin32proc.py or twisted/internet/win32eventreactor.py, in the signalProcess method in either case) So PythonWin needs to be installed on a Windows buildbot slave, right? unless someone hacks Twisted to use _subprocess.TerminateProcess instead of the win32all version... Twisted's Win32 process support also uses win32api, win32con, win32event, win32file, win32pipe, and win32security. ;P So the new subprocess module alone isn't quite a sufficient replacement... Jean-Paul ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Still looking for volunteer to run Windows buildbot
[Martin v. Löwis] So PythonWin needs to be installed on a Windows buildbot slave, right? It must, since that's what my wiki page says ;-): http://wiki.python.org/moin/BuildbotOnWindows ... o Install a matching version of pywin32. ... ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Still looking for volunteer to run Windows buildbot
[Trent Mick] I have a patch in the works that defaults to yes, this machine does have a soundcard if cscript.exe cannot be found on the PATH. However, one wrinkle: test_winsound.py is made up of three test cases: BeepTest MessageBeepTest PlaySoundTest only the last need be skipped if there is not soundcard. I'd say instead that they should never be skipped: the real difference on your box is the expected _outcome_ in the third category. After umpteen years we've got a universe of one machine where PlaySoundTest is known to fail, and now a little mound of VB code that presumably returns something different on that machine than on other machines. In reality, that's more code to test. We seem to be assuming here that the VB code says no sound device means PlaySoundTest will fail in a particular way, and have one box on which that's known to be true. So sure, skip the tests on that box, and the immediate buildbot failure on that box will go away. Other possiblities include that the test will also be skipped on boxes where it would actually work, because the VB code isn't actually a definitive test for some reason. Since we can't be sure from a universe of one exception, better to test that assumption too, by reworking the tests to say oh, but if the VB code thinks we don't have a sound card, then this test should raise RuntimeError instead. There's still a testable outcome here. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Still looking for volunteer to run Windows buildbot
[Mark Hammond] Maybe the following VBScript port of the above will work: -- check_soundcard.vbs rem Check for a working sound-card - exit with 0 if OK, 1 otherwise. set wmi = GetObject(winmgmts:) set scs = wmi.InstancesOf(win32_sounddevice) for each sc in scs set status = sc.Properties_(Status) wscript.Echo(sc.Properties_(Name) + / + status) if status = OK then wscript.Quit 0 rem normal exit end if next rem No sound card found - exit with status code of 1 wscript.Quit 1 -- eof Running cscript.exe check_soundcard.vbs and checking the return code should work. FYI, it works on my main box: C:\Codecscript.exe csc.vbs Microsoft (R) Windows Script Host Version 5.6 Copyright (C) Microsoft Corporation 1996-2001. All rights reserved. Creative Audigy Audio Processor (WDM)/OK C:\Codeecho %errorlevel% 0 ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Still looking for volunteer to run Windows buildbot
Tim Peters wrote: I'd say instead that they should never be skipped: the real difference on your box is the expected _outcome_ in the third category. That is indeed more reasonable than what I proposed. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Still looking for volunteer to run Windows buildbot
[Martin v. Loewis wrote] Tim Peters wrote: I'd say instead that they should never be skipped: the real difference on your box is the expected _outcome_ in the third category. That is indeed more reasonable than what I proposed. I'll do this tonight or tomorrow. Trent -- Trent Mick [EMAIL PROTECTED] ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Still looking for volunteer to run Windows buildbot
[Trent Mick, on test_winsound] I'll do this tonight or tomorrow. Cool! I see that your Win2K buildbot slave always dies in the compile step now, with -- Build started: Project: pythoncore, Configuration: Debug Win32 -- Compiling resources... generate buildinfo cl.exe -c -D_WIN32 -DUSE_DL_EXPORT -D_WINDOWS -DWIN32 -D_WINDLL -D_DEBUG -MDd ..\Modules\getbuildinfo.c -Fogetbuildinfo.o -I..\Include -I..\PC Microsoft (R) 32-bit C/C++ Optimizing Compiler Version 13.10.3077 for 80x86 Copyright (C) Microsoft Corporation 1984-2002. All rights reserved. getbuildinfo.c Linking... LINK : fatal error LNK1104: cannot open file './python25_d.dll' That happened to me once, but I still don't understand it. It turned out that the corresponding python_d.exe was still running (for hours, and hours, and hours, ...), and I had to manually kill the process. I'm not sure that was enough, because I coincidentally rebooted the box before the buildbot tests ran again. I am pretty sure that the symptom above won't fix itself. Possibly related: since we upgraded to a new bsddb (and this may be coincidence), I've seen two failure modes in test_shelve: test_bool (which is the first test) never completes, and test_bool does complete but fails. Turns out both are rare failure modes, and they haven't happened again since I prepared myself to dig into them 0.5 wink. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Still looking for volunteer to run Windows buildbot
[Tim Peters wrote] ... I see that your Win2K buildbot slave always dies in the compile step now, with -- Build started: Project: pythoncore, Configuration: Debug Win32 -- Compiling resources... generate buildinfo cl.exe -c -D_WIN32 -DUSE_DL_EXPORT -D_WINDOWS -DWIN32 -D_WINDLL -D_DEBUG -MDd ..\Modules\getbuildinfo.c -Fogetbuildinfo.o -I..\Include -I..\PC Microsoft (R) 32-bit C/C++ Optimizing Compiler Version 13.10.3077 for 80x86 Copyright (C) Microsoft Corporation 1984-2002. All rights reserved. getbuildinfo.c Linking... LINK : fatal error LNK1104: cannot open file './python25_d.dll' That happened to me once, but I still don't understand it. It turned out that the corresponding python_d.exe was still running (for hours, and hours, and hours, ...), and I had to manually kill the process. I'm not sure that was enough, because I coincidentally rebooted the box before the buildbot tests ran again. I am pretty sure that the symptom above won't fix itself. Yes I've noticed it too. I've had to kill python_d.exe a few times. I haven't yet had the chance to look into it. I am NOT getting this error on another Windows Python build slave that I am running in-house for play. Trent -- Trent Mick [EMAIL PROTECTED] ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Still looking for volunteer to run Windows buildbot
[Trent Mick] Yes I've noticed it too. I've had to kill python_d.exe a few times. I haven't yet had the chance to look into it. I am NOT getting this error on another Windows Python build slave that I am running in-house for play. The last run on your Win2K slave that got beyond the compile step: http://www.python.org/dev/buildbot/all/x86%20W2k%20trunk/builds/16/step-test/0 Looks like it was running test_bsddb at the time, and the test framework gave up after waiting 20 minutes for more output. I had one of those recently that waited 20 minutes for output after starting test_shelve, but it's scrolled off the page. Berkeley DB is fishy. Looks like the buildbot doesn't know how to kill a process on Windows either (SIGKILL sure ain't gonna do it ;-)). The good news is that at least we're not seeing the random segfaults plaguing the Mac slave :-) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Still looking for volunteer to run Windows buildbot
[Uncle Timmy] ... Looks like it was running test_bsddb at the time, and the test framework gave up after waiting 20 minutes for more output. I had one of those recently that waited 20 minutes for output after starting test_shelve, but it's scrolled off the page. Berkeley DB is fishy. Well speak of the devil, and the Canadians appear ;-) Your _current_ Win2K test run crapped out after waiting 20 minutes for test_shelve to finish: http://www.python.org/dev/buildbot/all/x86%20W2k%20trunk/builds/23/step-test/0 I don't recall this ever happening before we moved to the newer bsddb, Now it's happened on at least two machines. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Still looking for volunteer to run Windows buildbot
[Tim Peters wrote] This touches on something we (including Martin) should think about: it's very painful to build a full Python on Windows because of these external packages... Yup. That is part of what I meant by updating PCBuild\readme.txt below: to improve the instructions so the build-dance for all these external packages is a little more cut n' paste. Something we might be able to do instead is have just one person per external project endure the pain of building it, and then have them check in the whole post-compilation post-test project directory tree. Everyone else (presumably including Windows buildbot slaves) using the same compiler could then just check out the result. If people don't mind having prebuilt binaries checked in, yes I suppose we could do this. For some projects here at work we have a prebuilt dir at the root of the source tree: prebuilt/ win32-x86/ foo/# Windows prebuilt bits for package foo linux-x86/ foo/ solaris8-sparc/ foo/ ... does-svn up python/ Whoa, someone is way ahead of me here. :) What's worse is when checkins cause both trunk and branch rebuilds. That can bring my zippy box to its knees, particularly when two of the disk-intensive tests (like test_largefile) run at the same time. That would require some kind of Scheduler work in the build master config that knows how to queue up scheduled builds from multiple sources to only allow one per machine at a time. I'd find this very useful for other projects at work and I'm sure other buildbot users would as well. Maybe buildbot may already have something like this? Remaining TODOs: - make sure the python24-maint branch one works Works for me, although (as the Wiki page says) you also need to copy zlib-1.2.3 into the branch build area. Yup. Looks like my build worked. Another TODO now though: - Figure out why usage of: winsound.PlaySound(something, winsound.SND_ALIAS) fails on my Win2k box. This is why the test suite fails on that box. - get the build slaves running as a Windows service Mark Hammond has a patch to the buildbot project toward this end: http://sf.net/tracker/index.php?func=detailaid=1401121group_id=73177atid=537003 Cool. I'll look into that. Trent -- Trent Mick [EMAIL PROTECTED] ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Still looking for volunteer to run Windows buildbot
Trent Mick wrote: Yup. Looks like my build worked. Another TODO now though: - Figure out why usage of: winsound.PlaySound(something, winsound.SND_ALIAS) fails on my Win2k box. This is why the test suite fails on that box. Doesn't that always fail when there is no soundcard in the machine? Thomas ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Still looking for volunteer to run Windows buildbot
[Thomas Heller wrote] Trent Mick wrote: Yup. Looks like my build worked. Another TODO now though: - Figure out why usage of: winsound.PlaySound(something, winsound.SND_ALIAS) fails on my Win2k box. This is why the test suite fails on that box. Doesn't that always fail when there is no soundcard in the machine? I don't know. The *other* winsound tests work and I *can* hear that rising tone goop coming out of the box's tinny speakers when the test suite is run. http://www.python.org/dev/buildbot/all/x86%20W2k%202.4/builds/5/step-test/0 http://msdn.microsoft.com/library/default.asp?url=/library/en-us/multimed/htm/_win32_playsound.asp PlaySound ... SND_ALIAS The pszSound parameter is a system-event alias in the registry or the WIN.INI file. ... These aliases are apparently supposed to be in the registry or WIN.INI, but the docs don't say where. Perhaps my Win2k box is missing some registry entries. No, I think it is the config of the box -- read on. I do have a sound card in that box, however, the Sounds and Multimedia Properties dialog (off Control Panel) says that there are No Playback Devices for Sound Playback. So I guess that is it. Maybe the sound card in that box is not hooked up. Grrr. I certainly don't care about the sound card for that box but I don't want the test suite to keep reporting a spurious failure. Trent -- Trent Mick [EMAIL PROTECTED] ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Still looking for volunteer to run Windows buildbot
Trent Mick wrote: This touches on something we (including Martin) should think about: it's very painful to build a full Python on Windows because of these external packages... Yup. That is part of what I meant by updating PCBuild\readme.txt below: to improve the instructions so the build-dance for all these external packages is a little more cut n' paste. I took an approach with a little more automation: Tools/buildbot/external.bat gradually learns how to fetch and build the necessary prerequisites; to avoid moving URLs, these come from the external/ directory of the projects svn (in case of bsddb, this already has the patches, and the VS project files converted). If people don't mind having prebuilt binaries checked in Well, I do mind having prebuilt binaries checked in. They take up space for little gain, plus they might increase the maintenance burden. That would require some kind of Scheduler work in the build master config that knows how to queue up scheduled builds from multiple sources to only allow one per machine at a time. I'd find this very useful for other projects at work and I'm sure other buildbot users would as well. Maybe buildbot may already have something like this? I could not easily find it. There is the AnyBranchScheduler, but it seems to serve a different purpose. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Still looking for volunteer to run Windows buildbot
[Martin v. Loewis wrote] I took an approach with a little more automation: Tools/buildbot/external.bat gradually learns how to fetch and build the necessary prerequisites; to avoid moving URLs, these come from the external/ directory of the projects svn (in case of bsddb, this already has the patches, and the VS project files converted). If people don't mind having prebuilt binaries checked in Well, I do mind having prebuilt binaries checked in. They take up space for little gain, plus they might increase the maintenance burden. I like you approach better. Very nice. Trent -- Trent Mick [EMAIL PROTECTED] ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Still looking for volunteer to run Windows buildbot
Trent Mick wrote: I do have a sound card in that box, however, the Sounds and Multimedia Properties dialog (off Control Panel) says that there are No Playback Devices for Sound Playback. So I guess that is it. Maybe the sound card in that box is not hooked up. Grrr. I certainly don't care about the sound card for that box but I don't want the test suite to keep reporting a spurious failure. Now, if there was a reliable check whether a soundcard is present, that check could be run as a prerequisite, then raising TestSkipped if no soundcard is present. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Still looking for volunteer to run Windows buildbot
[Martin v. Loewis wrote]
Trent Mick wrote:
I do have a sound card in that box, however, the Sounds and Multimedia
Properties dialog (off Control Panel) says that there are No Playback
Devices for Sound Playback. So I guess that is it. Maybe the sound card
in that box is not hooked up. Grrr. I certainly don't care about the
sound card for that box but I don't want the test suite to keep
reporting a spurious failure.
Now, if there was a reliable check whether a soundcard is present, that
check could be run as a prerequisite, then raising TestSkipped if no
soundcard is present.
Roger on python-win32 had an answer which works for me:
[Roger Upole wrote]
WMI can list sound devices.
import win32com.client
wmi=win32com.client.GetObject('winmgmts:')
scs=wmi.InstancesOf('win32_sounddevice')
for sc in scs:
print sc.Properties_('Name'), sc.Properties_('Status')
However, that requires PyWin32 so can't really use that for
test_winsound.py. My understanding of ctypes is that it can NOT replace
win32com, but I'd be happy to be wrong here. Thomas?
Trent
--
Trent Mick
[EMAIL PROTECTED]
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Still looking for volunteer to run Windows buildbot
Roger on python-win32 had an answer which works for me:
[Roger Upole wrote]
WMI can list sound devices.
import win32com.client
wmi=win32com.client.GetObject('winmgmts:')
scs=wmi.InstancesOf('win32_sounddevice')
for sc in scs:
print sc.Properties_('Name'), sc.Properties_('Status')
However, that requires PyWin32 so can't really use that for
test_winsound.py. My understanding of ctypes is that it can NOT replace
win32com, but I'd be happy to be wrong here. Thomas?
Maybe the following VBScript port of the above will work:
-- check_soundcard.vbs
rem Check for a working sound-card - exit with 0 if OK, 1 otherwise.
set wmi = GetObject(winmgmts:)
set scs = wmi.InstancesOf(win32_sounddevice)
for each sc in scs
set status = sc.Properties_(Status)
wscript.Echo(sc.Properties_(Name) + / + status)
if status = OK then
wscript.Quit 0 rem normal exit
end if
next
rem No sound card found - exit with status code of 1
wscript.Quit 1
-- eof
Running cscript.exe check_soundcard.vbs and checking the return code
should work. cscript.exe comes with all modern Windows variants, and
although there may be ways to install Windows without it, I think we can
safely assume it exists for these purposes.
Cheers,
Mark
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Still looking for volunteer to run Windows buildbot
[Mark Hammond wrote] Maybe the following VBScript port of the above will work: ... Cool, yes that works. Running cscript.exe check_soundcard.vbs and checking the return code should work. cscript.exe comes with all modern Windows variants, and although there may be ways to install Windows without it, I think we can safely assume it exists for these purposes. I have a patch in the works that defaults to yes, this machine does have a soundcard if cscript.exe cannot be found on the PATH. However, one wrinkle: test_winsound.py is made up of three test cases: BeepTest MessageBeepTest PlaySoundTest only the last need be skipped if there is not soundcard. However, TestSkipped only works add the module level. So, which is better: 1. Use TestSkipped and skip all three test cases if there is not sound card. Running the test suite will actually show that something is being skipped. 2. Conditionally define class PlaySoundTest only if there is a soundcard. Running the test suite on a machine without a soundcard will not show that something is being skipped but *will* run the Beep tests. 3. Break out test_winsound.py into two test modules: one with the beep tests and one with PlaySoundTest (the latter using TestSkipped). Trent -- Trent Mick [EMAIL PROTECTED] ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Still looking for volunteer to run Windows buildbot
Trent Mick wrote: 1. Use TestSkipped and skip all three test cases if there is not sound card. Running the test suite will actually show that something is being skipped. This is best. The sound tests are not that important that they absolutely need to be run. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Still looking for volunteer to run Windows buildbot
Tim Peters wrote: That would be great. A dupe of WinXP would also be great: I'm not going to keep my buildbot slave up all the time, or anywhere near all the time. It seems like the buildbot needs even more hand-holding on Windows: it apparently doesn't survive a master stop/start cycle. While the Unix buildbots reconnect, the Windows one doesn't. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Still looking for volunteer to run Windows buildbot
[Martin v. Löwis] It seems like the buildbot needs even more hand-holding on Windows: it apparently doesn't survive a master stop/start cycle. While the Unix buildbots reconnect, the Windows one doesn't. What makes us believe that? My box was hibernating from 03:12 to 11:54 EST today, and couldn't have responded to anything during that period. When I woke the box up, I was pleased to see that Twisted figured out that it hadn't heard from the master in 8+ hours, and established a new connection, all by itself. OTOH, I see the new connection soon got wedged all by itself too ;-), while running trunk tests just now. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Still looking for volunteer to run Windows buildbot
Tim Peters wrote: It seems like the buildbot needs even more hand-holding on Windows: it apparently doesn't survive a master stop/start cycle. While the Unix buildbots reconnect, the Windows one doesn't. What makes us believe that? The slave was reported as idle just before I restarted the master, and offline after the restart. From that, I inferred that the restart broke the connection. My box was hibernating from 03:12 to 11:54 EST today, and couldn't have responded to anything during that period. I see. So it is rather that the master doesn't see the slaves go away. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Still looking for volunteer to run Windows buildbot
[Tim Peters wrote] Setup is hellish, Agreed, though I have everything going with my own testing buildbot master (everything for the trunk build that is). My only remaining problem is that I can't connect to python.org's master. (Following up with Martin.) The second-worst part was figuring out which parts of various software docs could be ignored. :) Did you apply the Berkeley DB patches to your db-4.2.52 sources? I recorded all that remained here: http://wiki.python.org/moin/BuilbotOnWindows correction for others: http://wiki.python.org/moin/BuildbotOnWindows x86 XP-2 trunk. I'd still like to give it a go. The machine I'd use (initially at least) would be Win2k -- so not just a dupe of Tim's WinXP box. That would be great. A dupe of WinXP would also be great: I'm not going to keep my buildbot slave up all the time, or anywhere near all the time. I'm worried about the load this is going to cause on this machine with a new build for every checkin (granted they are serialized so not for *every* checkin). The full (doubled) test suite takes a *long* time to run on Windows. It looks like it took about 25 minutes on your box. It is taking over 40 minutes on one of my machines here. :( This Windows box needs an enema. Remaining TODOs: - get connection to python.org master working - make sure the python24-maint branch one works - see about the load issue - get the build slaves running as a Windows service - update PCbuild/readme.txt Trent -- Trent Mick [EMAIL PROTECTED] ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Still looking for volunteer to run Windows buildbot
[Tim Peters wrote] Setup is hellish Any objections to: Index: Tools/buildbot/build.bat === --- build.bat (revision 42982) +++ build.bat (working copy) @@ -1,3 +1,3 @@ @rem Used by the buildbot compile step. call %VS71COMNTOOLS%vsvars32.bat -devenv.com /build Debug PCbuild\pcbuild.sln +devenv.com /useenv /build Debug PCbuild\pcbuild.sln Adding the /useenv means that one's PATH actually gets through. This is important for the _ssl.vproj build. It calls build_ssl.py which tries to find a Perl to use. Without /useenv Visual Studio is getting a PATH from somewhere else (presumably from its internal environment configuration). The result is that build_ssl.py fallsback to its well-known locations for a Perl install. On one mahcine I was trying this on I didn't have Perl installed to C:\Perl but to C:\Perl58. People who install to an alternate drive might also get surprised. Trent -- Trent Mick [EMAIL PROTECTED] ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Still looking for volunteer to run Windows buildbot
Trent Mick wrote: Adding the /useenv means that one's PATH actually gets through. This is important for the _ssl.vproj build. It calls build_ssl.py which tries to find a Perl to use. Without /useenv Visual Studio is getting a PATH from somewhere else (presumably from its internal environment configuration). The result is that build_ssl.py fallsback to its well-known locations for a Perl install. Go ahead. The above makes a good check-in message. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Still looking for volunteer to run Windows buildbot
[Tim] Setup is hellish, [Trent] Agreed, though I have everything going with my own testing buildbot master (everything for the trunk build that is). My only remaining problem is that I can't connect to python.org's master. (Following up with Martin.) Looks like that got fixed. The second-worst part was figuring out which parts of various software docs could be ignored. :) Did you apply the Berkeley DB patches to your db-4.2.52 sources? Ah, _which_ patches? As with my buildbot Wiki page, I write down everything I do if there's a good chance I may need to do it again. So, e.g., these are my words in PCbuild\readme.txt: As of 11-Apr-2004, you also need to download and manually apply two patches before proceeding (and the sleepycat download page tells you about this). Cygwin patch worked for me. cd to dist\db-4.2.52 and use patch -p0 patchfile once for each downloaded patchfile. It's possible that there are more patches needed since then, but if so I wouldn't know about that (last time I built the externals in the _bsddb part was indeed 11-Apr-2004). This touches on something we (including Martin) should think about: it's very painful to build a full Python on Windows because of these external packages, each with its own unique and involved compile dance, and I expect it's a _huge_ barrier for getting Windows buildbot volunteers. You have to do real, messy, tedious work to get beyond this part. Something we might be able to do instead is have just one person per external project endure the pain of building it, and then have them check in the whole post-compilation post-test project directory tree. Everyone else (presumably including Windows buildbot slaves) using the same compiler could then just check out the result. I recorded all that remained here: http://wiki.python.org/moin/BuilbotOnWindows correction for others: http://wiki.python.org/moin/BuildbotOnWindows The URL I gave was correct at the time I sent it ;-) You're right that I renamed the page later, and good eye! I'm worried about the load this is going to cause on this machine with a new build for every checkin (granted they are serialized so not for *every* checkin). The full (doubled) test suite takes a *long* time to run on Windows. It looks like it took about 25 minutes on your box. It is taking over 40 minutes on one of my machines here. :( This Windows box needs an enema. Its current second test run is pretty pointless. Would be fine by me if we changed test.bat from call rt.bat -d -uall -rw to call rt.bat -d -q -uall -rw -q would cause it to run the tests only once. What's worse is when checkins cause both trunk and branch rebuilds. That can bring my zippy box to its knees, particularly when two of the disk-intensive tests (like test_largefile) run at the same time. Remaining TODOs: - get connection to python.org master working - make sure the python24-maint branch one works Works for me, although (as the Wiki page says) you also need to copy zlib-1.2.3 into the branch build area. - see about the load issue - get the build slaves running as a Windows service Mark Hammond has a patch to the buildbot project toward this end: http://sf.net/tracker/index.php?func=detailaid=1401121group_id=73177atid=537003 - update PCbuild/readme.txt Anything in particular need changing there? ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Still looking for volunteer to run Windows buildbot
Martin v. Löwis wrote: Josiah Carlson told me had has given up getting a Windows buildbot running, because every time he installed VS.NET on his machine, the installation would immediately crash. So if anybody wants to contribute both a machine and time to operate it (including the likely very tedious phase to get any results out of this at all), please contact me. Are there any estimates how much network traffic a buildbot would generate? And how must it be connected to the internet - I assume it must be reachable from the outside. Thomas ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Still looking for volunteer to run Windows buildbot
[Thomas Heller] Are there any estimates how much network traffic a buildbot would generate? It should be trivial except for the initial checkout of the Python code base. And how must it be connected to the internet - I assume it must be reachable from the outside. The slave opens a socket connection to the master, so you must be able to reach the python.org box _from_ the slave. I don't expect you'll have serious problems if you can do that much. For example, my home box has a dynamic IP (assigned by my ISP to my router), my router gives a different dynamic IP to my box (NAT'ed), and I'm running behind both software and hardware (SPI) firewalls. Because my box initiated the connection, nothing in the chain objects. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Still looking for volunteer to run Windows buildbot
Thomas Heller wrote: Are there any estimates how much network traffic a buildbot would generate? It will need to download the entire source code twice (once for the trunk, and once for 2.4). After that, it currently does only svn up, on each commit. And how must it be connected to the internet - I assume it must be reachable from the outside. No: it opens a connection itself, and then keeps that connection permanently open. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Still looking for volunteer to run Windows buildbot
[Trent Mick] I'm keen. However, it looks like Tim is most of the way there already: The first 100% clean (ignoring _ctypes warnings) Windows builbot test run just finished. Yippee! Setup is hellish, although you've already done the worst of it if you regularly build+test full Python on Windows from a checkout (full == everything we ship on Windows, including bsddb, ssl, Tcl/Tk and bz2). The second-worst part was figuring out which parts of various software docs could be ignored. I recorded all that remained here: http://wiki.python.org/moin/BuilbotOnWindows Reading that should save you several months ;-) x86 XP-2 trunk. I'd still like to give it a go. The machine I'd use (initially at least) would be Win2k -- so not just a dupe of Tim's WinXP box. That would be great. A dupe of WinXP would also be great: I'm not going to keep my buildbot slave up all the time, or anywhere near all the time. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Still looking for volunteer to run Windows buildbot
Josiah Carlson told me had has given up getting a Windows buildbot running, because every time he installed VS.NET on his machine, the installation would immediately crash. So if anybody wants to contribute both a machine and time to operate it (including the likely very tedious phase to get any results out of this at all), please contact me. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
