Re: [Python-Dev] Python REPL doesn't work on Windows over remote powershell session (winrm)
ZHU Xiang writes: > === > Steps to reproduce > > # 1/ pre-install python on server1 (server 1 is a windows os) > # 2/ from a powershell console on server0, type below 2 commands: > enter-pssession server1 > python > > Expected behavior > # The python >>> prompt appears > > Actual behavior > # Nothing, it is still the powershell prompt > === Still the powershell prompt or nothing at all? If the latter, try using "python -i" instead. The "-i" will force interactive mode if stdin isn't otherwise detected as interactive (under the covers, isatty() is false for stdin), which is where I believe the issue is. I've used that under Windows ssh sessions (though with cygwin rather than powershell as well as some of my own remoting tools) for as long as I can remember (certainly back to XP and Python 2.x - maybe 1.x) for an interactive prompt when operating without a local windows console. I'm not sure if there's any better way for Python to detect a remote shell as being interactive under Windows that would cover such cases. Perhaps some of the newer pty changes I read Microsoft is making might help, assuming it flows through to the isatty() test. -- David ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Can we clean up the buildbots please?
Larry Hastings la...@hastings.org writes: Is MSVS 2015 the only supported compiler for Python 3.5 on Windows? What's the other buildbot using MSVS 2015? For a while I think the only buildbot was my 8.1 slave, but I believe at this point Jeremy may also have it on his 7 slave. The latest on my 7 slave is still 2010 (which is still working, sans recent test failures). I'll answer my own question here. According to PCbuild/readme.txt: This script will use the env.bat script to detect one of Visual Studio 2015, 2013, 2012, or 2010, any of which may be used to build Python, though only Visual Studio 2015 is officially supported. I'll admit I'm puzzled by the wisdom of using unsupported compilers on buildbots. I guess it's a historical thing. But I gently suggest that we should either upgrade those buildbots to a supported compiler or remove them entirely. Definitely we should remove unsupported the two unsupported platforms from the buildbots--that's just crazy. To be fair, VS 2015 hasn't been officially released yet. It only recently (as in a few weeks ago) reached RC stage. Given the size of installing it, and earlier uncertainty about upgrading during the pre-release cycle, plus some early issues with the build process, for my part I've opted to hold off with my older slaves until it hits release status, using only the 8.1 slave until then. (Arguably the current RC is supposed to be at most a minor update away from full release, so we're probably close) Along the way it was concluded that XP just wasn't worth making work for the 3.5+ development, but the slave was still valuable for the 2.7 branch, so would be left around for now for that purpose. It is a bit misleading to still be trying to build the 3.x branch on it but I suspect eliminating the branch from that slave is just an oversight, or nobody with the proper access has had time yet. -- David ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Buildbot x86 XP-4 3.x doesn't compile anymore: drop it?
Victor Stinner victor.stin...@gmail.com writes: Would it be possible to fix this buildbot, or to turn it off? (...) By the way, do we seriously want to support Windows XP? I mean, *who* will maintain it (no me sorry!). I saw recent changes to explicitly *drop* support for Windows older than Visa (stop using GetTickCount, always call GetTickCount64, for time.monotonic). Disabling the 3.x branch (or 3.5+ going forward) on the XP buildbot seems plausible to me, assuming the issue is unlikely to be fixed (or someone found with time to look into it). If, as Tim suggests in his response, 3.5+ need not be officially supported on XP anyway, it may not be worth much effort to address. At some point (such as switching to VS 2015), we aren't going to be able to compile newer releases under XP anyway. I'm assuming you aren't suggesting turning off the XP buildbot entirely, correct? As long as any fixes are still getting applied to 2.x or earlier 3.x releases, I do think having a buildbot covering those branches for XP would be useful, if only to catch possible issues in back-ported fixes. There's still an amazing number of XP systems in service. While I doubt very many of those running XP also need the latest 3.x release of Python, I expect there are quite a few active 2.x users. -- David ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] ubuntu buildbot
On Mon, Nov 24, 2014 at 11:07 AM, Benjamin Peterson benja...@python.org wrote: On Mon, Nov 24, 2014, at 00:33, David Bolen wrote: Yeah, it definitely needs it. Historically it was intentional as my own servers were all on 8.04, but the last of those moved 12.04 last year. I think there's already a 12.04 buildbot, so perhaps 14.04 would be better? I do prefer sticking with an LTS. 14.04 would be good. It'll need to move to 64-bit given the hosting environment, at least for the kernel. I could do 32-bit userspace via multiarch if keeping a 32-bit buildbot in the mix is attractive. It'd be nice to keep a 32-bit bot around. Took a bit longer than anticipated, but the slave upgrade is complete. The bolen-ubuntu slave is now a 32-bit Ubuntu 14.04.2 LTS system. I've re-run the most recent 2.7, 3.4 and 3.x builds which all pass (though with a few new compiler warnings in some cases). -- David ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] ubuntu buildbot
Yeah, it definitely needs it. Historically it was intentional as my own servers were all on 8.04, but the last of those moved 12.04 last year. I think there's already a 12.04 buildbot, so perhaps 14.04 would be better? I do prefer sticking with an LTS. It'll need to move to 64-bit given the hosting environment, at least for the kernel. I could do 32-bit userspace via multiarch if keeping a 32-bit buildbot in the mix is attractive. -- David On Sun, Nov 23, 2014 at 10:18 PM, Benjamin Peterson benja...@python.org wrote: Hi David, I noticed you run the Builder x86 Ubuntu Shared buildbot. It seems it's running a very old version of Ubuntu. Is there any chance of getting that updated? Regards, Benjamin ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Who's using VS/Windows to work on Python?
Steve Dower steve.do...@microsoft.com writes: Also, who currently owns the Windows buildbots and are you willing/able to add a VS 2015 Preview installation (or give me access so I can do it)? (...) I've got several of the Windows buildbots, and could add this. Is there benefit to just starting with one (I've got XP, Win7 and Win8) first to save some time? My only real concern is to verify that this will co-exist properly with the existing VS installations and buildbot build process. I note that the download page says not to use on production computers. That's probably just a CYA, and I don't mind if VS itself has issues, but if it risks compromising other aspects of the buildbot, then I think testing on a different machine would be preferred. Is anything aside from VS 2015 itself (e.g., the Ultimate 2015 download on the preview page) needed? I'm assuming not. -- David ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Who's using VS/Windows to work on Python?
Steve Dower steve.do...@microsoft.com writes: Starting with just the Win7 or Win8 one would be fine. Python 3.5 won't support XP, and VS 2015 doesn't support XP (though I believe it will still be able to build for XP, just not *on* XP). Ok, I'll probably try the Win8 buildbot first then. I'll let you know when it's available. FYI, the VS 2015 requirements page seems to include XP as a supported client OS (if I'm reading it correctly), and says the requirements are the same as for VS 2013. Of course, as you say it won't matter if we're dropping XP for Python 3.5. VS 2010 should be fine. The most likely issues are with VS 2013 (for teams that haven't updated their setup authoring yet), but these should have been ironed out already. Ok, the buildbots are just VS 2008 and VS 2010, so no 2013 to worry about. Nope, that's it. (...) Sounds good. -- David ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] XP buildbot problem cloning from hg.python.org
Ned Deily n...@acm.org writes: Update: after consulting with Donald on IRC, it appears that the problem was on the python.org end and is now fixed. David, is it now working again for you? Sorry for the delay - yes, it appears to be working again for me as well. And it looks like clones during the buildbot tests were working again as of tests yesterday. -- David ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] XP buildbot problem cloning from hg.python.org
Donald Stufft don...@stufft.io writes: What version of OpenSSL is it using. I'm using the pre-built Windows Mercurial installer, but if I unpack the included library.zip, the SSLEAY32.DLL shows version 0.9.8r. This is from the 3.1.2 install I just did a few hours ago. It appears that hg 2.5.2 on my other XP box also has 0.9.8r. The prior buildbot version (1.6.2) looks like it had 0.9.8o. I also got around to trying a manual clone on the Windows 7 buildbot, and it worked fine, even with the older hg 1.6.2. So it seems to correlate with XP more than anything else at the moment. -- David ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] XP buildbot problem cloning from hg.python.org
Do you mean your local repo? If so, I don't have a local repo at this point - the failure is during the first clone. -- David On Sat, Oct 25, 2014 at 1:19 AM, Steve Dower steve.do...@microsoft.com wrote: I was seeing this recently and had to run recover on my repo (not sure what the command line is for that - TortoiseHg had a menu). YMMV, but the symptoms sound the same. Cheers, Steve ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] XP buildbot problem cloning from hg.python.org
Donald Stufft don...@stufft.io writes:
I have an idea, can you run https://bpaste.net/show/c5d7cd102f5b and
tell me what it outputs? Both on a machine that works and one that
doesn’t.
All but Linux (so XP/7 buildbots, XP standalone, OSX) return:
('DHE-RSA-AES128-SHA', 'TLSv1/SSLv3', 128)
My Linux (Ubuntu 12.04) returns:
('ECDHE-RSA-AES128-SHA', 'TLSv1/SSLv3', 128)
The script was run under a default Python on each box (2.6 on Windows,
2.7 on OSX and Linux). I tried 2.6 through 3.1 on my standalone XP with
no change, so I don't think it differs by Python version. Its not
precisely the same as running hg, since it has its own embedded Python
under Windows, but I installed a source install on my XP box under 2.7
and it fails a clone the same way.
In new news though, I just the same failure on the Win7 buildbot in a
clone test. In repeated attempts, that's the only one so far.
I also realized that one shared feature is that the XP boxes were using
IPv4 while the other boxes were all IPv6 (an HE tunnel on my side).
Though my earlier Win7 failure was also IPv6. I manually forced the
Win7 box to use IPv4, but didn't see much difference. It certainly didn't
start failing like the XP boxes.
Anecdotally, the failing XP attempts appear to be running slower in
general (with lower transfer rates as monitored by my router). I have
had slow clones work on other boxes, so that's not automatically bad.
But I wonder if it's still some sort of timeout somewhere.
I don't think I currently have an active ssh account, but if there were
a way to test a clone over ssh rather than http perhaps that would be a
useful data point, in terms of eliminating some middlemen processing.
-- David
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] XP buildbot problem cloning from hg.python.org
As another data point, I've tried cloning randomly selected other repositories from hg.python.org, and smaller repositories (distutils2, peps, jython to name a few) are all working fine under XP, even though with jython for example, the clone takes longer in terms of wall time than I'll often see cpython fail.(*) A test of what I presumed was a more comparably sized repository (features/cdecimal) dies like cpython. -- David (*) Overall clone time is probably unrelated anyway since the XP buildbot traditionally needed 10+min for clones in the past (such as when the new build script changes were in place and every test used a clone) and was working fine with that. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] XP buildbot problem cloning from hg.python.org
Starting yesterday, my XP buildbot began failing to execute clone operations against hg.python.org. There's not a lot of data being given aside from a transaction abort message (and my buildbot log showing the hg command exiting), and I'm wondering if something may be amiss on the server or its configuration? Note that this is a full clone (which for some reason the Windows buildbots seem to fall back on with some frequency) and can take quite a while. My Windows 7 buildbot is ok so far but it's still doing incremental pulls over the same time period. I've got two separate Internet connections here and have tried routing over both so I don't think it's a network issue. I've completely flushed the local build trees and rebooted the buildbot. Is there anything that might be available on the server to see if there are errors being logged? Or anything that could have changed configuration wise recently (maybe timeout related or something)? I'm running a bit low of items to try to change or reset on the buildbot side. Thanks. -- David ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] XP buildbot problem cloning from hg.python.org
Donald Stufft don...@stufft.io writes: Is this using HTTPS or SSH. Um, good question - whatever the buildbot build process uses. Looking at the slave log on buildbot.python.org (I don't get the hg output locally), appears to be http (it's cloning http://hg.python.org/cpython) - though I thought I saw it using https (port 443) in some traffic monitoring I was doing, so maybe it gets redirected? Oh yeah, the log also shows real URL is https://hg.python.org/cpython; as the first output from hg. -- David ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] XP buildbot problem cloning from hg.python.org
Antoine Pitrou solip...@pitrou.net writes: Have you tried running the hg clone manually from the buildbot? You could try to add --debug to get more info where the thing breaks. Yes, I had but pretty much got the same output as the buildbot slave. But I just tried --traceback and it's definitely complaining about the connection being terminated. Regular test: hg clone --verbose --noupdate http://hg.python.org/cpython test real URL is https://hg.python.org/cpython requesting all changes adding changesets adding manifests transaction abort! rollback completed abort: connection ended unexpectedly Traceback: hg clone --traceback --verbose --noupdate http://hg.python.org/cpython test real URL is https://hg.python.org/cpython requesting all changes adding changesets adding manifests transaction abort! rollback completed Traceback (most recent call last): File mercurial\dispatch.pyc, line 54, in _runcatch File mercurial\dispatch.pyc, line 490, in _dispatch File mercurial\dispatch.pyc, line 351, in runcommand File mercurial\dispatch.pyc, line 541, in _runcommand File mercurial\dispatch.pyc, line 495, in checkargs File mercurial\dispatch.pyc, line 488, in lambda File mercurial\util.pyc, line 420, in check File mercurial\commands.pyc, line 725, in clone File mercurial\hg.pyc, line 334, in clone File mercurial\localrepo.pyc, line 1853, in clone File mercurial\localrepo.pyc, line 1206, in pull File mercurial\localrepo.pyc, line 1695, in addchangegroup File mercurial\revlog.pyc, line 1239, in addgroup File mercurial\changegroup.pyc, line 31, in chunkiter File mercurial\changegroup.pyc, line 20, in getchunk File mercurial\util.pyc, line 924, in read File mercurial\httprepo.pyc, line 22, in zgenerator IOError: [Errno None] connection ended unexpectedly abort: connection ended unexpectedly I also stuck on --debug which generates a metric ton of output, but the final portion is: hg clone --debug --traceback --verbose --noupdate http://hg.python.org/cpython test (...) manifests: 5271/93170 chunks (5.66%) manifests: 5272/93170 chunks (5.66%) manifests: 5273/93170 chunks (5.66%) manifests: 5274/93170 chunks (5.66%) manifests: 5275/93170 chunks (5.66%) manifests: 5276/93170 chunks (5.66%) manifests: 5277/93170 chunks (5.66%) manifests: 5278/93170 chunks (5.66%) transaction abort! rollback completed Traceback (most recent call last): File mercurial\dispatch.pyc, line 54, in _runcatch File mercurial\dispatch.pyc, line 490, in _dispatch File mercurial\dispatch.pyc, line 351, in runcommand File mercurial\dispatch.pyc, line 541, in _runcommand File mercurial\dispatch.pyc, line 495, in checkargs File mercurial\dispatch.pyc, line 488, in lambda File mercurial\util.pyc, line 420, in check File mercurial\commands.pyc, line 725, in clone File mercurial\hg.pyc, line 334, in clone File mercurial\localrepo.pyc, line 1853, in clone File mercurial\localrepo.pyc, line 1206, in pull File mercurial\localrepo.pyc, line 1695, in addchangegroup File mercurial\revlog.pyc, line 1239, in addgroup File mercurial\changegroup.pyc, line 31, in chunkiter File mercurial\changegroup.pyc, line 20, in getchunk File mercurial\util.pyc, line 924, in read File mercurial\httprepo.pyc, line 22, in zgenerator IOError: [Errno None] connection ended unexpectedly abort: connection ended unexpectedly which appears to die mid-stream while receiving the manifests. So I'm sort of hoping there might be some record server-side as to why things are falling apart mid-way. -- David ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] XP buildbot problem cloning from hg.python.org
David Bolen db3l@gmail.com writes: which appears to die mid-stream while receiving the manifests. So I'm sort of hoping there might be some record server-side as to why things are falling apart mid-way. Just to follow-up to myself, I get the same same error trying to do a clone from my own personal XP machine rather than the buildbot (which is a VM). I've had the issue with hg 1.6.2, 2.5.2 and 3.1.2. However, the same clones completely successfully under OSX and Linux. So that's sort of strange. -- David ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Questions regarding Windows buildbots
Brian Curtin br...@python.org writes: On Mon, May 12, 2014 at 5:16 PM, Claudiu Popa pcmantic...@gmail.com wrote: (...) - If we can acquire the privilege by elevating our process, does the Windows buildbots have UAC enabled and if so, how's the notification setting configured? For instance, elevating a process will trigger a new UAC window with the message Do you want to allow the following program from an unknown publisher to make changes to this computer? on the recommended configuration, but this doesn't happen when the configuration is set to Never notify. That probably depends on how each machine is setup. If they happen to get blocked on any individual slave, we'll just have to ask the owner to change that setting. For reference, my Windows 7 slave (bolen-windows7) is currently running with stock UAC settings, so I believe would prompt in such a scenario. The slave does run all builds with Windows error dialogs disabled, but I doubt that covers UAC. It's certainly no problem to disable the UAC prompting if it would help or if that becomes a more useful setting for the buildbot environment. It probably fits better with my existing model of disabling other sorts of error popups anyway, but just isn't something we've run up against yet in the build process. Currently there are no Windows build slaves running as administrator. I used to have one but the machine died and I never replaced it. I also said a few months ago that I would get one setup again, but that hasn't happened yet. I can get a new machine up and running but probably not until next week as I'm at a conference. Yes - my slaves (XP and Win7) are running under administrative accounts, but not the literal Administrator account. Though I'm guessing this topic may be moot for the XP slave as it doesn't have UAC. Claudiu, if you've got any specific test code you'd like to have executed on either of my slaves to see how it behaves, I'd be happy to help. Just contact me directly. -- David ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] cpython (merge 3.3 - default): Merge fix from 3.3 into default.
Terry Reedy tjre...@udel.edu writes: On 9/1/2013 5:04 PM, Antoine Pitrou wrote: Probably because they don't have the hgeol extension enabled. Yes, I believe that's correct, at least for my Windows buildbots. Since the tests also failed on installed Python, it seems that the .msi installer is created in a repository with the extension enabled. If so, I think that the buildbots should have the extension enabled also, so that they are testing Python as it will be distributed. If it should be enabled, I believe it will have to be done globally, so affecting all buildbot branches (not that it sounds like that would be an issue). The individual branch repositories are physically removed at times (such as during a failure to update, which escalates to a removal followed by full clone, as well as I believe during a custom build), so I don't believe persistent local changes to the repository's own hgrc are possible in a reliable way. In the event we did want per-repository control, then I think we'd need to have some sort of additional script that ran as part of the buildbot build process to ensure the right contents for the local repository hgrc following a clone, or just prior to the update step. -- David ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Green buildbot failure.
Richard Oudkerk shibt...@gmail.com writes: On 11/08/2013 11:00am, Antoine Pitrou wrote: You've got the answer at the bottom: program finished with exit code 0 So for some reason, the test suite crashed, but with a successful exit code. Buildbot thinks it ran fine. Was the test terminated because it took too long? Yes, it looks like it. This test (and one on the XP-4 buildbot in the same time frame) was terminated by an external watchdog script that kills python_d processes that have been running for more than 2 hours. I put the script in place (quite a while back) as a workaround for failures that would strand a python process, blocking future tests due to files remaining in use. It's a last ditch, crude, sledge-hammer. Historically, if this code ran, the buildbot had already itself timed out, so the exit code (which I can't control) wasn't very important. 2 hours had been conservative (and a trade-off as longer values also risks failing more future tests) but it may need to be increased. In this particular case it was a false alarm - the host was heavily loaded during this time frame, which I think prolonged the test time by an unusually large amount. -- David ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Issue 18312 fix broke buildbots
Eric V. Smith e...@trueblade.com writes: http://buildbot.python.org/all/builders/x86%20Tiger%202.7/builds/2030 http://buildbot.python.org/all/builders/x86%20Tiger%203.3/builds/742 http://buildbot.python.org/all/builders/x86%20Tiger%203.x/builds/6522 http://buildbot.python.org/all/builders/bolen-dmg-3.x/builds/278 (...) I've looked at a few of these, and they don't seem related to my change. I'll review it some more this afternoon. The above set (which are all on the x86 Tiger buildbot) appear due to some sort of filesystem corruption on the buildbot machine, so it appears that the timing of the failures and particular changesets is coincidental. I've repaired the filesystem and the disk passes a surface scan, so hopefully it'll be cleaned up. I've also re-queued the most recent builds in each branch. -- David ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] hg.python.org Mercurial upgrade
Antoine Pitrou solip...@pitrou.net writes: I've upgraded the Mercurial version on hg.python.org. If there any problems, don't hesitate to post here. (apart from the connectivity problems we seem to have from time to time and which shouldn't be related) I'm not sure if this is related to the upgrade specifically (or just some other hg issue0, but yesterday my Windows 7 buildbot failed an hg pull operation on the 3.x branch and automatically removed it's local repository. Ever since then an attempt to start fresh (with an hg clone) is dying during the clone operation. I'm running the clone operation manually now, and after sitting there for 10 minutes or so at the adding changesets stage, it died just like in the buildbot logs (transaction abort! followed by rollback completed). Nothing was happening on the buildbot during the adding changesets delay so I'm assuming that was all server-side. -- David ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Proposed schedule for Python 3.4
mar...@v.loewis.de writes: We once had nightly builds of the Windows installers. It required a dedicated buildbot operator, since the process tended to break. For what it's worth, I'd be willing to try to re-institute the daily Windows installer builds if they'd see usage, but I suspect I'll need assistance with any initial issues with the packaging process, which is where the issues are most likely to be. At the time we shut them down (Dec 2008 I think) they had been failing for about 6 months without anyone noticing, so it didn't seem repairing it again. -- David ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Daily DMG builds
Ned Deily n...@acm.org writes: In article 20121001211812.4c40a...@pitrou.net, Antoine Pitrou solip...@pitrou.net wrote: Hello, It seems that the daily DMG builds have been failing for some time on the default branch: http://buildbot.python.org/daily-dmg/ Since there has been no report or complaint about that, should we just stop producing those builds? It's failing on a checksum error of the cached download ncurses tarball which is kind of odd. I've pinged David about it. Ned's email got to me before I got to the list today - the initial issue does appear to be a bad ncurses external download from a while back. The download was Aug 6 so that seems to match when the 3.x dmg stopped being produced. I flushed that and let it grab a new copy. The 3.3 branch then had a separate issue since it was brand new and the Python-ast.[ch] rebuild process doesn't work right under 10.4, but touching the files included in hg resolves that. (If I recall it's because it uses the system python which is ancient - I had done the same fix a while back for the 3.x builder). So I believe things should be ok at this point, or at least back to the way they were previously. I must admit I don't pay attention to the dmg buildbot but I probably should. I know I try to catch slave-related failures, but staying on top of all of the builders can be a little tough. I don't think there's a good summary page for the slaves I want to follow, so it's tricky to stay current (plus the trunk stuff tends to have more transient failures). Not sure if there's a way to have per-owner summary pages or anything on the build master. I do use the main build slaves summary page for overall status, but scanning the individual builders is 30+ other pages (the per-slave pages don't often have enough history to cover all the builders), or trying to find each of my builders in the overall waterfall page. Neither approach tends to survive very long in terms of my regularly doing it :-) Anyway, I always welcome (and appreciate) any head's up for failures that seem to be slave-related... I know at least one person (Raymond) has reported using the dmg builds regularly and the 2.7 builds are still going. There is a bit of an issue now for 3.3 and default: the buildbot-slave is apparently running OS X 10.4 so it is no longer capable of building either of the two installer variants that we release on python.org (one now requires at least 10.5 and the other at least 10.6). However, we don't want to break anything on 10.4, if possible, and the installers still work on later systems so there is some value in producing them - as long as someone is paying attention when they break and, hopefully, someone uses them. It's been a few years since setting this up, but at least at one point I think it was safer to build an installer on the older platform in terms of compatibility, but it wouldn't surprise me if that process has become dated, or no longer necessary. In fact, if I recall correctly, I think Ronald was already using a later platform for his own building when the daily buildbot was set up. At the time I needed a 10.4 platform for local application building (I had an app supporting 10.4 clients and couldn't build it on a later OSX and still have it install on older systems), but it was otherwise idle so available as a buildbot. The physical machine is unlikely to get upgraded beyond 10.4 though. It's never been entirely clear how useful the daily build slaves are. We used to do daily Windows builds too but when they ran into some problems the demand (or lack thereof) didn't seem to justify the time to troubleshoot so they were discontinued. My guess is the same will probably happen to the OSX daily build slave once it hits something difficult to troubleshoot/fix. Although until it does I suppose there's no need to explicitly turn it off. It would still be interesting to know who may still be making use of the images being produced. -- David ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] OS X buildbots missing
Antoine Pitrou solip...@pitrou.net writes: Well, the reason it can't qualify for the stable list right now is that there's a recurrent test_logging failure on it: http://bugs.python.org/issue14644 Yeah, I don't know that I'm necessarily suggesting it be in the stable set as just mentioning that there is at least an OS X buildbot available, if only to reference pending another coming on-line. In the past I think it was tough to keep Tiger green. Though several people put a decent amount of effort into cleaning things up and I believe it's been pretty good for a while now, the current 3.x issue notwithstanding (and which is probably at least in part a chicken and egg problem in terms of awareness). The other branches look good, at least for the limited history on the web interface, and it looks like the 3.x change worked. I know I still build some of my OS X applications under Tiger (since it's easier to support later OS X versions from an earlier build than vice versa), but I don't know if Python necessarily wants to require that for releases, ala stable. -- David ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] OS X buildbots missing
Antoine Pitrou solip...@pitrou.net writes: For the record, we don't have any stable OS X buildbots anymore. If you want to contribute a build slave (I hear we may have Apple employees reading this list), please take a look at http://wiki.python.org/moin/BuildBot I realize it may not qualify for the official stable list as it's a Tiger-based buildbot, but osx-tiger is an OS X buildbot that's still chugging along quite nicely (including doing the daily DMG builds). -- David ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Buildbot status web pages
I've been receiving 503 errors from the buildbot web status pages beneath www.python.org/dev/buildbot for a day or two now - is there perhaps something that needs a bit of a kick-start? Thanks. -- David ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Buildbots and regrtest timeout
Victor Stinner victor.stin...@haypocalc.com writes: But today I saw a a buildbot timeout without any traceback: a possible hang in test_io on x86 FreeBSD 7.2 3.x buildbot, command timed out: 3900 seconds without output. I realized that some buildbots (all buildbots?) override the TESTOPTS variable (make buildbottest TESTOPTS= TESTPYTHONOPTS= for x86 FreeBSD 7.2 3.x). (...) Please, don't disable the TESTTIMEOUT option for your buildbot. You may want to use a shorter or longer timeout, (...) Just a note, given the phrasing above. None of this is anything that I for example, as a buildbot operator, am actively controlling. That command, including the environment variable overrides, is exactly as provided by the master for a given test run. So I'd think you could adjust it if needed through changes in the master build configuration and probably without having to add an environment variable or Per Paul's follow-up on Windows, buildbot under Windows seems to impose a 1200s idle timeout (just for no output), but I'm not positive how it's calculated. The test process itself has never, I'm pretty sure, specified a timeout to regrtest (via the test.bat, rt.bat, regrtest.py path). (Oh, I guess the --timeout option itself in regrtest is fairly new) So if there's a change in defaults for regrtest that will change Windows behavior implicitly, and I believe at this point the buildbots will be inconsistent, since you're only overriding the regrtest default in a subset of buildbot types. -- David ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Notification for buildbot builder changes for slaves?
I was wondering if it might be possible to have a channel (message here, email to a list of slave owners or whatever) to mention when the set of builders for the slaves is getting adjusted? A new builder tree can burn a good deal of disk space in some cases for example (each tree adds in the neighborhood of .5GB for each of my Windows buildbots), and while it's generally not an issue, the recent conversion to hg actually exhausted my disk space on several slaves, for example. I'm not looking to get in the way of any process - just have a way to hear about such changes in advance when possible in case I need to make adjustments. Thanks. -- David ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Have we lost changeset info in the buildbots
Victor Stinner victor.stin...@haypocalc.com writes: Le lundi 14 mars 2011 à 15:36 -0400, David Bolen a écrit : Speaking of bbreport, I sometimes use the published page on that site (http://code.google.com/p/bbreport/wiki/PythonBuildbotReport) to check over things, but looking at it today, it seems to most recently have been generated back in January. Or is the generated date line wrong? I ran a cron task to regenerate this page each hour, but I didn't get any feedback and I stopped to use it, so I just removed the cron task. Do you want that I restart this cron task? If it's a hassle to maintain, it's probably not worth it. I can't say it's critical to me, but I do find it helpful to review periodically given how it summarizes issues that may aggregate up to a builder or slave. I just keep it open on a tab along with the build slave summary page which is my primary check, so it's a bit simpler than having to re-run a command line tool for quick checks. I did submit one feature request (issue 16) early on when I first started using it about seeing if it was possible to include slave (and not just builder) information. -- David ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] buildbot VM exhausted in test_crashers
I've run out of virtual memory on my Ubuntu buildbot today (and I think the FreeBSD buildbots a few days ago). Those have physically OOMd/restarted the VM. The Windows buildbots have been giving me VM warnings, but appear to have gotten through it. I believe it's the test_crashers test, which when I observed the Ubuntu buildbot when trying to bring it back online suddenly spikes to over 500MB allocated to the Python process (which is the limit of physical memory on that box), then causes an overall OOM and crash. The problem is that the failing build is automatically retried when I try to reconnect the buildbot, so it just get rights into trouble again. The comment in that build's changeset looks like it might be getting rolled back? Assuming that's the case, is there a way to flush any pending builds for my slaves before I bring them back online? Thanks. -- David PS: Neither FreeBSD nor my Ubuntu slave are actually up though the master thinks they are. The pinging option worked once for the Ubuntu slave but at the moment I can't seem to get the master to think that any of them are really disconnected. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] buildbot VM exhausted in test_crashers
Antoine Pitrou solip...@pitrou.net writes: I've enabled the button to cancel pending builds in the Web UI. You should be able to bring back the Ubuntu slave online. You can do the same (cancel pending builds) for other slaves if needed. Thanks. Is that a single button somewhere or do I need to cancel the pending builds individually when there are several waiting? In any event it looks like the retries of the failing 3.x build stopped on their own. I suspect, however, that as soon as something else commits to the 3.x branch and causes a new test, that test_crashers (presumably from changeset 9c6dcb5d8f01) will likely just take the slaves down again. Perhaps the test is just too hard to execute reliably on the entire build farm? -- David ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] buildbot VM exhausted in test_crashers
Nick Coghlan ncogh...@gmail.com writes: I don't want to give up completely on the idea just yet, but I'll experiment in the sandbox before I turn it back on. If you get to that point again, I'd also be willing to pick a time to manually check out the right branch or whatever and try it manually on one of two of my slaves while I'm watching. Typically I don't pay too much attention to various failures created by testing, but this particular case breaks in a way that the slaves don't recover, so is probably better to put a toe in the water slowly while watching :-) -- David ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] cpython: Exercise crashers to ensure they are still covering known error cases
Nick Coghlan ncogh...@gmail.com writes: On Thu, Mar 17, 2011 at 10:19 AM, Antoine Pitrou solip...@pitrou.net wrote: This has broken many buildbots. I'm pretty sure I know which one is the problem child, so I just pushed a fix which should get that one dying reliably on more platforms. On my Windows buildbots it appears as if this change started creating new debug RTL pop-up windows that my AutoIt script wasn't expecting, so it probably affected the build step in some subsequent builds too since the dialogs kept the python_d process live. I've reset those dialogs and updated my script to automatically acknowledge these dialogs too going forward, but failures prior to this point may not be true failures but just fallout from the prior issue. -- David ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Have we lost changeset info in the buildbots
Antoine Pitrou solip...@pitrou.net writes: I suggest you try http://code.google.com/p/bbreport/, which provides a very nice command-line interface. Speaking of bbreport, I sometimes use the published page on that site (http://code.google.com/p/bbreport/wiki/PythonBuildbotReport) to check over things, but looking at it today, it seems to most recently have been generated back in January. Or is the generated date line wrong? -- David ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] funky buildbot
Bill Janssen jans...@parc.com writes: I'm trying to get a new buildbot in the swim of things, and it keeps getting into this state where the buildslave process seems caught in an endless loop. Perhaps someone here knows why? Do you have any information as to what it is doing while in the loop? I think the only obvious looping I've seen with any of my buildbots is a loop while trying to attach to the master, either because the master was down/unreachable, rejecting connections (bad user/password), not yet configured for my buildbot, or occasionally because it thought the buildbot was still attached. In that latter case I just had to wait long enough for the status page to switch from idle to disconnected and then try again. -- David ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] funky buildbot
Bill Janssen jans...@parc.com writes: David Bolen db3l@gmail.com wrote: Bill Janssen jans...@parc.com writes: I'm trying to get a new buildbot in the swim of things, and it keeps getting into this state where the buildslave process seems caught in an endless loop. Perhaps someone here knows why? Do you have any information as to what it is doing while in the loop? Not much. The twistd log shows that it is, in fact, trying to reconnect to the build master. It's just chewing up 100% CPU. Could just be updates to the master that are doing this, but my other (PPC) buildbots seem OK with it. When I look at the slave status page[*], it seems to show all of your slaves as idle, so if one of them is looping trying to connect, I suspect you may have run into that last case I've seen (mismatch between slave and master). Try shutting down the build slave for a while and watching the status page until it shows disconnected. Then you should be able to restart things and have it attach successfully. Leaving the slave continually trying to reattach, I believe, prevents the master from ever declaring it disconnected. I'm guessing the master is rejecting the new connection since it thinks the slave is still attached (though the request seems to defeat the status from correcting). I'm not sure how it gets into that state but it's happened a handful of times to me (typically after some network interruptions, so maybe it just needs enough time for buildbot's own keepalives to notice what is otherwise a quiet TCP channel). There used to be a way to request a ping from the master side (I think on the same page you could manually run a build from) that I would used to force it to recognize a slave was really down, but after the web interface got rearranged a while back, I can't seem to find the option any more. -- David [*] http://www.python.org/dev/buildbot/buildslaves ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] funky buildbot
Bill Janssen jans...@parc.com writes: David Bolen db3l@gmail.com wrote: There used to be a way to request a ping from the master side (I think on the same page you could manually run a build from) that I would used to force it to recognize a slave was really down, but after the web interface got rearranged a while back, I can't seem to find the option any more. Looks like the ping option is back now :-). Hey, how about that. (I had taken a quick peek before writing the response to verify it was still unavailable so it's definitely recent) Thanks to whomever restored that. -- David ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Stable buildbots
Paul Moore p.f.mo...@gmail.com writes:
Presumably, you're inserting a pskill command somewhere into the
actual build process. I don't know much about buildbot, but I thought
that was controlled by the master and/or the Python build scripts,
neither of which I can change.
If I want to add a pskill command just after a build/test has run
(i.e., about where kill_python runs at the moment) how do I do that?
I haven't been able to - as you say there's no good way to hook into
the build process in real time as the changes have to be external or
they'll get zapped on the next checkout. I suppose you could rapidly
try to monitor the output of the build slave log file, but then you
risk killing a process from a next step if you miss something or are
too slow. And I've had cases (after long periods of continuous
runtime) where the build slave log stops being generated even while
the slave is running fine.
Anyway, in the absence of changes to the build tree, I finally gave up
and now run an external script (see below) that whacks any python_d
process it finds running for more than 2 hours (arbitrary choice). I
considered trying to dig deeper to identify processes with no logical
test parent (more similar to the build kill_python itself), but
decided it was too much effort for the minimal extra gain. So not
terribly different from your once a day pskill, though as you say if
you arbitrarily kill all python_d processes at any given point in
time, you risk interrupting an active test.
So the AutoIt script covers pop-ups and the script below cleans up
hung processes. On the subject of pop-ups, I'm not sure but if you
find your service not showing them try enabling the Allow service to
interact with the desktop option in the service definition. In my
experience though if a service can't perform a UI interaction, the
interaction just fails, so I wouldn't expect the process to get stuck
in that case.
Anyway, in my case the kill script itself is Cygwin/bash based, but
using the pstools tools, and conceptually just kills (pskill) any
python_d process identified as having been running for 2 or more hours
of wall time (via pslist):
- - - - - - - - - - - - - - - - - - - - - - - - -
#!/bin/sh
#
# kill_python.sh
#
# Quick 'n dirty script to watch for python_d processes that exceed a few
# hours of runtime, then kill then assuming they're hung
#
PROC=python_d
TIMEOUT=2
while [ 1 ]; do
echo `date` Checking...
PIDS=`pslist 21 | grep ^$PROC | awk -v TIMEOUT=$TIMEOUT
'{split($NF,fields,:); if (int(fields[1]) = int(TIMEOUT)) {print $2}}'`
if [ $PIDS ]; then
echo = `date`
for pid in $PIDS; do
pslist $pid 21 | grep ^$PROC
pskill $pid
done
echo =
fi
sleep 300
done
- - - - - - - - - - - - - - - - - - - - - - - - -
It's a kludge, but as you say, for us to impose this on the build
slave side requires it to be outside of the build tree. I've been
running it for about a month now and it seems to be doing the job. I
run a similar script on OSX (my Tiger slave also sometimes sees stuck
processes, though they just burn CPU rather than interfere with
tests), but there I can identify stranded python_d processes if they
are owned by init, so the script can react more quickly.
I'm pretty sure the best long term fix is to move the kill processing
into the clean script (as per issue 9973) rather than where it
currently is in the build script, but so far I don't think the idea
has been able to attract the interest of anyone who can actually
commit such a change. (See also the Dec continuation of this thread -
http://www.mail-archive.com/python-dev@python.org/msg54389.html)
I had also created issue 10641 from when I thought I found a problem
with kill_python, but that turned out incorrect, and in subsequent
tests kill_python in the build tree always worked, so the core issue
seems to always be the failure to run it at all as opposed to it not
working.
For now though, these two external monitors seem to have helped
contain the number of manual operations I have to do on my two Windows
slaves. (Though recently I've begun seeing two new sorts of pop-ups
under Windows 7 but both related to memory, so I think I just need to
give my VM a little more memory)
-- David
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] 3.2b2 fails test suite on (my) Windows XP
Brian Curtin brian.cur...@gmail.com writes: http://bugs.python.org/issue9116 covers this issue. The reason it doesn't fail on any of the build slaves is because they modify a registry value for Windows Error Reporting to not display the pop-up window, or at least mine does. I think I got the idea from one of the other Windows build slave maintainers. (delayed message as I was traveling) Note that the buildbot handling only prevents the pop-up dialogs. The underlying test that would have generated the dialog should still fail if the pop-up would have represented a failure, either through the Win32 API call failure, or the C RTL assertion termination the process with an error exit code. How that happens varies (OS dialogs are prevented from ever occurring, while C RTL dialogs have a scripted OK button press). There have been experiments with disabling the C RTL within Python itself in the past but never quite became consistent enough to trust on the buildbot (at least for me). But if somehow this is preventing a test failure from being detected on the buildbots, that's certainly an issue, and might indicate that some parent code to that causing the error isn't properly detecting it, either through an API error result, or process termination code. Of course, if the pop-up is not due to something considered a failure, then yes, the buildbot won't reflect what an actual user may see. I guess 9116 is an error in a child process which under Linux just terminates but under Win32 generates the pop-up, so perhaps aside from the pop-up, the test is assuming such an erroneous termination is acceptable? In such a case, then the approach in 9116 to permit control over the RTL is probably the only choice, but I don't think I see any way to accurately test that fact via the buildbots, since permitting such pop-ups are so disastrous to the overall build process. -- David ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Issue #10348: concurrent.futures doesn't work on BSD
Martin v. Löwis mar...@v.loewis.de writes: 1. Does it still fail on FreeBSD 7.3+? Yes, it still fails. The limits (30 semaphores) haven't changed. It also remains untunable. Yeah, my recollection about 7.3 appears to have been remembering when the kernel module was included by default as opposed to needing to be explicitly loaded. From the older discussion (http://mail.python.org/pipermail/python-dev/2010-November/105380.html) it would appear that 7.x remains fixed sans a kernel build (not necessarily a big deal for FreeBSD users), but 8.1+ can be tuned with sysctl. -- David ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Issue #10348: concurrent.futures doesn't work on BSD
Martin v. Löwis mar...@v.loewis.de writes: I don't have a good suggestion (or a computer with a keyboard anywhere near me) right now, but making a migration/fallback to SYSV style semaphores a release blocker seems like a mistake to me. And indeed, I don't propose to make that a release blocker. Instead, I propose to disable support for the module (either multiprocessing or concurrent.futures only) on FreeBSD, and make such disabling a release blocker. Or, I'll make the same offer I think I made in the multiprocessing case, which is I can build a kernel on the buildbot with a higher limit, if that's needed just to ensure test completion. Yes, it would also mean that users would need to do the same (or in later FreeBSD releases it can be done dynamically via sysctl), but that didn't seem to be considered a big hurdle in the prior discussion for multiprocessing. That would essentially switch this to a documentation issue, to document that on older FreeBSD platforms a user needs to take some steps, either as a startup configuration, or a kernel rebuild depending on release. That assumes that normal use of the module will need as many semaphores as the tests, but if not, presumably the documentation comments could become more a caution than a requirement. I've also been considering retiring the 6.x buildbot in favor of an 8.x (I only have resources for 2, and even that is slow). Updating the 7.x buildbot would also include dynamic adjustment of the limit, which I think based on the prior discussion her was mentioned as being in 7.3. Though 7.4 releases late in January so could jump right to that. -- David PS: In regards to another comment in this thread, while 7.2 doesn't load the POSIX support by default, the buildbot does have the appropriate loader configuration to include it. PPS: I am about to be traveling on business through 1/10, so unlikely to make any buildbot changes before then. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] kill_python on windows buildbots
Hirokazu Yamamoto ocean-c...@m2.ccsnet.ne.jp writes: Yes, but test can freeze. In that case, I'm worried that (snip) rt.bat # freeze here (will be halt by buildbot) vcbuild kill_python_d # Will this be called? in test.bat. Yeah, you're right. It may be impossible to completely eliminate the risk of a process getting stuck without doing something external to the build process, so long as the first thing a new build tries to do is an svn checkout that will fail if the process is still running. Having the kill operation in clean.bat covers the vast majority of the common cases with a minimum change, so seems the simplest. -- David ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] kill_python on windows buildbots
Tres Seaver tsea...@palladion.com writes: Maybe belt-and-suspenders it in both places. The clean batch file is also called from the build step, so relocating it there should maintain the existing behavior as well. Hirokazu (ocean-city) pointed out in my new issue an earlier issue he created (#9973) that included a patch for such a change. In thinking about it some more, I suppose there's still a small window for a loss of communication during a test which results in clean not being run (which could then block the next svn checkout without an opportunity to kill the processes), so maybe the right place is actually at the end of the test batch file which is the step during which such hung processes might get generated? I don't know the history, if any, of it's current location in the flow. Relocating the use of kill_python seems reasonable to me, after which we could sort of wait and see if we run into any other hangs that interfere with the builds. I can't make such a change myself though. -- David ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] kill_python on windows buildbots (was Re: Stable buildbots)
I previously wrote: I suspect the problem may be on the identify which process to kill rather than the kill it part, but it's definitely going to take time to figure that out for sure. While the approach kill_python takes is much more appropriate, since we don't currently have multiple builds running simultaneously (and for me the machines are dedicated as build slaves, so I won't be having my own python_d), a more blanket kill operation is safe enough. For anyone interested, I caught (well, Georg Brandl caught it first) a case on Saturday with some hung processes on the Win7 buildbot that I was able to verify kill_python failed to kill. This was after having a few instances where it did work fine. I've created issue 10641 to track. I also noticed another recent issue (10136) that is also related to kill_python missing processes, but given that it works in my case some of the time, and is always called via the same path, not sure if that can be my problem. I also realized that some fraction of the other cases I have seen might not have truly been an issue, since from what I can see kill_python is only run at the start of a build process, so hung processes (even killable ones) from a prior build hang around until the next build takes place. They can however, interfere with the svn checkout so things never get to the point of using kill_python. So maybe kill_python's use should be moved to the clean stage? -- David ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Stable buildbots
Trent Nelson tr...@snakebite.org writes: That's interesting. (That kill_python.exe doesn't kill the wedged processes, but pskill does.) kill_python is pretty simple, it just calls TerminateProcess() after acquiring a handle with the relevant PROCESS_TERMINATE access right. (...) Are you calling pskill with the -t flag? i.e. kill process and all dependents? That might be the ticket, especially if killing the child process that wedged select() is waiting on causes it to return, and thus, makes it killable. Nope, just pskill python_d. Haven't bothered to check the pskill source but I'm assuming it's just a basic TerminateProcess. Ideally my quickest workaround would just be to replace the kill_python in the buildbot tools script with that command but of course they could get updated on checkouts and I'm not arguing it's generally appropriate enough to belong in the source. I suspect the problem may be on the identify which process to kill rather than the kill it part, but it's definitely going to take time to figure that out for sure. While the approach kill_python takes is much more appropriate, since we don't currently have multiple builds running simultaneously (and for me the machines are dedicated as build slaves, so I won't be having my own python_d), a more blanket kill operation is safe enough. Otherwise, if it happens again, can you try kill_python.exe first, then pskill, and confirm if the former fails but the latter succeeds? Yeah, I've got a temporary tree with a built-binary around, but still have to make sure of the right way to run it manually in a way that it will do the identification right (which I think also means I need to figure out from which build tree the hung process started). Up until now, typically when I've found a hung setup, the rest of the build tree which originally applied to that process has been cleaned. I definitely sympathize with Martin's position though - it wasn't the simplest tool to write (and I still have some email from him about the week+ it took just to test the process identification part remotely through buildbots at the time), so I regret not jumping right in to try to fix it. But it's just way more effort than typing pskill python_d, at least with my current availability. -- David ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Stable buildbots
Brian Curtin brian.cur...@gmail.com writes: Is the dialog closer script available somewhere? I'm guessing this is the same script that closes the window which pops up during test_capi's crash? Not sure about that specific test, as I won't normally see the windows. If the failure is causing a C RTL pop-up, then yes, the script will be closing it. If the test is generating an OS level pop-up (process error dialog from the OS, not RTL) then that is instead suppressed for any of the child processes run on my slave, so it never shows up at all. The RTL script is trivial enough that I'll just include it inline: - - - - - - - - - - - - - - - - - - - - - - - - - ; buildbot.au3 ; Forceably acknowledge any RTL pop-ups that may occur during testing $MSVCRT = Microsoft Visual C++ Runtime Library while 1 ; Wait for any RTL pop-up and then acknowledge WinWait($MSVCRT) ControlClick($MSVCRT, , [CLASS:Button; TEXT:OK]) ; Safety check to avoid spinning if it doesn't go away Sleep(1000) WEnd - - - - - - - - - - - - - - - - - - - - - - - - - Execute with AutoIt3 (http://www.autoitscript.com/autoit3/). I just use the plain autoit3.exe against this script from the Startup folder. The error mode buildbot patch was discussed in the past on this list (or it might have been the python-3000-devel list at the time). Originally it just used pywin32, but I added a fallback to ctypes if available. When first done, we were still building pre-2.5 builds - I suppose at this point it could just assume the presence of ctypes. The patch below is from 0.7.11p3: - - - - - - - - - - - - - - - - - - - - - - - - - --- commands.py 2009-08-13 11:53:17.0 -0400 +++ /cygdrive/d/python/2.6/lib/site-packages/buildbot/slave/commands.py 2009-11-08 02:09:38.0 -0500 @@ -489,6 +489,23 @@ if not self.keepStdinOpen: self.pp.closeStdin() +# [db3l] Under Win32, try to control error mode +win32_SetErrorMode = None +if runtime.platformType == 'win32': +try: +import win32api +win32_SetErrorMode = win32api.SetErrorMode +except: +try: +import ctypes +win32_SetErrorMode = ctypes.windll.kernel32.SetErrorMode +except: +pass + +if win32_SetErrorMode: +log.msg( Setting Windows error mode) +old_err_mode = win32_SetErrorMode(7) + # win32eventreactor's spawnProcess (under twisted = 2.0.1) returns # None, as opposed to all the posixbase-derived reactors (which # return the new Process object). This is a nuisance. We can make up @@ -509,6 +526,10 @@ if not self.process: self.process = p +# [db3l] +if win32_SetErrorMode: +win32_SetErrorMode(old_err_mode) + # connectionMade also closes stdin as long as we're not using a PTY. # This is intended to kill off inappropriately interactive commands # better than the (long) hung-command timeout. ProcessPTY should be - - - - - - - - - - - - - - - - - - - - - - - - - -- David ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Stable buildbots
Nick Coghlan ncogh...@gmail.com writes: Do we have any idea why the workaround to avoid the popup windows stopped working? (assuming it ever worked reliably - I thought it did, but that impression may have been incorrect) Oh, the pop-up handling for the RTL dialogs still seems to be working fine (at least I haven't seen any since I put it in place). That, plus the original buildbot tweaks to block any OS popups still looks solid for avoiding any dialogs that block a test process. This is a completely separate issue, though probably around just as long, and like the popup problem its frequency changes over time. By hung here I'm referring to cases where something must go wrong with a test and/or its cleanup such that a python_d process remains running, usually several of them at the same time. So I end up with a bunch of python_d processes in the background (but not with any dialogs pending), which eventually cause errors during attempts the next time the same builder is used since the file remains in use. I expect some of this may be the lack of a good process group cleanup under Windows, though the root cause may not be unique to Windows. I see something very similar reasonable frequency on my OSX Tiger buildbot as well. But since the filesystem there can let the build tree get cleaned and rebuilt even with a stranded executable, the impact is minimal on subsequent tests than on Windows, though the OSX processes do burn a ton of CPU. I run a script on OSX to kill them off, but that was quick to whip up since in those cases the stranded processes all end up getting owned by init so it's a simple ps grep and kill. In the Windows case I'll probably just set a time limit so if the processes have been around more than a few hours I figure they're safe to kill. -- David ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Stable buildbots
Martin v. Löwis mar...@v.loewis.de writes: This is what kill_python.exe is supposed to solve. So I recommend to investigate why it fails to kill the hanging Pythons. Yeah, I know, and I can't say I disagree in principle - not sure why Windows doesn't let the kill in that module work (or if there's an issue actually running it under all conditions). At the moment though, I do know that using the sysinternals pskill utility externally (which is what I currently do interactively) definitely works so to be honest, automating that is a guaranteed bang for buck at this point with no analysis involved. Looking into kill_python or its use can be a follow-on. -- David ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Stable buildbots
Paul Moore p.f.mo...@gmail.com writes: Do you run your slave as a service? (And for that matter, what do other Windows slave owners do?) Are there any best practices for ongoing admin of a Windows buildslave that might be worth collecting together? (I'll try to put some notes on what I've found together - maybe a page on the Python wiki would be the best place to collect them). I've always run my slave interactively under Windows (well, started it interactively). Not sure if I tried a service in the beginning or not, it was a while ago. So your slave is probably the guinea pig for service operation. There is http://wiki.python.org/moin/BuildbotOnWindows (for which I can't take any credit). It could probably use a little love and updating, and it's largely aimed at setting things up, but not as much operating it. I think the only stuff I'm doing on my slave above and beyond the basic setup is a small patch to buildbot (circa 2007, couldn't get it back upstream at the time) to use SetErrorMode to disable OS pop-ups, and the AutoIt script (from earlier this year) to auto-acknowledge C RTL pop-ups. The kill script in this thread as a safety net above kill_python would be a third tweak. There was a buildbot fix for uploading that was only needed for the short-lived MSI generation, and which I think later buildbot versions have their own changes for. I'd be happy to work with you if you're willing to combine/edit our bits of information. Probably something we can take off-list, so just let me know. -- David ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Stable buildbots
Antoine Pitrou solip...@pitrou.net writes: (even though the Windows buildbots give a rather unconventional meaning to the word stability). Nag, nag, nag :-) There's been a bit of an uptick in the past few weeks with hung python_d processes (not a new issue, but it ebbs and flows), so I'm going to try to pull together a monitor script this weekend to start killing them off automatically. Should at least get rid of some of the low hanging fruit that interferes with subsequent builds. -- David ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Too many open files errors on x86 FreeBSD 7.2 3.x buildbot
Jeroen Ruigrok van der Werven asmo...@in-nomine.org writes: -On [20101108 00:36], David Bolen (db3l@gmail.com) wrote: Well, I think the SYSV semaphores are either less limited or at least more adjustable. They've certainly been around longer in FreeBSD. The POSIX semaphore support is not enabled by default in FreeBSD 7, so I added loader.conf stuff to load them (as part of issue7272). It is enabled by default on FreeBSD 8 at least. Looking through the repository it seems 7-STABLE has it enabled by default as well in the GENERIC kernel (the standard one it boots with after its first install). It seems this was added for 7.3 and onward. So 7.2 and before need an options P1003_1B_SEMAPHORES added to their kernel at least. The SYSV options are already present in the entire 7.x line. My use of enabled may not have been the best word choice since I didn't mean to imply a kernel option. I'm still using GENERIC on the 7.2 buildbot, so I didn't need to recompile the kernel in that release either. The issue was that the POSIX semaphore module wasn't loaded by default (something I thought only changed in 8.x), so the buildbot currently has a 'sem_load=YES' loader.conf entry to ensure that's done. -- David ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Too many open files errors on x86 FreeBSD 7.2 3.x buildbot
Victor Stinner victor.stin...@haypocalc.com writes: You wrote that the POSIX semaphore are very limited. Do it mean that there is another kind of semaphore will an higher limit? Well, I think the SYSV semaphores are either less limited or at least more adjustable. They've certainly been around longer in FreeBSD. The POSIX semaphore support is not enabled by default in FreeBSD 7, so I added loader.conf stuff to load them (as part of issue7272). I don't think the Python internals are using the SYSV semaphores though. SYSV functions have no underscore (e.g., semget) whereas POSIX do (sem_get). Also, I believe only POSIX has named semaphores. I don't think that skipping the test is a good idea: it looks like a real bug in (a limitation of) the ProcessPoolExecutor implementation on FreeBSD. Eg. test_map fails on FreeBSD 7.2 with ProcessPoolExecutorTest which uses self.executor = futures.ProcessPoolExecutor(max_workers=1): only one worker process! It looks like it is possible to tune semaphore limits on FreeBSD, without recompiling the kernel, by using boot loader option (kern.ipc.sem* options). But ask the FreeBSD user to tune its boot loader options to use the concurrent.futures module is not pratical :-) Yeah, I guess the key question is if changing the limit is just needed to get around an artifact of the test process (which I'm willing to do for the buildbot), or if it would be needed to be able to use the regular modules in practice. If the latter, I doubt too many users are going to jump through such hoops, particularly if it needs a kernel rebuild, so we may need to make other choices in terms of support under FreeBSD. I'm also not entirely sure just what is the limiting factor. I think the kern.ipc.sem* options are for the SYSV semaphores, not POSIX, though some of them do have a similar limit. Some are adjustable by sysctl, others by loader.conf. The references I found were talking about a limit set explicitly (#define SEM_MAX) in the kernel source (uipc_sem.c) which exports its value (at least in 7.2) via the sysctl p1003_1b.sem_nsems_max, which is read-only. I got the impression they weren't adjustable even in loader.conf, but haven't actually tried it yet myself. It may be different in 8.x, but one email thread I found indicated that the changes proposed to make the POSIX limits adjustable didn't make the 8.1 cut (current release), though might make it in the next 8.x release. -- David ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Too many open files errors on x86 FreeBSD 7.2 3.x buildbot
On Sat, Nov 6, 2010 at 7:19 AM, Victor Stinner victor.stin...@haypocalc.com I noticed OSError: [Errno 23] Too many open files in system errors on your FreeBSD buildbot. I would like to know if you configured a limit on the open files or maybe of child processes on this buildbot or not, or if it is a failure in Python? The first error always occurs in the first test of test_concurrent_futures. It's maybe because this test uses a lot of open files or processes? I couldn't find the matching failures that you're talking about, but then I figured out you mean the FreeBSD7 (7.2) buildbot, not the FreeBSD (6.4) buildbot I haven't configured any specific limits with respect to open files. On both FreeBSD buildbots, kern.maxfiles is 3600 and kern.maxfilesperproc is 3060. Both have limits of 1530 processes. The latter also agrees with the maximum descriptors as shown by limit. In regards to R. David Murray's response, the buildbots are VMs with limited memory, so the dynamic calculation he references for descriptors is much lower than his system. Looks like the reason FreeBSD is ok, and FreeBSD7 is because the relevant tests don't run due to lack of POSIX semaphore support. I manually enabled their use on FreeBSD7 a while back (11/2009, issue7272) since they aren't on by default. I'd be surprised if at least test_multiprocessing didn't pass at that point (since that's what the issue was for) but even it seems to be generating the open files error now. The buildbots haven't changed, but I suppose the tests might just have grown in the number of files they need over time. I noticed that the failures seem to always be on a semaphore call. Some quick googling found a few references that seems to imply that the number of posix semaphores are very limited (like 30), and can't be changed without recompiling the kernel from source. So that's not so big a threshold for the tests to have perhaps started crossing since issue7272 was fixed. Certainly seems more likely than 3000+ files or 1500+ processes. I wonder if it's possible to deduce if this started recently or not? The web buildbot interface doesn't go back that far, and an additional complexity is that the FreeBSD builds tend to have various errors somewhat consistently over time, but perhaps there are server logs we can grep for this particular error? Not sure if the best approach at this point is to see if the tests can use fewer semaphores, skip these tests under FreeBSD 7 like 6, or if it's important enough to compile a new kernel with a higher semaphore limit. -- David ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] SSH access against buildbot boxes
Giampaolo Rodolà g.rod...@gmail.com writes: In such cases I would find more easy to be able to connect to the machine and test myself rather than create a separate branch, commit, schedule a buildbot run, wait for it to complete and see whether everything is green. I agree with both Stephen and Martin's prior responses. For me, I'm happy to arrange for individual access on a case by case basis, but am less comfortable leaving access enabled permanently. I've arranged access to both my Windows and FreeBSD buildbots in the past, and while I suspect my OSX Tiger buildbot may be a little less interesting than the other OSX boxes, the offer remains open for any of my buildbots. -- David ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Regular scheduled releases
Martin v. Löwis mar...@v.loewis.de writes: People will never ever test nightly builds. Been there, done that. Instead, the nightly build process will break, and nobody will fix it for months (or even complain, for that matter). Certainly seems to be past experience. I know Martin knows this, but for other readers who may not, my Windows XP buildbot built nightly windows installers (the basic MSI package, close but not necessarily a fully signed new release as Martin makes) starting in September, 2007. It ran successfully for about 6 months, at which point it started to fail fairly consistently. Nobody noticed, and Martin and I finally just shut it down in December, deciding it wasn't worth the effort to try to fix. My OSX buildbot has been building nightly DMG images (though again, I suspect Ronald has a few extra steps beyond what its doing for a full release) since April. I'd actually be interested in knowing if anyone is using them - I suspect perhaps not. In both cases, getting the process going actually took quite a bit of effort (even stuff like having to fix the buildbot upload code in the Windows case), not just on my part, but with the help of Martin and Ronald. But without actual use of the result, it's hard to think it was worth it. I'm pretty sure my default reaction to a break-down in the current OSX build process at this point would be to first suggest disabling it unless there were real users. -- David ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Stable build slaves authority
Martin v. Löwis mar...@v.loewis.de writes: Please be aware that Windows poses its own challenges. Often, builds or testsuite runs end up with popup windows, which then hang subsequent builds. You often get dozens of them to click away. So operating a Windows slave is much more tedious than a Unix one. For anyone else considering a Windows buildbot, recently I started using an AutoIt script in the background to automatically clear any RTL dialogs (which seems to come and go as an issue but got really bad recently in one of the branches, and I lost track of when Python itself stopped disabling them during tests), and it's been working well so far. I also have a tweak in the local buildbot code to disable all Windows-based (non-RTL) dialogs during test runs. I'd be happy to provide both bits to anyone else starting a Windows buildbot. There's still a regular problem of stranded python_d processes in the background (something shared with my OSX tiger slave, but there I can run a script to detect processes owned by init and kill them). So periodic manual checks/cleanup is still definitely needed. -- David ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Fwd: Builder: x86 Windows7 3.x OpenSSL compile error
Hirokazu Yamamoto ocean-c...@m2.ccsnet.ne.jp writes: Hello. I've sent following mail to buildbot manager, but I found that buildbot page saids the problem of unsable bot should be sent to python-...@python.org. So I'll do it. (I'm fine with direct email for any of my build slaves, and it's probably the quickest way to get me to pay attention, but this issue has been around a bit and likely also of general interest) Nice catch on this - the Win7 3.x branch has failed to build OpenSSL for a bit now and I hadn't figured out why. Flushing out the OpenSSL tree hadn't fixed it, and it was weird that it didn't happen on XP, nor did the pre-1.0 OpenSSL releases have a problem on Win7. So I was thinking it might just be an issue with OpenSSL. I hadn't even thought of the possibility of it being the assembler. I've bumped the slave up to the latest nasm (2.09.02) and restarted the most recent build and it's gotten past the OpenSSL stage, so it looks like 2.06 may be the specific bad version. I guess I just happened to construct this builder in the relative small interval when 2.06 was current. Thanks again for figuring this out. -- David ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Builder: x86 Windows7 3.x OpenSSL compile error
Antoine Pitrou solip...@pitrou.net writes:
(...)
test test_ssl failed -- Can't read certificate file
b'D:\\cygwin\\home\\db3l\\buildarea\\3.x.bolen-windows7\\build\\lib\\test\\keycert.pem'
But the aforementioned certificate file is in the SVN tree and other
buildslaves have no problem reading it. Can you try to check if there's
something wrong on your buildslave?
I suspect it's an actual Python bug, though perhaps Win7-specific.
The file is in fact in the tree, and I can read it just fine from the
command line. If I'm reading the test right, and regardless of the
actual error message, it's really failing an existence test
(os.path.exists) rather than any failure to open or read the file.
Looks like some issue with using bytes for os.path.exists - I
borrowed a Python 3.x build currently in the tree as part of build
1611 and tried:
d...@buildbot-win7 ~
$ buildarea/3.x.bolen-windows7/build/pcbuild/python_d
Python 3.2a2+ (py3k, Sep 20 2010, 18:49:03) [MSC v.1500 32 bit (Intel)] on win32
Type help, copyright, credits or license for more information.
import os
[50683 refs]
os.path.exists('d:\\cygwin\\home\\db3l\\buildarea\\3.x.bolen-windows7\\build\\lib\\test\\keycert.pem')
True
[50689 refs]
os.path.exists(b'd:\\cygwin\\home\\db3l\\buildarea\\3.x.bolen-windows7\\buil
d\\lib\\test\\keycert.pem')
False
[50718 refs]
So the bug appears to be why those two calls return a different result.
-- David
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Windows buildbots MSVC RTL popups
Since the recent history of my two Windows buildbots has turned ugly, I figured I'd mention that they both (XP and Windows 7) have started generating quite a few GUI C++ RTL runtime pop-up assertions, which has been throwing a wrench into things until they get manually cleared. I first noticed them Sunday night but they were probably there 24-48 hours at that point. These are R6034 An application has made an attempt to load the C runtime library incorrectly I glanced through recent commits and I don't think I see anything obviously related but was wondering if anyone who might have been committing changes in the past few days might have an thought? The source filename is truncated in the dialog (what genius at MS thought that would be the one field worth truncating?!) so I am not absolutely sure if they are limited to a specific branch or not. I've thrown on an autoit script to try to automatically acknowledge these dialogs so hopefully the affected test runs will just fail rather than timing out and blocking later runs. -- David ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] proto-pep: plugin proposal (for unittest)
Michael Foord fuzzy...@voidspace.org.uk writes: I would be interested in hearing from other Mac users as to where they would look for configuration files for command line tools - in ~ or in ~/Library/Preferences? My primary personal machine has been OSX for years now, and as someone who lives in the command line (well, or Emacs shells), I think either approach is workable for command line only tools. I will say that if I need to find preferences or application-specific files my first inclination is absolutely to look under ~/Library. That's the platform location (just as looking for dot files under Linux is my first choice, or Documents and Settings/user for Windows). I don't think I shift mental gears automatically for command line tools versus not, unless I have some prior knowledge about the tool. With that said, given all the third party and Unix-oriented stuff I install under OSX, it's hardly rare (as has been pointed out elsewhere) to be working with ~/.something either, so it's not like I'd consider it that unusual to find that's where I need to go. In glancing at my current system, it does appear command line only tools are more commonly using ~/.something files rather than under ~/Library (which tends to be stuff packaged up as an application in /Applications, even if they can also be run from the command line). Though it might be a biased sample set since I'm more likely to have brought in command line tools to OSX from the Unix side of things, I suspect that's true of other users of command line tools as well. I will say that it's rarer to find a native (Cocoa/Carbon) GUI application that doesn't store preferences or application settings beneath ~/Library, and in such a case I'd feel they were more wrong and non-conforming if they didn't do that. So it depends on how native an application wishes to be perceived. I guess in thinking about it while writing this, having something installed in /Applications is more strongly linked with ~/Library in my mind than other tools. Of course, even with /Applications, non-native GUI apps are more of a mixed bag. For example, the X versions of Gimp and Inkscape - Gimp properly uses ~/Library/Application Support while Inkscape still uses ~/.inkscape. Of course, as X apps, neither truly feels native or conforming anyway. So that probably helps make things as clear as mud :-) -- David ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] MSDN subscription extensions?
As one of the beneficiaries of the efforts (much appreciated) last year to obtain Microsoft MSDN subscriptions for developers/testers (in my case, primarily buildbot operation), I was wondering if anyone might know if those subscriptions will be able to be renewed this year? -- David ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] red buildbots on 2.7
Paul Moore p.f.mo...@gmail.com writes: Thanks for the alert. I've killed the stuck test and should see some runs going through now. Shame, really, I was getting used to seeing a nice page of all green results... In my experience, my OSX and Windows buildbots need some manual TLC on an ongoing basis. I kill off stranded python processes several times a week on both platforms. OSX actually seems as bad as Windows in this regard, which is strange given its *nix heritage, but perhaps its how some of the test processes are created. Most of the time the stranded processes aren't hurting anything but local resource, but sometimes they can lock directories, or hang a build/test for a particular builder. My windows buildbots also have a tendency to fill up temp, or even if there's room, get sluggish due to all the cruft left in that directory, so I periodically clean that out manually as well. -- David ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] please take a look at buildbot result
Bill Janssen jans...@parc.com writes: Ronald Oussoren ronaldousso...@mac.com wrote: As Antoine noted the test failures are unexpected, could you check if the tests pass if you do the build and testrun manually? What about readline? Darwin doesn't have it (or rather, it has a different one). Why is that 'skip' unexpected? For what it's worth, for the libraries, I used the build-installer script to build the same versions of libraries that it uses for the binary installer, and installed them in /usr/local for my buildbot to use, until there's enough time to have the buildbot build such local libraries within its own tree. That at least matches up the external dependencies for buildbot builds with that used by the eventual binary installer. For my buildbot, generally the only unexpected skips are for test_bsddb185 (which feels right to me - I don't have that version of the library, nor do I think has the binary installer) and test_ioctl (which I have no idea on yet). IMHO it would be better to do a framework build for at least some of the OSX buildbots because that's what is in the binary installer for OSX. Yes, that sounds good to me, too. But how do we make that a standard test so that appropriately enabled buildbot slaves will do it automatically? Something that gets configured in the build master? This came up in the earlier discussion, and while I do still think that's a better long term approach, I suspect that with respect to test coverage and code generation the non-framework build for the interpreter is a fair representation for testing. I suspect much of this is just a builder change on the master (to use the right Makefile targets for generating the framework build), but just given experience to date getting the binary installer building under buildbot, there may be some unexpected environmental stuff. Ideally we'd work up a way to do a universal framework build (though I'm not sure what if any extra support may be needed to have the system use the framework if not installed in a standard location), and then for full testing, maybe even use the makefile target that tests both the Intel and PPC path (the latter via Rosetta on an Intel system). What I'm not sure about at the moment is how much different in terms of testing such a setup might be (if, for example, any extra work is needed to be able to test a framework build while still local in the buildbot tree and not in a normal framework location), so how much energy is worth putting into it, especially if that might use resource among those able to resolve some of the open code issues. For my part, if there's free cycles I'd personally rather address the external library issue first, as opposed to the framework build stuff, since that feels more fragile to me (in terms of the buildbot environment for the Mac) at the moment. Over the past few weeks I'm also fairly sure that I'm finding stranded python test processes (not too dissimilar as on the Windows buildbots) that are bogging down the buildbot and thus more detrimental to ongoing buildbot operation than the lack of framework builds, so there may be other unknown issues more valuable to hit as we get more experience. -- David ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Fwd: Broken link to download (Mac OS X)
Michael Foord fuzzy...@voidspace.org.uk writes: 10.6.3 and yes I have Tcl and Tk in /Library/Frameworks. How do I determine which versions they are? You can use info patchlevel in tclsh - assuming you're running a tclsh linked to your /Library version (a normal Tcl install puts this in /usr/local/bin I think). Or, tcl.h (in the Headers folder beneath the framework install) has TCL_VERSION and TCL_PATCH_LEVEL defines near the top of the file. Given that your error is a failure to build and not a skip, it sounds like setup is finding Tcl/Tk. From a quick glance, it looks like tkinter may also require the X11 headers (you'd have to have installed X11 separately) - do you have output in your log from what exactly is failing when that module attempts to build? -- David ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Fwd: Broken link to download (Mac OS X)
Michael Foord fuzzy...@voidspace.org.uk writes: Hmmm... looks like a 32 / 64 bit issue, which I believe may be the expected result when trying to build on Snow Leopard (?). I think so - I haven't tried a 64-bit build myself, but there's a comment in setup.py indicating that none of the Tcl/Tk framework builds support 64-bit. So I suppose you'd have to build 32-bit if you wanted a working _tkinter. -- David ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] MSDN licenses available for python-dev
Antoine Pitrou solip...@pitrou.net writes: Does it include a license for Windows itself? Does it allow me to install and run it in a VM? If so, I'm interested. Yes, in fact, it's due to the availability of this license that I was able to set up the Win7 buildbot. -- David ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Automatic installer builds (was Re: Fwd: Broken link to download (Mac OS X))
Martin v. Löwis mar...@v.loewis.de writes: I do currently have a DMG built for 2.7 Beta 1, if it would be useful. As I said before: if you would plan to do this on a regular basis, for all upcoming releases, that would certainly be a good thing. No argument - just figured I'd offer to get past the near term resource issue, presuming that having a beta installer available is better than not, even with some build-environment difference risk. I could probably commit to a longer term too, but it sounds like Ronald is still on board for that. I've also been a little hesitant since there's clearly a lot of effort that has been put into the Mac build stuff and I didn't want to be the inexperienced bull in the china shop compared to those who have been doing it for a while. I'm happy to stick with the buildbot side of things for now, and remain willing to assist in any changes to better match the buildbot builds to the final distribution process for more aligned testing. -- David ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Fwd: Broken link to download (Mac OS X)
Ronald Oussoren ronaldousso...@mac.com writes: On 18 Apr, 2010, at 17:17, Martin v. Löwis wrote: Ronald Oussoren wrote: That *is* trivial: use Mac/BuildScript/build-installer.py on OSX 10.5. Hmm. When I tried it (on some 2.5 release), it took me two days until it produced something. It should be much improved in 2.6 and later, please file a bug if it doesn't work for you. For what it's worth, the trunk currently takes ~25 minutes on my relatively modest Mini, when run in parallel with some buildbot stuff, and with third-party sources already downloaded. More or less evenly split among third-party packages, the interpreter itself, and then docs/framework/disk image creation. -- David ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Automatic installer builds (was Re: Fwd: Broken link to download (Mac OS X))
Martin v. Löwis mar...@v.loewis.de writes: This actually happened on Windows - some people now recommend to run the buildbot scripts on a regular developer checkout, because they supposedly do the right things. I have to admit that I'm guilty of this (though to be fair most of my test builds are buildbot-related), if only because it takes care of all the external stuff automatically and self-contained in the build tree. I would rather prefer to have the buildbot run the same commands that we recommend developers to run. The knowledge about these should be in the README, and it should be stable knowledge, i.e. require infrequent changes. This is true for Unix: configure/make/make test/make clean had been the correct procedure for ten years now. The Unix buildbot only deviates slightly, to have the slaves run a more expensive version of the test suite. In digging around a bit, it would appear that there's actually quite a bit of OSX support already in the Makefile (either the main one or the one in Mac). There's even a target that tests both halves of a universal build (using rosetta for the PPC version) on an Intel box. I suspect it's just a question of setting up a Mac-appropriate builder, using the universal SDK in the configure step, and whatever Makefile targets are deemed best and current, perhaps with the addition of support for pulling in the third party stuff through externals or whatever. A first pass could just be to factor that process into a separate stage of build-installer that could be run independently of the rest of the installer build process. In the interim, I've generated the third-party libraries via the current trunk build-installer script and installed them in /usr/local on my buildbot, so they are found by any of the buildbot builds using the stock configure/make approach. Other than a slight update to the bzip version, looks like the dependency versions in the installer script haven't changed for over a year, so I suspect the libraries are fine for any of the branches currently being built. I also updated to the latest 8.4.19 Tcl/Tk in /Library/Frameworks since I saw some interpreter crashes in tests in what appeared to be a Tcl code path. It had been building against my system 8.4.7 Tcl. The Windows buildbot uses Tcl 8.5 - not sure if that should be preferred for the Mac buildbot as well, but will leave it at 8.4 for now. I think this should create test builds more representative of the final binaries. It's not testing a universal framework build, but the test target only tests the Intel path anyway, the generated code should still be the same, and the same libraries are in use. If anyone more familiar with this side of things for OSX has some spare time down the road, I'd be happy to help work on improving the process for the buildbot. I'd be interested in setting up daily builds then. For these, I think it's fine that they may differ slightly from the official binaries - people would recognize that they are testing a different set of binaries. We can certainly start by just trying to automate the execution of build-installer, something I suspect can be completely controlled from the master side, and then uploading the resulting dmg file. I'd be happy to help coordinate any experiments offline if you're interested. I do currently have a DMG built for 2.7 Beta 1, if it would be useful. -- David ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Automatic installer builds (was Re: Fwd: Broken link to download (Mac OS X))
Ronald Oussoren ronaldousso...@mac.com writes: Speaking of which... I have a mac-mini that could be used for a buildbot. How much work is needed to kickstart a buildbot, taking into account that I'd prefer to have a buildbot with different configure-flags that the default unix build (that is, I want to test a framework build that is a universal binary). Sort of picking this message to enter the thread ... Interestingly enough, I had made an offer to Martin to host an OSX build slave last week (before this thread), since I had a Mini whose disk crashed and would be reasonably free after I rebuilt it, and I had noticed there were no longer any OSX build slaves around. It's online now, though it certainly need not be the only one. There's not much to setting up a build slave - just easy_install buildbot on top of a basic Python install (you'll need Twisted), use the build-slave command to create a tree, edit the files in info, and buildbot start tree-root to get it started. Of course, there's some overhead to monitoring things over time. My new slave is a Tiger box, since that's what I still need for my own application builds. So it won't help with building/testing x64 builds. Per the startup on reboot part of the thread, I typically use the LaunchAgents setup on my Macs for that purpose, although to be honest, not many of my build slaves start automatically anyway, as the boxes don't tend to get restarted other than manually. In the first builds, I have noticed that the build master seems to execute the builds as a Unix system, so isn't building with the universalsdk option or as a framework, though the latter would probably need a bit of glue to make the framework available just locally to the buildbot process. On Windows, the buildbot tasks use scripts that keep the third party stuff in the buildbot branch tree itself. I wonder if perhaps with a little TLC, we could come up with some Mac-specific buildbot scripts to better have an OSX build slave mimic the final build process. It might be as simple as using build-installer with a specified directory within the build tree, though utilitizing the built framework for the tests probably needs some support. The build master could then be set to execute those scripts (much as it does on Windows) rather than the stock Unix approach. I anticipate some tuning to do with respect to external libraries. Rather than use system libraries or my own (or finks or MacPorts) version of libraries, I suspect it would be better to download the same releases of the third party stuff that the installer script downloads and install them locally for normal buildbot builds, to best match what would be packaged up with a final binary. Of course, if so, then there's the question of keeping the buildbot environment up to date with the installer script. As I believe Martin mentioned elsewhere in the thread, he used to have my Windows slave generate nightly MSI builds and upload them, but the process became fragile over time, and didn't seem to be missed when we discontinued them. But something similar could probably be worked out for building nightly DMGs for OSX if it were deemed useful. Creating the Mac installer is easy: just run Mac/BuildScript/build-installer.py on an OSX 10.5 system where a local version of Tcl/Tk 8.4 is installed in /Library/Frameworks. The system should also not have fink or darwinports and a clean /usr/local tree to avoid contaminating the build. Yes, I've successfully used the script to create a DMG of the latest 2.7 out of trunk, and at least the basic sniff test (install it locally, run a few things) seems fine. Did run into one quirk where if you have already built Python within the tree you run the script from, it doesn't rebuild everything beneath /tmp/_py/_bld - such as the pgen stuff. So you want a clean source tree too. To the extent that the output (DMG) of the build-installer script is what is currently needed for OSX, I can generate one if it is still needed, though I can't promise much beyond just executing the script. The new build slave can also be made available for RMs (or whomever) to generate the DMG when needed if that might be helpful. Though there should probably be some basic installation test on other systems prior to publishing any such generated files. -- David ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Automatic installer builds (was Re: Fwd: Broken link to download (Mac OS X))
Martin v. Löwis mar...@v.loewis.de writes: Not sure what you mean by make available - I thought this is just a matter of configure options? Building as a framework, yes. But I think there's some steps to take to then have the test python binary use the locally built framework while running the tests (since the framework won't be in a system location like /Library/Frameworks during the test). Probably similar to whatever py2app does to use packaged frameworks contained within the application bundle. I doubt it's overly complex, but just an extra step that might need to be incorporated into any buildbot test rules or script to ensure the just built framework is used in the tests. Would you like to see such a build on your machine in addition, or instead of, the Unix build? I suppose an argument could be made to test both, since others have indicated here they often do a normal Unix-oriented build on OSX, but I think if I only had to pick one, I'd go with framework to match what gets published as a binary for downloading. Theoretically it should probably also be a universal build, though of course only one variant would actually get tested on a build slave depending on its platform. Certainly anything we could do to make the buildbot generated builds match as closely to possible the distribution binary process would be beneficial, I'd think. On Windows, the buildbot tasks use scripts that keep the third party stuff in the buildbot branch tree itself. I wonder if perhaps with a little TLC, we could come up with some Mac-specific buildbot scripts to better have an OSX build slave mimic the final build process. No. I'd rather create a separate builder on the master. I'm wondering if I'm saying the same thing but just not using the right vernacular. Something on the master (a builder?) for Windows instructs it to run Tools\buildbot\build.bat for building and test.bat for testing, rather than each of the individual commands. Whether we encapsulate the needed logic in separate makefile targets for the Mac, or independent build scripts like on Windows, the build master would only need to execute some Mac appropriate command on a given branch (not sure if that's separate builders), with the makefile/script in the given checkout handling the third party stuff and environmental setup. Keeping the knowledge in the makefile or script in the source tree would let the rules change across branches without affecting the build master. Though if having more specific rules in the master was easier, I'd be fine with that too. It would be possible to commit these packages into the externals repository, just as we do for Windows. It would then be possible to check them out over again all the time, or somehow to detect when the URL changes so they export a different subversion directory. Sure - that's the sort of thing I figured a build script could take care of, much as it does on Windows. Or, the current Mac build-installer script already has the necessary information, and can be instructed where to place the third party stuff, so it might only be necessary to have the buildbot process run that script with appropriate build tree-relative paths. For 2.7, I would do that only if the very same build process is also going to be used for the final release. There is no point in testing builds when then the final release uses some other process. Agreed, thus my caveat as to the output of the build-installer script in fact being what has been published for downloads. In my brief tests it looks like it created a production DMG ready for publishing on the download page, and did in fact install correctly, but I'm still new to the Mac binary building process. It would also be good if anybody who commits to producing OSX binaries now would also produce them throughout the 2.7 lifetime (which could be a bit longer than the traditional 18 months). To the extent that the installer script is in fact the definitive process, and if the machine (10.4 Intel) is considered suitable as a binary build platform, then worst case we could probably have the buildbot run the script and upload the result when needed (sort of a one-shot version of the old MSI daily builder). I don't plan on the machine going away any time soon. Of course, this is just the DMG construction. Not sure what amount of test the installer testing should be required prior to publication. -- David ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Status of 2.7b1?
Brian Curtin brian.cur...@gmail.com writes: The tests are run on a native Win32 build as compiled by VS2008. The functionality is Win32 specific and wouldn't work on Cygwin, so the tests are skipped there. I believe Cygwin is used for kicking off the tests and other buildbot stuff, but they don't actually run through Cygwin. Yes, that's correct. Cygwin is on my buildbot just for management convenience. The build slave happens to be started from a Cygwin bash shell (and beneath a Cygwin-based home directory, which is why you see it in various path references in logs), but the buildbot code itself is responsible for executing the python process directly for testing. And python itself is built normally with VS 2008 as a pure win32 build, no Cygwin dependency. I did also verify over the weekend that the failures occur whether the python_d process is started from a Cygwin bash shell or from the normal Windows cmd process, and even if test_os is used directly, without running through rt.bat. The case that ran successfully was not via test_os, but only by interactively replicating the test. I'm back at this point, and to the extent my buildbot is the only place currently replicating the problem, am working with Brian for whatever access or resources might be helpful. -- David ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] __file__
Steven D'Aprano st...@pearwood.info writes: Personally, I can't imagine ever wanting to ship a .pyc module without the .py, but since Python already gives people the opportunity to shoot themselves in the foot, meh, we're all adults here. Not sure I've seen it mentioned in this thread, but for myself, I've certainly used (indirectly) such a distribution many times when packaging applications with py2exe for installation on Windows clients. That puts all the pyc files into a single support zip file from which the application runs. That seems a perfectly useful use case, and not due to any issues with security/obfuscation. The matching interpreter is being packaged with the application, so there's no version worries with the pyc. The files are internal to a zip, so why complicate things with recompiling and writing locally on the user's machine, particularly when on newer versions of Windows the installation directory might not be writable anyway. As long as executing from pyc files continues to work, presumably py2exe can be updated to collect those files from any new cache location during the build process. But I do think it's useful to continue to support executing them directly outside of any new cache location, which it sounds like is the direction being taken. -- David ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] /trunk test_distutils failing on Mac OS X 10.5
Mark Dickinson dicki...@gmail.com writes: I'm not seeing the test_distutils failure you report on my own 10.5 machine, for some reason. I don't think it's OSX specific. My FreeBSD slaves (both 6.4 and 7.2) have been getting the same error. It seems related to the sticky bit on /tmp and the fact that the account running the build slave is in the wheel(0) group, so directories/files created beneath /tmp get a group of 0 rather than the user's gid. That affects the temporary files in the distutils test, which is expecting a default of os.getgid(). -- David ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Building a Windows MSI for Python /trunk
Martin v. Löwis mar...@v.loewis.de writes: It was too difficult to get right, plus nobody was interested in using the daily MSI files. At some point, it was broken for several months (IIRC), with nobody reporting that breakage. So when we noticed, we just turned the service off. Sounds right - the daily MSIs were built on my slave from Sep 2007 into 2008 somewhere, but it had been failing for 6 months or more when the task was turned off last December. I'm not sure precisely when the failures started (the buildbot status page was also down for a while starting in March), but it had been failing for a long time. I believe the failures at the time were tcl related - maybe some of the MSI buildbot-related scripts getting out of date with ongoing development or something. Sounds like it might be the same stuff Titus is running into. It didn't seem worth the effort to track down at the time, given that nobody had seemed to notice the lack of builds. -- David ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Buildslave gets intermittent errors in the svn step
Paul Moore p.f.mo...@gmail.com writes: buildbot.interfaces.BuildSlaveTooOldError: This buildslave (moore-windows) does not know about multiple branches, and using mode=update would probably build the wrong tree. Refusing to build. Please upgrade the buildslave to buildbot-0.7.0 or newer. This is very odd, as (a) a simple rerun fixes the issue, and (b) I'm already on buildbot 0.7.0 or newer (can't recall the exact version). It's a Windows XP system, if that helps. Does anyone have any suggestions? In my experience this error just means that someone tried to manually trigger a build but put the wrong branch information (or left it blank) on the submission form. -- David ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Buildslave gets intermittent errors in the svn step
Paul Moore p.f.mo...@gmail.com writes: Hmm, wasn't me :-) The reason field on the build is blank, so this looks like it's an automatic build. If it's got an SVN tag indicated, and you can checkout that tag on your slave, then it's probably not the scenario that I was thinking of. In my case, it's almost always me trying to initiate a test build, and forgetting the right branch tag or what its relative to (historically forgetting to put branches/ in it). I'm getting odd failures with some of the tests as well, at the moment - test_smtplib and others failing but when Re-running failed tests in verbose mode there were no failures. I wonder if it could be a flaky network connection...? I suppose. Certainly, for this specific error, it's pretty clear that it's not that precise, and various SVN checkout problems can probably be trapped by the error, so it's pretty unlikely it's actually a buildbot version error. What you might try is locating the buildbot log entries (twistd.log) on your slave that correspond to the failed build. It'll show the exact svn command being issued, and you can try it manually in a test tree. If it succeeds then you'll know it was some sort of transient issue - if it fails, it ought to provide useful data as well. PS Is there a better list for this type of question? I'm not sure how interesting it is to general python-dev readers. I don't think there's anywhere separate, and generally (recent history being a bit different) buildbot related stuff has been very low volume. So this list is probably appropriate (given what the buildbots are being used for). I expect if it gets too noisy we'll hear about it :-) -- David ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Add an optional timeout to lock operations
Antoine Pitrou solip...@pitrou.net writes: I've submitted a patch (*) to add an optional timeout to locking operations (Lock.acquire() etc.). Since it's a pretty basic functionality, I would like to know if there was any good reason for not doing it. I always assumed it was because as a least-common-denominator set of functionality, some platforms didn't have the necessary support. Providing the discussion on this ends up with the an implementation being accepted, I'd absolutely love to see this then leveraged by threading.Condition.wait() rather rather than the current poll-with-timed-sleep approach. The overhead (CPU, but most notably latency) of that approach (which also directly impacts Queues) has historically been my top reason in various applications on Windows to have to implement my own Queue or Condition structures using native Windows calls. -- David ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] External module tests skipping on Windows build slaves?
While working with the Windows (and now Windows 7) build slaves I've been noticing that a number of unexpected test skips are occurring, mostly for external modules like ssl, bz2, tk, etc... However, I know for a fact that those libraries are being fetched and built, so I was a little perplexed that they were being skipped for testing. I don't think it's a regression due to the recent slave work, but something that may have been happening for a while, if not always. Some manual testing I did would seem to indicate that there's a problem related to loading the right MS CRT when loading these modules. But they seem to reference both debug and non-debug versions, so even if I copy the CRT DLLs (from the SxS tre) into pcbuild manually, then I get a CRT assertion about a bad initialization. I suspect this wouldn't be a problem when everything is built in release mode. I would have sworn they used to get run, but now I'm not so sure. Perhaps I'm remembering older Python releases with VS.NET 2003, since the MSVC9 versions of the CRT and the SXS stuff was new with VS 2008 I think. Does anyone happen to know if these tests have ever run recently on the Windows build slaves? -- David ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] External module tests skipping on Windows build slaves?
David Bolen db3l@gmail.com writes: (...) I would have sworn they used to get run, but now I'm not so sure. Perhaps I'm remembering older Python releases with VS.NET 2003, since the MSVC9 versions of the CRT and the SXS stuff was new with VS 2008 I think. Does anyone happen to know if these tests have ever run recently on the Windows build slaves? Just to follow up - after finding an old 2.6a0 tree I had manually built that worked fine (so clearly it wasn't just VS 2008), I worked my way through some 2.6 tags and this broke as of 2.6.1, so I think the tests have been getting skipped since then. The root problem appears to be the change (r67121 for 2.6 merged from r67120 in trunk) that removed embedded manifests. I'll follow up on the issue tracker. -- David ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Status of the Buildbot fleet and related bugs
R. David Murray rdmur...@bitdance.com writes: The buildbot pages appear to be pretty messed up now. I get many 404s (ex: the above url, the all stable builders page), although some seem to work (ex: the all builders page), and if I stick an 'all' into the URL for my buildbot page I can get to it, though that's is not the version of the URL linked from the 'all builders' table header. Yes, I think this just started happening. I'm guessing that the main site proxies the buildbot URL requests to the buildbot master process, and when it's down you get the 404s from the main server. I figured someone might be working on the master, though perhaps it just burped on its own :-) -- David ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Updates to XP-4 and FreeBSD build slaves
I think in a prior discussion, it was suggested that build slave updates were ok for this list - I apologize to those who may not be interested. I've just completed some updates to my two build slaves. XP-4 is now running XP Pro SP3 (was SP2) and is using the full version of VS 2008 (was previously using Express). FreeBSD is now running FreeBSD 6.4-RELEASE (was 6.2-RELEASE). Both are new VMs, and also now use the latest buildbot (0.7.11p3) release. For at least the next few days they are being run on a different host while I do some long-awaited maintenance to the old host. I've done manual builds on both, as well as reissuing a few builds through the master, but if anyone notices anything strange please feel free to drop me a note. -- David PS: I did discover one interesting bug by starting fresh on Windows. It turns out that one of the distutils tests assumes Notepad entries in the registry that aren't actually present unless you run Notepad at least once. I hadn't needed to run it, but then noticed the build failures so ran it just to get the registry populated. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Possible language summit topic: buildbots
Martin v. Löwis mar...@v.loewis.de writes: The remaining issue is the popups; if a process still has a popup, you can't even terminate it properly. There are two kinds of popups: system-generated ones, and CRT-generated ones. For the CRT ones, we once had a way to turn them off, but I'm not sure whether that mechanism might have been removed. For the system messages, there is a way to turn them off in the parent process. David Bolen (IIRC) had developed a patch, but I think this patch only runs on his system(s). Yes, process-stopping dialogs have probably been the single most annoying issue over time running a Windows build slave - certainly from my perspective in terms of maintenance and detection they have taken up the largest amount of time. I believe the CRT disabling is still active in the 3.x branches (the -n flag to regrtest in the buildbot's test.bat), after you restored it this past March (it had regressed during an earlier py3k branch set of patches and caused a bunch of problems for a bit) but not in the 2.x branches or trunk. So there's still a bit of exposure there and I'd certainly be in favor of porting the regrtest -n support over to all current development branches. I think the other issue most likely to cause a perceived downtime with the Windows build slave that I've had a handful of cases over the past two years where the build slave appears to be operating properly, but the master seems to just queue up jobs as if it were down. The slave still shows an established TCP link to the master, so I generally only catch this when I happen to peek at the status web page, or catch a remark here on python-dev, so that can reduce availability. My build slave (based on 0.7.5 I think) runs with local patches to: 1. Protect again Win32 pop-up error boxes in child processes. 2. A fix for a recursive chaining of Twisted Deferreds during uploads which could break for large file transfers. This only came up when my build slave was generating daily MSI builds and uploading them to the master. 3. Handle clock jumps. It's a general flaw in the presence of system clock adjustments, but I only encountered it while in my FreeBSD build slave under VMWare with a Linux host. (2) and (3) are useful, but not likely to be an issue with most build slaves in normal operation. (2) probably isn't needed on my end any more now that the daily MSI builds aren't run, and it's possible that it got corrected in later buildbot updates, since I did report it on the development list at the time. (1) is a pretty trivial patch, but added a dependency on pywin32, so passing it back up to the buildbot maintainers (back in 2007) stalled while determining if that was ok, and I don't I ever closed the loop. I did later make it fall back to ctypes if pywin32 was missing, but then I think buildbot was using a minimum of Python 2.3 at the time, so even ctypes was a new dependency. Anyway, it became less crucial when Python's regrtest started executing similar code, though the buildbot patch covers anything run under it and not just the python process. I'd of course be happy to pass along the patch to anyone interested. I believe that Thomas Heller had run his Windows buildbot with some similar local code, but implemented with a modified buildbot script for building Python, rather than tweaking buildbot itself. Of course, the patch only protects against system pop-ups - it can't control the CRT assertion dialogs when Python is built in debug mode, which is why I've argued in the past that the test process for Python should ensure those are disabled. The CRT errors themselves are still important, but can be redirected to stderr rather than a blocking GUI dialog. -- David ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Possible language summit topic: buildbots
exar...@twistedmatrix.com writes: This sounds like something that should be reported upstream. Particularly if you know how to reproduce it. Has it been? No, largely because I can't reproduce it at all. It's happened maybe 4-5 times in the past 2 years or so. All that I see is that my end looks good yet the master end seems not to be dispatching jobs (it never shows an explicit disconnect for my slave though). My best guess is that something disrupted the TCP connection, and that the slave isn't doing anything that would let it know its connection was dropped. Although I thought there were periodic pings even from the slave side. Given the frequency, it's not quite high priority to me, though having the master let the owner of a slave know when it's down would help cut down on lost availability due to this case, so I suppose I could suggest that feature to the buildbot developers. -- David ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Possible language summit topic: buildbots
MRAB pyt...@mrabarnett.plus.com writes: Couldn't you write a script to check the status periodically? Sure, I suppose scraping the web status page would work. If it happened frequently I'd probably be forced to do something like that, but it's relatively low frequency (though I guess it does have a big impact in terms of availability) makes it hard to dedicate time to that compared to my real world work :-) -- David ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] thinking about 2.7
exar...@twistedmatrix.com writes: It's easy for someone to volunteer to set up a new slave. It's even easy to make sure it keeps running for 6 months. But it's not as easy to keep it running indefinitely. This isn't about the software involved (at least not entirely). It's about someone paying attention to whether the slave restarts on reboots, and about paying attention to whether the slave host has lots its network connection, or been decomissioned, or whether a system upgrade disabled the slave, or whatever other random administrative-like tasks are necessary to keep things running. Casual volunteers generally just won't keep up with these tasks. One suggestion from my perspective as a buildbot maintainer who, IMHO, has maintained generally stable and fairly well monitored buildbots (XP-4 and FreeBSD) for a while now I don't know if it's possible but it would be great if the buildbot master could send an email to a buildbot maintainer if it thinks a buildbot is down. There have actually been times in the past when from the perspective of my buildbot things were just dandy, but the master showed it as down. Stopping and restarting the buildbot, forcing it to reconnect, cleaned things up, but I have to monitor the web status page which is a manual process. And since it's the master's perspective that really matters when it comes to a buildbot being available, getting status from the master would be very helpful. -- David ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] VC++ versions to match python versions?
Chris Withers ch...@simplistix.co.uk writes: Is the Express Edition of Visual C++ 2008 suitable for compiling packages for Python 2.6 on Windows? (And Python 2.6 itself for that matter...) Yes - it's currently being used on my buildbot, for example, to build Python itself. Works for 2.6 and later. Ditto for 2.5, 3.1 and the trunk (which I guess becomes 3.2?) 2.5 needs VS 2003. -- David ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Microsoft MSDN
Christian Heimes li...@cheimes.de writes: Steve Holden wrote: I sent fourteen requests for licenses in to Microsoft. I've asked them to let me know which they grant (since they may choose to limit the number) and will inform you all personally when I hear their decision. I've received my MSDN subscription today. Everybody watch out for a message from MSDN! I almost confused the email with spam. Thanks for your work and please forward my gratitude to James Rice. Ditto from me (my subscription info arrived yesterday afternoon). Many thanks to all involved! -- David ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] py3k buildbots
Antoine Pitrou solip...@pitrou.net writes: Only one of the py3k buildbots seems up: http://www.python.org/dev/buildbot/3.x.stable/ Strange - everything looks good on my buildbot end (XP-4), including an established TCP session back to dinsdale. Not sure why the master thinks it's offline. Although I'm pretty sure I've seen this a handful of times in the past too. The fact that it also shows my FreeBSD buildbot offline (which I rarely have to touch compared to the Windows version and they're on the same host machine) seems to imply some wierd communication breakdown. I've restarted my buildbots. -- David ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Google Wave as a developer communication tool
Aahz a...@pythoncraft.com writes: It sounds like Wave requires a high-powered browser, similar to Google Maps. That makes me -1 because I want to continue using Lynx. I'm not sure - I think you can implement your own choices at different points. What's interesting to me so far is less the current UI/flashiness (drag 'n drop photos, real-time keystrokes, etc...) they are showing, than the openness of the protocol and implementation, opening the door to all sorts of potential uses and/or integrations with existing systems. Other than convenience for interactive use, and as a reference client, I'm not sure how critical a browser really is to the system. Fundamentally, wave seems to be rich, versioned (or at least time tracked), content management system with well defined server to server and server to client communication protocols, including conflict resolution. They even brought up VCS-like concepts (currently unimplemented) of having waves branch and evolve independently, fold changes back into a parent wave and then re-distribute down to the other children over time, which points to a reasonable central data structure. For example, if you jump to about 1:08:13 in the presentation you can see a more textual display implemented by an independent wave server/client - almost newsreader like. I presume such a client has to make similar decisions as to display of non-textual data as Lynx has to with the web. Or if you embed the wave in a web page, it could still be handled by text browsers (the blog integration at 20:43 for example) or perhaps with RSS. Server robots could help too - a robot participating in the wave could produce a more text-friendly reader version, perhaps published elsewhere on the web or in a different system. They demoed, for example, integrating with Google Code's bug tracking back end at 1:02:30 where comments in the wave were reflected into the bug comments (though the reverse direction wasn't implemented yet), or even the twitter integration at 57:53. -- David ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [buildbot] some build slaves in bad shape
Tarek Ziadé ziade.ta...@gmail.com writes: - x86 XP-4 (trunk and 3x) is throwing an no space left on device error when it compiles the sqlite module in its temp dir Ooops, that's mine. Geez - it's a VM, but has a 10GB C: drive, and the actual build slave has its working directory on a separate virtual drive. Wonder what the heck has filled up the system drive. I'm working on it now though. -- David ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [buildbot] some build slaves in bad shape
David Bolen db3l@gmail.com writes: Ooops, that's mine. Geez - it's a VM, but has a 10GB C: drive, and the actual build slave has its working directory on a separate virtual drive. Wonder what the heck has filled up the system drive. I'm working on it now though. Well, looks like it was 5+GB of temporary files of some sort. It's cleaned up now and back online. -- David ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Test failures under Windows?
Kristján Valur Jónsson krist...@ccpgames.com writes: Btw, I am working on finding out the test suite failures for test_multiprocessing. This is all well and good, but I still haven't seen any plausible reason for not preventing these popups (in favor of stderr failures) during buildbot test runs? I don't get it - it just seems a no brainer to help protect the Windows buildbots. Surely there may be similar issues down the road that also introduce the potential for assertions? I guess I'll stop asking after this note, but can anyone give a final verdict on whether the older -n option can be restored to the buildbot test.bat (from the revision history I'm not actually sure it was intentionally removed in the first place)? Unlike normal Windows popups I can't protect against it easily through local modifications to my build slave, since test.bat gets updated as part of the source tree. -- David ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Test failures under Windows?
Martin v. Löwis mar...@v.loewis.de writes: Notice, however, that the feature was never present in the trunk. Yep - would be nice if it were to get backported to trunk at some point but that's a separate discussion ... presumably at some point py3k will be the trunk anyway, and for better or worst (perhaps due to the sorts of changes made) the assertions seem to have hit the py3k branch more than others. Thanks for the test.bat change. -- David ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Test failures under Windows?
Kristján Valur Jónsson krist...@ccpgames.com writes: But again, it shows how useful assertions can be and why we ought not to disable them. Note that just to be clear, I'm certainly not advocating the disabling of CRT assertions - just the redirection of them so they don't prevent unattended test runs from completing. -- David ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Test failures under Windows?
Hirokazu Yamamoto ocean-c...@m2.ccsnet.ne.jp writes: CRT Assertion was totally disabled before, but recently was enabled, and workarounds were patched for problematic functions. (ex: fdopen and dup) Probably this *patch* is not perfect. See http://bugs.python.org/issue4804 Ah - that ticket may explain why my buildbot hadn't been having problems until recently, even though the -n option in test.bat was removed quite a while back. I'm now +3/4 for the idea disabling assertion by default, and enabling by startup option or environment variable. (Or enabling by default and disabling by environment variable?) This is a bit more than I'm looking to have changed. I'm only discussing disabling the GUI assertions during buildbot test runs. It's fine if there are other circumstances when one might also want a mechanism to disable them, but I'd rather not intermingle such cases since there may be different pros and cons to different cases. The nature of the development process is that you're going to have issues at times which is why you're testing, and given the consequences of a pop-up box on an unattended build slave, it just seems very strange to me that there could be any good reason to leave a potential for such popups (which at this point is easy to disable) in the buildbot environment. -- David ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Test failures under Windows?
David Bolen db3l@gmail.com writes: From what I can see though, the tools/buildbot/test.bat file no longer adds the -n option that it used to, although I'm unclear on why it might have been removed. Perhaps this was just a regression that was accidentally missed, as it appears to have disappeared during a large merger from trunk to the py3k branch mid-2008 (r64273) when the batch file line ending was changed to CRLF. It would be nice to also have this in the other active release branches. This thread sort of died out ... would there be any objections to restoring the -n option in the buildbot test.bat file for Windows buildbots? I just went through clearing a ton of popups again on my build slave, but in the end had to again manually kill off all the individual python_d processes, as the popups just seemed to keep getting created anew. I don't know why they are happening so frequently now when there was a reasonable period when they weren't an issue (something about new I/O support in 3.x perhaps?), but without preventing them it seems the Windows build slaves are going to become (if not already) quite a bit less useful. Don't know about anyone else's but I can't watch mine 7x24. -- David ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Test failures under Windows?
Mark Hammond skippy.hamm...@gmail.com writes: The issue was that Python unconditionally changed the behaviour of the CRT, not only during the test suite. Hmm... I was more or less referring to the state of the py3k tree as of, say, r57823 back in 2007. It appeared to just add access to the necessary functions in the msvcrt module, and a -n/--nowindow option to regrtest.py which then used those functions to disable stuff before running the test, and finally added -n to the tools/buildbot/test.bat file, so things got disabled in buildbot test runs. (By disabled, I believe that all of the CRT warning/error/assert went to stderr) So nothing permanently built into Python, nor enabled outside of a buildbot regression test. I agree permanently disabling them wouldn't make sense, but I thought we also agreed to that back in 2007? -- David PS: The dialogs are showing up again on my build slave. I tried clearing it but then like 6+ of them showed up and kept showing up even though I kept aborting them (I guess the tests kept running new python_d instances or something). I finally managed to catch up and then whack the python_d processes manually through task manager. Really seems like disabling them can only make things more robust, otherwise the Windows build slaves are just going to keep ending up unresponsive. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Test failures under Windows?
Curt Hagenlocher c...@hagenlocher.org writes: The variation that goes through assert.c should write to stderr for a console-mode application, so it's reasonable to assume that we're hitting the other code path -- and that Mark's suggestion to use CrtSetReportMode would address the issue. Which is pretty much what the regrtest.py -n option does/did in the py3k branch (and release30-maint). It enables the following code: import msvcrt msvcrt.SetErrorMode(msvcrt.SEM_FAILCRITICALERRORS| msvcrt.SEM_NOALIGNMENTFAULTEXCEPT| msvcrt.SEM_NOGPFAULTERRORBOX| msvcrt.SEM_NOOPENFILEERRORBOX) try: msvcrt.CrtSetReportMode except AttributeError: # release build pass else: for m in [msvcrt.CRT_WARN, msvcrt.CRT_ERROR, msvcrt.CRT_ASSERT]: msvcrt.CrtSetReportMode(m, msvcrt.CRTDBG_MODE_FILE) msvcrt.CrtSetReportFile(m, msvcrt.CRTDBG_FILE_STDERR) which seems like it would still be the right sort of code to use. For my buildbot the SetErrorMode is redundant because I already do that in the buildbot code itself (and the setting is inherited) but it won't hurt. From what I can see though, the tools/buildbot/test.bat file no longer adds the -n option that it used to, although I'm unclear on why it might have been removed. Perhaps this was just a regression that was accidentally missed, as it appears to have disappeared during a large merger from trunk to the py3k branch mid-2008 (r64273) when the batch file line ending was changed to CRLF. It would be nice to also have this in the other active release branches. -- David ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Test failures under Windows?
Antoine Pitrou solip...@pitrou.net writes: Am I the only one getting those? By the way, what happened to the Windows buildbots? Oops - sorry, mine (XP-4) apparently got stuck with C++ debug assertion dialogs (an assertion from the internal close.c module) from python_d during some earlier build. I try to catch these but have been a bit preoccupied this month. Should be back now. -- David ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
