[issue44078] Output relative path when using PurePath.relative_to
New submission from Mark Hammond : Comparing two paths, PurePath.relative_to fails with ValueError if the second path is not in the first. >>> Path('/a/b').relative_to('/a/b/c') Traceback (most recent call last): File "", line 1, in File "/usr/lib/python3.9/pathlib.py", line 928, in relative_to raise ValueError("{!r} is not in the subpath of {!r}" ValueError: '/a/b' is not in the subpath of '/a/b/c' OR one path is relative and the other is absolute. Please extend PurePath.relative_to so it is able to output a relative path in form of '..' instead of an error. Currently, the only way to do this is to use relpath from os.path but it would be very useful if Path was able to output the relative path. -- components: Library (Lib) messages: 393262 nosy: mhammondr priority: normal severity: normal status: open title: Output relative path when using PurePath.relative_to type: enhancement versions: Python 3.9 ___ Python tracker <https://bugs.python.org/issue44078> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue41771] bdist_wininst doesn't execute postinstall script
Change by Mark Hammond : -- keywords: +patch pull_requests: +21265 stage: -> patch review pull_request: https://github.com/python/cpython/pull/22210 ___ Python tracker <https://bugs.python.org/issue41771> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue41771] bdist_wininst doesn't execute postinstall script
New submission from Mark Hammond : install.c tries to dynamically load PyCFunction_New at https://github.com/python/cpython/blob/fb2718720346c8c7a0ad2d7477f20e9a5524ea0c/PC/bdist_wininst/install.c#L686 - however, since 3.8 this no longer exists - PyCFunction_New is a #define which calls PyCFunction_NewEx -- assignee: mhammond messages: 376771 nosy: mhammond priority: normal severity: normal status: open title: bdist_wininst doesn't execute postinstall script type: behavior versions: Python 3.10, Python 3.8, Python 3.9 ___ Python tracker <https://bugs.python.org/issue41771> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue41526] Python 3.9.0rc1 "setup successful" dialog box overflow
Mark Hammond added the comment: Yes, while I appreciate the gesture, I am somewhat embarrassed these days - as you say, many others deserve this more than me these days. So please remove it. -- ___ Python tracker <https://bugs.python.org/issue41526> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40740] Missing api-ms-win-core-path-l1-1.0.dll for python-3.9.0b1-amd64.exe Under Win7
Mark Hammond added the comment: Can we get this mentioned somewhere? Eg, https://www.python.org/downloads/release/python-390b3/ doesn't mention anything about minimum versions. https://www.python.org/downloads/windows/ also has prominent notices like "Note that Python 3.8.3rc1 cannot be used on Windows XP or earlier." but nothing for 3.9 (This threw me for some time - I still have a win7 VM I used for testing and figured there was something broken in the VM before finding this) -- nosy: +mhammond ___ Python tracker <https://bugs.python.org/issue40740> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39631] Fix file association MIME type on Windows
Change by Mark Hammond : -- nosy: +mhammond ___ Python tracker <https://bugs.python.org/issue39631> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue33944] Deprecate and remove pth files
Mark Hammond added the comment: pywin32, up until recently, just listed 3 directories in its .pth file - these were for directories which pre-dated packages and were never converted. Eg, "import win32api" actually loads win32api.pyd from the "site-packages/win32" directory. Earlier this year, via https://github.com/mhammond/pywin32/issues/1151, I also added the line: import os;os.environ["PATH"]+=(';'+os.path.join(sitedir,"pywin32_system32")) which is to support pywin32 being installed from wheels - this is due to pywin32 shipping with various shared DLLs which implement many pywin32 types - eg, pywintypesXX.dll is used by (almost) every single .pyd shipped with pywin32, and disutils doesn't offer any way of copying files as part of a post-install script or any other way of ensuring these .dll files are on the PATH or otherwise next to pythonXX.dll/.exe I'm happy to replace both of these with alternatives when they exist. -- ___ Python tracker <https://bugs.python.org/issue33944> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue27966] PEP-397 documents incorrect registry path
New submission from Mark Hammond: Received via email: PEP-397 (PEP 397 -- Python launcher for Windows) says: """ The launcher installation is registered in HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\CurrentVersion\SharedDLLs with a reference counter. """ There is no such entry. It should be HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\SharedDLLs -- assignee: docs@python components: Documentation messages: 274525 nosy: docs@python, mhammond priority: normal severity: normal status: open title: PEP-397 documents incorrect registry path ___ Python tracker <http://bugs.python.org/issue27966> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue27410] DLL hijacking vulnerability in Python 3.5.2 installer
Mark Hammond added the comment: While I agree the risk is fairly low and it will require effort to actually do, it still sounds worth fixing at some point. A user might be tricked into downloading a DLL - eg, Firefox will happily save it without any scary UI - it's just a file. Later they run our "trusted" download from the same directory and we screw them - even if the attacker can't elevate they can do damage. -- nosy: +mhammond ___ Python tracker <http://bugs.python.org/issue27410> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue27417] Call CoInitializeEx on startup
Mark Hammond added the comment: > > This may well break things like pythonwin until they also grow support > > for the new param > I expect that, which is why I'm only proposing it for 3.6 onwards. While > adding support for a new major version of Python should be fairly cheap, > it isn't entirely free and so it's the right time to add new complications. My point with that is that pythonwin is a GUI app rarely started by the command-line. It isn't that adding the cmdline support is difficult, more that it's difficult for users to specify it. This will be true for any GUI installed into the start menu (eg, idle) > Nikita: > > COM should be initialized on demand by C modules that need it. They might > > need STA or > > MTA, it's their choice to make. > And if the C module is the core interpreter (see issue26137 for an example of > where > this would be necessary)? If we require user code to initialize COM, this > whole > proposal is moot as that is the current state of the world and it does not > require > any changes to achieve. I'd be surprised if issue26137 ended up unconditionally doing a malware scan on everything Python ever executes. Thus, I don't see why "I'd like to enable calling CoInitializeEx on Python startup for 3.6" is necessary - just attempting to initialize it immediately before that feature is invoked would be fine and may sidestep the entire issue. Instead of command-line flags to control COM initialization we should add new flags to disable these new features that require COM (and thus also implicitly control whether COM is initialized or not.) IOW, I think it makes sense for the core to initialize COM immediately before it needs to use COM, under the assumption that executing "python" or "python myscript.py" isn't going to need to do that by default. I think initializing COM by default at process startup on the off-chance that some COM-using feature will be invoked is more problematic. -- ___ Python tracker <http://bugs.python.org/issue27417> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue27417] Call CoInitializeEx on startup
Mark Hammond added the comment: I've a few reservations here: * CoInitialize will load a number of COM DLLs into the process, which isn't free and will have some memory and performance costs for programs that don't use COM. I see around 10 such DLLs loaded. * pythoncom uses sys.coinit_flags because some COM objects simply don't work with the wrong apartment model. IOW, it is the objects you want to use on a particular thread that dictates the model you should use for that thread. Thus scripts written to interact with a particular COM object could set this flag before importing pythoncom so the correct threading model is setup. If this is done at Python startup, the script has lost the chance to influence this - insisting that Python be run with a particular set of flags for the script to work sounds painful. * pythoncom defaults to COINIT_APARTMENTTHREADED as the apartment threading model is a special snowflake - if you need to use apartment model objects, the main thread *must* be apartment threading (even though other threads can use free threading.) COM objects with a UI (eg, MSOffice, IE) typically required apartment threading. Most new objects probably allow free threading, but I think we want to be careful about defaulting to a model that might avoid common objects from being used without an obscure command-line param. * This may well break things like pythonwin until they also grow support for the new param - but new params for GUI applications somewhat suck as people tend to start them from an icon instead of the command-line. * Each thread that wants to use COM must also make this call. If *any* object you want to use uses apartment model threading, then you really have no choice other than to init the main thread with this model - but you are then free to spin up other threads using free-threading, and life is good. So to make this sane, you really want to expose CoInitialize to Python code so new threads can do the right thing. So in this case, I wonder why we don't just expose it and let it be called manually? IOW, it seems the potential risks of this outweighs the cost of requiring it to be called manually in a controlled way. -- ___ Python tracker <http://bugs.python.org/issue27417> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue26071] bdist_wininst created binaries fail to start and find 32bit Pythons
Mark Hammond added the comment: welp, I hadn't actually tested the patch on x64 yet ;) > #ifdef WIN_64 should read: > #ifdef _WIN64 WIN_64 is a Python invented name and this stub doesn't Python pull the headers in... -- ___ Python tracker <http://bugs.python.org/issue26071> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue26071] bdist_wininst created binaries fail to start and find 32bit Pythons
Mark Hammond added the comment: > > The directory of the DLL can be added to the search path if you > > use LoadLibraryEx with the flag LOAD_WITH_ALTERED_SEARCH_PATH and use an > > absolute path. > That sounds like the better fix then. I thought there was something along > those lines available. That does work. I didn't do it that way initially as I felt your earlier comment: > Setting the working directory is likely to lead to a more reliable initial > state. Was an additional nice outcome of just changing the cwd. This patch does the LoadLibraryEx() and works (but obviously leaves the CWD being the downloaded directory, probably %temp%) I'll check this in once I get the green-light, thanks. -- Added file: http://bugs.python.org/file41575/bdist.patch ___ Python tracker <http://bugs.python.org/issue26071> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue26071] bdist_wininst created binaries fail to start and find 32bit Pythons
Mark Hammond added the comment: > Mark said the "built binary links against the DLL version of the CRT > but I just checked in 3.5.1 that you actually didn't switch that > one to link against vcruntime140.dll like you did with everything else Yeah, I didn't dig too much further here, but I'm fairly sure that "static crt" has been a thing for quite a while now for that stub (there are comments about that around the SetStdHandle calls in that file). I suspect it was accidentally lost in one of the build system fixups that happened (which seem great and a vast improvement!) -- ___ Python tracker <http://bugs.python.org/issue26071> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue26070] Launcher fails to find in-place built binaries from earlier Python versions
Mark Hammond added the comment: > When is the launcher ever going to find Python built in-place? On the machine that built Python in-place :) I have a dev environment where all Python versions and pywin32 are built and that's the environment where py.exe is failing. My build scripts use py.exe to build for a specific Python version. It sounds like you are fine with it, so I'll try and work out how to get a new key up and push it unless I hear otherwise, thanks. -- ___ Python tracker <http://bugs.python.org/issue26070> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue26071] bdist_wininst created binaries fail to start and find 32bit Pythons
New submission from Mark Hammond: Binaries created by bdist_wininst fail on 3.5+ for 2 reasons: * The built binary links against the DLL version of the CRT. This means the binary will typically fail to start as the CRT DLL isn't found. * When looking for 32bit Python versions, it fails to take the recent "-32" change for the registry key, so it tells you no Python is installed. This patch: * Uses the static CRT, which IIRC, was the case in previous Python versions. * Changes directory to the directory with pythonxx.dll before attempting to loadlibrary it - this allows the loadlibrary to succeed as the CRT DLL in that directory is found. * Appends "-32" to registry keys on 32bit builds. With these patches I can successfully get a pywin32 build working and installing. Steve (or anyone), what do you think? (Note that the attached patch does *not* include a newly built wininst-14.exe, but that would obviously need to be updated before checking in.) -- components: Windows files: bdist.patch keywords: patch messages: 257879 nosy: mhammond, paul.moore, steve.dower, tim.golden, zach.ware priority: normal severity: normal status: open title: bdist_wininst created binaries fail to start and find 32bit Pythons type: behavior versions: Python 3.5, Python 3.6 Added file: http://bugs.python.org/file41561/bdist.patch ___ Python tracker <http://bugs.python.org/issue26071> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue26070] Launcher fails to find in-place built binaries from earlier Python versions
New submission from Mark Hammond: The launcher was recently updated to look in PCBuild/win32 to support the win32 binaries being built in this directory instead of the top-level PCBuild directory. However, when this new launcher tries to find a binary for, say, Python 2.7, it doesn't find an executable because it's directly in PCBuild, not one of the win32 or amd64 sub-directories. Note this only impacts Python when it is built in the directory - it doesn't impact installed versions. The fix I came up with is to continue looking in PCBuild if it isn't found in one of those dirs. Vinay, what do you think? -- components: Windows files: launcher.patch keywords: patch messages: 257878 nosy: mhammond, paul.moore, steve.dower, tim.golden, vinay.sajip, zach.ware priority: normal severity: normal status: open title: Launcher fails to find in-place built binaries from earlier Python versions type: behavior versions: Python 3.5, Python 3.6 Added file: http://bugs.python.org/file41560/launcher.patch ___ Python tracker <http://bugs.python.org/issue26070> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue25921] project files for wininst-14.0*.exe don't exist
Mark Hammond added the comment: Awesome, thanks, sorry I missed that. -- ___ Python tracker <http://bugs.python.org/issue25921> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue25921] project files for wininst-14.0*.exe don't exist
New submission from Mark Hammond: Revision 34b35aa1967d pushed new versions of wininst-14.8*.exe, but didn't include the VS project files required to build it, thus meaning it can't reasonably be built from the source tree. -- components: Build messages: 256823 nosy: mhammond, steve.dower priority: normal severity: normal status: open title: project files for wininst-14.0*.exe don't exist ___ Python tracker <http://bugs.python.org/issue25921> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue25148] Windows registry PythonCore key changed inconsistent with other releases
Mark Hammond added the comment: It appears bdist_wininst wasn't updated for this so installed created by distutils don't work in 32bit versions - is there a different bug open for that? (Or I'm doing something wrong, but the installer fails and a quick scan of the source doesn't seem to handle it) -- ___ Python tracker <http://bugs.python.org/issue25148> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue25148] Windows registry PythonCore key changed inconsistent with other releases
Mark Hammond added the comment: > and the launcher was actually updated to match the registry key > directly, rather than special-casing the "-32" suffix It appears this has broken the launcher. A freshly-built py.exe from Python 3.5 doesn't seem to find older 32bit versions, while py.exe from older versions doesn't find the 32bit 3.5. Doesn't this break the entire reason for having the launcher in the first place? -- ___ Python tracker <http://bugs.python.org/issue25148> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue25148] Windows registry PythonCore key changed inconsistent with other releases
Mark Hammond added the comment: > The original naming ("3.5") can't be used because you can't > simultaneously install 32-bit and 64-bit per-user Pythons with the same > key name. They will be in different nodes - the 32bit version would be under Wow6432Node. The py.exe launcher had no issues locating 64 and 32 bit versions, so I don't understand any of the justifications here. I believe this change is most certainly a bug and will break many tools that looked in the registry for Python using the sane approach used previously. -- nosy: +mhammond ___ Python tracker <http://bugs.python.org/issue25148> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1602] windows console doesn't print or input Unicode
Mark Hammond added the comment: > File redirection has nothing to do with win-unicode-console Thank you, that comment is spot on - there are multiple issues being conflated here. This bug is purely about the tty/console behaviour. -- ___ Python tracker <http://bugs.python.org/issue1602> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1602] windows console doesn't print or input Unicode
Mark Hammond added the comment: > The crash you see is maybe not a crash at all. I'd call it a "crash" - the repl shouldn't exit. But it's not necessarily part of *this* bug. -- ___ Python tracker <http://bugs.python.org/issue1602> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue21354] PyCFunction_New no longer exposed by python DLL breaking bdist_wininst installers
New submission from Mark Hammond: Python 3.3 and earlier have in methodobject.c: /* PyCFunction_New() is now just a macro that calls PyCFunction_NewEx(), but it's part of the API so we need to keep a function around that existing C extensions can call. */ #undef PyCFunction_New PyAPI_FUNC(PyObject *) PyCFunction_New(PyMethodDef *, PyObject *); which means PyCFunction_New is exported from the DLL. Python 3.4 does not have this (which seems a bug in its own right given the comment in 3.3 and earlier) but PC/bdist_wininst/install.c has code that attempts to dynamically load this function from the DLL and fails, causing 3rd party installers to fail. Assuming the removal of this API was intentional so the problem is that install.c needs to be updated, the following patch fixes the issue. -- components: Windows files: t.patch keywords: patch messages: 217185 nosy: mhammond priority: normal severity: normal status: open title: PyCFunction_New no longer exposed by python DLL breaking bdist_wininst installers type: behavior Added file: http://bugs.python.org/file35043/t.patch ___ Python tracker <http://bugs.python.org/issue21354> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue14302] Rename Scripts directory to bin and move python.exe to bin
Mark Hammond added the comment: I get the impression the first link just punted and the second supports the fact many people would see it as an improvement. So a patch that both works and addresses the concerns would probably be most welcome! -- ___ Python tracker <http://bugs.python.org/issue14302> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue19141] Windows Launcher fails to respect PATH
Mark Hammond added the comment: > I am trying to draw attention to the situation where the script has no > shebang line, and there is no other explicit configuration info for > py.exe. In that case, the user should just type "python scriptname.py" - py.exe is for cases where just specifying "python" doesn't do the right thing. -- ___ Python tracker <http://bugs.python.org/issue19141> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue18491] Add "exe wrapper" functionality to Windows launcher
Mark Hammond added the comment: Vinay's idea makes sense to me. Paul can also subtly change the patch such that when SCRIPT_WRAPPER is defined, failure to find the wrapper is fatal and prints a message specific to this fact rather than just starting an interactive Python (assuming I read the patch correctly, that is :) Then a new .exe is built named something innocuous, installed in a single well-known location as the "stub" and that pip-style code creating the wrapper has a nice easy job :) -- ___ Python tracker <http://bugs.python.org/issue18491> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue18491] Add "exe wrapper" functionality to Windows launcher
Mark Hammond added the comment: Obviously I'm missing a little context, but it seems a little wrong for the same launcher to be doing this double-duty. It seems we only want to use the launcher in this way as it already has some of the interesting code we need - but the vast majority of users aren't going to want to have it behave this way implicitly (ie, when they start what they consider the "launcher", they aren't going to expect this magic. Conversely, when someone executes "foo", they are going to be expecting exactly 1 thing to happen and that isn't "do what py.exe with no args does") So maybe this could be behind a #define, and have another .exe built from the same sources? One exe sticks with the launcher semantics (ie, never looks for foo-script.py), and the other sticks with the wrapper semantics (ie, terminates with an error if foo-script.py can't be found) All that said though, I'm not involved in the distutils discussions and acknowledge the general requirements as real, so do whatever you need to do :) -- ___ Python tracker <http://bugs.python.org/issue18491> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue18491] Add "exe wrapper" functionality to Windows launcher
Mark Hammond added the comment: I don't understand the motivation for this - how will it be used in practice? -- ___ Python tracker <http://bugs.python.org/issue18491> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17290] pythonw - loading cursor bug when launching scripts
Mark Hammond added the comment: If anyone would like to test the fix out, I've put a 32bit pyw.exe with the fix at http://starship.python.net/crew/skippy/downloads/pyw.exe -- ___ Python tracker <http://bugs.python.org/issue17290> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17290] pythonw - loading cursor bug when launching scripts
Mark Hammond added the comment: The problem is that Explorer displays the IDC_APPSTARTING icon when an app launches, until that app does something ui-ish (eg, creating a window, fetching a windows message). I could reproduce the problem on XP, and pushed a fix for the launcher to https://bitbucket.org/vinay.sajip/pylauncher/commits/639582f0bebbabbf63fe8746986122173e765baf which solves the problem for me. I'm not sure what the process is for getting that fix into the Python repo - hopefully Vinay knows :) -- ___ Python tracker <http://bugs.python.org/issue17290> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue15751] Support subinterpreters in the GIL state API
Mark Hammond added the comment: To clarify, I wrote: > can be associated with a thread-state specified by the > embedding application Where I meant to say: Can be associated with an interpreter state and corresponding thread-state ... Or something like that - it's been a while since I've looked at that code. -- ___ Python tracker <http://bugs.python.org/issue15751> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue15751] Support subinterpreters in the GIL state API
Mark Hammond added the comment: The GIL state api was mainly interested in the case of a thread which has (possibly) never been seen before calling into Python. IIUC, the proposal here is so that a thread that *has* been seen before can be associated with a thread-state specified by the embedding application (and the degenerate case would be to assume the thread hasn't been seen, and as such should get the default interpreter) If that isn't too wide of the mark, I agree it sounds workable and worthwhile. -- nosy: +mhammond ___ Python tracker <http://bugs.python.org/issue15751> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue15321] bdist_wininst installers may terminate with "close failed in file object destructor:\nsys.excepthook is missing\nlost sys.stderr"
New submission from Mark Hammond : Note the error message in the title is only for Python 2.x - Python 3.x shows an empty string instead, but otherwise seems identical. This was first brought to my attention via http://sourceforge.net/tracker/?func=detail&aid=352&group_id=78018&atid=551954. I can reproduce it (see below) with the pywin32 installer but believe the problem will exist with any such installer. All tests done on a Win7x64 VM. Symptoms are: * Run the installer with UAC enabled - reliably works. All "print" statements from the post-install script are shown in the final dialog. * Disable UAC. The installation reliably ends with the "close failed..." message shown in the final dialog. No print output is shown. The installation did actually complete (but is likely to fail if the output fills stdout buffering). The UAC is probably relevant as when UAC is enabled, the installer re-spawns itself with elevation. * The problem with UAC disabled can be reproduced running in the VS debugger. The problem is that Python sees stdout and stderr with a FILE object that has a fileno of -2 (ie, of _NO_CONSOLE_FILENO). Then, when Python is finalizing it tries to close these handles and fails due to the invalid fileno. The other messages are a consequence of the fact stderr itself is what is being closed (ie, Python is trying to report a problem closing stderr to stderr.) Python 3 seems to handle this case better courtesy of issue 1415, but the fact the output is lost remains in both versions. * Looking over the installer code, I notice that Python is initialized and terminated twice to compile .pyc/.pyo, then initialized a third time in preparation to run the install script - the handles are redirected prior to initializing the 3rd time. On a hunch, I hacked the code to perform the redirection *before* Python is initialized for the first time. This solved the problem when run under the debugger but still failed when run stand-alone (*sigh*). I fiddled with some params to CreateFile in the redirection code but could not get things working. Note that avoiding the multiple init/finalize calls may be problematic as Py_OptimizeFlag is set differently between runs. * I looked over the CRT init code to try and determine why _NO_CONSOLE_FILENO was being used. It appears the CRT attempts to use the process STARTUPINFO and only falls back to GetStdHandle if that STARTUPINFO doesn't provide valid handles. So I tried both using GetStartupInfo and putting the redirect handles in the result, and explicitly closing the old std handles, but neither of them worked. Next step is probably to arrange to step through the CRT init code to see what is going on there. I doubt I will get back to this any time soon, so this is my attempt at detailing what I've discovered to date incase someone else has a look... -- assignee: eric.araujo components: Distutils messages: 165213 nosy: eric.araujo, mhammond, tarek priority: normal severity: normal status: open title: bdist_wininst installers may terminate with "close failed in file object destructor:\nsys.excepthook is missing\nlost sys.stderr" type: behavior versions: Python 2.7, Python 3.2, Python 3.3, Python 3.4 ___ Python tracker <http://bugs.python.org/issue15321> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1677] Ctrl-C will exit out of Python interpreter in Windows
Changes by Mark Hammond : -- nosy: +mhammond ___ Python tracker <http://bugs.python.org/issue1677> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue14302] Move python.exe to bin/
Mark Hammond added the comment: Tools that use the registry will typically look up the InstallPath key and look for python.exe there. They will not look in sub-directories (except some may look in PCBuild and PCBuild/amd64). It is exactly those tools I'm concerned about. -- ___ Python tracker <http://bugs.python.org/issue14302> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue14302] Move python.exe to bin/
Mark Hammond added the comment: To clarify the second comment from Eric: it is actually the first of the proposals that I consider controversial - moving where python.exe lives (regardless of to where) will break 3rd party tools. If a decision was made to move it anyway, then renaming "Scripts" to "bin" and installing it there makes sense IMO. -- ___ Python tracker <http://bugs.python.org/issue14302> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue8170] Wrong Paths for distutils build --plat-name=win-amd64
Mark Hammond added the comment: Those instructions in section 5.4 do seem wrong. On first reading they imply that you need to start with a clean source tree on your 32bit Windows, then build the 64bit version of Python. So far so good. However, then you need to run python.exe - which must be a 32bit version of Python or it wouldn't start. So I *guess* the instructions really should say to build *both* the 32 and 64bit versions in the same source tree - in which case I can see why your patch would be necessary. Can you confirm the process you used? I assume you did build both versions in the same tree, but you also mentioned copying the .lib files around - was that an attempt to work around this bug? But yeah, sorry for the distraction - given 5.4 I think this bug is probably valid, but the patch should probably include updated instructions in 5.4 to make it more clear. -- ___ Python tracker <http://bugs.python.org/issue8170> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue8170] Wrong Paths for distutils build --plat-name=win-amd64
Mark Hammond added the comment: Can't you just install a 64bit Python in a different directory and use that executable to build the 64bit installer? It seems less error prone and should work without manually copying stuff or changing any code. -- ___ Python tracker <http://bugs.python.org/issue8170> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue8170] Wrong Paths for distutils build --plat-name=win-amd64
Mark Hammond added the comment: I don't quite understand how the order will be wrong. Which earlier entry is causing the problem? OTOH though, I also don't quite understand how the build would work at all if a 32bit Python is used to invoke distutils with --plat-name=win-amd64 as the 32bit Python install wont have that PCBuild/AMD64 directory at all - so some more context is needed before I can understand this. -- ___ Python tracker <http://bugs.python.org/issue8170> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13938] 2to3 fails to convert types.StringTypes appropriately
New submission from Mark Hammond : test_types.py converts "types.StringTypes" to "str" - but types.StringTypes is a tuple, so expressions like "type(x) in type.StringTypes" fails after conversion with "TypeError: argument of type 'type' is not iterable" Attaching a fix and test. Note that the fixer still seems "strange" after this fix - types.StringType gets converted to "bytes" but types.StringTypes uses str. This means the expression "type.StringType in type.StringTypes" evaluates to True in 2.x but False once converted - however, that should probably be tackled in a different bug - the fact the expression now causes a TypeError once converted is more blatantly wrong and the focus of this bug. -- components: 2to3 (2.x to 3.x conversion tool) files: fix_stringtypes_fixer.patch keywords: patch messages: 152592 nosy: mhammond priority: normal severity: normal status: open title: 2to3 fails to convert types.StringTypes appropriately versions: Python 3.2, Python 3.3, Python 3.4 Added file: http://bugs.python.org/file24415/fix_stringtypes_fixer.patch ___ Python tracker <http://bugs.python.org/issue13938> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue8418] Crash 0xc0000417 STATUS_INVALID_CRUNTIME_PARAMETER on program exit
Mark Hammond added the comment: See also http://bugs.python.org/issue13038 - same exception but in a different content, but the underlying cause may be similar. -- nosy: +mhammond ___ Python tracker <http://bugs.python.org/issue8418> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7833] bdist_wininst installers fail to load extensions built with Issue4120 patch
Mark Hammond added the comment: Actually, I think this is OK - the reference to the "x86" is in the tests and those tests don't actually perform a build, just check the manifest is detected and stripped (ie, the test should still work fine on 64bit boxes). Ideally the test could also check a manifest with a 64bit architecture, but I don't think that's really necessary... -- ___ Python tracker <http://bugs.python.org/issue7833> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13892] distutils handling of windows manifest isn't optimal
Mark Hammond added the comment: I thought this shouldn't be a problem - that as pythonxx.dll contains a manifest with the references and also contains hoops to ensure its "activation context" is used when loading dynamic modules, that things should work correctly. The scenario you outline isn't that different to Python being used as a COM server - where a program like "cscript.exe", which comes with Windows but I'm pretty sure has no reference to the CRT manifest we care about, loads up a Python COM object, causing Python to be loaded and it to load extension modules - and that works correctly. That said though, I agree it would be nice if the manifest handling was optional, but it isn't immediately clear how that could be done given how distutils is structured. Indeed, there isn't even an obvious way to do it programatically other than by monkey-patching the compiler class. At least if you do take the monkey-patch route, you should find it necessary to only patch one fairly small method in the class. -- ___ Python tracker <http://bugs.python.org/issue13892> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7833] bdist_wininst installers fail to load extensions built with Issue4120 patch
Mark Hammond added the comment: ack - that is a really good point. IIRC it can be "*". I'll try and look at this over the next day or 2. -- ___ Python tracker <http://bugs.python.org/issue7833> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13486] msvc9compiler.py doesn't properly generate manifest files.
Mark Hammond added the comment: A manifest seems to be currently created fine - can you provide steps to reproduce the problem you see? -- ___ Python tracker <http://bugs.python.org/issue13486> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13320] _remove_visual_c_ref in distutils.msvc9compiler causes DLL load fail with embedded Python and multiple CRT versions
Mark Hammond added the comment: I can't explain why this might be happening given the Python dll is still build against vc9 - I'm guessing this can't be reproduced without vs10 in the mix? Re making the feature optional - distutils doesn't really lend itself to customizing the compiler best I can tell - the only answer I know of here is to provide your own compiler sub-class and override manifest_get_embed_info to return the manifest filename. -- ___ Python tracker <http://bugs.python.org/issue13320> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12989] Consistently handle path separator in Py_GetPath on Windows
Mark Hammond added the comment: The first chunk of that patch is for when pythonhome==NULL. There is also a similar block just under it when MS_WINDOWS is not defined. While I don't know in which cases this will be built without that define, it looks as though the *buf++ = DELIM; should also be added to that block? Also, there is an existing conditional: if (argv0_path) { which can never be false (as argv0_path is a char array). It could be changed to if (argv0_path[0]) but ISTM that it could also be removed completely - ie, the 2 lines left in that block should have no effect in the case where the buffer is empty. I haven't actually tested it though, but apart from the first comment above, it *looks* like it does the right thing :) -- nosy: +mhammond ___ Python tracker <http://bugs.python.org/issue12989> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4431] Distutils MSVC doesn't create manifest file
Mark Hammond added the comment: I don't think a buildbot will be necessary - like the earlier compilers, they may have basic support but they don't all get buildbot support. The "problem" isn't the lack of ability/will to get things working with VS2010 - it is more the lack of incentive to add another compiler to the mix. OTOH, seeing it is nearly 2012, a case could certainly be made to move 3.3 to that compiler :) I'm interested to see the patch required to get it working - almost every other upgrade has been almost handled completely by the VS upgrade process so it would be good to see how different this one is - so please CC me on any new issue you create. I'm also interested to know how mixing the CRTs is causing problems - are you using the very few Python APIs impacted by such mixing, or is the issue more about needing to redistribute both, or something else entirely? -- ___ Python tracker <http://bugs.python.org/issue4431> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7833] bdist_wininst installers fail to load extensions built with Issue4120 patch
Mark Hammond added the comment: I pushed the changes to 2.7, 3.2 and 3.3. I'm happy to help with distutils2/packaging but I'll need to do that later once I work out where to start :) Therefore I'm not yet closing this issue. -- ___ Python tracker <http://bugs.python.org/issue7833> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4431] Distutils MSVC doesn't create manifest file
Mark Hammond added the comment: My experience is that for VS2008 at least, the /MANIFESTFILE: option seems to be ignored if there is nothing to put in the manifest, and this tends to be true if you use a static CRT instead of the DLL based one (ie, if you use /MT) Issue 7833 has a patch designed to both (a) remove the manifest entirely if the only assembly reference is to the CRT and (b) give the setup.py author finer control over this behaviour. -- ___ Python tracker <http://bugs.python.org/issue4431> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7833] bdist_wininst installers fail to load extensions built with Issue4120 patch
Mark Hammond added the comment: New version of the patch with the small tweaks requested plus a NEWS entry. -- Added file: http://bugs.python.org/file23400/bug-7833-tweaks-plus-news.patch ___ Python tracker <http://bugs.python.org/issue7833> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7833] bdist_wininst installers fail to load extensions built with Issue4120 patch
Mark Hammond added the comment: Thanks for the review. One note: | +def manifest_setup_ldargs | I’d make all new methods private ones (i.e. leading underscore). They aren't strictly private and are designed to be overridden by subclasses (although in practice, subclassing the compiler is much harder than it should be, so pywin32 monkey-patches the instance.) This is actually the entire point of my updated patch - to give setup.py authors some level of control over the manifest behaviour. I do intend forward-porting to 3.3 and also to check it is is too late for 3.2 (a quick check before implied it might be OK, but I'm not sure) -- ___ Python tracker <http://bugs.python.org/issue7833> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7833] bdist_wininst installers fail to load extensions built with Issue4120 patch
Mark Hammond added the comment: My apologies Eric - I had completely overlooked those tests. Attaching a new patch with a test. Note the existing test doesn't actually perform a build so the new test also doesn't, but it does check the core logic (ie, that a manifest with only the msvcrt reference gets it scrubbed). -- Added file: http://bugs.python.org/file23341/bug-7833-overridable-manifest-settings-with-test.patch ___ Python tracker <http://bugs.python.org/issue7833> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7833] bdist_wininst installers fail to load extensions built with Issue4120 patch
Mark Hammond added the comment: I'm reluctant to commit to adding test infrastructure for the distutils build commands - if I've missed existing infrastructure and adding such tests would actually be relatively simple, please educate me! Or if someone else would like to help with the infrastructure so I can test just this patch, that would be awesome. But I don't think this fix should block on tests given it can easily be tested and verified manually. -- ___ Python tracker <http://bugs.python.org/issue7833> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13101] Module Doc viewer closes when browser window closes on Windows 8
Mark Hammond added the comment: For some reason, IE is struggling to even display the page - it just seems to sit there loading the page without displaying anything, but hitting "stop" then "refresh" usually brings it up. But if you kill IE (which best I can tell can only be done via the task manager - it has no other Windows controls) the doc server process does also terminate. If you run the doc server using python.exe, you will notice tracebacks in the console due to the socket connection being reset (which is probably related to the above problems - the socket should have been fully read by the time you manage to kill IE) - but using python.exe the process stays alive serving requests. I *guess* that the problem is pythonw.exe is hitting an error when it attempts to print to the invalid stderr handle. It might be possible that somehow under Windows 8, stderr isn't buffered (or has as large of a buffer) as other Windows versions, so dies when a small amount of data is written to stderr - but I suspect the same problem could be provoked on other Windows versions by arranging for > 8k of "connection reset by peer" tracebacks to be written, at which point the buffer is attempted to be flushed and fails. Here endeth my speculation for the day ;) -- nosy: +mhammond ___ Python tracker <http://bugs.python.org/issue13101> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7833] Bdist_wininst installers fail to load extensions built with Issue4120 patch
Mark Hammond added the comment: This is biting people (including me :) so I'm going to try hard to get this fixed. One user on the python-win32 mailing list resorts to rebuilding every 3rd party module he uses with this patch to get things working again (although apps which use only builtin modules or pywin32 modules - which already hacks this basic functionality in - don't see the problem.) I'm attaching a different patch that should have the same default effect as Christoph's, but also allows the behaviour to be overridden. Actually overriding it is quite difficult as distutils isn't setup to easily allow such compiler customizations - but at least it *is* possible. To test this out I hacked both the pywin32 and py2erxe build processes to use those customizations and it works fine and allows them both to customize the behaviour to meet various modules' requirements. Finally, this whole thing is still fundamentally broken for extensions which need a manifest (eg, to reference the common controls or the requestedExecutionLevel cruft). These extension will still need to include the CRT reference in their manifest and thus will need a copy of the CRT next to each of them. I *think* this also means they basically get a private copy of the CRT - they are not sharing the CRT with Python, which means they are at risk of hitting problems such as trying to share FILE * objects. In practice, this means such modules are probably better of just embedding the CRT statically. This is the route I've taken for one pywin32 module so the module can have a manifest and still work without a complete, private copy of the CRT needing to live next to it. But even with that problem I think this patch should land. It would be great if someone can review and test this version of the patch and I'll check it in. -- versions: +Python 3.3, Python 3.4 -Python 2.6 Added file: http://bugs.python.org/file23305/bug-7833-overridable-manifest-settings.patch ___ Python tracker <http://bugs.python.org/issue7833> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11629] Reference implementation for PEP 397
Mark Hammond added the comment: The most recent version of PEP397 has removed all mentioned of this reference implementation - the C implementation at https://bitbucket.org/vinay.sajip/pylauncher/ is now the reference. -- resolution: -> out of date status: open -> closed ___ Python tracker <http://bugs.python.org/issue11629> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12200] bdist_wininst install_script not run on uninstall
Mark Hammond added the comment: (OTOH though, I could tweak the patch to work in a built tree - it would mean appending "PCBuild" to the dir and retrying the DLL load if the other options fail...) -- ___ Python tracker <http://bugs.python.org/issue12200> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12200] bdist_wininst install_script not run on uninstall
Mark Hammond added the comment: Adding tests would be fairly painful - there is no test infrastructure in place for generating and running installers at all, and worse, the changes are likely to not work correctly when run from a Python "build" tree when the built DLL is not installed into System32 (the patch basically assumes an "installed" dir structure rather than a "built" one.) -- ___ Python tracker <http://bugs.python.org/issue12200> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12200] bdist_wininst install_script not run on uninstall
Changes by Mark Hammond : -- assignee: tarek -> mhammond keywords: +patch stage: -> patch review versions: +Python 2.7, Python 3.2, Python 3.3, Python 3.4 Added file: http://bugs.python.org/file22205/issue12200.patch ___ Python tracker <http://bugs.python.org/issue12200> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12200] bdist_wininst install_script not run on uninstall
Changes by Mark Hammond : Added file: http://bugs.python.org/file22162/hello-install.py ___ Python tracker <http://bugs.python.org/issue12200> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12200] bdist_wininst install_script not run on uninstall
Changes by Mark Hammond : Added file: http://bugs.python.org/file22161/hello.py ___ Python tracker <http://bugs.python.org/issue12200> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12200] bdist_wininst install_script not run on uninstall
New submission from Mark Hammond : Probably in all versions, but certainly in 2.7. If you create an installer with bdist_wininst and specify an install_script, that script is not run on uninstallation. See attached test case: setup.py specifies an install_script which just appends argv to %TEMP%/uninstall-test.txt. * Create the installer with "python setup.py bdist_wininst" * Execute the installer (ie, "dist\hello-0.1.win32.exe") * Check %TEMP%/uninstall-test.txt - it should have 1 line: ['o:\\src\\python-2.6\\Scripts\\hello-install.py', '-install'] * Uninstall the package. * Check %TEMP%/uninstall-test.txt - it *should* have a new line reflecting the uninstall. Instead it is unchanged. To get more info: * Reinstall the package. * Manually uninstall using the cmdline (with adjusted paths) C:\> "c:\python26\Removehello.exe" -u "c:\python26\hello-wininst.log" > %TEMP%\delme.out 2>&1 Check the contents of %TEMP%\delme.out - you will see: *** Could not load Python ** Could not run installation script *** Digging into the source code of install.c - the problem is that although the Python DLL name is parsed from the log file, this isn't reflected in the global variables python_dir or pythondll - so they remain empty strings and attempting to load the empty string as Python causes the error messages. -- assignee: tarek components: Distutils files: setup.py messages: 137105 nosy: eric.araujo, mhammond, tarek priority: normal severity: normal status: open title: bdist_wininst install_script not run on uninstall type: behavior Added file: http://bugs.python.org/file22160/setup.py ___ Python tracker <http://bugs.python.org/issue12200> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11717] conflicting definition of ssize_t in pyconfig.h
Mark Hammond added the comment: As Tim Roberts says on the python-win32 list: > Actually, on closer examination, it may be a bit difficult to sell this. The > Microsoft compilers do not define this symbol at all. The SDK defines > SSIZE_T (as a long). Nothing defines ssize_t. http://code.activestate.com/lists/python-win32/11231/ for the full post. So yeah, the conclusion is that we shouldn't define such symbols incase someone else does too. So conceptually I'm +1 on that patch (I'm yet to try it and probably will not get a chance for a week or 2) -- ___ Python tracker <http://bugs.python.org/issue11717> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6498] Py_Main() does not return on SystemExit
Mark Hammond added the comment: Good catch - new patch with PyRun_SimpleStringFlags() corrected too. -- keywords: +patch Added file: http://bugs.python.org/file21448/bug-6498.patch ___ Python tracker <http://bugs.python.org/issue6498> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6498] Py_Main() does not return on SystemExit
Mark Hammond added the comment: It will return 1 if you specify a script to run and that has an unhandled exception. -- ___ Python tracker <http://bugs.python.org/issue6498> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6498] Py_Main() does not return on SystemExit
Changes by Mark Hammond : -- keywords: +needs review -patch ___ Python tracker <http://bugs.python.org/issue6498> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6498] Py_Main() does not return on SystemExit
Mark Hammond added the comment: For completeness, here is a doc patch against 2.6 which corrects the documentation. -- keywords: +patch Added file: http://bugs.python.org/file21447/bug-6498.patch ___ Python tracker <http://bugs.python.org/issue6498> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6498] Py_Main() does not return on SystemExit
Mark Hammond added the comment: > What teh docs says currently about SystemError calling > exit() is just _WRONG_. Correct - the docs should be fixed - which is what this bug is currently addressing (see the "Components" and "Assigned To" fields) > Also, I am not asking for a new feature. I'm pointing that > there is something wrong, with teh docs or teh code, and > that it is probably teh code. Nope - it is the docs. The docs were written after the code and incorrectly describe it in 2 important ways. If you want the code to change the way it has been forever, then you are asking for a new feature. -- ___ Python tracker <http://bugs.python.org/issue6498> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6498] Py_Main() does not return on SystemExit
Mark Hammond added the comment: @Rogi - you might like to re-read my responses a couple more times: * I refer to SystemError as the docs *you quoted* refer to SystemError. Therefore, we should *not* make the implementation match the docs - the docs would be wrong *even if* we change Python to work how you want. IOW, the docs need changing regardless of the outcome (and this is the bug where that should be managed, given the Components and assignee already on this bug) * I understand your point that calling exit() directly is undesirable but that should be handled as a new separate feature request bug. But while that bug goes through the process, the docs should be fixed to reflect the current reality (this bug). Once your new bug is accepted and fixed, the docs should be changed again to reflect that new reality. -- ___ Python tracker <http://bugs.python.org/issue6498> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6498] Py_Main() does not return on SystemExit
Mark Hammond added the comment: @Rogi - you seem to have a problem with your keyboard - the 'h' and 'e' keys seem to have been swapped. The docs are wrong regardless - I don't think anyone would suggest the behaviour match the docs regarding SystemError - having Py_Main return on SystemError would be backwards incompatible. Given the discussion in those docs about SystemError, I'm confident that the person who wrote the docs was simply attempting to document the actual implementation and made an error. The implementation has, as far as I know, always worked the way it does (ie, there has never been a regression in this). So IMO there are 2 issues: * The documentation should be made to reflect the actual and historic behaviour. The fact it doesn't is a clear bug and it should be addressed ASAP and without changing the behaviour so people are not misled in the future. This change should be easy and non-controversial. * You may have a desire to change the actual and historic implementation, which I can understand. However, this should be tackled in a new "feature request". If the feature request is accepted, then both the behaviour and the docs should be changed to match the new agreed behaviour. This change would be harder and may well be controversial, depending on a perception on how many people rely on the current behaviour regardless of what the docs say. -- ___ Python tracker <http://bugs.python.org/issue6498> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6498] Py_Main() does not return on SystemExit
Mark Hammond added the comment: Note the quoted documentation in comment 1, the paragraph "Note that if an otherwise unhandled SystemError ..." I don't think that paragraph is correct - SystemError doesn't seem to terminate Py_Main - but if you replace "SystemError" with "SystemExit", that paragraph is correct, including the comment about Py_InspectFlag. So I think there are 2 simple documentation bugs (the component of this bug is "Documentation") and not necessarily a behaviour bug (even though the "type" is set to behaviour :) * The first paragraph should have references to sys.exit() removed (I didn't mention this in my previous comment!) * The second paragraph should s/SystemError/SystemExit/ and optionally a note that calling sys.exit() will result in a SystemExit exception. I just traced through this in Python 2.6 - PyRun_InteractiveOneFlags winds up calling PyErr_PrintEx() and this function explicitly checks for SystemExit and calls handle_system_exit, which calls exit(). There doesn't seem to be any special handling for SystemError at all. -- ___ Python tracker <http://bugs.python.org/issue6498> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6498] Py_Main() does not return on SystemExit
Mark Hammond added the comment: Isn't the only problem here that the docs refer to SystemError instead of SystemExit - eg 'raise SystemError("foo")' in an interactive session doesn't terminate the process at all (and I don't believe it should) whereas SystemExit obviously does. -- nosy: +mhammond ___ Python tracker <http://bugs.python.org/issue6498> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11629] Reference implementation for PEP 397
New submission from Mark Hammond : A tracking bug for the reference implementation of PEP397 - A Python launcher for Windows. -- assignee: mhammond components: Documentation files: pep-0397-reference.py messages: 131723 nosy: mhammond priority: normal severity: normal status: open title: Reference implementation for PEP 397 type: feature request Added file: http://bugs.python.org/file21330/pep-0397-reference.py ___ Python tracker <http://bugs.python.org/issue11629> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11424] logging fileConfig may not correctly detect children
New submission from Mark Hammond : fileConfig has code to detect existing "child" loggers and ensure they are enabled if the parent is configured. However, the approach it takes of sorting the log names can fail in some cases. eg, if 3 loggers exist with names like "bar", "bar.child" and "bar-etc", "bar.child" is not detected as a child of "bar" as "bar-etc" sorts in between "bar" and "bar.child". Attaching a test case demonstrating this - things work correctly for "foo" and "foo.child" but not for "bar" and "bar.child" given the fact that "bar-etc" exists. -- files: log_test.py messages: 130224 nosy: mhammond, vinay.sajip priority: normal severity: normal status: open title: logging fileConfig may not correctly detect children type: behavior versions: Python 2.5, Python 2.6, Python 2.7, Python 3.1, Python 3.2, Python 3.3 Added file: http://bugs.python.org/file21026/log_test.py ___ Python tracker <http://bugs.python.org/issue11424> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7833] Bdist_wininst installers fail to load extensions built with Issue4120 patch
Mark Hammond added the comment: I'm wondering if, in practice, extensions which need a manifest can have the manifest being generated completely by the linker - ie, I expect that in most cases where something other than the CRT or MFC is needed in the manifest, the author will want to specify an explicit manifest file rather than one generated by the linker. And worse, those extensions are going to be screwed anyway - the fact the manifest remains at all will mean they probably fail to load for reasons already discussed in this bug. IOW, not having a manifest at all is about the only way to ensure the module will be loaded correctly. Re adding a manifest back in - I've struck the exact same problem with pywin32 - any DLLs which are "entry points" into Python need to have a manifest - eg, pythoncomxx and a few other pywin32 ones. This left me with a dilemma for pythoncom - as it is both an "entry-point" (ie, COM loads that DLL) and a regular Python module, I simultaneously needed a manifest (to work when loaded by COM) and needed to *not* have one (to work when loaded by Python). My solution has been to introduce another DLL with a manifest and have COM servers register using that. This strategy seems to be working well in all my tests. -- ___ Python tracker <http://bugs.python.org/issue7833> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7833] Bdist_wininst installers fail to load extensions built with Issue4120 patch
Mark Hammond added the comment: Thinking more about this, I think the approach of this patch is more complex than necessary. I think a better patch would be one which *unconditionally* removes the manifest from extension modules. For maximum flexibility though, we should probably allow a hook so setup.py can specify the name of (or contents of) a manifest file to use when linking with the default being None. -- ___ Python tracker <http://bugs.python.org/issue7833> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7833] Bdist_wininst installers fail to load extensions built with Issue4120 patch
Mark Hammond added the comment: I'm failing to get a new pywin32 out of the door due to this issue. I've spent a few hours playing with this and I think the analysis is generally correct here. The key thing is that when using distutils, the extensions end up with a manifest (albeit one without a reference to the CRT) whereas the extensions shipped with Python have no manifest at all. I agree with Martin that it seems strange the CRT fails to be used even though the CRT is obviously already loaded, but it seems to be a fact. I can't find much on this, but suspect it relates to the different "activation contexts" in use and how the activation contexts are designed to allow side-by-side loading of DLLs; Windows doesn't know if the version of the DLL already loaded is suitable. I also guess that the fact the DLL has *any* manifest means they use a more strict interpretation of things (ie, refuse to use already loaded ones) whereas a dll with no manifest gets given a little more slack. I can confirm that with the attached patch, pywin32 builds and installs fine on a machine without the CRT installed globally - so I'm +1 on this patch with one caveat: The check for '.pyd' should either be expanded to include '.dll', or else the check should just use the 'target_desc == CCompiler.EXECUTABLE' condition already used. I'm happy to make the change once I get some feedback and/or guidance about where I should check this in - I believe it is too late for python 2.6 which is a shame... -- ___ Python tracker <http://bugs.python.org/issue7833> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue459007] Document sys.path on Windows
Mark Hammond added the comment: FWIW, I think the rules are fairly well explained in the comments in PC/getpathp.c; last time I looked at this, the only thing I couldn't really decide was where in the official docs these comments should be put (after reformatting and minor rewording etc) -- ___ Python tracker <http://bugs.python.org/issue459007> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue8954] wininst regression: errors when building on linux
Mark Hammond added the comment: With the caveat that I haven't tested it (I'm currently traveling), the patch looks good to me. -- ___ Python tracker <http://bugs.python.org/issue8954> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue8929] wininst: msvcr90 dependency in x64 build
Mark Hammond added the comment: A quick check of my tree shows that the 32 and 64 bit versions are the same in this regard - the raw stubs wininst-9.0.exe and wininst-9.0-amd64.exe do not depend on the CRT. A test install of 2.6.5-amd64, then using the 64bit versoion of 'depends' against wininst-9.0-amd64.exe also shows no dependency against the msvc* DLLs. You may be instead striking a slightly different problem whereby the extension(s) installed by bdist_wininst are failing to start due to this dependency and them not finding or being able to use the same DLLs which Python itself depends on? -- ___ Python tracker <http://bugs.python.org/issue8929> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue8870] --user-access-control=force produces invalid installer on Vista
Mark Hammond added the comment: Is it possible the installer is being run from a network share? A comment from PC/bdist_wininst/install.c: // interesting failure scenario that has been seen: initial executable // runs from a network drive - but once elevated, that network share // isn't seen, and ShellExecute fails with SE_ERR_ACCESSDENIED. SE_ERR_ACCESSDENIED is 5, so googling for that might offer more insights. -- nosy: +mhammond ___ Python tracker <http://bugs.python.org/issue8870> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7594] shlex refactoring
Mark Hammond added the comment: I tried to use this in place of shlex for parsing IMAP responses for the 'imapclient' package. A couple of things struck me. * The class no longer has a next() method but probably should be added for b/w compat. * The class no longer has a 'token' attribute, which people may use to record the current token. Sadly it isn't clear if this is a documented part of the API or not. * Typo: wordchards = property(_get_wordchars, _set_wordchars) 'wordchards' is wrong. This implies there are no tests which set wordchars. * I *think* the lexer is now returning an empty string token as an input source is rolled over whereas before it did not. In effect, I *think* the old lexer would allow a single token to span sources, where this patched version does not. Sadly I didn't confirm this is truly accurate - but tests for this behaviour would probably help. -- nosy: +mhammond ___ Python tracker <http://bugs.python.org/issue7594> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1284316] Win32: Security problem with default installation directory
Mark Hammond added the comment: Another consideration here will be how distutils will work in a python with restricted permissions - the pattern of "just run 'setup.py install'" will not work unless it is done from an elevated command-prompt. As I expect this would frustrate people we'd need some story to address this. My take is still that Python is a tool, not an app. People writing an app they with to distribute using Python should include Python in their package (ie, not rely on an installed version) and these apps should conform with the guidelines. -- ___ Python tracker <http://bugs.python.org/issue1284316> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7833] Bdist_wininst installers fail to load extensions built with Issue4120 patch
Mark Hammond added the comment: the pywin32 DLLs have 2 heads. They are Python extension modules as well as regular DLLs. They are built by distutils and therefore have no manifests - I think many packages use distutils to build their extension modules - it is just that they usually don't have a .dll extension. I fear that simply adding a manifest to those DLLs will put them in the same position we have before issue4120 was addressed, and these .dlls do need a way to be installed into System32 (or somewhere else on the global PATH) to function as COM servers etc. I need to experiment with this. -- ___ Python tracker <http://bugs.python.org/issue7833> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7833] Bdist_wininst installers fail to load extensions built with Issue4120 patch
Changes by Mark Hammond : -- nosy: +mhammond ___ Python tracker <http://bugs.python.org/issue7833> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6132] Implement the GIL with critical sections in Windows
Changes by Mark Hammond : -- nosy: +mhammond ___ Python tracker <http://bugs.python.org/issue6132> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue5799] Change ntpath functions to implicitly support UNC paths
Mark Hammond added the comment: Checked into the trunk as r72387 (after normalizing whitespace in ntpath.py after the precommit-hook failed). Thanks Larry and Eric! -- resolution: -> fixed status: open -> closed ___ Python tracker <http://bugs.python.org/issue5799> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue5799] Change ntpath functions to implicitly support UNC paths
Mark Hammond added the comment: This looks good to me. My take on Guido's earlier notes are that they caused a problem in practice, and the philosophical concerns added justification for removing it. I'm yet to meet a Windows user with a philosophical objection to this. I'm attaching a new patch; I've added a news entry, changed to a PendingDeprecation warning and moved the import of the warnings module to the splitunc function to avoid extra import overhead, even if tiny. Could anyone still awake and available please have a quick review of the news entry and my change? -- assignee: -> mhammond Added file: http://bugs.python.org/file13902/markh.ntpath.r72374.diff ___ Python tracker <http://bugs.python.org/issue5799> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue5799] Change ntpath functions to implicitly support UNC paths
Mark Hammond added the comment: Should the DeprecationWarning for splitunc be a PendingDeprectionWarning for 3.1 and get 'upgraded' in 3.2? -- ___ Python tracker <http://bugs.python.org/issue5799> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1109963] bdist_wininst ignores build_lib from build command
Mark Hammond added the comment: So it looks like I broke the ability to override --build-lib :( Without testing, I think you might also need to handle the cross-compile case - the version may be the same, but the platform different. I know distutils is a PITA so might make this difficult, but it might be better that we only set build_lib if the user didn't - ie, trust the user knows what they are doing if they specify that option, even in these 'mismatched version/platform' cases. -- ___ Python tracker <http://bugs.python.org/issue1109963> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue5731] bdist_wininst no longer works on non-Windows platforms
Mark Hammond added the comment: +1 on the idea - it's not the first time I've forgotten that works on platforms other than Windows. It appears the patch you attached is reversed though (or its just way too early for me...) -- ___ Python tracker <http://bugs.python.org/issue5731> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4015] [patch] make installed scripts executable on windows
Mark Hammond added the comment: > It could also point to a "python launcher", which reads the first line What would that first line look like on Windows? o:\src\python-2.6-svn\PCBuild\python.exe would be appropriate for my machine, but I wouldn't really be happy with installed scripts embedding this in their first line - if for no better reason than depending on the Virtual Machine I am using at the time, that exact same directory will be seen as being on a completely different device, and potentially under a different 'root' directory too. -- ___ Python tracker <http://bugs.python.org/issue4015> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1386675] _winreg specifies EnvironmentError instead of WindowsError
Mark Hammond added the comment: oops - this slipped off my radar. I agree with *both* Frederic and Tony :) The module promises to raise an EnvironmentError, which is does. However, the main killer from my POV is that a 'winerror' attribute is likely to be of interest, and this is only available if we promise a WindowsError. In other words, to write code which conformed to the docs, you would need to write: try: ... except WindowsError, exc: if exc.winerror==SOME_INTERESTING_CODE:... except EnvironmentError, exc: # hrmph.. The second except clause is not actually necessary based on the impl, but is according to the docs. For this reason I'd be happy to update the docs to reflect the implementation reality. -- assignee: mhammond -> fdrake ___ Python tracker <http://bugs.python.org/issue1386675> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue850997] mbcs encoding ignores errors
Mark Hammond added the comment: It is still present, but I'm not sure what problems can be seen due to this so can't comment on its desirability. It would also introduce a backwards compatability concern but I've not enough experience to know how much of a problem that would be in practice either. ___ Python tracker <http://bugs.python.org/issue850997> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue5182] str() on memoryview of bytearray failing on py3k
New submission from Mark Hammond : % py30 -c "str(memoryview(bytearray((1,2,3" Traceback (most recent call last): File "", line 1, in TypeError: __str__ returned non-string (type bytes) The expected behaviour is that a string representation be returned. -- messages: 81363 nosy: mhammond severity: normal status: open title: str() on memoryview of bytearray failing on py3k versions: Python 3.0, Python 3.1 ___ Python tracker <http://bugs.python.org/issue5182> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4804] Python on Windows disables all C runtime library assertions
Mark Hammond added the comment: > Python shouldn't (IMHO) crahs, even if you give bogus input to > python functions. But it doesn't actually crash does it? It throws an assertion dialog, which sucks when the machine is unattended - which is why it is a problem for the buildbots - but otherwise if you hit 'ignore', things keep working IIUC? > Making sure that it doesn't crash in the test suite is > not enough, I think. I think it is enough. The tests are explicitly checking insane input to these functions but "in the real world" such assertions blowing would almost certainly be demonstrating a real bug. pywin32 has a similar issue - one or 2 tests cause such an assertion failure, but outside of the test suite, such assertions should always be enabled IMO. I'd even advocate the assertions are disabled only for the individual tests known to throw such errors and enabled for the rest. Hirokazu Yamamoto's suggestion sounds reasonable too, but still doesn't change my mind that it *is* reasonable to leave these assertions enabled in the general case, and thus the idea isn't really necessary IMO. Martin has the final say on what actually gets accepted though :) ___ Python tracker <http://bugs.python.org/issue4804> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4804] Python on Windows disables all C runtime library assertions
Mark Hammond added the comment: Fair enough - but assertions are still disabled while your "reference count" is >0, which is still while other arbitrary code is running. While I agree this is an improvement, I'm afraid I'm not playing close enough attention to understand why your patch is better than, eg, simply disabling such assertions globally during the test run. Given the patch in issue5116, this could be implemented in far less lines of code - certainly less intrusive lines. Would this not achieve the same result? ___ Python tracker <http://bugs.python.org/issue4804> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com