[issue47203] ImportError: DLL load failed while importing binascii: %1 is not a valid Win32 application.
Eryk Sun added the comment: There is something fundamentally wrong with the way modules built into the interpreter DLL (python3x.dll) are loaded if anything in sys.path or the system PATH can cause an import to fail. -- ___ Python tracker <https://bugs.python.org/issue47203> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue47203] ImportError: DLL load failed while importing binascii: %1 is not a valid Win32 application.
Eryk Sun added the comment: The user site packages directory is architecture specific starting with 3.10, e.g. "%AppData%\Python\Python310-32\site-packages". The PYTHONPATH environment variable is shared by all versions. However, I don't understand how the binascii module could be a problem because it's built into the interpreter DLL itself, i.e. it's in sys.builtin_module_names. There is no binascii.pyd, and, even if there were, it would be ignored. -- nosy: +eryksun ___ Python tracker <https://bugs.python.org/issue47203> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue47198] os.stat on windows doesn't take an open file even though os.stat in os.supports_fd
Eryk Sun added the comment:
You're mistaken about what `fd` is. It's a TextIOWrapper, which wraps a
BufferedWriter, which buffers a FileIO raw file object, which accesses the open
file number fd.fileno(). For example:
>>> f = open('tmp.tmp','w')
>>> os.stat(f.fileno()).st_size
0
--
nosy: +eryksun
resolution: -> not a bug
stage: -> resolved
status: open -> closed
___
Python tracker
<https://bugs.python.org/issue47198>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue47170] py launcher on windows opens new terminal window when parsing python script with shebang
Eryk Sun added the comment: > This is Windows (shell) behaviour. To avoid this, you need to > add the .py extension to the PATHEXT environment variable. PowerShell reuses the current console session only if .PY is set in PATHEXT. Otherwise Python gets executed with a flag that tells the system to allocate a new console session. For CMD, PATHEXT only affects the file search, not how the file is executed. If the internal START command isn't used, CMD reuses the current console session. The START command defaults to making the system allocate a new console, unless the /B option is used. -- nosy: +eryksun ___ Python tracker <https://bugs.python.org/issue47170> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue47161] pathlib method relative_to doesnt work with // in paths
Eryk Sun added the comment:
> Hmm..., I get it, but Im not gonna lie it's pretty confusing given
> that in other places `//` works as a substitute for `/`. Maybe it
> should be mentioned in the documentation?
In Linux, the system resolves "//" as just "/". In other POSIX systems, such as
Cygwin or MSYS2 (running in Windows), "//" is a UNC path of the form
"//server/share/filepath". I would expect resolve() to handle this. For example:
Linux:
>>> p = pathlib.Path('//tmp')
>>> p
PosixPath('//tmp')
>>> p.resolve()
PosixPath('/tmp')
However, resolve() is broken for a UNC path in 3.9 under MSYS2:
>>> p = pathlib.Path('//localhost/C$/temp')
>>> p.exists()
True
>>> p.resolve()
PosixPath('/localhost/C$/temp')
>>> p.resolve().exists()
False
realpath() is also broken for this case in 3.9 under MSYS2:
>>> os.path.realpath(p)
'/localhost/C$/temp'
--
nosy: +eryksun
___
Python tracker
<https://bugs.python.org/issue47161>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39090] Document various options for getting the absolute path from pathlib.Path objects
Eryk Sun added the comment:
> Now a file that doesn't exist:
> >>> mike = Path("palin.jpg")
> >>> mike.resolve()
> WindowsPath('palin.jpg')
This is a bug in resolve(). It was fixed in 3.10+ by switching to
ntpath.realpath(). I don't remember why a fix for 3.9 was never applied. Work
on the PR may have stalled due to a minor disagreement.
> 'C:\Windows\..\Program Files' and '/usr/../bin' == relative path
No, a relative path depends on either the current working directory or, for a
symlink target, the path of the directory that contains the symlink.
In Windows, a rooted path such as r"\spam" is a relative path because it
depends on the drive of the current working directory. For example, if the
current working directory is r"Z:\eggs", then r"\spam" resolves to r"Z:\spam".
Also, a drive-relative paths such as "Z:spam" depends on the working directory
of the given drive. Windows supports a separate working directory for each
drive. For example, if the working directory of drive "Z:" is r"Z:\eggs", then
"Z:spam" resolves to r"Z:\eggs\spam".
--
nosy: +eryksun
___
Python tracker
<https://bugs.python.org/issue39090>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue47134] Document the meaning of the number in OverflowError
Eryk Sun added the comment:
The error code for `1e+300 ** 2` is ERANGE, from calling libm pow(). Since
pow() returns a double, there's no way to indicate an error in the return
value. Instead, C errno is set to 0 beforehand and checked for a non-zero error
value after the call. If the error code isn't ERANGE, then float.__pow__()
raises ValueError instead of OverflowError. Here's the relevant snippet from
float_pow() in Objects/floatobject.c:
/* Now iv and iw are finite, iw is nonzero, and iv is
* positive and not equal to 1.0. We finally allow
* the platform pow to step in and do the rest.
*/
errno = 0;
ix = pow(iv, iw);
_Py_ADJUST_ERANGE1(ix);
if (negate_result)
ix = -ix;
if (errno != 0) {
/* We don't expect any errno value other than ERANGE, but
* the range of libm bugs appears unbounded.
*/
PyErr_SetFromErrno(errno == ERANGE ? PyExc_OverflowError :
PyExc_ValueError);
return NULL;
}
return PyFloat_FromDouble(ix);
Here's a direct example using ctypes:
import ctypes
import errno
libm = ctypes.CDLL('libm.so.6', use_errno=True)
libm.pow.argtypes = (ctypes.c_double, ctypes.c_double)
libm.pow.restype = ctypes.c_double
>>> errno.ERANGE
34
>>> ctypes.set_errno(0)
0
>>> libm.pow(1e300, 2)
inf
>>> ctypes.get_errno() == errno.ERANGE
True
--
nosy: +eryksun
___
Python tracker
<https://bugs.python.org/issue47134>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue32642] add support for path-like objects in sys.path
Eryk Sun added the comment: > I've got in mind a PyListObject subclass with calls to PyOS_FSPath > before insert, append, extend and __getitem__. The sys module doesn't prevent rebinding sys.path. Most code I think is careful to update sys.path. But surely some code replaces it with a new list instead of updating via insert(), append(), extend(), or updating a slice such as `sys.path[:] = new_path`. For example, ModifiedInterpreter.transfer_path() in Lib/idlelib/pyshell.py rebinds sys.path. It doesn't have to, but it does. -- nosy: +eryksun ___ Python tracker <https://bugs.python.org/issue32642> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue47086] Include HTML docs with Windows installer instead of CHM
Change by Eryk Sun : -- nosy: -eryksun ___ Python tracker <https://bugs.python.org/issue47086> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue47086] Include HTML docs with Windows installer instead of CHM
Eryk Sun added the comment: Do you have any thoughts about distributing the docs in ePub format? -- nosy: +eryksun ___ Python tracker <https://bugs.python.org/issue47086> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46788] regrtest fails to start on missing performance counter names
Change by Eryk Sun : Removed file: https://bugs.python.org/file50695/loadtracker.py ___ Python tracker <https://bugs.python.org/issue46788> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46788] regrtest fails to start on missing performance counter names
Change by Eryk Sun : -- Removed message: https://bugs.python.org/msg415781 ___ Python tracker <https://bugs.python.org/issue46788> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue47096] Use the PDH API in WindowsLoadTracker
New submission from Eryk Sun : In bpo-44336, a new version of WindowsLoadTracker was implemented in Lib/test/libregrtest/win_utils.py. This version resolves issues with the previous implementation that spawned typeperf.exe. The new implementation uses the registry API's HKEY_PERFORMANCE_DATA pseudohandle to access the performance counters. This requires hard-coding "2" as the key name of the system object, 44 as the index of the "Processor Queue Length" counter, and also several struct definitions. The HKEY_PERFORMANCE_DATA 'key' just wraps the API as an alternate way to consume counter data. Instead of taking this detour through a minimalist interface just to use the API in a roundabout way, it would be better to implement a few of the Performance Data Helper functions in _winapi. I've implemented a prototype in ctypes to demonstrate this, which I'm attaching as "loadtracker.py". The functions that would need to be implemented are PdhOpenQueryW(), PdhAddEnglishCounterW(), PdhCollectQueryData(), PdhGetRawCounterValue(), and PdhCloseQuery(). -- components: Windows files: loadtracker.py messages: 415808 nosy: eryksun, jkloth, paul.moore, steve.dower, tim.golden, zach.ware priority: normal severity: normal status: open title: Use the PDH API in WindowsLoadTracker type: enhancement Added file: https://bugs.python.org/file50696/loadtracker.py ___ Python tracker <https://bugs.python.org/issue47096> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue47093] Documentation Fix: Remove .bat when activating venv on windows
Eryk Sun added the comment: Running `tutorial-env\Scripts\activate` should suffice. The .bat script is for CMD, and the .ps1 script is for PowerShell. The shell should run the right script without having to include the extension. In Windows 10+, if you use a case-sensitive directory for the virtual environment, note that the script name for PowerShell is "Activate.ps1". PowerShell 7+ checks the directory for any name that case-insensitively matches "activate", but you'll have to run `tutorial-env\Scripts\Activate` in PowerShell 5.1. -- components: +Windows keywords: +easy nosy: +eryksun, paul.moore, steve.dower, tim.golden, zach.ware stage: -> needs patch versions: +Python 3.10, Python 3.9 ___ Python tracker <https://bugs.python.org/issue47093> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46788] regrtest fails to start on missing performance counter names
Eryk Sun added the comment:
I implemented a ctypes prototype that replaces the registry-based
implementation with the API calls PdhOpenQueryW(), PdhAddEnglishCounterW(),
PdhCollectQueryData(), PdhGetRawCounterValue(), and PdhCloseQuery(). I'm
attaching the script, but here's the class itself without the ctypes
definitions:
class WindowsLoadTracker():
"""
This class asynchronously reads the system "Processor Queue Length"
counter to calculate the system load on Windows. A raw thread is
used to avoid interfering with tests of the threading module.
"""
def __init__(self):
self._values = []
self._load = None
self._hrunning = kernel32.CreateEventW(None, True, False, None)
self._hstopped = kernel32.CreateEventW(None, True, False, None)
self._hquery = wintypes.HANDLE()
self._hcounter = wintypes.HANDLE()
pdh.PdhOpenQueryW(None, None, ctypes.byref(self._hquery))
pdh.PdhAddEnglishCounterW(self._hquery,
r"\System\Processor Queue Length",
None,
ctypes.byref(self._hcounter))
pdh.PdhCollectQueryData(self._hquery)
_thread.start_new_thread(self._update_load, (), {})
def _update_load(self,
# Localize module access to prevent shutdown errors.
WaitForSingleObject=_winapi.WaitForSingleObject,
SetEvent=kernel32.SetEvent):
# run until signaled to stop
while WaitForSingleObject(self._hrunning, 1000):
self._calculate_load()
# notify stopped
SetEvent(self._hstopped)
def _calculate_load(self,
# Lcalize module access to prevent shutdown errors.
PdhCollectQueryData=pdh.PdhCollectQueryData,
PdhGetRawCounterValue=pdh.PdhGetRawCounterValue):
counter_type = wintypes.DWORD()
raw = PDH_RAW_COUNTER()
PdhCollectQueryData(self._hquery)
PdhGetRawCounterValue(self._hcounter,
ctypes.byref(counter_type),
ctypes.byref(raw))
if raw.CStatus < 0:
return
processor_queue_length = raw.FirstValue
# Use an exponentially weighted moving average, imitating the
# load calculation on Unix systems.
#
https://en.wikipedia.org/wiki/Load_(computing)#Unix-style_load_calculation
#
https://en.wikipedia.org/wiki/Moving_average#Exponential_moving_average
if self._load is not None:
self._load = (self._load * LOAD_FACTOR_1
+ processor_queue_length * (1.0 -
LOAD_FACTOR_1))
elif len(self._values) < NVALUE:
self._values.append(processor_queue_length)
else:
self._load = sum(self._values) / len(self._values)
def getloadavg(self):
return self._load
def close(self,
# Localize module access to prevent shutdown errors.
WaitForSingleObject=_winapi.WaitForSingleObject,
INFINITE=_winapi.INFINITE,
SetEvent=kernel32.SetEvent,
CloseHandle=_winapi.CloseHandle,
PdhCloseQuery=pdh.PdhCloseQuery):
if self._hrunning is None:
return
# Tell the update thread to quit.
SetEvent(self._hrunning)
# Wait for the update thread to stop before cleanup.
WaitForSingleObject(self._hstopped, INFINITE)
CloseHandle(self._hrunning)
CloseHandle(self._hstopped)
PdhCloseQuery(self._hquery)
self._hrunning = self._hstopped = None
def __del__(self):
self.close()
return
--
Added file: https://bugs.python.org/file50695/loadtracker.py
___
Python tracker
<https://bugs.python.org/issue46788>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46788] regrtest fails to start on missing performance counter names
Eryk Sun added the comment: I was just wondering whether it's worth implementing it using the API. To be clear, I wasn't implying to hold off on applying Jeremy's PR. The existing code is causing him problems, and he has a working solution. -- nosy: +eryksun ___ Python tracker <https://bugs.python.org/issue46788> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43702] [Windows] correctly sort and remove duplicates in _winapi getenvironment()
Eryk Sun added the comment: > which name should be stored if they are duplicated with case insensitive? Ideally os.environ would preserve the original case of the process environment, and os.environ.copy() would return a copy that's also case insensitive. That would prevent most problems with duplicates keys. See msg387676 in bpo-28824, and msg414319 in bpo-15373. In msg390038 I suggested keeping the first key that's encountered. However, dicts preserve insertion order nowadays, so one could assume that the last one is the one that the caller wants to keep. -- ___ Python tracker <https://bugs.python.org/issue43702> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue47037] Build problems on Windows
Eryk Sun added the comment: The main entry point for python[_d].exe should support a command-line -X option or environment variable that suppresses Windows error/assert/warn reporting, or redirects it to stderr in verbose mode. This would be useful to simplify everyone's automated testing. It should require opting in, both for backward compatibility and also because the dialogs are useful for interactive testing. -- ___ Python tracker <https://bugs.python.org/issue47037> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue47037] Build problems on Windows
Change by Eryk Sun : -- priority: normal -> critical ___ Python tracker <https://bugs.python.org/issue47037> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue47037] Build problems on Windows
Eryk Sun added the comment:
> Ouch, is Python crashes because of an unsupported strftime call?
It's not a crash. It's a CRT error report dialog, which is enabled by default
for the _CRT_ASSERT and _CRT_ERROR macros in debug builds. This dialog can be
helpful when debugging interactively. It gives a developer the option to abort,
retry, or ignore. If a debugger is attached, the retry option breaks into the
debugger. I attached a debugger to the test runner to diagnose the problem with
format code "%4Y". For example:
>>> time.strftime('%4Y')
Debug Assertion Failed!
[...]
(Press Retry to debug the application)
(1a6c.101c): Break instruction exception - code 8003 (first chance)
ucrtbased!_Wcsftime_l+0x5af:
7ff8`a582b9af cc int 3
backtrace:
0:000> kc 4
Call Site
ucrtbased!_Wcsftime_l
ucrtbased!_Strftime_l
ucrtbased!strftime
python311_d!time_strftime
locals:
0:000> dv
_Expr_val = 0n0
string = 0x0226`fe38d1c0 ""
max_size = 0x400
format = 0x0226`fe37e7d0 "%4Y"
timeptr = 0x0041`c1feeda0
lc_time_arg = 0x`
locale = 0x`
locale_update = class _LocaleUpdate
format_it = 0x0226`fe37e7d2 "4Y"
remaining = 0x400
lc_time = 0x7ff8`a5868550
failed = true
string_it = 0x0226`fe38d1c0 ""
resume, with the STATUS_BREAKPOINT (0x8003) exception handled:
0:000> gh
Traceback (most recent call last):
File "", line 1, in
ValueError: Invalid format string
>>>
For non-interactive testing the report mode needs to be configured to 0 (no
report) or to report to stderr. For example:
>>> msvcrt.CrtSetReportMode(msvcrt.CRT_ERROR, msvcrt.CRTDBG_MODE_FILE)
4
>>> msvcrt.CrtSetReportFile(msvcrt.CRT_ERROR, msvcrt.CRTDBG_FILE_STDERR)
18446744073709551615
>>> os.abort()
abort() has been called
For time.strftime('%4Y'), the source of the report is _VALIDATE_RETURN(false,
EINVAL, 0) in _Wcsftime_l(). This macro includes an _ASSERT_EXPR(expr, msg)
check. In a debug build, this calls _CrtDbgReportW(_CRT_ASSERT, ...) if the
expression is false. If the latter returns 1 (retry), the __debugbreak()
intrinsic is called to break into the debugger. To suppress the dialog, either
temporarily set the CRT_ASSERT report mode to 0, or set it to report to stderr.
For example:
>>> msvcrt.CrtSetReportMode(msvcrt.CRT_ASSERT, msvcrt.CRTDBG_MODE_FILE)
4
>>> msvcrt.CrtSetReportFile(msvcrt.CRT_ASSERT, msvcrt.CRTDBG_FILE_STDERR)
18446744073709551615
>>> time.strftime('%4Y')
minkernel\crts\ucrt\src\appcrt\time\wcsftime.cpp(1163) : Assertion failed:
false
Traceback (most recent call last):
File "", line 1, in
ValueError: Invalid format string
--
___
Python tracker
<https://bugs.python.org/issue47037>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue47037] Build problems on Windows
Eryk Sun added the comment:
bpo-46587 added top-level code to "Lib/test/support/__init__.py" to check
whether time.strftime("%4Y") works. Since "Lib/test/libregrtest/setup.py"
imports the support module, that code executes before suppress_msvcrt_asserts()
is called by setup_tests().
--
nosy: +eryksun
___
Python tracker
<https://bugs.python.org/issue47037>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46890] getpath problems with framework build
Eryk Sun added the comment: > This means that "python -S" doesn't work with a virtual environment. Does that matter? I've only used a virtual environment to override or extend the system site packages. Is there a reason to use one without site packages? -- nosy: +eryksun ___ Python tracker <https://bugs.python.org/issue46890> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue47025] bytes do not work on sys.path
Eryk Sun added the comment: > this is a regression from 3.2 In Windows, bytes paths in sys.path do not work in 3.2+. I didn't test 3.0 and 3.1, but practically one can say that bytes paths were never supported in Python 3 on Windows. -- nosy: +eryksun ___ Python tracker <https://bugs.python.org/issue47025> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46950] Windows 11, VENV not working with case sensitive windows paths
Eryk Sun added the comment:
In 3.10, you should be able to work around the problem for the venv
site-packages directory by setting the environment variable "PYTHONPLATLIBDIR"
to "Lib". This sets sys.platlibdir, which the site module uses to create the
site-packages directory. The default value is "lib", which isn't properly
capitalized.
In 3.11, sys.platlibdir defaults to "DLLs" in Windows, and the site module
ignores this attribute. However, the site module does hard code the properly
capitalized value "Lib".
In POSIX, sys.platlibdir (e.g. "lib" or "lib64") is a common base directory for
the standard library and its extension modules. It isn't just the directory for
extension modules, as might be implied by changing the value to "DLLs" in
Windows. The "DLLs" directory in Windows Python is split off from "Lib" (for
some legacy reason, I suppose), so Windows would need separate sys.platlibdir
("Lib") and sys.platextdir ("DLLs") values, if that mattered, which it doesn't
since the names are fixed.
--
nosy: +eryksun
___
Python tracker
<https://bugs.python.org/issue46950>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46785] On Windows, os.stat() can fail if called while another process is creating or deleting the file
Eryk Sun added the comment: I was following the pattern of StatAttributeTests.test_access_denied(), which uses the current user's temp directory to get a filesystem that supports security. It would probably be better to skip tests if the filesystem of the current working directory doesn't support the test, e.g. if it needs NTFS or needs support for security, hard links, or reparse points. For example, Win32JunctionTests should be skipped if reparse points aren't supported. The os_helper module could implement a function that calls GetVolumeInformationW() to get the filesystem name (e.g. "Ntfs") and flags (e.g. FILE_PERSISTENT_ACLS, FILE_SUPPORTS_HARD_LINKS, FILE_SUPPORTS_REPARSE_POINTS). -- ___ Python tracker <https://bugs.python.org/issue46785> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46785] On Windows, os.stat() can fail if called while another process is creating or deleting the file
Eryk Sun added the comment:
> Why catch ERROR_NOT_READY and ERROR_BAD_NET_NAME as well?
When os.stat() falls back on FindFirstFileW(), an error that means the file
doesn't exist should be kept. ERROR_BAD_NET_NAME is an obvious error to keep
because it's already mapped to ENOENT (i.e. file not found). This error
typically means that share was removed or the network went down.
ERROR_NOT_READY typically means that the media was removed from a volume (e.g.
ejected disk). pathlib.Path.exists() returns False for ERROR_NOT_READY.
> Why can't the filename of the "foo"-like file in the test be
> simply os_helper.TESTFN, as done in some other tests?
I suppose the current working directory will be fine. I was looking to keep the
test on a NTFS filesystem, with known behavior, but there's no hard guarantee
that the user's temp directory is on the system volume.
> I noticed some code snippets used in tests are wrapped in a
> seemingly-redundant `if 1: ...` statement. Why is that?
The `if 1` test is logically redundant but convenient for using a multiline
string that has an arbitrary indentation level. For example:
>>> exec('''
... print(42)
... ''')
Traceback (most recent call last):
File "", line 1, in
File "", line 2
print(42)
IndentationError: unexpected indent
>>> exec('''if 1:
... print(42)
... ''')
42
--
___
Python tracker
<https://bugs.python.org/issue46785>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue47001] deadlock in ctypes?
Eryk Sun added the comment:
Pointers to resource type/name strings use the lower 16-bit range for integer
identifiers such as RT_ICON (3) and RT_GROUP_ICON (14). C code checks for these
cases using the IS_INTRESOURCE() macro.
It's incorrect to use a simple C string pointer type such as ctypes.c_wchar_p
(i.e. wintypes.LPWSTR) for a resource type/name string. Simple types get
automatically converted to the corresponding Python type (e.g. str), but a
16-bit ID is not a valid pointer. Support for resource IDs can be implemented
by a subclass of ctypes.c_wchar_p, since subclasses of simple types don't get
converted automatically. For example:
class LPWSTR(ctypes.c_wchar_p):
@property
def value(self):
v = ctypes.c_void_p.from_buffer(self).value or 0
if v >> 16 == 0:
return v
return super().value
@value.setter
def value(self, v):
ctypes.c_wchar_p.value.__set__(self, v)
def __eq__(self, other):
if not isinstance(other, __class__):
return NotImplemented
return self.value == other.value
def __hash__(self):
return hash(self.value)
RT_ICON = LPWSTR(3)
RT_GROUP_ICON = LPWSTR(RT_ICON.value + 11)
> kernel32 = ctypes.windll.kernel32
I recommend using `kernel32 = ctypes.WinDLL('kernel32', use_last_error=True)`,
which creates function pointers that capture the last error value before they
return. Call ctypes.get_last_error() to get the captured error value. To get an
exception for the error value, use ctypes.WinError(ctypes.get_last_error()).
This is more reliable since there's a chance that the thread executes some
WinAPI code that modifies the last error before GetLastError() is called. This
is particularly likely when working with ctypes calls in the interactive shell.
Using a separate WinDLL() instance also isolates a library from other
libraries. Since ctypes.windll caches modules, which cache function pointers,
it's a problem when multiple libraries require different function prototypes
(i.e. restype, argtypes, errcheck). The last library to set the prototype wins.
The other libraries will probably be broken in some way. Their function calls
may raise a ctypes.ArgumentError, or their post-call errcheck() isn't used, or
their required restype isn't returned.
> hmod = GetModuleHandle(sys.executable)
This is unnecessary and possibly wrong since there's no guarantee that
sys.executable is a valid file, let alone a file that's loaded as a module in
the current process. Pass hModule as NULL (None) to use the main module of the
process (i.e. the process image / executable).
> context structure that is used to use regular python functions as callbacks
> where external C code expects C callbacks with a certain signature.
Generally there's no need to use C context parameters, which aren't necessary
in object-oriented and functional programming paradigms. The callback function
can defined as a nested function, with access to closure variables, or as a
method, with access to instance attributes.
--
nosy: +eryksun
___
Python tracker
<https://bugs.python.org/issue47001>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46988] if a python program is execute by subprocess, the python program can't output unicode characters and raise UnicodeEncodeError
Change by Eryk Sun : -- superseder: -> Encoding error running in subprocess with captured output ___ Python tracker <https://bugs.python.org/issue46988> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46785] On Windows, os.stat() can fail if called while another process is creating or deleting the file
Eryk Sun added the comment:
Itai, you can add a test to Win32NtTests in Lib/test/test_os.py. Maybe spawn a
child process that creates and unlinks a file in a loop. In the parent process
execute a loop that tries to stat the file and ignores errors when the file or
path isn't found. For example:
@support.requires_subprocess()
def test_stat_unlink_race(self):
# bpo-46785: the implementation of os.stat() falls back on reading
# the parent directory if CreateFileW() fails with a permission
# error. If reading the parent directory fails because the file or
# directory is subsequently unlinked or because the volume or
# share is no longer available, then the original permission error
# should not be restored.
fname = os.path.join(os.environ['TEMP'], os_helper.TESTFN + '_46785')
self.addCleanup(os_helper.unlink, fname)
command = '''if 1:
import os
import sys
fname = sys.argv[1]
while True:
try:
with open(fname, "w") as f:
pass
except OSError:
pass
try:
os.remove(fname)
except OSError:
pass
'''
ignored_errors = (
2, # ERROR_FILE_NOT_FOUND
3, # ERROR_PATH_NOT_FOUND
21, # ERROR_NOT_READY
67, # ERROR_BAD_NET_NAME
)
deadline = time.time() + 5
p = subprocess.Popen([sys.executable, '-c', command, fname])
try:
while time.time() < deadline:
try:
os.stat(fname)
except OSError as e:
if e.winerror not in ignored_errors:
raise
finally:
p.terminate()
As the above test shows, I think the error should also be kept if
attributes_from_dir() fails with ERROR_NOT_READY or ERROR_BAD_NET_NAME. For
example:
switch (error) {
case ERROR_ACCESS_DENIED: /* Cannot sync or read attributes. */
case ERROR_SHARING_VIOLATION: /* It's a paging file. */
/* Try reading the parent directory. */
if (!attributes_from_dir(path, , )) {
/* Cannot read the parent directory. */
switch (GetLastError()) {
// keep these error codes
case ERROR_FILE_NOT_FOUND:
case ERROR_PATH_NOT_FOUND:
case ERROR_NOT_READY:
case ERROR_BAD_NET_NAME:
break;
// restore the error from CreateFileW()
default:
SetLastError(error);
}
return -1;
}
--
___
Python tracker
<https://bugs.python.org/issue46785>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46966] c_void_p array is a footgun on I32LP64 systems
Eryk Sun added the comment: > I'm not sure what can be done here (maybe a truncation warning?) For a function pointer, the default argument conversion for Python integers is the platform int type. Ideally, Python integer arguments would be converted to a type that matches the platform word size, as is the default behavior for integer arguments in C. But the behavior can't be changed at this point. Ideally, it would behave the same in LP64 (Unix) and LLP64 (Windows) systems, but OverflowError is raised in LLP64 because ctypes first converts to a long int. OverflowError could be manually raised if `(unsigned long)value > UINT_MAX`, but I think it's also too late to make that change. Scripts have worked around the current behavior for about two decades. Raising a warning is really the best that could be done, if anything is done at all. The best solution is to not use bare function pointers without setting the prototype. If a function pointer is created as an attribute of a CDLL instance, the common way to define the prototype is by setting the function's `argtypes` and `restype` attributes. Another ctypes concept to be aware of is that subclasses of simple types do not get converted by default when accessed as C fields, array subscripts, or function results. For example: class my_void_p(ctypes.c_void_p): pass >>> a = (my_void_p * 1)() >>> isinstance(a[0], my_void_p) True -- nosy: +eryksun versions: -Python 3.7, Python 3.8 ___ Python tracker <https://bugs.python.org/issue46966> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue37609] support "UNC" device paths in ntpath.splitdrive
Change by Eryk Sun : -- Removed message: https://bugs.python.org/msg390391 ___ Python tracker <https://bugs.python.org/issue37609> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46916] There is a problem with escape characters in the path passed in by pathlib.Path.mkdir()
Eryk Sun added the comment: > I thought pathlib would solve this problem completely now, > without having to replace the slashes. pathlib has nothing to do with how the Python language compiles string literals. A string *literal* is Python source code that gets compiled and instantiated as a string object. There is no technical problem with using r'F:\ceven\test2'. Using a raw string literal disables backslash escapes. But many developers find it inconvenient to have to use raw strings or to have to escape backslashes by doubling them. Plus raw string literals can't represent a path that ends with a backslash, such as C:\. The Windows API supports forward slash as a path separator in many cases, but not always, and also not always for paths passed on the command line. So it's preferable to always normalize paths. pathlib takes care of this for you. You can initialize conveniently with forward slashes, but get a Windows path string that uses backslashes. -- ___ Python tracker <https://bugs.python.org/issue46916> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46916] There is a problem with escape characters in the path passed in by pathlib.Path.mkdir()
Eryk Sun added the comment:
> e = Path(r'F:\ceven\test2')
Using forward slashes in the string literal is preferred, e.g.
Path('F:/ceven/test2'). This avoids the problem of backslash escapes in string
literals. The Path constructor parses the path and stores it internally as
component parts. When the path object is needed as a string, os.fspath()
returns the path using the platform's preferred path separator. A WindowsPath
or PureWindowsPath uses backslash as the path separator. For example:
>>> os.fspath(pathlib.PureWindowsPath('F:/ceven/test2'))
'F:\\ceven\\test2'
--
___
Python tracker
<https://bugs.python.org/issue46916>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46916] There is a problem with escape characters in the path passed in by pathlib.Path.mkdir()
Eryk Sun added the comment: > FileNotFoundError: [WinError 3] The system cannot find > the path specified: 'F:\\ceven\\test2' The Windows error code, ERROR_PATH_NOT_FOUND (3), indicates that the parent path, r"F:\ceven", does not exist. Try e.mkdir(parents=True) [1]. [1] https://docs.python.org/3/library/pathlib.html#pathlib.Path.mkdir -- nosy: +eryksun ___ Python tracker <https://bugs.python.org/issue46916> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46905] winsound.PlaySound should accept pathlib.Path instances
Change by Eryk Sun : -- components: +Windows keywords: +easy (C) nosy: +paul.moore, steve.dower, tim.golden, zach.ware ___ Python tracker <https://bugs.python.org/issue46905> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46905] winsound.PlaySound should accept pathlib.Path instances
Eryk Sun added the comment: PlaySound() is implemented by the C function winsound_PlaySound_impl() in "PC/winsound.c". To support pathlike objects, it could use `soundname = PyOS_FSPath(sound)`, and require soundname to be a Unicode string via PyUnicode_Check(soundname). Or it could more generically support any buffer or pathlike object via PyUnicode_FSDecoder(sound, ). This call decodes byte strings using the filesystem encoding, which is UTF-8 unless legacy ANSI mode is enabled. -- nosy: +eryksun stage: -> needs patch versions: +Python 3.11 ___ Python tracker <https://bugs.python.org/issue46905> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46888] SharedMemory.close() destroys memory
Eryk Sun added the comment:
Putting words into action, here's an example of what a privileged process (e.g.
running as SYSTEM) can do if a script or application is written to call the
undocumented NT API function NtMakePermanentObject(). A use case would be a
script running as a system service that needs a shared-memory section object to
persist after the service is stopped or terminated, e.g. such that the section
is still accessible if the service is resumed. Anyway, this is just an example
of what's possible, from a technical perspective. Also, note that I granted
SeCreatePermanentPrivilege to my current user for this example, so it's
certainly not required to use the SYSTEM account. An administrator can grant
this privilege to any account.
By default, named kernel objects are temporary:
>>> import os, _winapi, ctypes
>>> from multiprocessing.shared_memory import SharedMemory
>>> name = f'spam_{os.getpid()}'
>>> m = SharedMemory(name, True, 8192)
>>> m.close()
>>> try: SharedMemory(name)
... except OSError as e: print(e)
...
[WinError 2] The system cannot find the file specified: 'spam_6592'
Global permanent example:
Enable privileges for the current thread:
>>> import win32api; from win32security import *
>>> ImpersonateSelf(SecurityImpersonation)
>>> da = TOKEN_QUERY | TOKEN_ADJUST_PRIVILEGES
>>> ht = OpenThreadToken(win32api.GetCurrentThread(), da, False)
>>> ps = [[0, SE_PRIVILEGE_ENABLED], [0, SE_PRIVILEGE_ENABLED]]
>>> ps[0][0] = LookupPrivilegeValue(None, 'SeCreatePermanentPrivilege')
>>> ps[1][0] = LookupPrivilegeValue(None, 'SeCreateGlobalPrivilege')
>>> AdjustTokenPrivileges(ht, False, ps)
((16, 0),)
Create a global section object, and make it permanent:
>>> name = rf'Global\spam_{os.getpid()}'
>>> m = SharedMemory(name, True, 8192)
>>> from win32con import DELETE
>>> h = _winapi.OpenFileMapping(DELETE, False, name)
>>> ntdll = ctypes.WinDLL('ntdll')
>>> ntdll.NtMakePermanentObject(h)
0
>>> _winapi.CloseHandle(h)
A permanent object persists after the last handle is closed:
>>> m.close()
>>> m = SharedMemory(name) # This works now.
Make the section object temporary again:
>>> h = _winapi.OpenFileMapping(DELETE, False, name)
>>> ntdll.NtMakeTemporaryObject(h)
0
>>> _winapi.CloseHandle(h)
>>> m.close()
>>> try: SharedMemory(name)
... except OSError as e: print(e)
...
[WinError 2] The system cannot find the file specified: 'Global\\spam_6592'
--
___
Python tracker
<https://bugs.python.org/issue46888>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue28824] os.environ should preserve the case of the OS keys ?
Change by Eryk Sun : -- Removed message: https://bugs.python.org/msg414332 ___ Python tracker <https://bugs.python.org/issue28824> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue28824] os.environ should preserve the case of the OS keys ?
Eryk Sun added the comment:
Putting words into action, here's an example of what a privileged process (e.g.
running as SYSTEM) can do if a script or application is written to call the
undocumented NT API function NtMakePermanentObject(). A use case would be a
script running as a system service that needs a shared-memory section object to
persist after the service is stopped or terminated, e.g. such that the section
is still accessible if the service is resumed. Anyway, this is just an example
of what's possible, from a technical perspective.
By default, named kernel objects are temporary:
>>> import os, _winapi, ctypes
>>> from multiprocessing.shared_memory import SharedMemory
>>> name = f'spam_{os.getpid()}'
>>> m = SharedMemory(name, True, 8192)
>>> m.close()
>>> try: SharedMemory(name)
... except OSError as e: print(e)
...
[WinError 2] The system cannot find the file specified: 'spam_6592'
Global permanent example:
Enable privileges for the current thread:
>>> import win32api; from win32security import *
>>> ImpersonateSelf(SecurityImpersonation)
>>> da = TOKEN_QUERY | TOKEN_ADJUST_PRIVILEGES
>>> ht = OpenThreadToken(win32api.GetCurrentThread(), da, False)
>>> ps = [[0, SE_PRIVILEGE_ENABLED], [0, SE_PRIVILEGE_ENABLED]]
>>> ps[0][0] = LookupPrivilegeValue(None, 'SeCreatePermanentPrivilege')
>>> ps[1][0] = LookupPrivilegeValue(None, 'SeCreateGlobalPrivilege')
>>> AdjustTokenPrivileges(ht, False, ps)
((16, 0),)
Create a global section object, and make it permanent:
>>> name = rf'Global\spam_{os.getpid()}'
>>> m = SharedMemory(name, True, 8192)
>>> from win32con import DELETE
>>> h = _winapi.OpenFileMapping(DELETE, False, name)
>>> ntdll = ctypes.WinDLL('ntdll')
>>> ntdll.NtMakePermanentObject(h)
0
>>> _winapi.CloseHandle(h)
A permanent object persists after the last handle is closed:
>>> m.close()
>>> m = SharedMemory(name) # This works now.
Make the section object temporary again:
>>> h = _winapi.OpenFileMapping(DELETE, False, name)
>>> ntdll.NtMakeTemporaryObject(h)
0
>>> _winapi.CloseHandle(h)
>>> m.close()
>>> try: SharedMemory(name)
... except OSError as e: print(e)
...
[WinError 2] The system cannot find the file specified: 'Global\\spam_6592'
--
___
Python tracker
<https://bugs.python.org/issue28824>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46888] SharedMemory.close() destroys memory
Eryk Sun added the comment: > The persistent mode sounds just like Python shared memory also works > on Linux (where I can have these /dev/shm/* files even after the > Python process ends) but I think on Windows, Python is not using > the persistent mode and thus the shared memory goes away, in contrast > to how it works on Linux. Unix has the notion that everything is a file as a central organizing concept. As such, Linux opts to implement POSIX shm_open() [1] with a tmpfs [2] filesystem that's mounted on /dev/shm. tmpfs filesystems exist in virtual memory. They're not persistent in the sense of being created on a physical disk that provides persistent storage. Windows has the notion that everything is an object as a central organizing concept. Internally, the system has a "\" root object directory, which contains other object directories and object symbolic links (unrelated to filesystem symlinks). Each object directory, including the root directory, contains named kernel objects. Named device objects are normally created in r"\Device", such as r"\Device\HarddiskVolume2", and global symbolic links to devices are created in r"\GLOBAL??", such as r"\GLOBAL??\C:" -> r"\Device\HarddiskVolume2" for the "C:" drive. The root registry key object is r"\REGISTRY", which contains dynamic key objects such as r"\REGISTRY\MACHINE" (referenced via the pseudohandle HKEY_LOCAL_MACHINE) and "\REGISTRY\USER" (referenced via the pseudohandle HKEY_USERS), which in turn contain other keys such as "\REGISTRY\MACHINE\SOFTWARE" on which registry hives are mounted. For naming global kernel objects and session 0 objects, the Windows API internally uses the directory r"\BaseNamedObjects". For interactive Windows sessions, it uses r"\Sessions\\BaseNamedObjects" and, for app containers, subdirectories of r"\Sessions\\AppContainerNamedObjects". To explicitly name an object in the global directory, use the path r"Global\". The "Global" prefix is implemented as an object symbolic link to r"\BaseNamedObjects". Of course, that's an internal implementation detail; the API just refers to the r"Global\\" prefix. Naming a kernel object in r"\BaseNamedObjects" is nearly equivalent to creating a file in a tmpfs filesystem in Linux, with one major difference. Objects are reference counted, with a handle reference count and a pointer reference count. Opening a handle increments both counters, but kernel code can use just a pointer reference instead of opening a handle. By default, objects are temporary. As such, when their pointer reference count is decremented to 0, they get unlinked from the object namespace, if they're named objects, and deallocated. It's possible to create a permanent object in the object namespace, either initially via the OBJ_PERMANENT attribute [3], or later on via NtMakePermanentObject(handle). However, making an object permanent is considered a super privilege in Windows, so privileged in fact that the Windows API doesn't support this capability, at least not as far as I know. By default, SeCreatePermanentPrivilege is only granted to the SYSTEM account. Also, creating a named section object (i.e. file mapping object) in a session-restricted directory such as r"\BaseNamedObjects" requires SeCreateGlobalPrivilege, which is only granted by default to administrators and system service accounts. Unix and Linux are less concerned about creating 'permanent' files and global shared memory in virtual filesystems such as tmpfs. The lifetime while the system is running is left up to the creator, which has to manually remove the file, such as via shm_unlink() [4]. Stale files could accumulate in "/dev/shm" when processes crash, which is why a separate resource tracker is required, implementing what the Windows kernel provides by default. --- [1] https://pubs.opengroup.org/onlinepubs/9699919799/functions/shm_open.html [2] https://www.kernel.org/doc/Documentation/filesystems/tmpfs.txt [3] https://docs.microsoft.com/en-us/windows/win32/api/ntdef/ns-ntdef-_object_attributes [4] https://pubs.opengroup.org/onlinepubs/9699919799/functions/shm_unlink.html -- ___ Python tracker <https://bugs.python.org/issue46888> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue15373] copy.copy() does not properly copy os.environment
Eryk Sun added the comment: In bpo-28824, I suggested preserving the case of environment variables in Windows by using a case-insensitive subclass of str in the encodekey() function. This is self-contained by the use of the encodekey() and decodekey() functions in the mapping methods such as __iter__(). The reason for this is mostly about aesthetics, but also about faithfully displaying the actual environment variable names. Opinions vary, but to me "WindowsSdkVerBinPath" is both easier on my eyes and easier to read than "WINDOWSSDKVERBINPATH". The eyesore factor gets amplified when it's a wall of all upper-cased names 'screaming' at me. A copy via dict(os.environ) would use regular str keys and lose the case-insensitive, case-preserving property. It would be more useful if os.environ.copy() were implemented to return a copy that keeps the case-insensitive property for keys but disables updating the process environment. This could be implemented by making putenv and unsetenv parameters in the constructor. If self.putenv or self.unsetenv is None, then __setitem__() or __delitem__() woud have no effect on the process environment. The copy() method would return a new _Environ instance for self._data.copy(), one which of course disables updating the process environment. -- nosy: +eryksun ___ Python tracker <https://bugs.python.org/issue15373> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46888] SharedMemory.close() destroys memory
Eryk Sun added the comment:
> Yes, named memory mappings only exist on Windows until the last
> reference is closed, so this is a difference due to the underlying OS.
That's true for the Windows API, so it's true for all practical purposes. In
the underlying NT API, creating a permanent kernel object is possible by
setting OBJ_PERMANENT in the initial object attributes [1], or subsequently via
the undocumented system call NtMakePermanentObject(handle). Creating a
permanent object requires SeCreatePermanentPrivilege, which normally is granted
to just the SYSTEM account. An administrator can grant this privilege to any
user, group, or well-known group, but creating permanent objects should
generally be limited to drivers and system services. An object can be reverted
back to a temporary object via NtMakeTemporaryObject(handle).
A named section object (i.e. file mapping) can also be created as a global
name, i.e. r"Global\{object name}", which is accessible to all sessions. This
requires SeCreateGlobalPrivilege, which by default is granted to system service
accounts and administrators. This is separate from whether the section is
temporary or permanent, but a permanent section object is more likely to be
needed in the global namespace.
---
[1]
https://docs.microsoft.com/en-us/windows/win32/api/ntdef/nf-ntdef-initializeobjectattributes
[2]
https://docs.microsoft.com/en-us/windows-hardware/drivers/ddi/wdm/nf-wdm-zwmaketemporaryobject
--
nosy: +eryksun
___
Python tracker
<https://bugs.python.org/issue46888>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46791] Allow os.remove to defer to rmdir
Eryk Sun added the comment:
glibc remove() has an optimization to skip calling rmdir() if the macro
condition IS_NO_DIRECTORY_ERROR is true for the unlink() error. For Linux, this
condition is `errno != EISDIR`. On other platforms (e.g. BSD systems), the
condition is `errno != EPERM`. The implementation is the following, from
"sysdeps/posix/remove.c":
int
remove (const char *file)
{
/* First try to unlink since this is more frequently the necessary
action. */
if (__unlink (file) != 0
/* If it is indeed a directory... */
&& (IS_NO_DIRECTORY_ERROR
/* ...try to remove it. */
|| __rmdir (file) != 0))
/* Cannot remove the object for whatever reason. */
return -1;
return 0;
}
WinAPI DeleteFileW() doesn't support the same distinction. In this case,
ERROR_ACCESS_DENIED is set for the following system status codes:
STATUS_ACCESS_DENIED (like EACCES)
STATUS_CANNOT_DELETE (like EPERM; readonly or image-mapped)
STATUS_DELETE_PENDING (like EPERM)
STATUS_FILE_IS_A_DIRECTORY (like EISDIR)
os.remove() could skip skip calling RemoveDirectoryW() for a sharing violation,
i.e. ERROR_SHARING_VIOLATION. If DeleteFileW() fails with a sharing violation,
the path is not a directory and, even if it were, an attempt to delete it would
fail.
--
___
Python tracker
<https://bugs.python.org/issue46791>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46791] Allow os.remove to defer to rmdir
Eryk Sun added the comment:
> For REMOVE_BOTH, I don't see the need of calling GetFileAttributes
I was thinking that the NtQueryAttributesFile() system call is relatively cheap
compared to a full open, especially if the attributes of a remote file are
cached locally. However, on second thought, it's probably not. An open can fail
as soon as it knows that there's a file/directory type mismatch. This should be
able to take advantage of a local attribute cache instead of incurring network
latency.
> so I think there's no reason to combine it with the other two.
REMOVE_DIR can be separate, for the current behavior. But I wanted all modes
handled in one function in case later on we decide to fix os.rmdir() in
Windows. It allows deleting a directory symlink. Note that the os.lstat()
result reports a directory symlink as an S_IFLNK file, not S_IFDIR, so the
os.rmdir() behavior is internally inconsistent. This could be corrected by
forcing the REMOVE_DIR mode to raise NotADirectoryError. For example:
} else { // mode != REMOVE_BOTH
WIN32_FIND_DATAW data;
BOOL isDir = FALSE;
BOOL isLnk = FALSE;
HANDLE hFind = FindFirstFileW(path->wide, );
if (hFind != INVALID_HANDLE_VALUE) {
FindClose(hFind);
isDir = data.dwFileAttributes & FILE_ATTRIBUTE_DIRECTORY;
if (data.dwFileAttributes & FILE_ATTRIBUTE_REPARSE_POINT) {
if (mode == REMOVE_DIR) {
isLnk = data.dwReserved0 == IO_REPARSE_TAG_SYMLINK;
} else {
isLnk = IsReparseTagNameSurrogate(data.dwReserved0);
}
}
}
if ((mode == REMOVE_DIR) && (isDir && isLnk)) {
SetLastError(ERROR_DIRECTORY); // POSIX ENOTDIR
} else if ((mode == REMOVE_DIR) || (isDir && isLnk)) {
success = RemoveDirectoryW(path->wide);
} else {
success = DeleteFileW(path->wide);
}
}
--
___
Python tracker
<https://bugs.python.org/issue46791>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46869] platform.release() and sys returns wrong version on Windows 11
Eryk Sun added the comment: platform.release() returns platform.uname().release, which comes from platform.win32_ver() in Windows [1]. The issue with Windows 11 is being discussed in bpo-45382, but no PR has been submitted yet to resolve the issue. > >>> sys.getwindowsversion().platform_version > (10, 0, 22000) This result is correct [2], but you should use platform.version() instead. The major.minor version of Windows 11 is 10.0. The build number for Windows 11 is 22000 and above. The latest build number for Windows 10, in contrast, is 19044. The system version number concerns API compatibility, not user-interface updates or marketing names. Windows XP, XP Professional x64 Edition, Vista, 7, 8, and 8.1 correspond respectively to Windows system versions 5.1, 5.2, 6.0, 6.1, 6.2, and 6.3. System versions 7.x, 8.x, and 9.x were skipped. Version 10.0 is the first system version with two client workstation releases, Windows 10 and Windows 11. It's thus the first time that the build number is required to differentiate the release name. The `platform_version` attribute of the sys.getwindowsversion() result is supposed to be the true OS version in case the API is using a compatibility-mode version. It's based on the product version of the system file "kernel32.dll". However, it turns out that this is also not reliably the true OS version. The sys documentation was updated with a note that suggests using the platform module instead [3]. --- [1] https://docs.python.org/3/library/platform.html#platform.win32_ver [2] https://en.wikipedia.org/wiki/Windows_11_version_history#Version_history [3] https://docs.python.org/3/library/sys.html#sys.getwindowsversion -- nosy: +eryksun resolution: -> duplicate stage: -> resolved status: open -> closed superseder: -> platform() is not able to detect windows 11 ___ Python tracker <https://bugs.python.org/issue46869> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue28824] os.environ should preserve the case of the OS keys ?
Eryk Sun added the comment: > I think there should be a public class like this. I wrote a basic implementation of _CaseInsensitiveString under the assumption that it's hidden behind the __getitem__(), __setitem__(), and __delitem__() methods of the _Environ class. I don't want to complicate the implementation. The problem of case-insensitive names in file/registry paths and environment variables should be addressed with ntpath.normcase(). This should be based on WinAPI LCMapStringEx() with LOCALE_NAME_INVARIANT and LCMAP_UPPERCASE. The current implementation of ntpath.normcase() uses str.lower(), which depends on Python's Unicode database. This possibly differs from what Windows considers to be lower case in the invariant locale. It's also wrong because Windows uses upper case for cases-insensitive comparisons, which matters when sorting names. See bpo-42658. > Right now, it's not a good workaround because it contains the > environment at the time the interpreter was started, not the > current environment. Note that in some cases the "current environment" is the process environment, which isn't necessarily consistent with os.environ on any platform. In POSIX, posix.environ is created from C environ, which is kept in sync with changes to posix.environ via C putenv() and unsetenv(). However, directly calling os.putenv() or os.unsetenv(), or the underlying C functions, modifies the process environment without changing posix.environ. If subprocess.Popen() is called without overriding env, the child inherits the process environment, not posix.environ. In Windows, os.environ is created from nt.environ, with variables names converted to upper case. nt.environ is created from C _wenviron, which is created from the process environment, as returned by WinAPI GetEnvironmentStringsW(), except with variable names that begin with "=" filtered out. os.putenv() and os.unsetenv() are based on C _wputenv(), which updates C _wenviron and also updates the process environment via WinAPI SetEnvironmentVariableW(). If either os.putenv() or os.unsetenv() is called directly, or _wputenv() at a lower level, then the variables in C _wenviron (not just the letter case of the names) will be out of sync with os.environ. Additionally, if SetEnvironmentVariableW() is called directly to set or unset a variable, then both os.environ and C _wenviron will be out of sync with the process environment. If subprocess.Popen() is called without overriding env, the child inherits the process environment, not os.environ or C _wenviron. -- ___ Python tracker <https://bugs.python.org/issue28824> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46861] os.environ forces variable names to upper case on Windows
Change by Eryk Sun : -- resolution: -> duplicate stage: -> resolved status: open -> closed superseder: -> os.environ should preserve the case of the OS keys ? ___ Python tracker <https://bugs.python.org/issue46861> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue28824] os.environ should preserve the case of the OS keys ?
Change by Eryk Sun : -- versions: +Python 3.11 ___ Python tracker <https://bugs.python.org/issue28824> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46862] subprocess makes environment blocks with duplicate keys on Windows
Eryk Sun added the comment: I suggest closing this issue as a duplicate of bpo-43702. -- ___ Python tracker <https://bugs.python.org/issue46862> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43702] [Windows] correctly sort and remove duplicates in _winapi getenvironment()
Change by Eryk Sun : -- versions: +Python 3.11 -Python 3.8 ___ Python tracker <https://bugs.python.org/issue43702> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46862] subprocess makes environment blocks with duplicate keys on Windows
Eryk Sun added the comment: This should be handled in _winapi.CreateProcess(). An environment block is technically required to be sorted. (Ages ago this was a MUST requirement for getting and setting variables to work correctly, since the implementation depended on the sort order, but I think nowadays it's a SHOULD requirement.) For example, see the documentation of CreateProcessW() [1]: If an application provides an environment block, ... explicitly create these environment variable strings, sort them alphabetically (because the system uses a sorted environment) "Changing Environment Variables" is more specific [2]: All strings in the environment block must be sorted alphabetically by name. The sort is case-insensitive, Unicode order, without regard to locale. CompareStringOrdinal() [3] implements a case-insensitive ordinal comparison. When a key compares as equal, either keep the current one in the sorted list, or replace it with the new key. --- [1] https://docs.microsoft.com/en-us/windows/win32/api/processthreadsapi/nf-processthreadsapi-createprocessw [2] https://docs.microsoft.com/en-us/windows/win32/procthread/changing-environment-variables [3] https://docs.microsoft.com/en-us/windows/win32/api/stringapiset/nf-stringapiset-comparestringordinal -- nosy: +eryksun versions: +Python 3.11, Python 3.9 ___ Python tracker <https://bugs.python.org/issue46862> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46858] mmap constructor resets the file pointer on Windows
Eryk Sun added the comment:
The resize() method also modifies the file pointer. Instead of fixing that
oversight, I think it should directly set the file's FileEndOfFileInfo and
FileAllocationInfo. For example:
// resize the file
if (!SetFileInformationByHandle(
self->file_handle, FileEndOfFileInfo,
_size, sizeof(max_size)) ||
!SetFileInformationByHandle(
self->file_handle, FileAllocationInfo,
_size, sizeof(max_size)))
{
// resizing failed. try to remap the file
file_resize_error = GetLastError();
max_size.QuadPart = self->size;
new_size = self->size;
}
This is cheaper in terms of system calls. The existing implementation makes
four system calls: one to set the file pointer in SetFilePointerEx() and three
in SetEndOfFile(), which queries the file pointer, sets the end-of-file info,
and sets the allocation info.
Note that this approach doesn't modify the file pointer in any case. This may
be surprising if the file size shrinks to less than the existing file pointer.
But os.ftruncate() behaves the same way, as does the resize() method in Linux.
--
nosy: +eryksun
versions: -Python 3.7, Python 3.8
___
Python tracker
<https://bugs.python.org/issue46858>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46855] printing a string with strange characters loops forever
Eryk Sun added the comment:
The ordinal range 0x80-0x9F is the C1 control code set [1]. Ordinal 0x9F is
"Application Program Command" (APC). The command must be terminated by ordinal
0x9C, "String Terminator" (ST). For example, "\x9f Some Command \x9c".
In Gnome Terminal, after executing print('\x9f'), an APC command without a
terminator, typing Ctrl+L still works to clear the screen and get back to a
prompt.
[1] https://en.wikipedia.org/wiki/C0_and_C1_control_codes#C1_controls
--
nosy: +eryksun
___
Python tracker
<https://bugs.python.org/issue46855>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue37426] getpass.getpass not working with on windows when ctrl+v is used to enter the string
Eryk Sun added the comment: > I have an idea to solve it. But I don't know how to get the > clipboard data. In Windows, using the window manager entails extending the process and the current thread with GUI-related structures in the kernel and then connecting the process to a window station (usually "WinSta0", which contains the clipboard) and connecting the thread to a desktop (usually "Default"). This permanently changes how the OS sees the process. I think whether or not the process should be a GUI process is something for the application to decide, not the standard library. Thus getpass should not read text from the clipboard. The docs could note that terminals may need to be configured to support Ctrl+Shift+C (copy) and Ctrl+Shift+V (paste) shortcuts, and that some terminals provide alternate ways to paste text, such as a right-click action or context menu. I don't think the docs should provide detailed explanations and configuration details for particular terminals. -- ___ Python tracker <https://bugs.python.org/issue37426> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue37426] getpass.getpass not working with on windows when ctrl+v is used to enter the string
Eryk Sun added the comment: > Clicking `Edit > Paste` from the window menu > Use right-click to paste In particular, if the console has quick-edit mode enabled, then you can paste text by right-clicking. Also, if text is selected in quick-edit mode, right-clicking copies to the clipboard. > Enabling `Properties > Options > Use Ctrl+Shift+C/V as Copy/Paste` from the > menu I prefer this setting because it matches the behavior of terminals on other platforms. I suggest setting it in the "Defaults" dialog instead of "Properties". Otherwise, you'll have to configure it individually for each shell link (.LNK file) or session title. (The console session title defaults to the executable path, unless set in STARTUPINFO.) > Using the new Windows Terminal Terminal allows configuring the actions for keyboard shortcuts. By default it grabs Ctrl+C (copy) and Ctrl+V (paste). I disable these mappings. I leave the default mappings in place for Ctrl+Shift+C (copy), Ctrl+Shift+V (paste), Ctrl+Insert (copy), and Shift+Insert (paste). > This behavior is the same Command Prompt and PowerShell The behavior has to be the same when the parent process is a normal console shell such as CMD or PowerShell. Python inherits its console session from the shell, and that's the extent of the shell's involvement. A console session is hosted by an instance of conhost.exe or openconsole.exe. If the host is running in pseudoconsole (headless) mode, then the user interface for the session is hosted by another application (e.g. Windows Terminal). Even in pseudoconsole mode, however, the console host a lot to do in order to manage the session state for the console API. -- ___ Python tracker <https://bugs.python.org/issue37426> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12165] [doc] clarify documentation of nonlocal
Eryk Sun added the comment: > Another problem with the current text is that it fails to exclude > enclosing class scopes The nonlocal statement is only disallowed in module code (i.e. "exec" compilation mode) because it can never be nested. It's allowed in a class definition that has an outer function scope. A class body itself is never a nonlocal scope; it just has access to them. Here's a slightly modified version that includes class definitions: "When the definition of a function or class is nested (enclosed) within the definitions of other functions, its nonlocal scopes are the local scopes of the enclosing functions. The nonlocal statement causes the listed identifiers to refer to names previously bound in nonlocal scopes. If a name is bound in more than one nonlocal scope, the nearest binding is used. If a name is not bound in any nonlocal scope, or if there is no nonlocal scope, a SyntaxError is raised. The nonlocal statement applies to the entire scope of a function or class body. A SyntaxError is raised if a variable is used or assigned to prior to its nonlocal declaration in the scope." -- nosy: +eryksun ___ Python tracker <https://bugs.python.org/issue12165> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46839] Process finished with exit code -1073741819 (0xC0000005)
Eryk Sun added the comment: I tried to get more information for you by installing 2.7.11 (64-bit because of the given fault) and unassembling python27.dll at the fault offset. But it doesn't help. Whatever the bug is, it ended up jumping to an address that's in the middle of an instruction. This final bit of nonsense is what killed the process with an access violation, but it's not directly related to the problem. Here's the address that triggered the access violation: 0:000> ? python27 + a4f81 Evaluate expression: 1368149889 = `518c4f81 The nearest function is rawiobase_readall(): 0:000> ln python27 + a4f81 (`518c4f10) python27!rawiobase_readall+0x71 | (`518c51a0) python27!resize_buffer But 0x518c4f81 isn't the address of an instruction in this function: 0:000> u python27!rawiobase_readall python27!rawiobase_readall+80 python27!rawiobase_readall: [...] `518c4f7f 48894318mov qword ptr [rbx+18h],rax `518c4f83 48894310mov qword ptr [rbx+10h],rax `518c4f87 48894320mov qword ptr [rbx+20h],rax `518c4f8b 48837f10fe cmp qword ptr [rdi+10h],0FFFEh `518c4f90 4c89642440 mov qword ptr [rsp+40h],r12 It's inside of the first MOV instruction. -- nosy: +eryksun ___ Python tracker <https://bugs.python.org/issue46839> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46733] pathlib.Path methods can raise NotImplementedError
Eryk Sun added the comment:
> pathlib does not allow to distinguish "path" from "path/".
os.path.normpath() and os.path.abspath() don't retain a trailing slash -- or a
leading dot component for that matter. Are you referring to os.path.join()? For
example:
>>> os.path.join('./spam', 'eggs/')
'./spam/eggs/'
>>> os.path.normpath('./spam/eggs/')
'spam/eggs'
>>> PurePath('./spam') / PurePath('eggs/')
PurePosixPath('spam/eggs')
A leading dot component is significant in a context that searches a set of
paths -- usually PATH. A trailing slash is significant in a context that has to
distinguish a device or file path from a directory path, of which there are
several cases in Windows.
I think it's a deficiency in pathlib that it lacks a way to require
conservative normalization in these cases. Path and PurePath objects could gain
a keyword-only parameter, and internal attribute if needed, that enables a more
conservative normalization.
--
___
Python tracker
<https://bugs.python.org/issue46733>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46791] Allow os.remove to defer to rmdir
Eryk Sun added the comment:
In Windows, checking for a directory in order to defer to either
os_rmdir_impl() or os_unlink_impl() would lead to a redundant directory check.
os_unlink_impl() already has to check for a directory in order to delete a
directory symlink or mountpoint. I suggest implementing a common _Py_remove()
function in Windows. For example:
typedef enum {
REMOVE_FILE,
REMOVE_DIR,
REMOVE_BOTH,
} _Py_remove_mode;
int
_Py_remove(path_t *path, _Py_remove_mode mode)
{
BOOL isDir = FALSE;
BOOL isLnk = FALSE;
BOOL success = FALSE;
if (mode != REMOVE_DIR) {
DWORD fileAttributes = GetFileAttributesW(path->wide);
if (fileAttributes != INVALID_FILE_ATTRIBUTES) {
isDir = fileAttributes & FILE_ATTRIBUTE_DIRECTORY;
}
if (isDir && (mode == REMOVE_FILE)) {
WIN32_FIND_DATAW data;
HANDLE hFind = FindFirstFileW(path->wide, );
if (hFind != INVALID_HANDLE_VALUE) {
FindClose(hFind);
if (data.dwFileAttributes & FILE_ATTRIBUTE_REPARSE_POINT) {
isLnk = IsReparseTagNameSurrogate(data.dwReserved0);
}
}
}
}
if (mode == REMOVE_DIR || isDir && (mode == REMOVE_BOTH || isLnk)) {
success = RemoveDirectoryW(path->wide);
} else {
success = DeleteFileW(path->wide);
}
return success ? 0 : -1;
}
The multiple opens for GetFileAttributesW(), FindFirstFileW() and DeleteFileW()
add up to make the delete more expensive and more race prone than it has to be.
It would be nice to use a single open for all operations. But the Windows API
just has CreateFileW(), which requires SYNCHRONIZE access. The latter can't be
granted by the parent directory, unlike FILE_READ_ATTRIBUTES and DELETE access.
We could implement a version that uses CreateFileW(), and fall back on the
above version when access is denied, similar to what we do for os.stat(). Also,
Python is limited to the Windows 8.1 SDK, which makes it awkward to use
FileDispositionInfoEx (POSIX delete) like DeleteFileW() does in Windows 10, but
it should still be possible.
--
nosy: +eryksun
stage: -> needs patch
versions: +Python 3.11
___
Python tracker
<https://bugs.python.org/issue46791>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46785] On Windows, os.stat() can fail if called while another process is creating or deleting the file
Eryk Sun added the comment:
Windows filesystems disallow new opens for a file that has its delete
disposition set (i.e. the file is marked for deletion). For example,
CreateFileW() fails with ERROR_ACCESS_DENIED (5) in this case. A file with its
delete disposition set is still visibly linked in its parent directory.
The trigger to unlink a file that's marked for deletion depends on the delete
mode. In Windows 10, DeleteFileW() uses a POSIX delete if it's supported by the
filesystem (e.g. NTFS). This delete mode unlinks the file as soon the file
object that was used to set the delete disposition is closed. There's still a
small window of time, however, in which attempts to open the file will fail
with an access-denied error.
In os.stat(), if CreateFileW() fails with access denied, FindFirstFileW() is
called to try to get the stat data from the file's parent directory. But in
this case it's likely that the file has been unlinked from the directory by the
time FindFirstFileW() is called.
The original error code from CreateFileW() gets restored if FindFirstFileW()
fails. This is generally the right thing to do. However, if FindFirstFileW()
fails with ERROR_FILE_NOT_FOUND (2) or ERROR_PATH_NOT_FOUND (3), then I suggest
that the previous error code should not be restored.
For example:
switch (error) {
case ERROR_ACCESS_DENIED: /* Cannot sync or read attributes. */
case ERROR_SHARING_VIOLATION: /* It's a paging file. */
/* Try reading the parent directory. */
if (!attributes_from_dir(path, , )) {
/* Cannot read the parent directory. */
DWORD dir_error = GetLastError();
if (dir_error != ERROR_FILE_NOT_FOUND &&
dir_error != ERROR_PATH_NOT_FOUND) {
SetLastError(error);
}
return -1;
}
--
nosy: +eryksun
stage: -> needs patch
type: -> behavior
versions: +Python 3.11, Python 3.9
___
Python tracker
<https://bugs.python.org/issue46785>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46763] os.path.samefile incorrect results for shadow copies
Eryk Sun added the comment:
Sample implementation:
import os
import msvcrt
import win32file
def samefile(f1, f2):
"""Test whether two paths refer to the same file or directory."""
s1 = os.stat(f1)
s2 = os.stat(f2)
return _common_same_file(f1, f2, s1, s2)
def sameopenfile(fd1, fd2):
"""Test whether two file descriptors refer to the same file."""
s1 = os.fstat(fd1)
s2 = os.fstat(fd2)
return _common_same_file(fd1, fd2, s1, s2)
def _common_same_file(f1, f2, s1, s2):
if s1.st_ino != s2.st_ino or s1.st_dev != s2.st_dev:
return False
# (st_dev, st_ino) may be insufficient on its own. Use the final
# NT path of each file to refine the comparison.
p = _get_final_nt_paths([f1, f2])
# The stat result is unreliable if the volume serial number (st_dev)
# or file ID (st_ino) is 0.
if 0 in (s1.st_dev, s1.st_ino):
if None in p:
return False
return p[0] == p[1]
# A volume shadow copy has the same volume serial number as the
# base volume. In this case, the device names have to be compared.
d = _get_device_names(p)
if any('volumeshadowcopy' in n for n in d if n):
return d[0] == d[1]
return True
def _get_final_nt_paths(files):
result = []
nt_normal = 0x2 # VOLUME_NAME_NT | FILE_NAME_NORMALIZED
nt_opened = 0xA # VOLUME_NAME_NT | FILE_NAME_OPENED
for f in files:
p = None
if f is not None:
try:
p = _getfinalpathname(f, nt_normal)
except OSError:
try:
p = _getfinalpathname(f, nt_opened)
except OSError:
pass
result.append(p)
return result
def _get_device_names(paths):
# Look for "\Device\{device name}[\]".
result = []
for p in paths:
d = None
if p is not None:
q = p.split('\\', 3)
if len(q) > 2 and q[1].lower() == 'device' and q[2]:
d = q[2].lower()
result.append(d)
return result
def _getfinalpathname(p, flags=0):
try:
if isinstance(p, int):
h = msvcrt.get_osfhandle(p)
else:
h = win32file.CreateFile(p, 0, 0, None, win32file.OPEN_EXISTING,
win32file.FILE_FLAG_BACKUP_SEMANTICS, None)
return win32file.GetFinalPathNameByHandle(h, flags)
except win32file.error as e:
strerror = e.strerror.rstrip('\r\n .')
raise OSError(0, strerror, p, e.winerror) from None
--
___
Python tracker
<https://bugs.python.org/issue46763>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46775] [Windows] OSError should unconditionally call winerror_to_errno
New submission from Eryk Sun : bpo-37705 overlooked fixing the OSError constructor. oserror_parse_args() in Objects/exceptions.c should unconditionally call winerror_to_errno(), which is defined in PC/errmap.h. winerror_to_errno() maps the Winsock range 1-11999 directly, except for the 6 errors in this range that are based on C errno values: WSAEINTR, WSAEBADF, WSAEACCES, WSAEFAULT, WSAEINVAL, and WSAEMFILE. Otherwise, Windows error codes that aren't mapped explicitly get mapped by default to EINVAL. -- components: Interpreter Core, Windows keywords: easy (C) messages: 413383 nosy: eryksun, paul.moore, steve.dower, tim.golden, zach.ware priority: normal severity: normal stage: needs patch status: open title: [Windows] OSError should unconditionally call winerror_to_errno type: behavior versions: Python 3.10, Python 3.11, Python 3.9 ___ Python tracker <https://bugs.python.org/issue46775> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46763] os.path.samefile incorrect results for shadow copies
Eryk Sun added the comment: Python uses the volume serial number (VSN) and file ID for st_dev and st_ino. The OS allows the file ID to be 0 if the filesystem doesn't support file IDs. Also, it does not require or force the VSN to be a unique ID in the system, though if it's not 0 it's almost always a random 32-bit number for which the chance of collision is vanishingly small (notwithstanding a volume shadow copy, apparently). This means that (st_dev, st_ino) by itself is not sufficient to check whether two paths are the same file in Windows. Proposal: When comparing two file paths, if their (st_dev, st_ino) values differ, then they're not the same file. If their (st_dev, st_ino) values are the same, use the final NT paths from calling GetFinalPathNameByHandleW() with the flags VOLUME_NAME_NT | FILE_NAME_NORMALIZED. If only one of the paths supports FILE_NAME_NORMALIZED, then they're not the same file. If neither supports FILE_NAME_NORMALIZED, fall back on VOLUME_NAME_NT | FILE_NAME_OPENED. If either st_dev is 0 or st_ino is 0, the files are the same only if the final NT paths are the same. Else split out each device path. If the device paths are the same, then the paths are the same file. Otherwise they're different files. We should probably special case the comparison of a multiple UNC provider path with a local volume path. For example r'\\localhost\C$\Windows' is the same as r'C:\Windows'. The corresponding NT paths are r'\Device\Mup\localhost\C$\Windows' and typically r'\Device\HarddiskVolume2\Windows'. The special case is that when one of the device paths is "\Device\Mup", the two device paths are not required to be the same. Of course, this is given that the (st_dev, st_ino) values are the same, and neither st_dev nor st_ino is zero. That said, we would need to exclude volume shadow copies from the special case. I suppose we could just look for "VolumeShadowCopy" in the device name. Maybe we can do better. I've noticed that querying IOCTL_STORAGE_GET_DEVICE_NUMBER fails for a volume shadow copy, but that's probably going overboard. -- nosy: +eryksun stage: -> needs patch versions: +Python 3.10, Python 3.11 ___ Python tracker <https://bugs.python.org/issue46763> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46733] pathlib.Path methods can raise NotImplementedError
Eryk Sun added the comment: > I'm planning to learn more heavily on posixpath + ntpath in > pathlib once bpo-44136 is done. I think that would be a good > time to introduce is_mount() support on Windows. In the long run, it would be better to migrate the implementations in the other direction. Rewrite genericpath, ntpath, posixpath, and parts of shutil to use PurePath and Path objects. Using path objects instead of generic strings should improve the implementation of os.path and shutil, in addition to ensuring consistency with pathlib. However, that's a long-term goal that doesn't preclude using os.path and shutil as needed now, such as relying on os.path.ismount(). -- ___ Python tracker <https://bugs.python.org/issue46733> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46733] pathlib.Path methods can raise NotImplementedError
Eryk Sun added the comment: WindowsPath.is_mount() should call ntpath.ismount(). This function needs a significant redesign, but it's fine to use it as long as it's documented that is_mount() is equivalent to os.path.ismount(). Since the owner() and group() methods return names instead of numeric IDs, technically we could implement them in WindowsPath. That said, in Windows, st_mode and chmod() are unrelated to a file's owner and group, plus object ownership is fundamentally different from POSIX. Unless chown() is also implemented, I don't think we would gain much from knowing the owner and group, other than making it easier to display them. -- nosy: +eryksun ___ Python tracker <https://bugs.python.org/issue46733> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46751] Windows-style path is not recognized under cygwin
Eryk Sun added the comment:
MSYS2 has basically the same problem when the script is passed as a Windows
path, except it uses "/c" for the "C:" drive instead of "/cygdrive/c".
# python3 -VV
Python 3.9.9 (main, Dec 28 2021, 11:05:23)
[GCC 11.2.0]
# echo $PWD
/proc
# python3 C:/Temp/test.py
python3: can't open file '/proc/C:/Temp/test.py': [Errno 2] No such file or
directory
Windows paths in the command-line arguments appear to be passed without
conversion:
# python3 -ic 1 C:/Temp/test.py
>>> import os
>>> open(f'/proc/{os.getpid()}/cmdline').read().split('\0')
['python3', '-ic', '1', 'C:/Temp/test.py', '']
They're generally supported:
>>> open('C:/Temp/test.py').read()
'import sys\nprint(sys.executable)\n\n'
>>> os.path.samefile('C:/Temp/test.py', '/c/Temp/test.py')
True
>>> os.path.abspath('C:/Temp/test.py')
'C:/Temp/test.py'
realpath() doesn't support them:
>>> os.path.realpath('C:/Temp/test.py')
'/:/Temp/test.py'
But the C API _Py_wrealpath() does:
>>> import ctypes
>>> path = (ctypes.c_wchar * 1000)()
>>> ctypes.pythonapi._Py_wrealpath('C:/Temp/test.py', path, 1000)
1484496
>>> path.value
'/c/Temp/test.py'
--
nosy: +eryksun
___
Python tracker
<https://bugs.python.org/issue46751>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46716] regrtest didn't respect the timeout when running test_subprocess on AMD64 Windows11 3.x
Eryk Sun added the comment: > 4) use Job objects to group Windows processes for termination I think a separate issue should be created for this enhancement. _winapi wrappers would be needed for CreateJobObjectW(), SetInformationJobObject(), AssignProcessToJobObject(), TerminatejobObject(), and ResumeThread(), plus the constant CREATE_SUSPENDED. I'd also prefer to make related changes to send_signal(), which would require GetConsoleProcessList() and GenerateConsoleCtrlEvent(). A new Popen option would be needed to configure whether the job allows descendants to break away via the process creation flag CREATE_BREAKAWAY_FROM_JOB. This should be allowed by default. --- send_signal(): SIGKILL, SIGTERM, SIBREAK, SIGINT Support Popen.kill(group=False) and Popen.terminate(group=False) on all platforms as Popen.send_signal(signal.SIGKILL, group=group) and Popen.send_signal(signal.SIGTERM, group=group). The Universal CRT (ucrt) in Windows doesn't define SIGKILL. Even when it's not defined by the platform, signal.SIGKILL should always be defined, preferably as 9 (unused by ucrt), else as an unused value in the range up to NSIG, else as NSIG + 1. The `group` keyword-only option broadens the scope to the process group or job. A process is a group leader if it was created with the flag CREATE_NEW_PROCESS_GROUP (save self._creationflags) and its process ID is in GetConsoleProcessList(). For SIGKILL, always use forced termination. For SIGTERM, use forced termination either if `group` is false or if `group` is true and the process is not a group leader. To force termination, call TerminateJobObject(self._job_handle, 1) if `group` is true, else TerminateProcess(self._handle, 1). For SIGTERM, SIGBREAK, and SIGINT, call GenerateConsoleCtrlEvent() if `group` is true and the process is a group leader. For SIGTERM and SIGBREAK, send CTRL_BREAK_EVENT. For SIGINT, send CTRL_C_EVENT. Behavior to deprecate: When `group` is false and `sig` is CTRL_C_EVENT (0) or CTRL_BREAK_EVENT (1), send_signal() always calls os.kill(). This legacy behavior tries to handle a process as if it's a process group, and it combines POSIX kill() with Windows API constants. subprocess should distance itself from the fundamental problems with os.kill() in Windows. -- ___ Python tracker <https://bugs.python.org/issue46716> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46716] regrtest didn't respect the timeout when running test_subprocess on AMD64 Windows11 3.x
Eryk Sun added the comment:
> I fear the potential 3rd-party breakage alone should bump
> this to its own issue.
The change to the DWORD converter in _winapi should only be in 3.11+. If this
causes problems for other projects, they're probably depending on undefined
behavior in the standard library. No third-party package or application should
use _winapi directly.
> 1) modify Windows Popen._wait() to raise on out of bounds
> values [< 0 or >= INFINITE]
I think raising ValueError would be best at this level. For example:
if timeout is None:
timeout_millis = _winapi.INFINITE
else:
timeout_millis = int(timeout * 1000)
if timeout_millis >= _winapi.INFINITE:
raise ValueError("'timeout' exceeds the platform limit")
if timeout_millis < 0:
raise ValueError("'timeout' must be non-negative")
--
___
Python tracker
<https://bugs.python.org/issue46716>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46726] Thread spuriously marked dead after interrupting a join call
Eryk Sun added the comment: > I didn't have that in mind at all. I understood what you had in mind, and I don't disagree. I was just highlighting that the only somewhat important work done in _stop() is to clean up the _shutdown_locks set, to keep it from growing too large. But that task is already handled when creating a new thread, and it's arguably more important to handle it there. -- ___ Python tracker <https://bugs.python.org/issue46726> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46726] Thread spuriously marked dead after interrupting a join call
Eryk Sun added the comment: > Anything at the Python level that cares whether the thread is > still alive will call _wait_for_tstate_lock() again It's nice that _maintain_shutdown_locks() gets called in _stop(), but the more important call site is in _set_tstate_lock(). I typed up the example initially with try/finally. Then I changed it to avoid the extra lock.locked() call when there's no exception, but I forgot to add a `raise` statement: try: if lock.acquire_and_release(block, timeout): self._stop except: if not lock.locked(): self._stop() raise -- ___ Python tracker <https://bugs.python.org/issue46726> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46726] Thread spuriously marked dead after interrupting a join call
Eryk Sun added the comment: > is there a bulletproof way to guarantee that `self._stop()` gets > called if the acquire_and_release() succeeds? I don't think it's critical. But we still should deal with the common case in Windows in which a KeyboardInterrupt is raised immediately after the method returns. For example: try: if lock.acquire_and_release(block, timeout): self._stop except: if not lock.locked(): self._stop() The proposed acquire_and_release() method can also be used in threading._shutdown(), when it iterates _shutdown_locks to join non-daemon threads. -- ___ Python tracker <https://bugs.python.org/issue46726> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46716] regrtest didn't respect the timeout when running test_subprocess on AMD64 Windows11 3.x
Eryk Sun added the comment: > the fix should be as simple as coercing the timeout values to >= 0. Popen._remaining_time() should return max(endtime - _time(), 0). Popen._wait() should raise OverflowError if the timeout is too large for the implementation. In Windows, the upper limit in milliseconds is `_winapi.INFINITE - 1` (about 49.7 days). It's important to only allow the timeout in milliseconds to be _winapi.INFINITE when `timeout is None`. The DWORD converter in _winapi needs to subclass unsigned_long_converter. The current implementation based on the legacy format unit "k" is too lenient. Negative values and values that are too large should fail. I updated it to use the following definition: class DWORD_converter(unsigned_long_converter): type = 'DWORD' This produces the following improved results: >>> h = _winapi.GetCurrentProcess() >>> _winapi.WaitForSingleObject(h, _winapi.INFINITE + 1) Traceback (most recent call last): File "", line 1, in OverflowError: Python int too large to convert to C unsigned long >>> _winapi.WaitForSingleObject(h, -1) Traceback (most recent call last): File "", line 1, in ValueError: value must be positive -- ___ Python tracker <https://bugs.python.org/issue46716> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46726] Thread spuriously marked dead after interrupting a join call
Eryk Sun added the comment:
> Wrap everything needed in a custom C function.
Maybe add an `acquire_and_release()` method:
static PyObject *
lock_PyThread_acquire_and_release_lock(
lockobject *self, PyObject *args, PyObject *kwds)
{
_PyTime_t timeout;
if (lock_acquire_parse_args(args, kwds, ) < 0)
return NULL;
PyLockStatus r = acquire_timed(self->lock_lock, timeout);
if (r == PY_LOCK_INTR) {
return NULL;
}
if (r == PY_LOCK_ACQUIRED) {
PyThread_release_lock(self->lock_lock);
self->locked = 0;
}
return PyBool_FromLong(r == PY_LOCK_ACQUIRED);
}
--
___
Python tracker
<https://bugs.python.org/issue46726>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46726] Thread spuriously marked dead after interrupting a join call
Eryk Sun added the comment: > If the acquire() in fact times out, but the store to the `acquired` > variable is interrupted, `if _WINDOWS and acquired is None` will > succeed, despite that the lock is still locked Yeah, my proposed workaround is no good, so we can't resolve this without a more fundamental solution. Are you looking into a way to prevent the STORE_FAST instruction from getting interrupted by an asynchronous exception? -- ___ Python tracker <https://bugs.python.org/issue46726> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46726] Thread spuriously marked dead after interrupting a join call
Eryk Sun added the comment: > The race on := is much smaller than the original race > and I suspect in practice will be very hard to hit. In Windows, the acquire() method of a lock can't be interrupted. Thus, in the main thread, an exception from Ctrl+C gets raised as soon as acquire() returns. This exception definitely will interrupt the assignment. Here's a workaround: global scope: _WINDOWS = _sys.platform == 'win32' in _wait_for_tstate_lock(): acquired = None try: if acquired := lock.acquire(block, timeout): lock.release() self._stop() except: if _WINDOWS and acquired is None: acquired = True if acquired: lock.release() self._stop() raise This doesn't help in POSIX if the STORE_FAST instruction that assigns `acquired` gets interrupted. This can't be distinguished from acquire() itself getting interrupted. But at least the window for this is as small as possible. -- nosy: +eryksun ___ Python tracker <https://bugs.python.org/issue46726> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46716] regrtest didn't respect the timeout when running test_subprocess on AMD64 Windows11 3.x
Eryk Sun added the comment: > test_call_timeout() or test_timeout() in test_subprocess.py. These tests don't override the standard files, and they only spawn a single child with no descendants. I don't see why this would hang. It shouldn't be a problem with leaked pipe handles (see bpo-43346). It probably will need to be diagnosed by attaching a debugger, or offline with a dump file. > process trees whereas terminating a parent automatically kills the children One can use a job object to manage a child process and all of its descendants, including resource usage and termination. A process can belong to multiple job objects in Windows 8+, which is required by Python 3.9+. For reliability, the child has to be created in a suspended state via CREATE_SUSPENDED. It can be resumed with ResumeThread() after adding it to the job with AssignProcessToJobObject(). You can try to terminate a job cleanly, which is similar in effect to sending SIGTERM to a process group in POSIX. In Windows, this has to be approached differently for console vs graphical processes. To handle console apps, assuming the child inherits the current console, spawn it as a new process group via creationflags=subprocess.CREATE_NEW_PROCESS_GROUP. You can request an exit by sending a Ctrl+Break event to the group via os.kill(p.pid, signal.CTRL_BREAK_EVENT) [1]. The request might be ignored, but typically the default handler is called, which calls ExitProcess(). To handle GUI apps, assuming the child inherits the current desktop (usually "WinSta0\Default"), first enumerate the top-level and message-only windows on the current desktop via EnumWindows() and FindWindowExW(). Use GetWindowThreadProcessId() to filter the list to include only windows that belong to the job. Post WM_CLOSE to each window in the job. A process might ignore a request to close. It could keep the window open or continue running in the background. After an internal timeout, you can call TerminateJobObject() to kill any process in the job that remains alive. This is a forced and abrupt termination, which is similar to sending SIGKILL to a process group in POSIX. --- [1] This usage of os.kill() is what we're stuck with. Rightfully, we should be using os.killpg(p.pid, signal.SIGBREAK) or os.kill(-p.pid, signal.SIGBREAK) (note the negative pid value). -- ___ Python tracker <https://bugs.python.org/issue46716> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46703] boolean operation issue (True == False == False)
Eryk Sun added the comment: > True == False == False is really True == False and False == False > wich is False and True which is False Moreover, since the left-hand comparison is `True == False`, which evaluates to False, the right-hand comparison doesn't even get evaluated. In the following example, print(3) doesn't get called because the left-hand comparison, `None != None`, is False: >>> print(1) != print(2) == print(3) 1 2 False -- nosy: +eryksun ___ Python tracker <https://bugs.python.org/issue46703> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46697] _ctypes_simple_instance returns inverted logic
Change by Eryk Sun : -- stage: -> patch review versions: -Python 3.7, Python 3.8 ___ Python tracker <https://bugs.python.org/issue46697> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46686] [venv / PC/launcher] issue with a space in the installed python path
Eryk Sun added the comment:
The venv launcher can quote the executable path either always or only when it
contains spaces. The following is a suggestion for implementing the latter.
Before allocating `executable`, use memchr() (include ) to search the
UTF-8 source for a space. If found, increment the character count by two. After
allocating `executable`, add the initial quote character, and increment the
pointer.
BOOL add_quotes = FALSE;
if (memchr(start, ' ', (size_t)len) != NULL) {
add_quotes = TRUE;
cch += 2;
}
executable = (wchar_t *)malloc(cch * sizeof(wchar_t));
if (executable == NULL) {
error(RC_NO_MEMORY, L"A memory allocation failed");
}
if (add_quotes) {
*executable++ = L'\"';
}
Later, after checking the existence via GetFileAttributesW(), add the trailing
quote and null, and decrement the pointer:
if (GetFileAttributesW(executable) == INVALID_FILE_ATTRIBUTES) {
error(RC_NO_PYTHON, L"No Python at '%ls'", executable);
}
if (add_quotes) {
size_t n = wcslen(executable);
executable[n] = L'\"';
executable[n + 1] = L'\0';
executable--;
}
--
___
Python tracker
<https://bugs.python.org/issue46686>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46686] [venv / PC/launcher] issue with a space in the installed python path
Eryk Sun added the comment: I checked the source code in PC/launcher.c process(). It turns out that `executable` is not getting quoted in the venv launcher case. CreateProcessW() tries to get around this. If the command isn't quoted, it has a loop that consumes up to a space (or tab) and checks for an existing file (not a directory). If it finds a file, it rewrites the command line to quote the path of the file. My test happened to work. But it's simple enough to create an example that fails. For example, as an elevated admin, create a file named "C:\Program". Now the venv launcher won't be able to execute a base interpreter that's installed in "C:\Program Files": C:\Temp>echo >C:\Program C:\Temp>"C:\Program Files\Python310\python.exe" -m venv env Error: Command '['C:\\Temp\\env\\Scripts\\python.exe', '-Im', 'ensurepip', '--upgrade', '--default-pip']' returned non-zero exit status 101. -- priority: normal -> critical stage: -> needs patch versions: +Python 3.11 ___ Python tracker <https://bugs.python.org/issue46686> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46686] [venv / PC/launcher] issue with a space in the installed python path
Eryk Sun added the comment: run_child() expects `cmdline` to be correctly quoted, and normally it is. I can't reproduce this problem with Python 3.10.2. I created a user account with a space in the account name, logged on, and installed 3.10.2 for the current user, with the option enabled to add Python to PATH. Next I opened a command prompt in the user profile directory and created a virtual environment via `python.exe -m venv .venv`. Running ".venv\Scripts\python.exe -X utf8" worked. The command line used by the venv "python.exe" launcher was properly quoted and thus properly parsed in the original list of command-line arguments, sys.orig_argv. -- nosy: +eryksun ___ Python tracker <https://bugs.python.org/issue46686> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46659] Deprecate locale.getdefaultlocale() function
Eryk Sun added the comment: > Oh. Serhiy asked me to use LC_TIME rather than LC_CTYPE. Since Locale*Calendar is documented as not being thread safe, __init__() could get the real default via setlocale(LC_TIME, "") when locale=None and the current LC_TIME is "C". Restore it back to "C" after getting the default. That should usually match the behavior from previous versions that called getdefaultlocale(). In cases where it differs, it's fixing a bug because the default LC_TIME is the correct default. -- ___ Python tracker <https://bugs.python.org/issue46659> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46659] Deprecate locale.getdefaultlocale() function
Eryk Sun added the comment:
> getdefaultlocale() falls back to LANG and LANGUAGE.
_Py_SetLocaleFromEnv(LC_CTYPE) (e.g. setlocale(LC_CTYPE, "")) gets called at
startup, except for the isolated configuration [1].
I think calendar.Locale*Calendar should try the LC_CTYPE locale if LC_TIME is
"C", i.e. (None, None). Otherwise, it's introducing new default behavior. For
example, with LC_ALL set to "ru_RU.utf8":
3.8:
>>> locale.getlocale(locale.LC_TIME)
(None, None)
>>> locale.getlocale(locale.LC_CTYPE)
('ru_RU', 'UTF-8')
>>> cal = calendar.LocaleTextCalendar()
>>> cal.formatweekday(0, 15)
' Понедельник '
3.11.0a5+:
>>> locale.getlocale(locale.LC_TIME)
(None, None)
>>> locale.getlocale(locale.LC_CTYPE)
('ru_RU', 'UTF-8')
>>> cal = calendar.LocaleTextCalendar()
>>> cal.formatweekday(0, 15)
' Monday'
>>> locale.setlocale(locale.LC_TIME, '')
'ru_RU.utf8'
>>> cal = calendar.LocaleTextCalendar()
>>> cal.formatweekday(0, 15)
' Понедельник '
---
[1] https://docs.python.org/3/c-api/init_config.html?#isolated-configuration
--
nosy: +eryksun
___
Python tracker
<https://bugs.python.org/issue46659>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46668] encodings: the "mbcs" alias doesn't work
Eryk Sun added the comment:
> I don't think that this fallback is needed anymore. Which Windows
> code page can be used as ANSI code page which is not already
> implemented as a Python codec?
Python has full coverage of the ANSI and OEM code pages in the standard Windows
locales, but I don't have any experience with custom (i.e. supplemental or
replacement) locales.
https://docs.microsoft.com/en-us/windows/win32/intl/custom-locales
Here's a simple script to check the standard locales.
import codecs
import ctypes
kernel32 = ctypes.WinDLL('kernel32', use_last_error=True)
LOCALE_ALL = 0
LOCALE_WINDOWS = 1
LOCALE_IDEFAULTANSICODEPAGE = 0x1004
LOCALE_IDEFAULTCODEPAGE = 0x000B # OEM
EnumSystemLocalesEx = kernel32.EnumSystemLocalesEx
GetLocaleInfoEx = kernel32.GetLocaleInfoEx
GetCPInfoExW = kernel32.GetCPInfoExW
EnumLocalesProcEx = ctypes.WINFUNCTYPE(ctypes.c_int,
ctypes.c_wchar_p, ctypes.c_ulong, ctypes.c_void_p)
class CPINFOEXW(ctypes.Structure):
_fields_ = (('MaxCharSize', ctypes.c_uint),
('DefaultChar', ctypes.c_ubyte * 2),
('LeadByte', ctypes.c_ubyte * 12),
('UnicodeDefaultChar', ctypes.c_wchar),
('CodePage', ctypes.c_uint),
('CodePageName', ctypes.c_wchar * 260))
def get_all_locale_code_pages():
result = []
seen = set()
info = (ctypes.c_wchar * 100)()
@EnumLocalesProcEx
def callback(locale, flags, param):
for lctype in (LOCALE_IDEFAULTANSICODEPAGE,
LOCALE_IDEFAULTCODEPAGE):
if (GetLocaleInfoEx(locale, lctype, info, len(info)) and
info.value not in ('0', '1')):
cp = int(info.value)
if cp in seen:
continue
seen.add(cp)
cp_info = CPINFOEXW()
if not GetCPInfoExW(cp, 0, ctypes.byref(cp_info)):
cp_info.CodePage = cp
cp_info.CodePageName = str(cp)
result.append(cp_info)
return True
if not EnumSystemLocalesEx(callback, LOCALE_WINDOWS, None, None):
raise ctypes.WinError(ctypes.get_last_error())
result.sort(key=lambda x: x.CodePage)
return result
supported = []
unsupported = []
for cp_info in get_all_locale_code_pages():
cp = cp_info.CodePage
try:
codecs.lookup(f'cp{cp}')
except LookupError:
unsupported.append(cp_info)
else:
supported.append(cp_info)
if unsupported:
print('Unsupported:\n')
for cp_info in unsupported:
print(cp_info.CodePageName)
print('\nSupported:\n')
else:
print('All Supported:\n')
for cp_info in supported:
print(cp_info.CodePageName)
Output:
All Supported:
437 (OEM - United States)
720 (Arabic - Transparent ASMO)
737 (OEM - Greek 437G)
775 (OEM - Baltic)
850 (OEM - Multilingual Latin I)
852 (OEM - Latin II)
855 (OEM - Cyrillic)
857 (OEM - Turkish)
862 (OEM - Hebrew)
866 (OEM - Russian)
874 (ANSI/OEM - Thai)
932 (ANSI/OEM - Japanese Shift-JIS)
936 (ANSI/OEM - Simplified Chinese GBK)
949 (ANSI/OEM - Korean)
950 (ANSI/OEM - Traditional Chinese Big5)
1250 (ANSI - Central Europe)
1251 (ANSI - Cyrillic)
1252 (ANSI - Latin I)
1253 (ANSI - Greek)
1254 (ANSI - Turkish)
1255 (ANSI - Hebrew)
1256 (ANSI - Arabic)
1257 (ANSI - Baltic)
1258 (ANSI/OEM - Viet Nam)
Some locales are Unicode only (e.g. Hindi-India) or have no OEM code page,
which the above code skips by checking for "0" or "1" as the code page value.
Windows 10+ allows setting the system locale to a Unicode-only locale, for
which it uses UTF-8 (65001) for ANSI and OEM.
The OEM code page matters because the console input and output code pages
default to OEM, e.g. for os.device_encoding(). The console's I/O code pages are
used in Python by low-level os.read() and os.write(). Note that the console
doesn't properly implement using UTF-8 (65001) as the input code page. In this
case, input read from the console via ReadFile() or ReadConsoleA() has a null
byte in place of each non-ASCII character.
--
___
Python tracker
<https://bugs.python.org/issue46668>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46668] encodings: the "mbcs" alias doesn't work
Eryk Sun added the comment: > The Python 3.6 and 3.7 "codecs.register(_alias_mbcs)" doesn't work > because "search_function()" is tested before and it works for "cpXXX" > encodings. Isn't the 3.6-3.10 ordering of search_function() and _alias_mbcs() correct as a fallback? In this case, Python doesn't support a cross-platform encoding for the code page. That's why the old implementation of test_mbcs_alias() mocked _winapi.GetACP() to return 123 and then checked that looking up 'cp123' returned the "mbcs" codec. I'd actually prefer to extend this by implementing _winapi.GetOEMCP() and using "oem" as a fallback for that case. -- nosy: +eryksun ___ Python tracker <https://bugs.python.org/issue46668> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46654] urllib.request.urlopen doesn't handle UNC paths produced by pathlib's as_uri() (but can handle UNC paths with additional slashes)
Change by Eryk Sun : -- assignee: docs@python -> components: -2to3 (2.x to 3.x conversion tool), Argument Clinic, Build, C API, Cross-Build, Demos and Tools, Distutils, Documentation, Extension Modules, FreeBSD, IDLE, IO, Installation, Interpreter Core, Parser, Regular Expressions, SSL, Subinterpreters, Tests, Tkinter, Unicode, Windows, XML, asyncio, ctypes, email, macOS stage: -> needs patch type: performance -> behavior versions: -Python 3.7, Python 3.8 ___ Python tracker <https://bugs.python.org/issue46654> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46654] urllib.request.urlopen doesn't handle UNC paths produced by pathlib's as_uri() (but can handle UNC paths with additional slashes)
Eryk Sun added the comment:
> The value of req.selector never starts with "//", for which file_open()
> checks, but rather a single slash, such as "/Z:/test.py" or
> "/share/test.py".
To correct myself, actually req.selector will start with "//" for a "file:"
URI, such as "file:host/share/test.py". For this example, req.host is an
empty string, so file_open() still ends up calling open_local_file(), which
will open "//host/share/test.py". In Linux, "//host/share" is the same as
"/host/share". In Cygwin and MSYS2 it's a UNC path. I guess this case should be
allowed, even though the meaning of a "//" root isn't specifically defined in
POSIX.
Unless I'm overlooking something, file_open() only has to check the value of
req.host. In POSIX, it should require opening a 'local' path, i.e. if req.host
isn't None, empty, or a local host, raise URLError.
In Windows, my tests show that the shell API special cases "localhost" (case
insensitive) in "file:" URIs. For example, the following are all equivalent:
"file:/C:/Temp", "file:///C:/Temp", and "file://localhost/C:/Temp". The shell
API does not special case the real local host name or any of its IP addresses,
such as 127.0.0.1. They're all handled as UNC paths.
Here's what I've experimented with thus far, which passes the existing urllib
tests in Linux and Windows:
class FileHandler(BaseHandler):
def file_open(self, req):
if not self._is_local_path(req):
if sys.platform == 'win32':
path = url2pathname(f'//{req.host}{req.selector}')
else:
raise URLError("In POSIX, the file:// scheme is only "
"supported for local file paths.")
else:
path = url2pathname(req.selector)
return self._common_open_file(req, path)
def _is_local_path(self, req):
if req.host:
host, port = _splitport(req.host)
if port:
raise URLError(f"the host cannot have a port: {req.host}")
if host.lower() != 'localhost':
# In Windows, all other host names are UNC.
if sys.platform == 'win32':
return False
# In POSIX, support all names for the local host.
if _safe_gethostbyname(host) not in self.get_names():
return False
return True
# names for the localhost
names = None
def get_names(self):
if FileHandler.names is None:
try:
FileHandler.names = tuple(
socket.gethostbyname_ex('localhost')[2] +
socket.gethostbyname_ex(socket.gethostname())[2])
except socket.gaierror:
FileHandler.names = (socket.gethostbyname('localhost'),)
return FileHandler.names
def open_local_file(self, req):
if not self._is_local_path(req):
raise URLError('file not on local host')
return self._common_open_file(req, url2pathname(req.selector))
def _common_open_file(self, req, path):
import email.utils
import mimetypes
host = req.host
filename = req.selector
try:
if host:
origurl = f'file://{host}{filename}'
else:
origurl = f'file://{filename}'
stats = os.stat(path)
size = stats.st_size
modified = email.utils.formatdate(stats.st_mtime, usegmt=True)
mtype = mimetypes.guess_type(filename)[0] or 'text/plain'
headers = email.message_from_string(
f'Content-type: {mtype}\n'
f'Content-length: {size}\n'
f'Last-modified: {modified}\n')
return addinfourl(open(path, 'rb'), headers, origurl)
except OSError as exp:
raise URLError(exp)
Unfortunately nturl2path.url2pathname() parses some UNC paths incorrectly. For
example, the following path should be an invalid UNC path, since "C:" is an
invalid name, but instead it gets converted into an unrelated local path.
>>> nturl2path.url2pathname('//host/C:/Temp/spam.txt')
'C:\\Temp\\spam.txt'
This goof depends on finding ":" or "|" in the path. It's arguably worse if the
last component has a named data stream (allowed by RFC 8089):
>>> nturl2path.url2pathname('//host/share/spam.txt:eggs')
'T:\\eggs'
Drive "T:" is from "t:" in "t:eggs",
[issue46654] urllib.request.urlopen doesn't handle UNC paths produced by pathlib's resolve() (but can handle UNC paths with additional slashes)
Eryk Sun added the comment: > file://server/host/file.ext on windows, even though > file:server/host/file.ext open just fine. For r"\\host\share\test.py", the two slash conversion "file://host/share/test.py" is correct according to RFC80889 "E.3.1. URI with Authority" [1]. In this case, req.host is "host", and req.selector is "/share/test.py". The four slash version "file:host/share/test.py" is a known variant for a converted UNC path, as noted in RFC8089 "E.3.2. URI with UNC Path". In this case, req.host is an empty string, and req.selector is "//host/share/test.py". There's another variant that uses 5 slashes for a UNC path, but urllib (or url2pathname) doesn't support it. --- [1] https://datatracker.ietf.org/doc/html/rfc8089 -- ___ Python tracker <https://bugs.python.org/issue46654> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46654] urllib.request.urlopen doesn't handle UNC paths produced by pathlib's resolve() (but can handle UNC paths with additional slashes)
Eryk Sun added the comment: In FileHandler.file_open(), req.host is the host name, which is either None or an empty string for a local drive path such as, respectively, "file:/Z:/test.py" or "file:///Z:/test.py". The value of req.selector never starts with "//", for which file_open() checks, but rather a single slash, such as "/Z:/test.py" or "/share/test.py". This is a bug in file_open(). Due to this bug, it always calls self.open_local_file(req), even if req.host isn't local. The distinction shouldn't matter in Windows, which supports UNC paths, but POSIX has to open a path on the local machine (possibly a mount point for a remote path, but that's irrelevant). In POSIX, if the local machine coincidentally has the req.selector path, then the os.stat() and open() calls will succeed with a bogus result. For "file://host/share/test.py", req.selector is "/share/test.py". In Windows, url2pathname() converts this to r"\share\test.py", which is relative to the drive of the process current working directory. This is a bug in open_local_file() on Windows. For it to work correctly, req.host has to be joined back with req.selector as the UNC path "//host/share/test.py". Of course, this need not be a local file in Windows, so Windows should be exempted from the local file limitation in file_open(). -- ___ Python tracker <https://bugs.python.org/issue46654> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46654] file_open doesn't handle UNC paths produced by pathlib's resolve() (but can handle UNC paths with additional slashes)
Change by Eryk Sun : -- Removed message: https://bugs.python.org/msg412604 ___ Python tracker <https://bugs.python.org/issue46654> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46654] file_open doesn't handle UNC paths produced by pathlib's resolve() (but can handle UNC paths with additional slashes)
Eryk Sun added the comment:
Builtin open() calls C open(). This C function supports whatever path types are
supported natively. In Windows, C open() calls WinAPI CreateFileW(), which does
not support "file://" URIs. The Windows API handles it as a relative path,
which gets resolved against the current working directory. For example:
>>> os.getcwd()
'C:\\Temp'
>>> nt._getfullpathname('file:host/share/file')
'C:\\Temp\\file:\\host\\share\\file'
>>> nt._getfullpathname('file://host/share/file')
'C:\\Temp\\file:\\host\\share\\file'
As to the resolved path somehow working, that generally will not be the case.
Most filesystems in Windows will reject a path component named "file:" as an
invalid name. The ":" character is usually disallowed in base file and
directory names, since some Windows filesystems use it as a delimiter in file
streams, e.g. "name:stream_name:stream_type". The default data stream in a
regular file has no stream name, and its stream type is "$DATA". Thus for base
name "file", the default data stream can be referenced explicitly as
"file::$DATA". But just "file:", with neither a stream name nor a stream type,
is an invalid name.
--
nosy: +eryksun
___
Python tracker
<https://bugs.python.org/issue46654>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46631] Implement a "strict" mode for getpass.getuser()
Eryk Sun added the comment:
Here's an example for the suggested changes to _winapi.
Include these headers:
#include // LsaGetLogonSessionData
#include // STATUS_SUCCESS
Add these argument-clinic macros to _winapi_functions:
_WINAPI_GETCURRENTPROCESSTOKEN_METHODDEF
_WINAPI_GETTOKENINFORMATION_METHODDEF
_WINAPI_LSAGETLOGONSESSIONDATA_METHODDEF
Add TokenStatistics in winapi_exec():
WINAPI_CONSTANT(F_DWORD, TokenStatistics);
Add minimal implementations that wrap the WinAPI functions:
/*[clinic input]
_winapi.GetCurrentProcessToken -> HANDLE
Return a handle for the access token of the current process.
[clinic start generated code]*/
static HANDLE
_winapi_GetCurrentProcessToken_impl(PyObject *module)
/*[clinic end generated code: output=cf8e8e20dd41dd6e
input=73a282cf3718af9e]*/
{
return GetCurrentProcessToken();
}
/*[clinic input]
_winapi.GetTokenInformation
handle: HANDLE
information_class: unsigned_long
/
Get information from an access token.
[clinic start generated code]*/
static PyObject *
_winapi_GetTokenInformation_impl(PyObject *module, HANDLE handle,
unsigned long information_class)
/*[clinic end generated code: output=caecec0a25658348
input=b277ad2414f1b03e]*/
{
if (information_class != TokenStatistics) {
return PyErr_Format(
PyExc_NotImplementedError,
"Unsupported information class: %d",
information_class);
}
DWORD returned_size;
TOKEN_STATISTICS info;
if (!GetTokenInformation(handle, information_class, , sizeof(info),
_size)) {
return PyErr_SetFromWindowsErr(0);
}
PyObject *result = PyDict_New();
if (!result) {
return NULL;
}
PyObject *value = PyLong_FromUnsignedLongLong(
(((uint64_t)info.AuthenticationId.HighPart) << 32) +
info.AuthenticationId.LowPart);
if (!value) {
goto error;
}
if (PyDict_SetItemString(result, "AuthenticationId", value) < 0) {
Py_DECREF(value);
goto error;
}
Py_DECREF(value);
return result;
error:
Py_CLEAR(result);
return NULL;
}
/*[clinic input]
_winapi.LsaGetLogonSessionData
logon_id: unsigned_long_long
/
Get data for the logon session identified by logon_id.
[clinic start generated code]*/
static PyObject *
_winapi_LsaGetLogonSessionData_impl(PyObject *module,
unsigned long long logon_id)
/*[clinic end generated code: output=680ac7725ef34527
input=01ff4216b89d01ef]*/
{
SECURITY_LOGON_SESSION_DATA *pdata;
LUID logon_luid;
logon_luid.HighPart = logon_id >> 32;
logon_luid.LowPart = logon_id & 0x;
NTSTATUS status = LsaGetLogonSessionData(_luid, );
if (status != STATUS_SUCCESS) {
return PyErr_SetFromWindowsErr(LsaNtStatusToWinError(status));
}
PyObject *result = PyDict_New();
if (!result) {
goto error;
}
PyObject *value = PyUnicode_FromWideChar(pdata->UserName.Buffer,
pdata->UserName.Length / sizeof(WCHAR));
if (!value) {
goto error;
}
if (PyDict_SetItemString(result, "UserName", value) < 0) {
Py_DECREF(value);
goto error;
}
Py_DECREF(value);
LsaFreeReturnBuffer(pdata);
return result;
error:
LsaFreeReturnBuffer(pdata);
Py_CLEAR(result);
return NULL;
}
--
___
Python tracker
<https://bugs.python.org/issue46631>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46631] Implement a "strict" mode for getpass.getuser()
New submission from Eryk Sun :
getpass.getuser() checks the environment variables LOGNAME (login name), USER,
LNAME, and USERNAME, in that order. In Windows, LOGNAME, USER, and LNAME have
no conventional usage. I think there should be a strict mode that restricts
getuser() to check only USERNAME in Windows and only LOGNAME in POSIX [1]. If
the login variable isn't defined, it should fall back on using the system API,
based on the user ID in POSIX and the logon ID in Windows.
For the fallback in Windows, the _winapi module could implement
GetCurrentProcessToken(), GetTokenInformation(), and LsaGetLogonSessionData().
For TokenStatistics, return a dict with just "AuthenticationId". For
LsaGetLogonSessionData(), return a dict with just "UserName".
GetCurrentProcessToken() returns a pseudohandle (-4), which should not be
closed.
For example, assuming _winapi wraps the required functions:
def getuser(strict=False):
"""Get the username from the environment or password database.
First try various environment variables. If strict, check only LOGNAME
in POSIX and only USERNAME in Windows. As a fallback, in POSIX get the
user name from the password database, and in Windows get the user name
from the logon-session data of the current process.
"""
posix = sys.platform != 'win32'
if strict:
names = ('LOGNAME',) if posix else ('USERNAME',)
else:
names = ('LOGNAME', 'USER', 'LNAME', 'USERNAME')
for name in names:
if user := os.environ.get(name):
return user
if posix:
import pwd
return pwd.getpwuid(os.getuid())[0]
import _winapi
logon_id = _winapi.GetTokenInformation(
_winapi.GetCurrentProcessToken(),
_winapi.TokenStatistics)['AuthenticationId']
return _winapi.LsaGetLogonSessionData(logon_id)['UserName']
Like WinAPI GetUserNameW(), the above fallback returns the logon user name
instead of the account name of the token user. As far as I know, the user name
and the account name only differ for the builtin service account logons
"SYSTEM" (999) and "NETWORK SERVICE" (996), for which the user name is the
machine security principal (i.e. the machine's NETBIOS name plus "$"). The user
name of the builtin "LOCAL SERVICE" logon (997), on the other hand, is just the
"LOCAL SERVICE" account name, since this account lacks network access.
Unlike GetUserNameW(), the above code uses the process token instead of the
effective token. This is like POSIX getuid(), whereas what GetUserNameW() does
is like geteuid(). getuser() could implement an `effective` option to return
the effective user name. In Windows this would switch to calling
GetCurrentThreadEffectiveToken() instead of GetCurrentProcessToken().
---
[1] https://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap08.html
--
components: Library (Lib), Windows
messages: 412495
nosy: eryksun, paul.moore, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
stage: needs patch
status: open
title: Implement a "strict" mode for getpass.getuser()
type: enhancement
versions: Python 3.11
___
Python tracker
<https://bugs.python.org/issue46631>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46490] Add "follow_symlinks=False" support for "os.utime()" on Windows
Eryk Sun added the comment:
In case you missed it, I implemented _Py_CreateFile2() in bpo-46506 and rewrote
os.stat() based on it. Check it out in case you're interested in moving forward
with a PR in bpo-46506.
For this issue, follow_symlinks is fairly simple to support with
_Py_CreateFile2(). We may as well add fd support, since that's trivial to add.
For example:
if (path->fd != -1) {
hFile = _Py_get_osfhandle(path->fd);
} else {
Py_BEGIN_ALLOW_THREADS
hFile = _Py_CreateFile2(path->wide, FILE_WRITE_ATTRIBUTES, 0,
OPEN_EXISTING, NULL, follow_symlinks, NULL);
Py_END_ALLOW_THREADS
}
if (hFile == INVALID_HANDLE_VALUE) {
if (path->fd == -1) {
path_error(path);
}
return NULL;
}
One also has to define the following macros to declare follow_symlinks and fd
support: UTIME_HAVE_NOFOLLOW_SYMLINKS and PATH_UTIME_HAVE_FD.
To announce support in os.supports_follow_symlinks and os.supports_fd, it
should be conditioned on MS_WINDOWS, i.e. _add("MS_WINDOWS", "utime"). The os
module is frozen, so changing these two sets requires rebuilding Python.
--
___
Python tracker
<https://bugs.python.org/issue46490>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46594] Windows "Edit with IDLE >" only has one selection
Eryk Sun added the comment:
> the Open With entries may not be being merged.
That would probably be a bug in the Windows shell API. The HKCU and HKLM
subkeys of "Software\Classes\Python.File\Shell\editwithidle\shell" are merged
in the HKCR view. The same key path can exist in both hives, for which the view
contains the union, with precedence for HKCU.
For example, with "Software\Classes\spam":
import winreg
hkm = winreg.CreateKey(winreg.HKEY_LOCAL_MACHINE, r'Software\Classes\spam')
winreg.SetValueEx(hkm, 'eggs', 0, winreg.REG_SZ, 'hklm')
winreg.SetValueEx(hkm, 'baz', 0, winreg.REG_SZ, 'hklm')
hku = winreg.CreateKey(winreg.HKEY_CURRENT_USER, r'Software\Classes\spam')
winreg.SetValueEx(hku, 'eggs', 0, winreg.REG_SZ, 'hkcu')
winreg.SetValueEx(hku, 'bam', 0, winreg.REG_SZ, 'hkcu')
hkr = winreg.OpenKey(winreg.HKEY_CLASSES_ROOT, 'spam')
>>> winreg.EnumValue(hkr, 0)
('bam', 'hkcu', 1)
>>> winreg.EnumValue(hkr, 1)
('baz', 'hklm', 1)
>>> winreg.EnumValue(hkr, 2)
('eggs', 'hkcu', 1)
>>> winreg.EnumValue(hkr, 3)
Traceback (most recent call last):
File "", line 1, in
OSError: [WinError 259] No more data is available
--
___
Python tracker
<https://bugs.python.org/issue46594>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46594] Windows "Edit with IDLE >" only has one selection
Eryk Sun added the comment: Check your settings in the registry. In "HKCU\Software\Microsoft\Windows\CurrentVersion\Explorer\FileExts" there should be a ".py" key, but not necessarily. It should have an "OpenWithList" subkey that contains an "MRUList" value (most recently used list). The value should be a sequence of letters, each of which should be a value name in the key. If the launcher was last used to open a ".py" file, the first letter in the list should be a value with the data "py.exe". Ideally there should also be a subkey named "UserChoice" that contains a "ProgId" (programmatic identifier) value with the data "Python.File". This sets the "Python.File" ProgID as the locked-in user choice for ".py" files. In the GUI, you can set this in the open-with dialog by selecting "always use this app to open .py files". The selected app should be "Python", with an icon that contains the Python logo and a rocket (the launcher). If ".py" isn't the locked-in user choice, the shell API will use the most recent user selection in the open-with menu. If there's no user selection, the default association is calculated from "HKCR\.py", which is a merged view of "[HKCU|HKLM]\Software\Classes\.py". The default value of "HKCR\.py" sets the default file association. Ideally it should be "Python.File". "HKCR\Python.File" is a merged view of "[HKCU|HKLM]\Software\Classes\Python.File". For the merged view, if a value name is defined in the same subkey of HKCU and HKLM, the view prefers the HKCU value. There should be a subkey named "shell\editwithidle\shell". It should define one or more subkeys named "edit3*", such as "edit310". Each should contain a "MUIVerb" value that sets the command description in the "open-with" menu. There should also be a "command" subkey that contains the template command as its default value, e.g. ""C:\Program Files\Python310\pythonw.exe" -m idlelib "%L" %*". -- nosy: +eryksun ___ Python tracker <https://bugs.python.org/issue46594> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue29688] Add support for Path.absolute()
Eryk Sun added the comment:
> I'd imagine that bug is reproducible with `Path('C:\\Temp', 'C:')`
> already, right? If that's the case, should it logged as a
> separate issue?
Yes, it's a separate issue that affects the _from_parts() call in absolute().
How about designing absolute() to create a new instance from an absolute path
that's created by os.path? For example:
join(abspath(self.drive) if self.drive else getcwd(), self)
Of course use accessor functions.
--
___
Python tracker
<https://bugs.python.org/issue29688>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue29688] Add support for Path.absolute()
Eryk Sun added the comment: > I'm not seeing what's wrong with your example. "C:" or "C:spam\\eggs" are not absolute paths. They depend on the effective working directory on the drive. An absolute path should never depend on a working directory, which can change at random. WinAPI SetEnvironmentVariableW() allows applications to set environment variables with names that begin with "=". These names are effectively reserved for special use by the OS, at least as documented. In particular, names of the form "=X:", where "X" is a drive letter, are used to store the working directory on a drive. The C runtime _[w]chdir() function sets these per-drive environment variables, as does Python's os.chdir(). As environment variables, they can be inherited by child processes. When then Windows API resolves a file path to access a file, or in GetFullPathNameW(), a drive-relative path such as "X:" or "X:spam\\eggs" is resolved against either the current working directory (if it's on the drive) or the value of the "=X:" environment variable for the drive. If the latter isn't defined, it defaults to the root directory, e.g. "X:\\". If the current working directory is on the drive, the system updates the value of the "=X:" environment variable, if it exists. > on Windows you have to resolve the drive separately from the > working directory and then concatenate them? No, if self.drive is defined, then abspath(self.drive) should be called instead of getcwd(). In Windows, ntpath.abspath() calls WinAPI GetFullPathNameW(), which resolves the working directory on the drive. -- ___ Python tracker <https://bugs.python.org/issue29688> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue29688] Add support for Path.absolute()
Change by Eryk Sun : -- Removed message: https://bugs.python.org/msg412220 ___ Python tracker <https://bugs.python.org/issue29688> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue29688] Add support for Path.absolute()
Eryk Sun added the comment: > I'm not seeing what's wrong with your example. "C:" or "C:spam\\eggs" are not absolute paths. They depend on the effective working directory on the drive. An absolute path should never depend on a working directory, which can change at random. WinAPI SetEnvironmentVariableW() allows applications to set environment variables with names that begin with "=". These names are effectively reserved for special use by the OS, at least as documented. In particular, names of the form "=X:", where "X" is a drive letter, are used to store the working directory on a drive. The C runtime _[w]chdir() function sets these per-drive environment variables, as does Python's os.chdir(). As environment variables, they can be inherited by child processes. When then Windows API resolves a file path to access a file, or in GetFullPathNameW(), a drive-relative path such as "X:" or "X:spam\\eggs" is resolved against either the current working directory (if it's on the drive) or the value of the "=X:" environment variable for the drive. If the latter isn't defined, it defaults to the root directory, e.g. "X:\\". If the current working directory is on the drive, the system updates the value of the "=X:" environment variable, if it exists. > on Windows you have to resolve the drive separately from the > working directory and then concatenate them? Yes, if self.drive is defined. This would be handled by os.path.abspath(self.drive), which calls WinAPI GetFullPathNameW(). -- ___ Python tracker <https://bugs.python.org/issue29688> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
