[issue34689] Lib/sysconfig.py expands non-variables
Change by Erwan Le Pape : -- pull_requests: +19696 pull_request: https://github.com/python/cpython/pull/20439 ___ Python tracker <https://bugs.python.org/issue34689> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue35545] asyncio.base_events.create_connection doesn't handle scoped IPv6 addresses
Erwan Le Pape added the comment:
Assuming similar configuration to the one in msg343430, a simple native
getaddrinfo test to check whether any scope ID is returned.
```
#include
#include
#include
#include
void test(char *addrstr) {
int status;
struct addrinfo *res;
struct addrinfo *iter;
struct sockaddr_in6 *addr;
status = getaddrinfo(addrstr, "80", NULL, );
if (status != 0) {
fprintf(stderr, "getaddrinfo(%s) returned %i\n", addrstr, status);
return;
}
for (iter = res; iter; iter = iter->ai_next) {
if (iter->ai_addr->sa_family != AF_INET6)
continue;
addr = (struct sockaddr_in6 *) iter->ai_addr;
if (addr->sin6_scope_id != 0) {
fprintf(stdout, "getaddrinfo(%s) return scope %u\n", addrstr,
addr->sin6_scope_id);
return;
}
}
}
int main() {
test("fe80::f8d1:81ff:fe81:ac05%2");
test("fe80::f8d1:81ff:fe81:ac05%en1");
return 0;
}
```
I've explicitly tested against numeric and named interfaces to ensure that this
isn't linked to AIX only handling named interfaces for scopes instead of
numeric ones (although given your `netstat` output, that's be surprising).
Since I've had to look for AIX programming documentation just to be sure, I
also stumbled upon this AIX bug
https://www-01.ibm.com/support/docview.wss?uid=isg1IV53671 (which is referenced
by the one I mentioned previously but I missed that). It seem to apply up to
7100-03 so you should be immune nonetheless.
I also noticed that it mentions SSH not working so I went and checked the
OpenSSH sources to see how they handle AIX.
While they have an explicit BROKEN_GETADDRINFO define, it doesn't check for the
specific scoped IPv6 address issue so I'm not sure they decided to special case
it.
https://github.com/openssh/openssh-portable/blob/85ceb0e64bff672558fc87958cd548f135c83cdd/configure.ac#L2341
If this is truly an "idiosyncrasy" in AIX, I'm not sure there is a better way
to handle it than skipping it since it's not really a Python bug if the
underlying `libc` doesn't work as intended.
--
___
Python tracker
<https://bugs.python.org/issue35545>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue35545] asyncio.base_events.create_connection doesn't handle scoped IPv6 addresses
Erwan Le Pape added the comment: Thanks for testing that. It's good that you used an actual address because that eliminates the possibility that AIX doesn't handle addresses it doesn't really know about. On the other hand, even when properly specified to a real scoped IPv6 address, `getaddrinfo` doesn't seem to get the necessary scope ID from the underlying C call which socket.getaddrinfo > _socket.getaddrinfo is pretty much mapped to. I'm looking at cpython/master for the socketmodule implementation: https://github.com/python/cpython/blob/6dbbe748e101a173b4cff8aada41e9313e287e0f/Modules/socketmodule.c#L6400 is `getaddrinfo` https://github.com/python/cpython/blob/master/Modules/socketmodule.c#L1294 is `makesockaddr` which actually creates the 4-tuple returned as the last element of the `getaddrinfo` tuples. The fourth element (ie. the scope ID) is clearly `a->sin6_scope_id` which should contain the scope ID. At this stage, I don't know if this is a bug from the socketmodule which I doubt or if the AIX `getaddrinfo` simply just doesn't handle scoped IP addresses properly. If you're still okay to proxy tests for AIX, I'll try and come up with either a simple C snippet to see what's in the returned structure or ctype the AIX `libc` `getaddrinfo`. -- ___ Python tracker <https://bugs.python.org/issue35545> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue35545] asyncio.base_events.create_connection doesn't handle scoped IPv6 addresses
Erwan Le Pape added the comment:
I don't have an AIX lying around to test so would you mind just running the
test on `getaddrinfo` for AIX. A simple `python3 -c 'import socket;
print(socket.getaddrinfo("fe80::1%1", 80))'` should fairly rapidly determine if
there is a legitimate reason for the test to fail (ie. this is internal to
`asyncio`) or if this is tied to the underlying AIX `getaddrinfo`.
The IPv6 Scoped Address Architecture RFC clearly indicates that `%`
should be supported although it isn't a must. Hopefully there's a subtlety to
`getaddrinfo` on AIX (maybe in the way the zone should be specified, I already
had to fallback to numeric interfaces so the test would work on both Linux &
Windows, I wouldn't be surprised if AIX had yet another syntax for it).
Also, it would be worthwhile to ensure that the patches mentioned by IBM
https://www-01.ibm.com/support/docview.wss?uid=isg1IV52116 are applied on the
machine running the test.
--
___
Python tracker
<https://bugs.python.org/issue35545>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue35545] asyncio.base_events.create_connection doesn't handle scoped IPv6 addresses
Change by Erwan Le Pape : -- keywords: +patch pull_requests: +10507 stage: -> patch review ___ Python tracker <https://bugs.python.org/issue35545> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue35545] asyncio.base_events.create_connection doesn't handle scoped IPv6 addresses
Erwan Le Pape added the comment: While the 3.7+ getaddrinfo isn't the best human representation of an IPv6 address, I believe it does make the most sense to keep it that way. In any case, this is a regression and changing return values of getaddrinfo for 3.7 isn't something that should be considered. The issue stems from the refactoring of the underlying socketmodule.c handling of IPv4/IPv6 addresses with dedicated make_ipv4_addr and make_ipv6_addr functions which returns proper tuples of: str/int for IPv4: https://github.com/python/cpython/blob/master/Modules/socketmodule.c#L1270 str/int/int/int for IPv6: https://github.com/python/cpython/blob/master/Modules/socketmodule.c#L1325 The actual issue is that _ensure_resolved naively assumes IPv4 and truncates the address to its first 2 members https://github.com/python/cpython/blob/3.7/Lib/asyncio/base_events.py#L1269 and never redefines them again in the case where they were set. I'd suggest passing the remaining elements of address in a packed *args or an optional flowinfo=0, scopeid=0 to _ipaddr_info since fundamentally, that's the method stripping valuable information. -- nosy: +lepaperwan ___ Python tracker <https://bugs.python.org/issue35545> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue34686] Add `-r`, as opposed to `-R` to Python core interpreter
Erwan Le Pape added the comment: As mentioned by Benjamin Peterson, this is a duplicate of #34722, which already has an implementation proposal by Peter Ebden. While implementation specifics change, the aim is the same and #34722 is the better solution (again, credit to Benjamin Peterson). -- resolution: -> duplicate stage: -> resolved status: open -> closed ___ Python tracker <https://bugs.python.org/issue34686> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue34686] Add `-r`, as opposed to `-R` to Python core interpreter
Erwan Le Pape added the comment: Given that marshal basically only just dumps code objects, the only viable solution I can see is adding a flag that can be passed all the way to the AST from `Python/bltinmodule.c:builtin_compile_impl` that would sort elements when creating code objects of unordered types. This could be automatically enabled when compiling a file if we assume imported files are trustworthy or add a flag to the `Lib/compileall.py` module. I only fear that this might break compiled code objects that make use of unordered types. Since I haven't sifted through the AST internals and the implications of such a change yet, so this is largely up for debate. -- ___ Python tracker <https://bugs.python.org/issue34686> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue34689] Lib/sysconfig.py expands non-variables
Change by Erwan Le Pape : -- keywords: +patch pull_requests: +8786 stage: -> patch review ___ Python tracker <https://bugs.python.org/issue34689> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue34686] Add `-r`, as opposed to `-R` to Python core interpreter
Erwan Le Pape added the comment: Great! My only concern with that is marshalling of untrusted data at runtime (I know, you shouldn't do that) can become a much more expensive operation. Is there any internal use of marshal beyond .pycs used at runtime by the core interpreter that might be affected by such a change? If not, it seems (to me) an acceptable modification of marshal and I'll submit a PR for it. -- ___ Python tracker <https://bugs.python.org/issue34686> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue34689] Lib/sysconfig.py expands non-variables
New submission from Erwan Le Pape :
Lib/sysconfig.py _parse_makefile needs to expand Makefile-style variables which
includes `$()` `${}` variables with a litteral `$` being `$$`.
That last particular point is properly accounted for in the first half of the
function when determining values that need expansion and those that don't.
However, the second half that actually performs variable expansion doesn't.
Sample breaking input:
VAR1= VALUE1
VAR2= VALUE2
VAR3=$${VAR1} ${VAR2}
Output:
Expect: {'VAR1': 'VALUE1', 'VAR2': 'VALUE2', 'VAR3': '$${VAR1} VALUE2'}
Actual: {'VAR1': 'VALUE1', 'VAR2': 'VALUE2', 'VAR3': '$VALUE1 VALUE2'}
I'm working on patching this but I'm putting this here as feedback is welcome.
As far as I could tell, this is a side effect of b2b1217, which patched #24705.
--
components: Library (Lib)
messages: 325393
nosy: lepaperwan
priority: normal
severity: normal
status: open
title: Lib/sysconfig.py expands non-variables
type: behavior
versions: Python 2.7, Python 3.5, Python 3.6, Python 3.7, Python 3.8
___
Python tracker
<https://bugs.python.org/issue34689>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue34686] Add `-r`, as opposed to `-R` to Python core interpreter
Erwan Le Pape added the comment: How would you suggest going about doing that? Without the proposed option, the alternative is leaving the build process vulnerable to environment variables potentially breaking the build process by patching configure.ac as follows: -PYTHON_FOR_BUILD='./$(BUILDPYTHON) -E' +PYTHON_FOR_BUILD='PYTHONHASHSEED=0 ./$(BUILDPYTHON) -E' Otherwise all environment variables affecting the Python interpreter would need to be cleared in addition to setting PYTHONHASHSEED=0. Without these `hacks`, making build outputs to be deterministic means fixing marshal to essentially sort elements when dumping unordered objects. Would you rather see a patch going in that direction? -- ___ Python tracker <https://bugs.python.org/issue34686> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue34686] Add `-r`, as opposed to `-R` to Python core interpreter
New submission from Erwan Le Pape : I'm attempting to leverage PEP 552 to make the core interpreter build process more deterministic. However, the standard Python Makefile uses `python -E` when calling the compileall (and all other setup scripts), which forces randomization since it can only be turned off through environment variables (which in turn leads to nondeterministic behaviour as noted in PEP 552 [1],[2]). Also, is adding a flag that disables randomization something that would be acceptable? Or are options to the core interpreter to be kept to a minimum and this does not represent a large enough use-case? This would basically hold in Modules/main.c in something as short as: + case 'r': + config->use_hash_seed = 1; + config->hash_seed = 0; + break; -- components: Interpreter Core messages: 325371 nosy: lepaperwan priority: normal severity: normal status: open title: Add `-r`, as opposed to `-R` to Python core interpreter type: enhancement versions: Python 3.7, Python 3.8 ___ Python tracker <https://bugs.python.org/issue34686> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue27803] ctypes automatic byref failing on custom classes attributes
Erwan Le Pape added the comment: I can confirm Eryk got what I meant. I didn't know if it was meant to work that way or if it was simply something that was overlooked so I thought I'd ask, I will look into the ctypes code to provide a patch sometime this week if I can. Terry, for a working example take the following (on a MS Windows): >>> from ctypes import * >>> from ctypes.wintypes import * >>> >>> class CustomPHKEY(object): ... def __init__(self, value): ... self._as_parameter_ = HKEY(value) ... >>> >>> function = ctypes.windll.advapi32.RegOpenKeyExW >>> function.argtypes = [HKEY, c_wchar_p, DWORD, DWORD, POINTER(HKEY)] >>> function.restype = LONG >>> result = CustomPHKEY(0) >>> function(0x8002, 'SOFTWARE', 0, 0x20019, result) Traceback (most recent call last): File "", line 1, in ctypes.ArgumentError: argument 5: : expected LP_c_void_p instance instead of c_void_p -- ___ Python tracker <rep...@bugs.python.org> <https://bugs.python.org/issue27803> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue27803] ctypes automatic byref failing on custom classes attributes
New submission from Erwan Le Pape: When using a custom class to store a ctype value, passing that class as a function argument explicitly declared to be a pointer type fails to pass the _as_parameter_ class attribute as a pointer and instead raises a TypeError. For example: >>> from ctypes import * >>> from ctypes.wintypes import * >>> >>> class CustomPHKEY(object): ... def __init__(self, value): ... self._as_parameter_ = HKEY(value) ... >>> >>> function = windll.function >>> function.argtypes = [POINTER(HKEY)] >>> function.restype = LONG >>> result = CustomPHKEY(0) >>> function(result) Traceback (most recent call last): File "", line 1, in ctypes.ArgumentError: argument 1: : expected LP_c_void_p instance instead of c_void_p Shouldn't ctypes apply the required byref() conversion automatically? Or is this behavior normal and automatic byref() only concerns native ctypes types? I only flagged Python 3.5 and Python 2.7 since they are the only ones I explicitly tested this on but I suspect other versions are affected. -- components: ctypes messages: 273149 nosy: amaury.forgeotdarc, belopolsky, lepaperwan, meador.inge priority: normal severity: normal status: open title: ctypes automatic byref failing on custom classes attributes type: behavior versions: Python 2.7, Python 3.5 ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue27803> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
