[issue43620] os.path.join does not use os.sep as documentation claims

[Fred Drake]

Fred Drake  added the comment:

Just reviewed the documentation for both os.sep and os.path.join().

The os.sep docs do not suggest it can be set, and the reference in the 
os.path.join() description is silent regarding that, so can be read as Jared 
did.  I don't recall this coming up before, but... there's a lot I don't 
remember at this point.  --sigh--

Removing the reference to os.sep from the os.path.join() description would not 
be confusing, IMO.  Clearly, the ambiguity can bite.

I'd be open to a patch to remove the reference to os.sep from the 
os.path.join() docs.  Reopening, but removing 3.6 and 3.7 since they're in 
security-fix-only mode.

--
keywords: +easy
resolution: not a bug -> 
stage: resolved -> needs patch
status: closed -> open
versions:  -Python 3.6, Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40066] Enum: modify __repr__, __str__; update docs

[Ethan Furman]

Change by Ethan Furman :


--
components: +Library (Lib)
title: Enum._convert should change __repr__ and/or __str__ to use module name 
instead of class name -> Enum: modify __repr__, __str__; update docs

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43629] fix _PyRun_SimpleFileObject create __main__ module and cache. Call this function multiple times, the attributes stored in the module dict will affect eachother.

[junyixie]

Change by junyixie :


--
keywords: +patch
pull_requests: +23773
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/25023

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43629] fix _PyRun_SimpleFileObject create __main__ module and cache. Call this function multiple times, the attributes stored in the module dict will affect eachother.

[junyixie]

New submission from junyixie :

fix _PyRun_SimpleFileObject create __main__ module and cache. Call this 
function multiple times, the attributes stored in the module dict will affect 
eachother.

create __main__ module, and cache it.

for example. 
if we run fileA, call _PyRun_SimpleFileObject will create __main__ module, 
fileA add some attribute in __main__ module dict.

now we run fileB. call _PyRun_SimpleFileObject will load cached __main__ 
module. now in __main__ module dict, we can get fileA's attribute.


dir(module), We got unexpected results
```
for name in dir(module):
...
```

in unittest, if we execute test, and don't exit. (unittest main.py 
TestProgram), set exit=False.
```
def __init__(self, module='__main__', defaultTest=None, argv=None,
testRunner=None, testLoader=loader.defaultTestLoader,
exit=True, verbosity=1, failfast=None, catchbreak=None,
buffer=None, warnings=None, *, tb_locals=False):
```

then when unittest load tests. if we use _PyRun_SimpleFileObject to run 
unittest, it will Repeated load test cases
```
for name in dir(module):
obj = getattr(module, name)
if isinstance(obj, type) and issubclass(obj, case.TestCase):
tests.append(self.loadTestsFromTestCase(obj))
```


```
int
_PyRun_SimpleFileObject(FILE *fp, PyObject *filename, int closeit,
PyCompilerFlags *flags)
{
PyObject *m, *d, *v;
int set_file_name = 0, ret = -1;

m = PyImport_AddModule("__main__");
if (m == NULL)
return -1;
Py_INCREF(m);
d = PyModule_GetDict(m);
```

--
components: C API
messages: 389538
nosy: JunyiXie
priority: normal
severity: normal
status: open
title: fix _PyRun_SimpleFileObject create __main__ module and cache. Call this 
function multiple times, the attributes stored in the module dict will affect 
eachother.
type: behavior
versions: Python 3.10

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40758] For 7.2.7. Code Examples, distinguish between the Tutorial and other documentation

[Raymond Hettinger]

Raymond Hettinger  added the comment:

I would think this would go without saying.  The word "tutorial" means example 
driven instruction.  Previous to now, no one has ever reported confusion about 
this.

--
nosy: +rhettinger

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43628] Incorrect argument errors for random.getstate()

[Raymond Hettinger]

Raymond Hettinger  added the comment:

Here's how to use getstate() and setstate():

>>> import random
>>> state = random.getstate()
>>> random.choices('ABCDE', k=8)
['E', 'A', 'B', 'B', 'A', 'A', 'E', 'D']
>>> # restore the previous state
>>> random.setstate(state)
>>> # now, replay the random selections
>>> random.choices('ABCDE', k=8)
['E', 'A', 'B', 'B', 'A', 'A', 'E', 'D']



> 1. Is the TypeError correct? This should be an inconsistent
> argument number error. There is nothing to do with Type. 

Yes, it is correct, but I agree that it is unintuitive.  It just happens to be 
the Python way to report an incorrect number of arguments as a TypeError.


> 2. Is the detailed error correct? Doc says random.getstate()
> takes 0 argument, the reported error says getstate() take 1
> positional argument. which is inconsistent. 

This is an artifact of how Python implements object orient programming.  No one 
really likes this, but at some level it can be viewed as being technically 
correct — methods prepend an instance argument before calling an underlying 
function which reports on the number of arguments that it sees.  Presumably, if 
there were a straight-forward way of improving the error message, it would have 
been done long ago.

> Besides, I pass one argument to random.getstate(), 
> but the reported error says 2 were given.

Don't pass any arguments into getstate().


> random.getstate() takes 0 argument and return 
> the current setting for the weekday to start each week.

This part doesn't make sense to me.  Why do you think it accepts or returns a 
weekday?

--
resolution:  -> not a bug
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43628] Incorrect argument errors for random.getstate()

[Karthikeyan Singaravelan]

Change by Karthikeyan Singaravelan :


--
nosy: +mark.dickinson, rhettinger

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43628] Incorrect argument errors for random.getstate()

[Yang Feng]

New submission from Yang Feng :

In documentation of random.getstate(), it says:
“random.getstate()
Return an object capturing the current internal state of the generator. This 
object can be passed to setstate() to restore the state.”

random.getstate() takes 0 argument and return the current setting for the 
weekday to start each week. However, when I give one argument to 
random.getstate(), the interpreter reports the following error:
--
>>> import random
>>> random.getstate(1)
Traceback (most recent call last):
File "", line 1, in 
TypeError: getstate() takes 1 positional argument but 2 were given
--

Here I have two doubts about the reported errors:
1. Is the TypeError correct? This should be an inconsistent argument number 
error. There is nothing to do with Type. 
2. Is the detailed error correct? Doc says random.getstate() takes 0 argument, 
the reported error says getstate() take 1 positional argument. which is 
inconsistent. Besides, I pass one argument to random.getstate(), but the 
reported error says 2 were given.


Environment: Python 3.10, Ubuntu 16.04

--
assignee: docs@python
components: Documentation
messages: 389535
nosy: CharlesFengY, docs@python
priority: normal
severity: normal
status: open
title: Incorrect argument errors for random.getstate()
versions: Python 3.10

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43624] Add underscore as a decimal separator for string formatting

[Eric V. Smith]

Eric V. Smith  added the comment:

If we do anything for float, we should do the same for decimal.Decimal.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43624] Add underscore as a decimal separator for string formatting

[Eric V. Smith]

Change by Eric V. Smith :


--
nosy: +eric.smith

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39418] str.strip() should have a means of adding to the default behaviour

[Raymond Hettinger]

Raymond Hettinger  added the comment:

Marking this as closed because the issue doesn't seem to have gathered much 
interest.  If someone wants to move this forward, I recommend discussing it on 
python-ideas.

--
resolution:  -> rejected
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43620] os.path.join does not use os.sep as documentation claims

[Jared Sutton]

Jared Sutton  added the comment:

> Perhaps Jared was expecting that modifying os.sep would affect the functions 
> in os.path?

This is precisely what I thought, because the documentation makes it sound like 
that variable named os.sep is read and used as the path delimiter when 
constructing something with join(). In fact, that variable isn't read *at all* 
because the path separator is hard-coded in both posixpath.py and ntpath.py. 
Since os.sep isn't used, I see no reason why it should be referenced in the 
documentation at all.

I'm not trying to be pedantic here (though we nerds are famous for that :) ), 
but what I see here is a disagreement between what is documented and what 
actually exists in the implementation. Yes the *value* of os.sep happens to be 
the same as the one hard-coded into either ntpath or posixpath, but since that 
variable is not referenced in the implementation, its presence in the doc only 
serves to confuse people who take the documentation at face value and then get 
an unexpected result.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40758] For 7.2.7. Code Examples, distinguish between the Tutorial and other documentation

[Irit Katriel]

Change by Irit Katriel :


--
assignee:  -> docs@python
components: +Documentation
nosy: +docs@python
versions:  -Python 3.5, Python 3.6, Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39418] str.strip() should have a means of adding to the default behaviour

[Irit Katriel]

Irit Katriel  added the comment:

In python 3.9 there is also str.removeprefix:
https://docs.python.org/3/library/stdtypes.html#str.removeprefix
which you can apply in a loop.

--
components: +Library (Lib)
nosy: +iritkatriel

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38241] doc: Pickle with protocol=0 in python 3 does not produce a 'human-readable' format

[Irit Katriel]

Change by Irit Katriel :


--
assignee:  -> docs@python
components: +Documentation
nosy: +docs@python
title: Pickle with protocol=0 in python 3 does not produce a 'human-readable' 
format -> doc: Pickle with protocol=0 in python 3 does not produce a 
'human-readable' format
versions: +Python 3.10, Python 3.8, Python 3.9 -Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21297] csv.skipinitialspace only skips spaces, not "whitespace" in general

[Irit Katriel]

Change by Irit Katriel :


--
versions: +Python 3.10, Python 3.8, Python 3.9 -Python 2.7, Python 3.4, Python 
3.5, Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35816] csv.DictReader, skipinitialspace does not ignore tabs

[Irit Katriel]

Change by Irit Katriel :


--
components: +Library (Lib)
versions: +Python 3.10 -Python 3.5

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43618] random.shuffle loses most of the elements

[Raymond Hettinger]

Raymond Hettinger  added the comment:

As an immediate fix to your problem, replace this line:

random.shuffle(questions_element)

with:

questions = list(questions_element)
random.shuffle(questions)
questions_element[:] = questions

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43624] Add underscore as a decimal separator for string formatting

[Dominic Davis-Foster]

Dominic Davis-Foster  added the comment:

ISO 8-1:2009 recommends groups of three digits either side of the decimal 
sign.

--
nosy: +domdfcoding

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43627] What are the requirements for a test_sundry-testable script in Tools/scripts?

[Skip Montanaro]

Change by Skip Montanaro :


--
title: What are the requirements for a test_sunry-testable script in 
Tools/scripts? -> What are the requirements for a test_sundry-testable script 
in Tools/scripts?

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43625] CSV has_headers heuristic could be improved

[Skip Montanaro]

Skip Montanaro  added the comment:

I assume the OP is referring to this sort of usage:

>>> sniffer = csv.Sniffer()
>>> raw = open("mixed.csv").read()
>>> sniffer.has_header(raw)
False

*sigh*

I really wish the Sniffer class had never been added to the CSV module. I can't 
recall who wrote it (the author is long gone). Though I am responsible for the 
initial commits, it wasn't me or the main authors of csvmodule.c. As far as I 
know, it never really worked well. I can't recall ever using it.

A simpler heuristic would be if the first row contains a bunch of strings and 
the second row contains a bunch of numbers, then the file has a header. That 
assumes that CSV files consist mostly of numeric data.

Looking at has_header, I see this:

for thisType in [int, float, complex]:

I think this particular problem would be solved if the order of those types 
were reversed. The attached diff suggests that as well. Note that the Sniffer 
class currently contains no test cases, so that the test I added failed before 
the change and passes after doesn't mean it doesn't break someone's mission 
critical Sniffer usage.

(Sorry, Raymond. My Github-foo is insufficient to allow me to fork, apply the 
diff and create a PR.)

--
keywords: +patch
Added file: https://bugs.python.org/file49915/csv.diff

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40521] [subinterpreters] Make free lists and unicode caches per-interpreter

[STINNER Victor]

STINNER Victor  added the comment:

I reopen the issue to remind me that collections.deque() freelist is shared by 
all interpreters.

--
resolution: fixed -> 
status: closed -> open

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43627] What are the requirements for a test_sunry-testable script in Tools/scripts?

[Skip Montanaro]

New submission from Skip Montanaro :

In my fork of python/cpython I recently created a simple script to help me with 
my work (I am messing around in the internals and sometimes get blindsided by 
opcode changes). I stuck the script in Tools/script which caused 
test_tools.test_sundry to hang. (I suspect it's because my script reads from 
sys.stdin, but I'm not certain. The old Unix pipeline ways die hard.)

Looking around to see how I could modify my script to make it acceptable to 
test_sundry, I saw nothing about requirements. I tossed it in the 
TestSundryScripts.other list and now that test completes. Still, it seems there 
should be a bit written about what it takes for a script to be amenable to the 
minimal testing test_sundry.py performs.

--
components: Tests
messages: 389526
nosy: skip.montanaro
priority: normal
severity: normal
status: open
title: What are the requirements for a test_sunry-testable script in 
Tools/scripts?
versions: Python 3.10

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40521] [subinterpreters] Make free lists and unicode caches per-interpreter

[Raymond Hettinger]

Raymond Hettinger  added the comment:


New changeset 3bb19873abd572879cc9a8810b1db9db1f704070 by Raymond Hettinger in 
branch 'master':
Revert "bpo-40521: Remove freelist from collections.deque() (GH-21073)" 
(GH-24944)
https://github.com/python/cpython/commit/3bb19873abd572879cc9a8810b1db9db1f704070


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43625] CSV has_headers heuristic could be improved

[Raymond Hettinger]

Change by Raymond Hettinger :


--
nosy: +rhettinger

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43625] CSV has_headers heuristic could be improved

[Raymond Hettinger]

Change by Raymond Hettinger :


--
nosy: +skip.montanaro

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43511] tkinter with Tk 8.6.11 is slow on macOS

[Christopher A. Chavez]

Change by Christopher A. Chavez :


--
nosy: +chrstphrchvz

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24605] segmentation fault at asciilib_split_char.lto_priv

[STINNER Victor]

Change by STINNER Victor :


--
resolution:  -> out of date
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24605] segmentation fault at asciilib_split_char.lto_priv

[STINNER Victor]

STINNER Victor  added the comment:

No activity for 6 years, the reporter didn't provide any way to reproduce the 
crash, I close the issue.

Note: Python 3.10 now dumps the list of third party C extensions on a fatal 
error, or when faulthandler handlers a crash ;-) Just enabled the Python 
Development Mode:
https://docs.python.org/dev/library/devmode.html

--
status: pending -> open

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43609] ast.unparse-ing a FunctionType gives ambiguous result

[midori]

midori  added the comment:

@BTaskaya I've seen this in third party ides and type checker. For example they 
are referring to "Union[Callable[[], Union[int, str]], None]" as "() -> (int | 
str) | None" where there are parentheses around the returns.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43624] Add underscore as a decimal separator for string formatting

[STINNER Victor]

Change by STINNER Victor :


--
nosy: +vstinner

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43626] SIGSEV in PyErr_SetObject

[STINNER Victor]

STINNER Victor  added the comment:

> What can be happening? How can I help to debug this?

Try to run your application in the Python Debug Mode: 
https://docs.python.org/dev/library/devmode.html

The best is if you can run your application with a Python built in debug mode.

Usually, it's a bug in a 3rd party C extension.

See also: https://pythondev.readthedocs.io/debug_tools.html

--
nosy: +vstinner

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43620] os.path.join does not use os.sep as documentation claims

[Eryk Sun]

Eryk Sun  added the comment:

Fred, do you think this needs to be reopened to clarify that os.sep and 
os.altsep are not parameters for the ntpath and posixpath modules? 

I would have thought that the note at the top (path name conventions, etc) 
would be sufficient to let someone know that os.path is meant for working with 
paths on the current platform. Working with Windows paths in POSIX, and vice 
versa, is not simply a matter of replacing os.sep. For example:

>>> posixpath.join('C:/spam', '/eggs')
'/eggs'
>>> ntpath.join('C:/spam', '/eggs')
'C:/eggs'

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43626] SIGSEV in PyErr_SetObject

[Abraham Macias]

New submission from Abraham Macias :

Hi, I'm dealing with random crashes when using pymongo in Python 3.7.3 in a 
Debian Buster.

This is the python backtrace:
(gdb) thread apply all py-bt

Thread 2 (Thread 0x7f9817d95700 (LWP 221)):
Traceback (most recent call first):
  File "/usr/local/lib/python3.7/dist-packages/gevent/_threading.py", line 80, 
in wait
waiter.acquire() # Block on the native lock
  File "/usr/local/lib/python3.7/dist-packages/gevent/_threading.py", line 162, 
in get
self._not_empty.wait()
  File "/usr/local/lib/python3.7/dist-packages/gevent/threadpool.py", line 270, 
in _worker
task = task_queue.get()
  File "/usr/local/lib/python3.7/dist-packages/gevent/threadpool.py", line 254, 
in __trampoline
g.switch()

Thread 1 (Thread 0x7f981fdfd740 (LWP 216)):
Traceback (most recent call first):
  
  File "/usr/local/lib/python3.7/dist-packages/bson/__init__.py", line 1089, in 
_decode_all_selective
return decode_all(data, codec_options)
  File "/usr/local/lib/python3.7/dist-packages/pymongo/message.py", line 1616, 
in unpack_response
self.payload_document, codec_options, user_fields)
  File "/usr/local/lib/python3.7/dist-packages/pymongo/cursor.py", line 1080, 
in _unpack_response
legacy_response)
  File "/usr/local/lib/python3.7/dist-packages/pymongo/server.py", line 131, in 
run_operation_with_response
user_fields=user_fields)
  File "/usr/local/lib/python3.7/dist-packages/pymongo/mongo_client.py", line 
1366, in _cmd
unpack_res)
  File "/usr/local/lib/python3.7/dist-packages/pymongo/mongo_client.py", line 
1471, in _retryable_read
return func(session, server, sock_info, slave_ok)
  File "/usr/local/lib/python3.7/dist-packages/pymongo/mongo_client.py", line 
1372, in _run_operation_with_response
exhaust=exhaust)
  File "/usr/local/lib/python3.7/dist-packages/pymongo/cursor.py", line 1001, 
in __send_message
address=self.__address)
  File "/usr/local/lib/python3.7/dist-packages/pymongo/cursor.py", line 1124, 
in _refresh
self.__send_message(q)
  File "/usr/local/lib/python3.7/dist-packages/pymongo/cursor.py", line 1207, 
in next
if len(self.__data) or self._refresh():
  File "/usr/local/lib/python3.7/dist-packages/pymongo/collection.py", line 
1319, in find_one
for result in cursor.limit(-1):
  File "/usr/local/lib/python3.7/dist-packages/gecoscc/userdb.py", line 119, in 
create_user
user = self.collection.find_one({'email': email})
  File 
"/usr/local/lib/python3.7/dist-packages/gecoscc/commands/create_adminuser.py", 
line 95, in command
{'is_superuser': self.options.is_superuser}
  File "/usr/local/lib/python3.7/dist-packages/gecoscc/management.py", line 90, 
in __call__
self.command()
  File "/usr/local/lib/python3.7/dist-packages/gecoscc/management.py", line 48, 
in main
command()
  File "/usr/local/bin/pmanage", line 10, in 
sys.exit(main())
(gdb) 

And this is the builtin-code backtrace:

Core was generated by `/usr/bin/python3 /usr/local/bin/pmanage 
/opt/gecosccui/gecoscc.ini create_admin'.
Program terminated with signal SIGSEGV, Segmentation fault.
#0  PyErr_SetObject (exception=0x7ff9c0 <_PyExc_AttributeError.lto_priv.2311>, 
value=0x7f1ab49cb098) at ../Python/errors.c:101
101 Py_INCREF(exc_value);
[Current thread is 1 (Thread 0x7f1abc823740 (LWP 370))]
(gdb) bt
#0  PyErr_SetObject (exception=, value="type object 
'dict' has no attribute '_type_marker'") at ../Python/errors.c:101
#1  0x0052c23b in PyErr_FormatV (vargs=0x7ffedff77c40, 
format=, exception=) at 
../Python/errors.c:852
#2  PyErr_Format (exception=, format=) 
at ../Python/errors.c:852
#3  0x0058717d in type_getattro (type=, name=) at ../Objects/typeobject.c:3223
#4  0x0054baae in _PyObject_LookupAttr (result=, 
name=, v=) at ../Objects/object.c:949
#5  builtin_getattr (self=, args=, 
nargs=) at ../Python/bltinmodule.c:1121
#6  0x0053 in _PyMethodDef_RawFastCallKeywords (method=0x89d160 
, self=, args=0x1237208, 
nargs=, kwnames=)
at ../Objects/call.c:651
#7  0x005463e3 in _PyCFunction_FastCallKeywords (kwnames=0x0, nargs=3, 
args=0x1237208, func=) at ../Objects/call.c:730
#8  call_function (kwnames=0x0, oparg=3, pp_stack=) at 
../Python/ceval.c:4568
#9  _PyEval_EvalFrameDefault (f=, throwflag=) at 
../Python/ceval.c:3124
#10 0x005cd68c in PyEval_EvalFrameEx (throwflag=0, 
f=Frame 0x1237088, for file 
/usr/local/lib/python3.7/dist-packages/bson/codec_options.py, line 35, in 
_raw_document_class (document_class=)) at 
../Python/ceval.c:547
#11 function_code_fastcall (globals=, nargs=, 
args=, co=) at ../Objects/call.c:283
#12 _PyFunction_FastCallKeywords (func=, stack=, 
nargs=, kwnames=) at ../Objects/call.c:408
#13 0x0054207c in call_function (kwnames=0x0, oparg=, 
pp_stack=) at ../Python/ceval.c:4616
#14 _PyEval_EvalFrameDefault (f=, throwflag=) at 
../Python/ceval.c:3124
#15 0x0053f732 in PyEval_EvalFrameEx (throwflag=0, 
f=Frame 0x17882e8, for 

[issue43609] ast.unparse-ing a FunctionType gives ambiguous result

[Batuhan Taskaya]

Batuhan Taskaya  added the comment:

Hey @cleoold! Technically the second AST you gave is invalid (fun2), 
considering that the FunctionType is not an expression;

mod = Module(stmt* body, type_ignore* type_ignores)
| FunctionType(expr* argtypes, expr returns)

expr = BoolOp(boolop op, expr* values)
 | NamedExpr(expr target, expr value)
 | BinOp(expr left, operator op, expr right)

Though I am curious about use case. How did you come with the fun2's AST?

--
nosy: +BTaskaya -Batuhan Taskaya

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43618] random.shuffle loses most of the elements

[Stefan Behnel]

Stefan Behnel  added the comment:

Yes, this is neither a bug in CPython (or its stdlib) nor in lxml. It's how 
things work. Don't use these two together.

--
resolution:  -> third party
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43624] Add underscore as a decimal separator for string formatting

[Raymond Hettinger]

Raymond Hettinger  added the comment:

Some brief research
===

""" in numbers four or more digits long, use commas to set off groups of three 
digits, counting leftward from the decimal point, in the standard American 
style. For long decimal numbers, do not use any digit-group separators to the 
right of the decimal point."""
— Google Style Guide https://developers.google.com/style/numbers

The CRC math handbook uses groups of five after the decimal point.
See §1.2.4 in 
http://dl.icdst.org/pdfs/files/2a2cbcfc89598fd83c315ce45c1ee663.pdf


NIST Guide for using SI units:  """The digits of numerical values having more 
than four digits on either side of the decimal marker are separated into groups 
of three using a thin, fixed space counting from both the left and right of the 
decimal marker. For example, 15 739.012 53 is highly preferred to 15739.01253. 
Commas are not used to separate digits into groups of three. (See Sec. 
10.5.3.)"""
— page vi in https://physics.nist.gov/cuu/pdf/sp811.pdf#10.5.2

StackExchange question on the topic:
https://math.stackexchange.com/questions/182775/convention-of-digit-grouping-after-decimal-point

The important reference, ISO 8:1 discusses this in section 7, "Printing 
rules", but the standard is not publicly available.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43616] random.shuffle() crashes with Unhandled exception

[Stefan Behnel]

Stefan Behnel  added the comment:

Almost certainly not a bug in CPython's stdlib.

Possibly something worth investigating in lxml, although, as stated in issue 
43618, random.shuffle() simply does not work on lxml.etree Elements.

--
resolution:  -> third party
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43625] CSV has_headers heuristic could be improved

[ejacq]

New submission from ejacq <0pyth...@jesuislibre.net>:

Here is an sample of CSV input:

"time","forces"
0,0
0.5,0.9

when calling has_header() from csv.py on this sample, it returns false.
Why? because 0 and 0.5 don't belong to the same type and thus the column is 
discarded by the heuristic.

I think the heuristic will better work if rather than just comparing number 
types, it would also consider casting the values in this order int -> float -> 
complex. If the values are similar then consider this upgraded type as the type 
of the column.

In the end, this file would be considered float columns with headers.

--
components: Library (Lib)
messages: 389515
nosy: ejacq
priority: normal
severity: normal
status: open
title: CSV has_headers heuristic could be improved
versions: Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43620] os.path.join does not use os.sep as documentation claims

[Fred Drake]

Fred Drake  added the comment:

Perhaps Jared was expecting that modifying os.sep would affect the functions in 
os.path?

os.sep was never intended to be updated.

Using the specific *path modules to work with "foreign" paths has long been 
advocated as the way to do this.  It isn't a work-around.

--
nosy: +fdrake

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37745] 3.8b3 - windows install gui/ inconsistent options

[Dimitri Merejkowsky]

Dimitri Merejkowsky  added the comment:

I know this is two years later so maybe this is not the best way to get 
answers, but Steve Dower said:

> We don't add Python to PATH by default (because it's bad), and people 
> couldn't find the option

Could someone clarify this for me? Because my experience as a Python teacher 
tells me the exact opposite.

For context, I've been teaching Python to absolute beginners (some of them do 
not even know how to *use* Windows to begin with).

I do tell them repeatedly to make sure to tick the "add python to Path", but 
every time, one of them forgot.

This gets even worse when the course is done remotely because I have to *tell* 
the students what to do through a screen-sharing app ...

Here's what I had to tell one of the students to do this morning.

Step 1: Tick "Display Hidden Items" in the Display option of the Windows File 
Explorer

Step 2: Go into c:\users\AppData

Step 3: Type `python.exe` in the search bar

Step 4: Copy/paste the full path 

Step 5: Open "Edit environment variables for my account"

Step 6: Click on "Edit environment variables"

Step 7: Click on Path

Step 8: Copy/paste the full path, without "python.exe"

Step 9: Copy/paste the full path, adding `\Scripts\` at the end

Step 10: restart cmd.exe


In insight, I wonder if I should have just told him to just re-run the 
installer and tick the box ...

Some notes: 

Fixing #14302 would help for step 9, which is nice.

I know very well that writing a Windows installer is far for trivial, but I'm 
ready to try and make a PR anyway. That being said, I understand there may be 
good reasons to *not* add python to PATH that I don't know about.

--
nosy: +Dimitri Merejkowsky

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

PyCA cryptography 3.4.7 released

[Paul Kehrer]

PyCA cryptography 3.4.7 has been released to PyPI. cryptography
includes both high level recipes and low level interfaces to common
cryptographic algorithms such as symmetric ciphers, asymmetric
algorithms, message digests, X509, key derivation functions, and much
more. We support Python 3.6+, and PyPy3.

Changelog (https://cryptography.io/en/latest/changelog.html#v3-4-7):
* Updated Windows, macOS, and manylinux wheels to be compiled with
OpenSSL 1.1.1k.

-Paul Kehrer (reaperhulk)
___
Python-announce-list mailing list -- python-announce-list@python.org
To unsubscribe send an email to python-announce-list-le...@python.org
https://mail.python.org/mailman3/lists/python-announce-list.python.org/
Member address: arch...@mail-archive.com

[issue43624] Add underscore as a decimal separator for string formatting

[Raymond Hettinger]

Raymond Hettinger  added the comment:

IIRC there is ISO recommending that after the decimal point, digits be arranged 
in groups of five.  I think is also how printed reference tables are typically 
formatted.

--
nosy: +mark.dickinson, rhettinger, serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24605] segmentation fault at asciilib_split_char.lto_priv

[Irit Katriel]

Irit Katriel  added the comment:

Is there anything we can do about this or should it be closed?

--
nosy: +iritkatriel
status: open -> pending

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43618] random.shuffle loses most of the elements

[Raymond Hettinger]

Raymond Hettinger  added the comment:

Interestingly, this isn't an LXML bug.  It is a documented difference from how 
the standard library works:

https://lxml.de/tutorial.html#elements-are-lists

So, if you want use random.shuffle(), you need the standard library ElementTree 
instead of lxml.

This:

from lxml.etree import Element

root = Element('outer')
root.append(Element('zero'))
root.append(Element('one'))
root.append(Element('two'))
root[0] = root[1]
print([e.tag for e in root])

Produces:

   ['one', 'two']

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43616] random.shuffle() crashes with Unhandled exception

[Raymond Hettinger]

Change by Raymond Hettinger :


--
nosy: +scoder -skrah

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43618] random.shuffle loses most of the elements

[Raymond Hettinger]

Raymond Hettinger  added the comment:

The standard library isn't at fault here.  Please file this an an LXML bug.

Reproducer:

from lxml.etree import Element

root = Element('outer')
root.append(Element('zero'))
root.append(Element('one'))
root.append(Element('two'))
print([e.tag for e in root])
root[1], root[0] = root[0], root[1]
print([e.tag for e in root])

This outputs:

   ['zero', 'one', 'two']
   ['one', 'two']

Replacing the import with:

   from xml.etree.ElementTree import Element

Gives the expected result:

   ['zero', 'one', 'two']
   ['one', 'zero', 'two']

--
nosy: +scoder -skrah

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43624] Add underscore as a decimal separator for string formatting

[Terry Davis]

New submission from Terry Davis :

Proposal:
Enable this
>>> format(12_34_56.12_34_56, '_._f')
'123_456.123_456'

Where now only this is possible
>>> format(12_34_56.12_34_56, '_.f')
'123_456.123456'


Based on the discussion in the Ideas forum, three core devs support this 
addition.
https://discuss.python.org/t/add-underscore-as-a-thousandths-separator-for-string-formatting/7407

I'm willing to give this a try if someone points me to where to add tests and 
where the float formatting code is. This would be my first CPython contribution.

The feature freeze for 3.10 is 2021-05-03.
https://www.python.org/dev/peps/pep-0619/#id5

--
components: Interpreter Core
messages: 389508
nosy: Terry Davis
priority: normal
severity: normal
status: open
title: Add underscore as a decimal separator for string formatting
type: enhancement
versions: Python 3.10

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43416] Add README files in Include/cpython and Include/internal

[STINNER Victor]

Change by STINNER Victor :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43416] Add README files in Include/cpython and Include/internal

[STINNER Victor]

STINNER Victor  added the comment:


New changeset 929c9039fe0468cb37e0811ddfb7b67c98353ea8 by Erlend Egeberg 
Aasland in branch 'master':
bpo-43416: Add Include/README.rst (GH-24884)
https://github.com/python/cpython/commit/929c9039fe0468cb37e0811ddfb7b67c98353ea8


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43582] SSLContext.sni_callback docs inaccurately describe available handshake info

[Andrew Dailey]

Andrew Dailey  added the comment:

Okay, that makes sense. I appreciate the overview! What do you feel is the best 
way to "solve" this issue, then? Should the docs be updated to reflect the fact 
that ALPN info isn't available in the sni_callback? Or should some code be 
modified to make the docs correct (even though that'd have to be a bit hacky 
since the OpenSSL handshake callback order seems fairly set in stone).

I've got the contributor agreement signed and ready to go. I can formalize my 
ideas for revising the docs into a patch if that would make sense and be 
helpful.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33164] Blake 2 module update

[Dong-hee Na]

Change by Dong-hee Na :


--
nosy: +corona10

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42136] [importlib] deprecate module_repr() methods

[Brett Cannon]

Change by Brett Cannon :


--
keywords: +patch
pull_requests: +23772
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/25022

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33164] Blake 2 module update

[Christian Heimes]

Change by Christian Heimes :


--
resolution: fixed -> 
stage: resolved -> needs patch
status: closed -> open
type:  -> behavior

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43620] os.path.join does not use os.sep as documentation claims

[Eryk Sun]

Eryk Sun  added the comment:

> certainly you can see this is a doc bug, since the doc clearly 
> states that os.sep is utilized to join the elements of the path,
> when it clearly isn't; right?

os.path is ntpath in Windows, which uses backslash as the path separator, which 
is the same as os.sep in Windows. os.path is posixpath in POSIX, which uses 
slash as the path separator, which is the same as os.sep in POSIX.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33164] Blake 2 module update

[STINNER Victor]

STINNER Victor  added the comment:

> This reverted commit 5940c535b06805960d008bcafec7260dae74d9b9

Context of the memcpy/memove change: https://github.com/BLAKE2/BLAKE2/issues/32

--
nosy: +vstinner

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43623] nouveauté 2021

[STINNER Victor]

Change by STINNER Victor :


--
Removed message: https://bugs.python.org/msg389503

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43623] nouveauté 2021

[STINNER Victor]

Change by STINNER Victor :


--
resolution:  -> not a bug
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43623] nouveauté 2021

[Emmanuel Miranda]

New submission from Emmanuel Miranda :

Avec un nouveau cycle de release annuel la communauté Python ne cesse d’aller 
de l’avant hâte de voir ce qu’ils nous réservent pour 2021 après la version 
3.10 ! En espérant que les applicatifs suivent pour supporter

https://webscre.com/

--
components: Library (Lib)
messages: 389503
nosy: Yanis77
priority: normal
severity: normal
status: open
title: nouveauté 2021

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33164] Blake 2 module update

[Petr Viktorin]

Petr Viktorin  added the comment:

This reverted commit 5940c535b06805960d008bcafec7260dae74d9b9

--
nosy: +petr.viktorin

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43244] Move PyArena C API to the internal C API

[STINNER Victor]

STINNER Victor  added the comment:

In short, this issue is a follow-up of bpo-40939.

These C API removal are related to the final step of the PEP 617 "New PEG 
parser for CPython": removal of the PyParser C API.

* The PyParser C API produced "node*" objects, like 
PyParser_SimpleParseFile(FILE*) => node*. This C API was removed in Python 
3.10: bpo-40939 "Remove the old parser".

* The AST C API takes "node*" as input: PyAST_FromNode(node*) => mod_ty. I 
removed these APIs.

* symtable takes "mod_ty" as input.

* ASDL and PyArena API were only useful for the removed AST C APIs.

I searched for removed functions in the top PyPI 4000 projects, there is a 
single project on 4000: typed_ast. This typed_ast project is special, it copies 
directly code from CPython. Even after the C API removed, typed_ast can 
continue to use the internal C API, it only has to define the 
Py_BUILD_CORE_MODULE macro.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43620] os.path.join does not use os.sep as documentation claims

[Jared Sutton]

Jared Sutton  added the comment:

I can understand your suggestion to just utilize the posixpath library on 
Windows if needed. That's a reasonable work-around. But certainly you can see 
this is a doc bug, since the doc clearly states that os.sep is utilized to join 
the elements of the path, when it clearly isn't; right?

--
components:  -Library (Lib)

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Let's celebrate: 20 years of EuroPython

[M.-A. Lemburg]

This year's conference will mark the 20th edition of the EuroPython
conference.

* EuroPython 2021 *

   https://ep2021.europython.eu/


Since we started touring Europe in 2002 in Charleroi, Belgium, we have
come a long way. The conference has grown from the initial 240 to around
1200-1400 attendees every year. The organization had started with a
small group of people in a somewhat ad-hoc way and has now grown into a
well structured team backed by the non-profit EuroPython Society (EPS),
while still keeping the fun spirit of the early days.

EuroPython 2002 was the first major all volunteer run event for Python.
A lot of other conferences have since emerged in Europe and we're
actively trying to help them wherever we can, with our grants program,
growing conference knowledge base and our Organizers' Lunch, for which
we regularly invite representatives of all European Python conferences
to get together to network, exchange experience in organizing events
community building and understand how we can most effectively help each
other.

To celebrate the anniversary, we took a deep look into our archives and
have put together live website copies of the last few years, going back
all the way to EP2009, which was held in Birmingham, UK. For previous
years, archive.org stored some pages of the websites, so let's do a
quick tour of 20 years of EuroPython ...

https://blog.europython.eu/20th-anniversary-of-europython/


Help spread the word


Please help us spread this message by sharing it on your social
networks as widely as possible. Thank you !

Link to the blog post:

https://blog.europython.eu/20th-anniversary-of-europython/

Tweet:

https://twitter.com/europython/status/1375077502851420160


Enjoy,
--
EuroPython 2021 Team
https://www.europython-society.org/

___
Python-announce-list mailing list -- python-announce-list@python.org
To unsubscribe send an email to python-announce-list-le...@python.org
https://mail.python.org/mailman3/lists/python-announce-list.python.org/
Member address: arch...@mail-archive.com

Let's celebrate: 20 years of EuroPython

[M.-A. Lemburg]

This year's conference will mark the 20th edition of the EuroPython
conference.

* EuroPython 2021 *

   https://ep2021.europython.eu/


Since we started touring Europe in 2002 in Charleroi, Belgium, we have
come a long way. The conference has grown from the initial 240 to around
1200-1400 attendees every year. The organization had started with a
small group of people in a somewhat ad-hoc way and has now grown into a
well structured team backed by the non-profit EuroPython Society (EPS),
while still keeping the fun spirit of the early days.

EuroPython 2002 was the first major all volunteer run event for Python.
A lot of other conferences have since emerged in Europe and we're
actively trying to help them wherever we can, with our grants program,
growing conference knowledge base and our Organizers' Lunch, for which
we regularly invite representatives of all European Python conferences
to get together to network, exchange experience in organizing events
community building and understand how we can most effectively help each
other.

To celebrate the anniversary, we took a deep look into our archives and
have put together live website copies of the last few years, going back
all the way to EP2009, which was held in Birmingham, UK. For previous
years, archive.org stored some pages of the websites, so let's do a
quick tour of 20 years of EuroPython ...

https://blog.europython.eu/20th-anniversary-of-europython/


Help spread the word


Please help us spread this message by sharing it on your social
networks as widely as possible. Thank you !

Link to the blog post:

https://blog.europython.eu/20th-anniversary-of-europython/

Tweet:

https://twitter.com/europython/status/1375077502851420160


Enjoy,
--
EuroPython 2021 Team
https://www.europython-society.org/

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: convert script awk in python

[Dan Ciprus (dciprus) via Python-list]

... funny thing is that OP never contributed to this discussion. Several people 
provided very valuable inputs but OP did not even bother to say "thank you".


just saying ...

On Wed, Mar 24, 2021 at 11:22:02AM -0400, Avi Gross via Python-list wrote:

Cameron,

I agree with you. I first encountered AWK in 1982 when I went to work for
Bell Labs.

I have not had any reason to use AWK since before the year 2000 so I was not
sure that unused variables were initialized to zero. The code seemed to
assume that. I have learned quite a few languages since and after a while,
they tend to blend into each other.

I think it would indeed have been more AWKthonic (or should that be called
AWKward?) to have a BEGIN section in which functions were declared and
variables clearly initialized but the language does allow some quick and
dirty ways to do things and clearly the original programmer used some.

Which brings us back to languages like python. When I started using AWK and
a slew of other UNIX programs years ago, what I found interesting is how
much AWK was patterned a bit on the C language, not a surprise as the K in
AWK is Brian Kernighan who had a hand in C. But unlike C that made me wait
around as it compiled, AWK was a bit more of an interpreted language and I
could write one-liner shell scripts (well, stretched over a few lines if
needed) that did things. True, if you stuck an entire program in a BEGIN
statement and did not actually loop over data, it seems a tad wasteful. But
sometimes it was handy to use it to test out a bit of C code I was writing
without waiting for the whole compile thing. In a sense, it was  bit like
using the python REPL and getting raid feedback. Of course, when I was an
early adopter of C++, too many things were not in AWK!

What gets me is the original question which made it sound a bit like asking
how you would translate some fairly simple program from language A to
language B. For some fairly simple programs, the translation effort could be
minimal. There are often trivial mappings between similar constructs. Quite
a bit of python simply takes a block of code in another language that is
between curly braces, and lines it up indented below whatever it modifies
and after a colon. The reverse may be similarly trivial. There are of course
many such changes needed for some languages but when some novel twist is
used that the language does not directly support, you may need to innovate
or do a rewrite that avoids it. But still, except in complicated
expressions, you can rewrite x++ to "x += 1" if that is available or "x = x
+ 1" or "x -> x + 1" or whatever.

What gets me here is that AWK in his program  was being used exactly for
what it was designed. Python is more general-purpose. Had we been asked (not
on this forum) to convert that AWK script to PERL, it would have been much
more straightforward because PERL was also designed to be able to read in
lines and break them into parts and act on them. It has constructs like the
diamond operator or split that make it easy.

Hence, at the end, I suggested Tomasz may want to do his task not using just
basic python but some module others have already shared that emulates some
of the filter aspects of AWK. That may make it easier to just translate the
bits of code to python while largely leaving the logic in place, depending
on the module.

Just to go way off the rails, was our annoying cross-poster from a while
back also promising to include a language like AWK into their universal
translator by just saving some JSON descriptions?

-Original Message-
From: Python-list  On
Behalf Of Cameron Simpson
Sent: Tuesday, March 23, 2021 6:38 PM
To: Tomasz Rola 
Cc: Avi Gross via Python-list 
Subject: Re: convert script awk in python

On 23Mar2021 16:37, Tomasz Rola  wrote:

On Tue, Mar 23, 2021 at 10:40:01AM -0400, Avi Gross via Python-list wrote:
[...]

I am a tod concerned as to where any of the variables x, y or z have
been defined at this point. I have not seen a BEGIN {...}
pattern/action or anywhere these have been initialized but they are
set in a function that as far as I know has not been called. Weird.
Maybe awk is allowing an uninitialized variable to be tested for in
your code but if so, you need to be cautious how you do this in python.


As far as I can say, the type of uninitialised variable is groked from
the first operation on it. I.e., "count += 1" first initializes count
to 0 and then adds 1.

This might depend on exact awk being used. There were few of them
during last 30+ years. I just assume it does as I wrote above.


I'm pretty sure this behaviour's been there since very early times. I think
it was there when I learnt awk, decades ago.


Using BEGIN would be in better style, of course.


Aye. Always good to be up front about initial values.


There is a very nice book, "The AWK Programming Language" by Aho,
Kernighan and Weinberger. First printed in 1988, now free and in pdf
format. Go search.


Yes, a really nice book. 

[issue43622] TLS 1.3, client polling returns event without data

[Christian Heimes]

Christian Heimes  added the comment:

It looks like your code is treating a SSLSocket like an ordinary Kernel socket. 
SSLSocket are implemented in user space and behave differently. 
https://docs.python.org/3/library/ssl.html#ssl-nonblocking explains some of the 
aspects of non-blocking I/O for TLS connections. For non-blocking I/O it is 
generally easier to use sans-I/O approach and make use of MemoryBIO 
https://docs.python.org/3/library/ssl.html#memory-bio-support instead of 
SSLSocket.

3.6 and 3.7 only receive security updates and are out of scope anyway.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: convert script awk in python

[Loris Bennett]

Peter Otten <__pete...@web.de> writes:

> On 25/03/2021 08:14, Loris Bennett wrote:
>
>> I'm not doing that, but I am trying to replace a longish bash pipeline
>> with Python code.
>>
>> Within Emacs, often I use Org mode[1] to generate date via some bash
>> commands and then visualise the data via Python.  Thus, in a single Org
>> file I run
>>
>>/usr/bin/sacct  -u $user -o jobid -X -S $start -E $end -s COMPLETED -n  | 
>> \
>>xargs -I {} seff {} | grep 'Efficiency' | sed '$!N;s/\n/ /' | awk '{print 
>> $3 " " $9}' | sed 's/%//g'
>>
>> The raw numbers are formatted by Org into a table
>>
>>| cpu_eff | mem_eff |
>>|-+-|
>>|96.6 |   99.11 |
>>|   93.43 |   100.0 |
>>|91.3 |   100.0 |
>>|   88.71 |   100.0 |
>>|   89.79 |   100.0 |
>>|   84.59 |   100.0 |
>>|   83.42 |   100.0 |
>>|   86.09 |   100.0 |
>>|   92.31 |   100.0 |
>>|   90.05 |   100.0 |
>>|   81.98 |   100.0 |
>>|   90.76 |   100.0 |
>>|   75.36 |   64.03 |
>>
>> I then read this into some Python code in the Org file and do something like
>>
>>df = pd.DataFrame(eff_tab[1:], columns=eff_tab[0])
>>cpu_data = df.loc[: , "cpu_eff"]
>>mem_data = df.loc[: , "mem_eff"]
>>
>>...
>>
>>n, bins, patches = axis[0].hist(cpu_data, bins=range(0, 110, 5))
>>n, bins, patches = axis[1].hist(mem_data, bins=range(0, 110, 5))
>>
>> which generates nice histograms.
>>
>> I decided rewrite the whole thing as a stand-alone Python program so
>> that I can run it as a cron job.  However, as a novice Python programmer
>> I am finding translating the bash part slightly clunky.  I am in the
>> middle of doing this and started with the following:
>>
>>  sacct = subprocess.Popen(["/usr/bin/sacct",
>>"-u", user,
>>"-S", period[0], "-E", period[1],
>>"-o", "jobid", "-X",
>>"-s", "COMPLETED", "-n"],
>>   stdout=subprocess.PIPE,
>>  )
>>
>>  jobids = []
>>
>>  for line in sacct.stdout:
>>  jobid = str(line.strip(), 'UTF-8')
>>  jobids.append(jobid)
>>
>>  for jobid in jobids:
>>  seff = subprocess.Popen(["/usr/bin/seff", jobid],
>>  stdin=sacct.stdout,
>>  stdout=subprocess.PIPE,
>>  )
>
> The statement above looks odd. If seff can read the jobids from stdin
> there should be no need to pass them individually, like:
>
> sacct = ...
> seff = Popen(
>   ["/usr/bin/seff"], stdin=sacct.stdout, stdout=subprocess.PIPE,
>   universal_newlines=True
> )
> for line in seff.communicate()[0].splitlines():
> ...

Indeed, seff cannot read multiple jobids.  That's why had 'xargs' in the
original bash code.  Initially I thought of calling 'xargs' via
Popen, but this seemed very fiddly (I didn't manage to get it working)
and anyway seemed a bit weird to me as it is really just a loop, which I
can implement perfectly well in Python.

Cheers,

Loris


>>  seff_output = []
>>  for line in seff.stdout:
>>  seff_output.append(str(line.strip(), "UTF-8"))
>>
>>  ...
>>
>> but compared the to the bash pipeline, this all seems a bit laboured.
>>
>> Does any one have a better approach?
>>
>> Cheers,
>>
>> Loris
>>
>>
>>> -Original Message-
>>> From: Cameron Simpson 
>>> Sent: Wednesday, March 24, 2021 6:34 PM
>>> To: Avi Gross 
>>> Cc: python-list@python.org
>>> Subject: Re: convert script awk in python
>>>
>>> On 24Mar2021 12:00, Avi Gross  wrote:
 But I wonder how much languages like AWK are still used to make new
 programs as compared to a time they were really useful.
>>>
>>> You mentioned in an adjacent post that you've not used AWK since 2000.
>>> By contrast, I still use it regularly.
>>>
>>> It's great for proof of concept at the command line or in small scripts, and
>>> as the innards of quite useful scripts. I've a trite "colsum" script which
>>> does nothing but generate and run a little awk programme to sum a column,
>>> and routinely type "blah  | colsum 2" or the like to get a tally.
>>>
>>> I totally agree that once you're processing a lot of data from places or
>>> where a shell script is making long pipelines or many command invocations,
>>> if that's a performance issue it is time to recode.
>>>
>>> Cheers,
>>> Cameron Simpson 
>>
>> Footnotes:
>> [1]  https://orgmode.org/
>>
>
-- 
Dr. Loris Bennett (Hr./Mr.)
ZEDAT, Freie Universität Berlin Email loris.benn...@fu-berlin.de
-- 
https://mail.python.org/mailman/listinfo/python-list

[issue43622] TLS 1.3, client polling returns event without data

[gchauvel]

gchauvel  added the comment:

that's not the polling in itself,
it's: polling returning event + setblocking(False) + recv() which blocks

note: the behavior is present for python3.6

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43622] TLS 1.3, client polling returns event without data

[Christian Heimes]

Christian Heimes  added the comment:

Could you please explain why you consider this a bug?

TLS 1.3 works differently than TLS 1.2. You must always assume that an 
application level read can result in a protocol level write operation and the 
other way around. This could happen with TLS 1.2, but it was less common to 
occur. With TLS 1.3 rekeying, client cert authentication with post handshake 
authentication, and sessions are most common examples.

--
versions:  -Python 3.6, Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43622] TLS 1.3, client polling returns event without data

[gchauvel]

New submission from gchauvel :

A simple test in test_ssl.py [1][2] with following context:
- client connects and listens to data without sending any first
- traces to make sure no data is written at test level from server or client
- TLSv1.3 is allowed or not using "context.options |= ssl.OP_NO_TLSv1_3"

produces this result:

TLSv1.2:
- no event on FD, no issue

TLSv1.3:
- event on FD without any write at test level
- recv() blocks, even with setblocking(False)

[1] master: 
https://github.com/g-chauvel/cpython/commit/8c95c4f67367ea43c508ea62a0cdbe120a3fed9b
[2] 3.6: 
https://github.com/g-chauvel/cpython/commit/7cd4b4ac22efea7c61a9f8f57b6f2315567f5742

--
assignee: christian.heimes
components: SSL
messages: 389496
nosy: christian.heimes, gchauvel
priority: normal
severity: normal
status: open
title: TLS 1.3, client polling returns event without data
type: behavior
versions: Python 3.10, Python 3.6, Python 3.7, Python 3.8, Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: convert script awk in python

[Peter Otten]

On 25/03/2021 08:14, Loris Bennett wrote:


I'm not doing that, but I am trying to replace a longish bash pipeline
with Python code.

Within Emacs, often I use Org mode[1] to generate date via some bash
commands and then visualise the data via Python.  Thus, in a single Org
file I run

   /usr/bin/sacct  -u $user -o jobid -X -S $start -E $end -s COMPLETED -n  | \
   xargs -I {} seff {} | grep 'Efficiency' | sed '$!N;s/\n/ /' | awk '{print $3 " 
" $9}' | sed 's/%//g'

The raw numbers are formatted by Org into a table


   | cpu_eff | mem_eff |
   |-+-|
   |96.6 |   99.11 |
   |   93.43 |   100.0 |
   |91.3 |   100.0 |
   |   88.71 |   100.0 |
   |   89.79 |   100.0 |
   |   84.59 |   100.0 |
   |   83.42 |   100.0 |
   |   86.09 |   100.0 |
   |   92.31 |   100.0 |
   |   90.05 |   100.0 |
   |   81.98 |   100.0 |
   |   90.76 |   100.0 |
   |   75.36 |   64.03 |

I then read this into some Python code in the Org file and do something like

   df = pd.DataFrame(eff_tab[1:], columns=eff_tab[0])
   cpu_data = df.loc[: , "cpu_eff"]
   mem_data = df.loc[: , "mem_eff"]

   ...

   n, bins, patches = axis[0].hist(cpu_data, bins=range(0, 110, 5))
   n, bins, patches = axis[1].hist(mem_data, bins=range(0, 110, 5))

which generates nice histograms.

I decided rewrite the whole thing as a stand-alone Python program so
that I can run it as a cron job.  However, as a novice Python programmer
I am finding translating the bash part slightly clunky.  I am in the
middle of doing this and started with the following:

 sacct = subprocess.Popen(["/usr/bin/sacct",
   "-u", user,
   "-S", period[0], "-E", period[1],
   "-o", "jobid", "-X",
   "-s", "COMPLETED", "-n"],
  stdout=subprocess.PIPE,
 )

 jobids = []

 for line in sacct.stdout:
 jobid = str(line.strip(), 'UTF-8')
 jobids.append(jobid)

 for jobid in jobids:
 seff = subprocess.Popen(["/usr/bin/seff", jobid],
 stdin=sacct.stdout,
 stdout=subprocess.PIPE,
 )


The statement above looks odd. If seff can read the jobids from stdin 
there should be no need to pass them individually, like:


sacct = ...
seff = Popen(
  ["/usr/bin/seff"], stdin=sacct.stdout, stdout=subprocess.PIPE,
  universal_newlines=True
)
for line in seff.communicate()[0].splitlines():
...



 seff_output = []
 for line in seff.stdout:
 seff_output.append(str(line.strip(), "UTF-8"))

 ...

but compared the to the bash pipeline, this all seems a bit laboured.

Does any one have a better approach?

Cheers,

Loris



-Original Message-
From: Cameron Simpson 
Sent: Wednesday, March 24, 2021 6:34 PM
To: Avi Gross 
Cc: python-list@python.org
Subject: Re: convert script awk in python

On 24Mar2021 12:00, Avi Gross  wrote:

But I wonder how much languages like AWK are still used to make new
programs as compared to a time they were really useful.


You mentioned in an adjacent post that you've not used AWK since 2000.
By contrast, I still use it regularly.

It's great for proof of concept at the command line or in small scripts, and
as the innards of quite useful scripts. I've a trite "colsum" script which
does nothing but generate and run a little awk programme to sum a column,
and routinely type "blah  | colsum 2" or the like to get a tally.

I totally agree that once you're processing a lot of data from places or
where a shell script is making long pipelines or many command invocations,
if that's a performance issue it is time to recode.

Cheers,
Cameron Simpson 


Footnotes:
[1]  https://orgmode.org/




--
https://mail.python.org/mailman/listinfo/python-list

[issue43613] gzip.compress and gzip.decompress are sub-optimally implemented.

[Ruben Vorderman]

Ruben Vorderman  added the comment:

I created bpo-43621 for the error issue. There should only be BadGzipFile. Once 
that is fixed, having only one error type will make it easier to implement some 
functions that are shared across the gzip.py codebase.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43621] gzip._GzipReader should only throw BadGzipFile errors

[Ruben Vorderman]

Change by Ruben Vorderman :


--
type:  -> behavior

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43612] zlib.compress should have a wbits argument

[Ruben Vorderman]

Change by Ruben Vorderman :


--
components: +Extension Modules -Library (Lib)

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43621] gzip._GzipReader should only throw BadGzipFile errors

[Ruben Vorderman]

New submission from Ruben Vorderman :

This is properly documented: 
https://docs.python.org/3/library/gzip.html#gzip.BadGzipFile . 

It now hrows EOFErrors when a stream is truncated. But this means that upstream 
both BadGzipFile and EOFError need to be catched in the exception handling when 
opening a gzip file for reading. When a gzip file is truncated it is also a 
"bad gzip file" in my opinion, so there is no reason to have an extra class of 
errors.
Also it throws zlib.error's when zlib craches for some reason. This means there 
is some corruption in the raw deflate block. Well that means it is a "bad gzip 
file" as well and the error message should reflect that. 

This won't break people's code. If they are already catching EOFError 
zlib.error and BadGzipFile it changes nothing. If they only catch BadGzipFile, 
they will have less annoying errors that pop through.

I can make the PR, but of course not without any feedback. I am curious what 
other people think.

--
components: Library (Lib)
messages: 389494
nosy: rhpvorderman
priority: normal
severity: normal
status: open
title: gzip._GzipReader should only throw BadGzipFile errors
versions: Python 3.10, Python 3.6, Python 3.7, Python 3.8, Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: convert script awk in python

[Christian Gollwitzer]

Am 25.03.21 um 00:30 schrieb Avi Gross:

It [awk] is, as noted, a great tool and if you only had one or a few tools like 
it
available, it can easily be bent and twisted to do much of what the others
do as it is more programmable than most. But following that line of
reasoning, fairly simple python scripts can be written with python -c "..."
or by pointing to a script


The thing with awk is that lots of useful text processing is directly 
built into the main syntax; whereas in Python, you can certainly do it 
as well, but it requires to load a library. The simple column summation 
mentioned before by Cameron would be


   awk ' {sum += $2 } END {print sum}'

which can be easily typed into a command line, with the benefit that it 
skips every line where the 2nd col is not a valid number. This is 
important because often there are empty lines, often there is an empty 
line at the end, some ascii headers whatever.


The closest equivalent I can come up with in Python is this:

==
import sys

s=0
for line in sys.stdin:
try:
s += float(line.split()[1])
except:
pass
print(s)
===


I don't want to cram this into a python -c " "  line, if it even is 
possible; how do you handle indentation levels and loops??


Of course, for big fancy programs Python is a much better choice than 
awk, no questions asked - but awk has a place for little things which 
fit the special programming model, and there are surprisingly many 
applications where this is just the easiest and fastest way to do the job.


It's like regexes - a few simple characters can do the job which 
otherwise requires a bulky program, but once the parsing gets to certain 
complexity, a true parsing language, or even just handcoded Python is 
much more maintainable.


Christian

PS: Exercise - handle lines commented out with a '#', i.e. skip those. 
In awk:


gawk '!/^\s*#/ {sum += $2 } END {print sum}'

--
https://mail.python.org/mailman/listinfo/python-list

Re: convert script awk in python

[Loris Bennett]

"Avi Gross"  writes:

> Just to be clear, Cameron, I retired very early and thus have had no reason
> to use AWK in a work situation and for a while was not using UNIX-based
> machines. I have no doubt I would have continued using WK as one part of my
> toolkit for years albeit less often as I found other tools better for some
> situations, let alone the kind I mentioned earlier that are not text-file
> based such as databases.
>
> It is, as noted, a great tool and if you only had one or a few tools like it
> available, it can easily be bent and twisted to do much of what the others
> do as it is more programmable than most. But following that line of
> reasoning, fairly simple python scripts can be written with python -c "..."
> or by pointing to a script
>
> Anyone have a collection of shell scripts that can be used in pipelines
> where each piece is just a call to python to do something simple?

I'm not doing that, but I am trying to replace a longish bash pipeline
with Python code.

Within Emacs, often I use Org mode[1] to generate date via some bash
commands and then visualise the data via Python.  Thus, in a single Org
file I run

  /usr/bin/sacct  -u $user -o jobid -X -S $start -E $end -s COMPLETED -n  | \   


  xargs -I {} seff {} | grep 'Efficiency' | sed '$!N;s/\n/ /' | awk '{print $3 
" " $9}' | sed 's/%//g' 
 
   
The raw numbers are formatted by Org into a table

  | cpu_eff | mem_eff |
  |-+-|
  |96.6 |   99.11 |
  |   93.43 |   100.0 |
  |91.3 |   100.0 |
  |   88.71 |   100.0 |
  |   89.79 |   100.0 |
  |   84.59 |   100.0 |
  |   83.42 |   100.0 |
  |   86.09 |   100.0 |
  |   92.31 |   100.0 |
  |   90.05 |   100.0 |
  |   81.98 |   100.0 |
  |   90.76 |   100.0 |
  |   75.36 |   64.03 |

I then read this into some Python code in the Org file and do something like

  df = pd.DataFrame(eff_tab[1:], columns=eff_tab[0])
  cpu_data = df.loc[: , "cpu_eff"]  

 
  mem_data = df.loc[: , "mem_eff"]  



  ...

  n, bins, patches = axis[0].hist(cpu_data, bins=range(0, 110, 5))  

 
  n, bins, patches = axis[1].hist(mem_data, bins=range(0, 110, 5)) 

which generates nice histograms.

I decided rewrite the whole thing as a stand-alone Python program so
that I can run it as a cron job.  However, as a novice Python programmer
I am finding translating the bash part slightly clunky.  I am in the
middle of doing this and started with the following:

sacct = subprocess.Popen(["/usr/bin/sacct",
  "-u", user,
  "-S", period[0], "-E", period[1],
  "-o", "jobid", "-X",
  "-s", "COMPLETED", "-n"],
 stdout=subprocess.PIPE,
)

jobids = []

for line in sacct.stdout:
jobid = str(line.strip(), 'UTF-8')
jobids.append(jobid)

for jobid in jobids:
seff = subprocess.Popen(["/usr/bin/seff", jobid],
stdin=sacct.stdout,
stdout=subprocess.PIPE,
)
seff_output = []
for line in seff.stdout:
seff_output.append(str(line.strip(), "UTF-8"))

...

but compared the to the bash pipeline, this all seems a bit laboured. 

Does any one have a better approach?

Cheers,

Loris


> -Original Message-
> From: Cameron Simpson  
> Sent: Wednesday, March 24, 2021 6:34 PM
> To: Avi Gross 
> Cc: python-list@python.org
> Subject: Re: convert script awk in python
>
> On 24Mar2021 12:00, Avi Gross  wrote:
>>But I wonder how much languages like AWK are still used to make new 
>>programs as compared to a time they were really useful.
>
> You mentioned in an adjacent post that you've not used AWK since 2000.  
> By contrast, I still use it regularly.
>
> It's great for proof of concept at the command line or in small scripts, and
> as the innards of quite useful scripts. I've a trite "colsum" script which
> does nothing but generate and run a little awk programme to sum a column,
> and routinely type "blah  | colsum 2" or the like to get a tally.
>
> I totally agree that once you're processing a lot of data from places