Re: [Python-Dev] Evil reference cycles caused Exception.__traceback__

[Martin (gzlist) via Python-Dev]

Thanks for working on this and writing up the details Victor.

For those confused about why this matters, routinely having every
object in your application participating in one (or more) giant
reference cycles makes reasoning about and fixing resource issues very
difficult.

For instance, a while back I made some changes in Bazaar so that each
TestCase instance was dereferenced after being run:



Which mattered when running ~25000 tests, to keep peak memory usage
sane. Also there were various other object lifetime issues, and
tracking down which specific test failed to join a thread, or close a
file on Windows was much simpler after making sure cleanup actually
happened at the time a test was deleted.

Keeping more stuff alive for longer periods also makes peak memory
requirements higher, and the gc has to do more work each time.

On 18/09/2017, Victor Stinner  wrote:
>
> Ideally, CPython 3.x should never create reference cycles. Removing
> Exception.__traceback__ is the obvious "fix" for the issue. But I
> expect that slowly, a lot of code started to rely on the attribute,
> maybe even for good reasons :-)
>
> A more practical solution would be to log a warning. Maybe the garbage
> collector can emit a warning if it detects an exception part of a
> reference cycle? Or maybe detect frames?

Logging a warning is unlikely to be practical. I had an optional test
flag that complained about reference cycles, and it produced a lot of
output.

Martin
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Replacement for array.array('u')?

[Martin (gzlist) via Python-Dev]

On Fri, 22 Mar 2019 at 16:12, Steve Dower  wrote:
>
> On 22Mar2019 0433, Antoine Pitrou wrote:
> > The question is: why would you use a array.array() with a Windows C API?
>
> I started replying to this with a whole lot of examples, and eventually
> convinced myself that you wouldn't (or shouldn't).
>
> That said, I see value in having a type for array/struct/memoryview that
> "is the same size as wchar_t", since that will avoid people having to
> guess (the docs for array in particular are deliberately vague about the
> actual size of the various types).

This is pretty much what ctypes provides for dealing with unicode?

https://docs.python.org/3/library/ctypes.html#ctypes.create_unicode_buffer

Seems a fine place to have things that help with win32 api interactions.

Martin
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Re: PEP 638: Syntactic macros

[Martin (gzlist) via Python-Dev]

On Fri, 16 Oct 2020 at 23:22, Guido van Rossum  wrote:
>
> Dima,
>
> Do you have a link to "babel macros"? Searching for that brought up several 
> different things; not being a frequent JS user I don't know how to filter 
> these.

These links should help:

https://babeljs.io/blog/2017/09/11/zero-config-with-babel-macros
https://github.com/kentcdodds/babel-plugin-macros
https://github.com/jgierer12/awesome-babel-macros

That's a general intro, the code repo for the macro plugin, and a repo
that lists implemented macros.

Martin
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/HJR6UF2XPOC6UUY6QRLKYCYZOWK2BYDD/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Integer concatenation to byte string

[Martin (gzlist) via Python-Dev]

On Tue, 2 Mar 2021 at 21:44, Memz  wrote:
>
> foo+= 255 # Works the same as 
> bytesvariable+=b"ÿ"

foo = b"%b%d" % (foo, 255)

> foo+= a"\x255\x00"  # Concatenation with itself

foo = b"%b%b" % (foo, foo)

See PEP461:

Adding % formatting to bytes and bytearray

"With Python 3 and the split between str and bytes, one small but
important area of programming became slightly more difficult, and much
more painful -- wire format protocols."

Martin
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/TM2XN4P6JMOFP3RUSFJLFPTDC5J77J7O/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: On the migration from master to main

[Martin (gzlist) via Python-Dev]

On Tue, 23 Mar 2021 at 21:39, Python Steering Council
 wrote:
>
> This isn’t happening because GitHub/Microsoft made a political decision. It’s 
> happening because it is incredibly easy to make this move, many projects have 
> already done this, and it reflects badly on any project not making this 
> change.

"We do this not because it's right, but because it's easy."

Great to see Python adopting the motto of our new century.

Martin
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/DDYQXHYATCMPD3W2OHETZLTVN5DJ4VIO/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Delayed evaluation of f-strings?

[Martin (gzlist) via Python-Dev]

On Thu, 24 Jun 2021 at 17:25, Eric Nieuwland  wrote:
>
> class DelayedFString(str):
> def __str__(self):
> vars = inspect.currentframe().f_back.f_globals.copy()
> vars.update(inspect.currentframe().f_back.f_locals)
> return self.format(**vars)

This isn't quite right as the semantics between f-strings and
str.format() are not actually the same (though this isn't well
documented):

>>> f'{1 + 2}'
'3'
>>> str(DelayedFString('{1 + 2}'))
Traceback (most recent call last):
  File "", line 1, in 
  File "", line 5, in __str__
KeyError: '1 + 2'

>>> d = dict(a=1)
>>> f'{d["a"]}'
'1'
>>> str(DelayedFString('{d["a"]}'))
Traceback (most recent call last):
  File "", line 1, in 
  File "", line 5, in __str__
KeyError: '"a"'

Basically, f-strings rely on eval-like semantics.

Martin
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/HGITYZPPRXJM4Y7YVJKUSVXUB75W5Z2L/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Delayed evaluation of f-strings?

[Martin (gzlist) via Python-Dev]

On Thu, 24 Jun 2021 at 17:37, Martin (gzlist)  wrote:
>
> >>> d = dict(a=1)
> >>> f'{d["a"]}'
> '1'
> >>> str(DelayedFString('{d["a"]}'))
> Traceback (most recent call last):
>   File "", line 1, in 
>   File "", line 5, in __str__
> KeyError: '"a"'

And the other side of the attribute lookup:

>>> d = dict(a=1)
>>> str(DelayedFString('{d[a]}'))
'1'
>>> f'{d[a]}'
Traceback (most recent call last):
  File "", line 1, in 
NameError: name 'a' is not defined

Yes, having three different ways of doing string interpolation (not
counting other things you can import, like string.Template) is a bit
confusing.

Martin
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/A7FQUWQND56VGHCTTCFW6XDNCWP5MVNM/
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-Dev] PEP 538 (review round 2): Coercing the legacy C locale to a UTF-8 based locale

[Martin (gzlist) via Python-Dev]

On 09/05/2017, Nick Coghlan  wrote:
>
> Enough changes have accumulated in PEP 538 since the start of the
> previous thread that it seems sensible to me to start a new thread
> specifically covering the current design (which aims to address all
> the concerns raised in the previous thread).
>
> I haven't requoted the PEP in full since it's so long, but will
> instead refer readers to the web version:
> https://www.python.org/dev/peps/pep-0538/

I did try to follow along via the mailing list threads, and have now
read over the PEP again. Responding now as I'm actually touching code
relevent to this again.

Broadly the proposal looks good to me. It does help one of the two
cases I care about, and does no serious harm.

For a command line Python script, making sure Python itself uses UTF-8
for the C locale is sufficient, and setting LC_CTYPE so spawned
processes that aren't Python have a chance at doing the right thing
too is a reasonable upgrade. This is probably good enough to drop one
hack[1] rather than porting it to Python 3.

For hosted Python code this does nothing (apart from print to stderr),
so mod_wsgi for instance is still going to need the same kind dance to
get users to set LANG as configuration themselves. Ideally this PEP
would have a C api or something so I could file bugs to make it just
do the right thing.

A few notes on specifics, I'll try not to stray too much into choices
already made.

The PEP doesn't persuade me that Py_Initialize actually is too late to
switch *specifically* from ascii to utf-8. Any preceeding operations
that operate on unicode would have been a safe subset. There might be
issues with other internals, or surrogateescape, or it's just a pain?

I don't like the side effect of changing the standard stream error
handler to surrogateescape if LANG=C.UTF-8 is actually set. Obviously
bad data vs exception is a trade off anyway, but means to get a Python
script that will always output valid data or exit, you have to set an
arbitrary language like en_US. Yes, that's true of the change as
implemented in 3.5 anyway.

Not setting LANG and only setting LC_CTYPE seems fine. Obviously,
things can go wrong based on odd behaviours of spawned processes, but
it works for the normal idioms.

I'm not sold on adding the PYTHONCOERCECLOCALE runtime configuration.
All it really does is turn off stderr kipple if you must use the C
locale for other reasons? Anyone with the ability to set that variable
could just set LANG instead. I was going to suggest just documenting
LC_ALL=C as the override instead of adding a python specific variable,
but note looking around that Debian discourage that[3].

That's all, though I will also grumble a bit about how long the PEP is.

Martin


[1] Override Py_FileSystemDefaultEncoding to utf-8 from ascii for the bzr script

[2] WSGIDaemonProcess lang and locale options

[3] "Using LC_ALL is strongly discouraged as it overrides everything"

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Handle errors in cleanup code

[Martin (gzlist) via Python-Dev]

On 12/06/2017, Serhiy Storchaka  wrote:
>
> But if an error is raised when execute undo_something(), it replaces the
> original exception which become chaining as the __context__ attribute.
> The problem is that this can change the type of the exception. If
> do_something_other() raises SystemExit and undo_something() raises
> KeyError, the final exception has type KeyError.

For Python 2.7, I've used the following idiom, which always masks
errors from the undo:

  do_something()
  try:
  do_something_other()
  except:
  try:
  undo_something()
  finally:
  raise

Unfortunately, that breaks down on Python 3, as the new exception
propogates with the original as context..

> Does it mean that we should rewrite every chunk of code similar to the
> above? And if such cumbersome code is necessary and become common, maybe
> it needs syntax support in the language? Or changing the semantic of
> exceptions raised in error handlers and finally blocks?

What I want is a way to chain exceptions in the other direction,
raising the original error, but attaching a related one. Unfortunately
neither __cause__ or __context__ really help.

Martin
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 538 (review round 2): Coercing the legacy C locale to a UTF-8 based locale

[Martin (gzlist) via Python-Dev]

Thanks for replying to my points!

On 12/06/2017, Nick Coghlan  wrote:
>
> `PYTHONIOENCODING=:strict` remains the preferred way of forcing strict
> encoding checks on the standard streams, regardless of locale.

Then the user of my script has to care that it's written in Python and
set that specifically in their crontab or so on...

> In addition to providing a reliable escape hatch with no other
> potentially unwanted side effects (for when folks actually want the
> current behaviour), the entry for the off switch in the CLI usage docs
> also provides us with a convenient place to document the *default*
> behaviour.

The documentation aspect is an interesting consideration.

Having thought about it a bit more, my preferred option is having the
disable be if either LC_ALL or LC_CTYPE vars are exactly 'C', then
don't override. Otherwise (including for LANG=C), force C.UTF-8. The
CLI usage docs could have a LC_CTYPE entry that goes into details of
why.

Martin
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Re: Something wrong with the bug tracker?

[Martin (gzlist) via Python-Dev]

Logging in with openid using launchpad just worked for me, so
transient or related to google's openid support presumably.

Martin

On Sun, 22 Dec 2019 at 15:13, Mark Shannon  wrote:
>
> Hi,
>
> For some reason I can't log into the bug tracker with Google.
> I just get the error message:
>
> """
> An error has occurred
>
> A problem was encountered processing your request.
> The tracker maintainers have been notified of the problem.
> """
>
> Whether I'm logged in or out of Google, or using a private tab, it makes
> no difference.
>
> Anyone else having problems?
>
> Cheers,
> Mark.
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at 
> https://mail.python.org/archives/list/python-dev@python.org/message/BXWXJYB2NDDRCM7CDWL2X3IWVDW7ZE7D/
> Code of Conduct: http://python.org/psf/codeofconduct/
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/KADR37G3JOAJWNB6VOJQDFNHSTZ2PH7Q/
Code of Conduct: http://python.org/psf/codeofconduct/