Thanks for adding your input, Gregory! It's much appreciated.
I'll shuffle your comments around a bit, as I'd rather address the
themes than each individual point.
On 15May2019 2134, Gregory Szorc wrote:
PyPreConfig_INIT and PyConfig_INIT as macros that return a struct feel
weird to me. Specifically, the `PyPreConfig preconfig =
PyPreConfig_INIT;` pattern doesn't feel right.
I see Victor agreed here, but I think this is the right pattern for
PreConfig. The "_INIT" macro pattern is common throughout as a way to
initialize a stack-allocated struct - we really can't change it to be
anything than "{ .member = static value }" without breaking users, but
if you have another way to initialize it correctly then that is fine.
The important factor here is that this struct has to be allocated
_without_ any memory management provided by Python.
That said, I don't particularly like this approach for PyConfig. As you
said:
Also, one thing that tripped me up a few times when writing PyOxidizer
was managing the lifetimes of memory that various global variables point
to.
My preference here is for PreConfig to get far enough that we can
construct the full configuration as regular Python objects (e.g. using
PyDict_New, PyUnicode_FromString, etc.)[1] rather than a brand new C
struct with a new set of functions. That will cost copies/allocations at
startup, but it will also ensure that the lifetime of the configuration
info is managed by the runtime.
I assume you already have code/helpers for constructing Python strings
and basic data structures, so I wonder whether it would be helpful to be
able to use them to create the configuration info?
([1]: Yes, this requires implementation changes so they work
pre-interpreter and cross-interpreter. This work probably has to happen
anyway, so I don't see any harm in assuming it will happen.)
I'm a little confused about the pre-initialization functions that take
command arguments. Is this intended to only be used for parsing the
arguments that `python` recognizes? Presumably a custom application
embedding Python would never use these APIs unless it wants to emulate
the behavior of `python`? (I suppose this can be clarified in the API
docs once this is implemented.)
One feature that I think is missing from the proposal (and this is
related to the previous paragraph) is the ability to prevent config
fallback to things that aren't PyConfig and PyPreConfig.
This is certainly my intent, and I *think* Victor is coming around to it
too ;)
My preference is that embedding by default does not use any information
outside of the configuration provided by the host application. Then our
"python" host application can read the environment/argv/etc. and convert
it into configuration. Since some embedders also want to do this, we can
provide helper functions to replicate the behaviour.
Does this sound like a balance that would suit your needs? Would you
expect an embedded Python to be isolated by default? Or would you assume
that it's going to pick up configuration from various places and that
you (as an embedder) need to explicitly suppress that?
(Some parts of the stdlib and some 3rd-party libraries use their own
environment variables at runtime. We may not be able to convert all of
those to configuration, but at least they're read lazily, and so Python
code can override them by setting the variables.)
What about PyImport_FrozenModules?
FWIW I /might/ be interested in a mechanism to better control importlib
initialization because PyOxidizer is currently doing dirty things at
run-time to register the custom 0-copy meta path importer.
Controlling imports early in initialization is one of our broader goals
here, and one that I would particularly like to figure out before we
commit to a new public API. Registering new importers should not have to
be "dirty".
Cheers,
Steve
_______________________________________________
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
