Bobby Bruce has submitted this change. (
https://gem5-review.googlesource.com/c/public/gem5/+/70898?usp=email )
Change subject: sim,python: follow the new CPython startup sequence
......................................................................
sim,python: follow the new CPython startup sequence
Currently, gem5 suffers from several bugs related
to Python interpreter's locale encoding issues.
gem5 will crash when the working directory contains
Non-ASCII characters.
The reason is that Python 3.8+ introduces a new
interpreter startup sequence [1]. The startup
sequence consists of three phases:
1. Python core runtime preinitialization
2. Python core runtime initialization
3. Main interpreter configuration
Stage 1 determining the encodings used for system
interfaces.
However, gem5 doesn't preinitialize the Python
interpreter. Thus, the locale settings do not take
effect. This patch preinitialize the Python for
Python 3.8+.
Also, this patch avoid the use of `Py_SetProgramName`,
which is deprecated since Python 3.11[3].
[1] https://peps.python.org/pep-0432/
[2] https://peps.python.org/pep-0587/
[3] https://docs.python.org/3/c-api/init.html#c.Py_SetProgramName
Change-Id: I08a2ec6ab2b39a95ab194909932c8fc578c745ce
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/70898
Tested-by: kokoro <noreply+kok...@google.com>
Reviewed-by: Bobby Bruce <bbr...@ucdavis.edu>
Maintainer: Bobby Bruce <bbr...@ucdavis.edu>
Reviewed-by: Jason Lowe-Power <power...@gmail.com>
Maintainer: Jason Lowe-Power <power...@gmail.com>
Reviewed-by: Roger Chang <rogerycch...@google.com>
---
M src/python/gem5py.cc
M src/sim/main.cc
2 files changed, 33 insertions(+), 0 deletions(-)
Approvals:
Bobby Bruce: Looks good to me, approved; Looks good to me, approved
Roger Chang: Looks good to me, but someone else must approve
kokoro: Regressions pass
Jason Lowe-Power: Looks good to me, but someone else must approve; Looks
good to me, approved
diff --git a/src/python/gem5py.cc b/src/python/gem5py.cc
index f2d8759..37ddee2 100644
--- a/src/python/gem5py.cc
+++ b/src/python/gem5py.cc
@@ -51,6 +51,21 @@
int
main(int argc, const char **argv)
{
+#if PY_VERSION_HEX >= 0x03080000
+ // Preinitialize Python for Python 3.8+
+ // This ensures that the locale configuration takes effect
+ PyStatus status;
+ PyPreConfig preconfig;
+ PyPreConfig_InitPythonConfig(&preconfig);
+
+ preconfig.utf8_mode = 1;
+
+ status = Py_PreInitialize(&preconfig);
+ if (PyStatus_Exception(status)) {
+ Py_ExitStatusException(status);
+ }
+#endif
+
py::scoped_interpreter guard;
// Embedded python doesn't set up sys.argv, so we'll do that ourselves.
diff --git a/src/sim/main.cc b/src/sim/main.cc
index 81a691d..1c42891 100644
--- a/src/sim/main.cc
+++ b/src/sim/main.cc
@@ -50,6 +50,7 @@
// Initialize gem5 special signal handling.
initSignals();
+#if PY_VERSION_HEX < 0x03080000
// Convert argv[0] to a wchar_t string, using python's locale and
cleanup
// functions.
std::unique_ptr<wchar_t[], decltype(&PyMem_RawFree)> program(
@@ -59,6 +60,23 @@
// This can help python find libraries at run time relative to this
binary.
// It's probably not necessary, but is mostly harmless and might be
useful.
Py_SetProgramName(program.get());
+#else
+ // Preinitialize Python for Python 3.8+
+ // This ensures that the locale configuration takes effect
+ PyStatus status;
+
+ PyConfig config;
+ PyConfig_InitPythonConfig(&config);
+
+ /* Set the program name. Implicitly preinitialize Python. */
+ status = PyConfig_SetBytesString(&config, &config.program_name,
+ argv[0]);
+ if (PyStatus_Exception(status)) {
+ PyConfig_Clear(&config);
+ Py_ExitStatusException(status);
+ return 1;
+ }
+#endif
py::scoped_interpreter guard(true, argc, argv);
--
To view, visit
https://gem5-review.googlesource.com/c/public/gem5/+/70898?usp=email
To unsubscribe, or for help writing mail filters, visit
https://gem5-review.googlesource.com/settings?usp=email
Gerrit-MessageType: merged
Gerrit-Project: public/gem5
Gerrit-Branch: develop
Gerrit-Change-Id: I08a2ec6ab2b39a95ab194909932c8fc578c745ce
Gerrit-Change-Number: 70898
Gerrit-PatchSet: 2
Gerrit-Owner: Luming Wang <wlm199...@126.com>
Gerrit-Reviewer: Andreas Sandberg <andreas.sandb...@arm.com>
Gerrit-Reviewer: Bobby Bruce <bbr...@ucdavis.edu>
Gerrit-Reviewer: Jason Lowe-Power <ja...@lowepower.com>
Gerrit-Reviewer: Jason Lowe-Power <power...@gmail.com>
Gerrit-Reviewer: Roger Chang <rogerycch...@google.com>
Gerrit-Reviewer: kokoro <noreply+kok...@google.com>
_______________________________________________
gem5-dev mailing list -- gem5-dev@gem5.org
To unsubscribe send an email to gem5-dev-le...@gem5.org
