[issue44913] segfault in call to embedded PyModule_New

[hai shi]

hai shi  added the comment:

This bpo isn't a bug, so I close it now. If there are some enhancements need to 
be discussed, we can reopen it:)

--
resolution:  -> not a bug
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44913] segfault in call to embedded PyModule_New

[hai shi]

hai shi  added the comment:

> Or are you saying something can be done in Python's components to
handle this use case transparently? so the user won't have to put GIL
locks in his code.

Sorry, I don't explain it good enough. You have said what I wanted to say:)

> Yes, but if the extension is loaded into a C program (i.e. not from
python then the extension (libfoo1.so) has to call Py_Initialize at
that point.

You are right. I just only saw your dload.py, so I don't know how you start the 
python prcoess in your env;).
I haven't the experience of the pam-python:(.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44913] segfault in call to embedded PyModule_New

[Madhu]

Madhu  added the comment:

*  hai shi  
<1629695372.88.0.481350414879.issue44...@roundup.psfhosted.org>
Wrote on Mon, 23 Aug 2021 05:09:32 +

> hai shi  added the comment:
>> which can be wrapped within the calls to PyGILState_Ensure/Release.
> OK, it's a good idea. But I think this enhancement will break the back
> compatibility.

I don't understand.

Wherever I make a Py C call from the extension (libfoo1.so), I have to
wrap it up within calls to PyGILState_Ensure/Release to avoid
segfaults right?

That's how I understood your comment.

So this is on the user to avoid the segfaults, right?

And you saying my extension will not be backward compatible, i.e. it
will not work on older versions of python? (pam-python still supports
py27, I think)

Or are you saying something can be done in Python's components to
handle this use case transparently? so the user won't have to put GIL
locks in his code

(They aren't required in the normal extension case AFAICT)

>> python is initialized within that entrypoint
> python is initialized when you run python :)
[Yes, but if the extension is loaded into a C program (i.e. not from
python then the extension (libfoo1.so) has to call Py_Initialize at
that point.]

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44913] segfault in call to embedded PyModule_New

[hai shi]

hai shi  added the comment:

> which can be wrapped within the calls to PyGILState_Ensure/Release.

OK, it's a good idea. But I think this enhancement will break the back 
compatibility.

> python is initialized within that entrypoint

python is initialized when you run python :)

--
type:  -> enhancement

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44913] segfault in call to embedded PyModule_New

[Madhu]

Madhu  added the comment:

*  hai shi  
<1629603770.32.0.682495978946.issue44...@roundup.psfhosted.org>
Wrote on Sun, 22 Aug 2021 03:42:50 +

> hai shi  added the comment:

> How about this one?
>
>   if (!Py_IsInitialized()) {
> return -1;
>   }

[When this extension is loaded into a C program rather than a python
script, then Python should be initialized here rather than quitting]

>   PyGILState_STATE state = PyGILState_Ensure();
>   PyObject *module_handle = PyModule_New(module_name);
>   if (module_handle == 0)
>   {
> fprintf(stderr,"PyModule_New(module_data_name=%s) failed\n",
> module_name);
> exit(1);
>   }
>   fprintf(stdout, "point is: %p\n", module_handle);
>   fprintf(stderr, "foo END\n");
>   PyGILState_Release(state);

Yes. Thanks. Calling PyGILState_Ensure and PyGILState_Release around
the call to PyModule_New does avoid the segfault.

It is a little tricky to do this in pam-python - because although
there is one entrypoint into the extension (which can be wrapped
within the calls to PyGILState_Ensure/Release), python is initialized
within that entrypoint, but I think this can be solved another way.

Thanks

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44913] segfault in call to embedded PyModule_New

[hai shi]

hai shi  added the comment:

How about this one?

int foo(void)
{
  printf(stderr, "foo BEGIN\n");
  if (!Py_IsInitialized()) {
fprintf(stderr, "python already initialized\n");
return -1;
  }
  const char *module_name = "foo";
  PyGILState_STATE state = PyGILState_Ensure();
  PyObject *module_handle = PyModule_New(module_name);
  if (module_handle == 0)
  {
fprintf(stderr,"PyModule_New(module_data_name=%s) failed\n",
module_name);
exit(1);
  }
  fprintf(stdout, "point is: %p\n", module_handle);
  fprintf(stderr, "foo END\n");
  PyGILState_Release(state);
  return 0;
}

--
nosy: +shihai1991

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44913] segfault in call to embedded PyModule_New

[Madhu]

New submission from Madhu :

Attached zip file has a test case which illustrate the problem:

A python process (`dload.py') loads up a shared library (`libfoo1.so')
and makes a call to a foreign function `foo'. `foo' Initializes Python
and creates makes a call to PyModule_New at which point dload.py
crashes.

If the calling process is not python(`dload.c'), there is no crash

This sort of situation occurs with python-pam.  I'm not sure if this
is a programmer error and would welcome correction

[I'm supplying a zip file because I can't attach multiple files
Steps to repeat
1. compile libfoo1.so according to comment
2. Run ./dload.py
3. Optionally compile and run dload.c

--
components: Extension Modules
files: test-case-embedded-1.zip
messages: 399591
nosy: enometh
priority: normal
severity: normal
status: open
title: segfault in call to embedded PyModule_New
versions: Python 3.9
Added file: https://bugs.python.org/file50217/test-case-embedded-1.zip

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com