Erik Bray <erik.m.b...@gmail.com> added the comment: On Feb 8, 2018 12:55, "Jeroen Demeyer" <rep...@bugs.python.org> wrote:
New submission from Jeroen Demeyer <jdeme...@cage.ugent.be>: Displaying the source code in tracebacks for Cython-compiled extension modules in IPython no longer works due to PEP 302. Various fixes are possible, the two most obvious ones are: To be clear this is nothing to do with PEP 302, but has more to do with details of the import system reimplementation of Python 3.3, which PEP 302 predates by quite a lot. linecache should continue searching for the source file even if loader.get_source() returns None. I don't necessarily agree here. For some modules there may not be a real file associated with it in the first place, much less on sys.path. I'm not exactly sure why that fallback is there in the first place, but if a module does have a __loader__ that should have the say of where the module's source code is found (if at all). 2. the method ExtensionFileLoader.get_source() should be removed (instead of being implemented and returning None). Why? What would that help with? PEP 302 says get_source() can return None of no sources are found. That ExtensionFileLoader does this is not wrong (though it might be nice if it had a way to show C sources). It certainly doesn't know anything about Cython. If anything, as you and I discussed, Cython should be providing its own loader for Cython modules (and perhaps have a way to better distinguish Cython modules from other extension modules). Now why was this broken and how do the above fix that? When IPython needs to display a traceback, it uses linecache.getlines() to get the source code to display. For Cython extensions, the filename is a correct *relative* filename (it must be a relative filename since Cython does not know where the sources will be after installation). Since the filename is relative, linecache does not immediately find it, so it looks for a PEP 302 loader. For extension modules (Cython or not), it finds an instance of ExtensionFileLoader. If the loader has a get_source() method, then it uses that to get the sources. Since ExtensionFileLoader.get_source() returns None, linecache stops looking further for sources. Instead, what should happen is that linecache continues looking for the sources in sys.path where it has a chance of finding them (if they are installed somewhere in sys.path). The problem with this analysis is that the fact that this used to work at all was relying on undocumented behavior. Also in the case of Sage, which was using this to find Cython sources, it's because we were putting Cython sources on a sys.path entry which is not a normal thing to do. I don't think there's a bug in Python here, and that this is a problem that needs to be solved on the Cython end. ---------- _______________________________________ Python tracker <rep...@bugs.python.org> <https://bugs.python.org/issue32797> _______________________________________ _______________________________________________ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com