Re: python packages versus pydoc -k
- Original Message - Sorry, I should have tried pdb first, because this one has nothing to do with rpm-python. I can see modname='PyQt4.uic.pyuic', and prior to the exception site is a line 'loader = importer.find_module(modname)', which is where the None came from. I can confirm this one in a clean mock root with just PyQt4 added (and its dependencies). This one might be a bug in the way pydoc -k iterates, because I can't even target that name directly: $ pydoc PyQt4.uic.pyuic no Python documentation found for 'PyQt4.uic.pyuic' $ python -c 'import PyQt4.uic.pyuic' Traceback (most recent call last): File string, line 1, in module ImportError: No module named pyuic It seems that PyQt4/uic/pyuic.py is a symlink to /usr/bin/pyuic4, which is in PyQt4-devel. This means that the file is not importable when PyQt4-devel is not installed. IMO this is both: 1) packaging problem, since the symlink shouldn't be there if the file it points to isn't installed. 2) pydoc problem, because it should try/except broken and unimportable files I opened a rhbz#1076346 for 1) And I'll try to figure out a good patch for upstream and send it there to fix 2) Slavek. -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: python packages versus pydoc -k
On 03/14/2014 08:05 AM, Bohuslav Kabrda wrote: - Original Message - Sorry, I should have tried pdb first, because this one has nothing to do with rpm-python. I can see modname='PyQt4.uic.pyuic', and prior to the exception site is a line 'loader = importer.find_module(modname)', which is where the None came from. I can confirm this one in a clean mock root with just PyQt4 added (and its dependencies). This one might be a bug in the way pydoc -k iterates, because I can't even target that name directly: $ pydoc PyQt4.uic.pyuic no Python documentation found for 'PyQt4.uic.pyuic' $ python -c 'import PyQt4.uic.pyuic' Traceback (most recent call last): File string, line 1, in module ImportError: No module named pyuic It seems that PyQt4/uic/pyuic.py is a symlink to /usr/bin/pyuic4, which is in PyQt4-devel. This means that the file is not importable when PyQt4-devel is not installed. IMO this is both: 1) packaging problem, since the symlink shouldn't be there if the file it points to isn't installed. 2) pydoc problem, because it should try/except broken and unimportable files I opened a rhbz#1076346 for 1) And I'll try to figure out a good patch for upstream and send it there to fix 2) Thanks for taking care of this, guys! Florian -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: python packages versus pydoc -k
On 03/12/2014 06:12 PM, Toshio Kuratomi wrote: On Wed, Mar 12, 2014 at 12:18:17PM -0700, Josh Stone wrote: Do we have any packaging requirements or guidelines for python modules to behave nicely with pydoc? I've seen this break a number of times, and sometimes the bugs I've filed have been fixed, sometimes ignored. Before I go through another round, I'd like to know if we have (or should have) some official policy on this. We don't currently have any official guidelines on this. I know that pydoc can be broken. Because of how it works I'm not certain that we can fix it and keep it fixed. AIUI, pydoc works by importing the named module, then displaying its docstrings. Then pydoc -k does this for all modules in sys.path, looking for the specified keyword. A problem then arises if something in the path does protect itself with 'if __name__ == __main__:' to know when it's acting as a module or a script. (And if some module really doesn't want to support normal importing, it should not be installed in the path!) There's also packages that need a non-default version of a dependency in order to work. We've worked out ways to do this so that the module can be imported when we use them in an application but it will probably break with the way pydoc -k works. setuptools entrypoints can break unrelated code. It's probably another way that pydoc -k could be broken. [..] Of course, these are just the first exceptions I hit. Experience shows that fixing these will likely find more behind them. Yeah, I think there's a never ending treadmill here. Alright, I'll try not to let it bother me then. Thanks for your input. Josh -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: python packages versus pydoc -k
On Thu, Mar 13, 2014 at 09:11:12AM -0700, Josh Stone wrote: On 03/12/2014 06:12 PM, Toshio Kuratomi wrote: On Wed, Mar 12, 2014 at 12:18:17PM -0700, Josh Stone wrote: Of course, these are just the first exceptions I hit. Experience shows that fixing these will likely find more behind them. Yeah, I think there's a never ending treadmill here. Alright, I'll try not to let it bother me then. Yeah... which is not to stop you from filing bugs and working on fixes if you want... but I don't think we'll be able to maintain a working pydoc -k... there's jsut too many ways it can go wrong whenever a python package updates or is added. -Toshio pgpWmeCX6s4Sm.pgp Description: PGP signature -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: python packages versus pydoc -k
On 03/12/2014 08:18 PM, Josh Stone wrote: For instance, right now I get: $ pydoc -k xyzzy lib2to3.fixes.fix_repr - Fixer that transforms `xyzzy` into repr(xyzzy). Traceback (most recent call last): File /usr/bin/pydoc, line 5, in module pydoc.cli() File /usr/lib64/python2.7/pydoc.py, line 2292, in cli apropos(val) File /usr/lib64/python2.7/pydoc.py, line 1992, in apropos ModuleScanner().run(callback, key, onerror=onerror) File /usr/lib64/python2.7/pydoc.py, line 1973, in run module = loader.load_module(modname) AttributeError: 'NoneType' object has no attribute 'load_module' It's hard to track that down, but with strace -e open it looks like somewhere in site-packages/rpm/. I couldn't figure out exactly which subpackage is triggering this. I really would like to get this fixed if it really is a problem in rpm-python. But someone needs to come up with some better error messages or other way of finding out what the actual problem is. After a quick view in the pydoc doc I am no longer sure that this is an rpm problem, though. This could also be a bug in pydoc or pkgutil or some other python module being processed. Florian -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: python packages versus pydoc -k
On 03/13/2014 10:39 AM, Florian Festi wrote: On 03/12/2014 08:18 PM, Josh Stone wrote: For instance, right now I get: $ pydoc -k xyzzy lib2to3.fixes.fix_repr - Fixer that transforms `xyzzy` into repr(xyzzy). Traceback (most recent call last): File /usr/bin/pydoc, line 5, in module pydoc.cli() File /usr/lib64/python2.7/pydoc.py, line 2292, in cli apropos(val) File /usr/lib64/python2.7/pydoc.py, line 1992, in apropos ModuleScanner().run(callback, key, onerror=onerror) File /usr/lib64/python2.7/pydoc.py, line 1973, in run module = loader.load_module(modname) AttributeError: 'NoneType' object has no attribute 'load_module' It's hard to track that down, but with strace -e open it looks like somewhere in site-packages/rpm/. I couldn't figure out exactly which subpackage is triggering this. I really would like to get this fixed if it really is a problem in rpm-python. But someone needs to come up with some better error messages or other way of finding out what the actual problem is. After a quick view in the pydoc doc I am no longer sure that this is an rpm problem, though. This could also be a bug in pydoc or pkgutil or some other python module being processed. Sorry, I should have tried pdb first, because this one has nothing to do with rpm-python. I can see modname='PyQt4.uic.pyuic', and prior to the exception site is a line 'loader = importer.find_module(modname)', which is where the None came from. I can confirm this one in a clean mock root with just PyQt4 added (and its dependencies). This one might be a bug in the way pydoc -k iterates, because I can't even target that name directly: $ pydoc PyQt4.uic.pyuic no Python documentation found for 'PyQt4.uic.pyuic' $ python -c 'import PyQt4.uic.pyuic' Traceback (most recent call last): File string, line 1, in module ImportError: No module named pyuic FWIW, a clean mock root with python3-PyQt4 is fine with pydoc3 -k. -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: python packages versus pydoc -k
On Wed, Mar 12, 2014 at 12:18:17PM -0700, Josh Stone wrote: Do we have any packaging requirements or guidelines for python modules to behave nicely with pydoc? I've seen this break a number of times, and sometimes the bugs I've filed have been fixed, sometimes ignored. Before I go through another round, I'd like to know if we have (or should have) some official policy on this. We don't currently have any official guidelines on this. I know that pydoc can be broken. Because of how it works I'm not certain that we can fix it and keep it fixed. AIUI, pydoc works by importing the named module, then displaying its docstrings. Then pydoc -k does this for all modules in sys.path, looking for the specified keyword. A problem then arises if something in the path does protect itself with 'if __name__ == __main__:' to know when it's acting as a module or a script. (And if some module really doesn't want to support normal importing, it should not be installed in the path!) There's also packages that need a non-default version of a dependency in order to work. We've worked out ways to do this so that the module can be imported when we use them in an application but it will probably break with the way pydoc -k works. setuptools entrypoints can break unrelated code. It's probably another way that pydoc -k could be broken. [..] Of course, these are just the first exceptions I hit. Experience shows that fixing these will likely find more behind them. Yeah, I think there's a never ending treadmill here. pgprsxW9r6Dwn.pgp Description: PGP signature -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
