[Python-Dev] Re: Debug C++
You can also attach gdb to a running (python) process .. but the interpreter needs to have the symbol table (eg. not stripped). I gave a presentation few years back on this exact topic (sorry for the quality, I was very in-experienced). https://skillsmatter.com/skillscasts/4313-intro-to-python-debug I hope this helps. PS> I'm working on a tiny "relocatable" python interpreter + toolchain using docker (sort of miniconda but for plain python) https://bitbucket.org/cav71/builder3/src/master that might be useful. On Mon, 30 Mar 2020 at 20:07, Rhodri James wrote: > On 30/03/2020 23:20, Leandro Müller wrote: > > Hi > > Are there any away to debug C module during python debug? > > In a word, gdb. > > I've been doing this quite a lot lately. The trick is to start Python > up under gdb, set a pending breakpoint in your C module, then carry on > as normal. For example: > > > rhodri@Wildebeest:~/hub_module$ gdb python3 > GNU gdb (Ubuntu 8.1-0ubuntu3.2) 8.1.0.20180409-git > Copyright (C) 2018 Free Software Foundation, Inc. > > [...large amounts of boilerplate omitted for brevity...] > > Reading symbols from python3...Reading symbols from > > /usr/lib/debug/.build-id/28/7763e881de67a59b31b452dd0161047f7c0135.debug...done. > done. > (gdb) b Hub_new > Function "Hub_new" not defined. > Make breakpoint pending on future shared library load? (y or [n]) y > Breakpoint 1 (Hub_init) pending. > (gdb) run > Starting program: /home/rhodri/Work/Lego/hub_module/hat_env/bin/python3 > [Thread debugging using libthread_db enabled] > Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1". > Python 3.6.9 (default, Nov 7 2019, 10:44:02) > [GCC 8.3.0] on linux > Type "help", "copyright", "credits" or "license" for more information. > >>> from hub import hub > > Breakpoint 1, Hub_new (type=0x76236340 , args=(), kwds=0x0) > at src/hubmodule.c:164 > 164 HubObject *self = (HubObject *)type->tp_alloc(type, 0); > (gdb) > > > ...and off you go! > > > -- > Rhodri James *-* Kynesim Ltd > ___ > Python-Dev mailing list -- python-dev@python.org > To unsubscribe send an email to python-dev-le...@python.org > https://mail.python.org/mailman3/lists/python-dev.python.org/ > Message archived at > https://mail.python.org/archives/list/python-dev@python.org/message/4JVUDRD5YZVVZ4FWC3BYN3RJDXBNBTIF/ > Code of Conduct: http://python.org/psf/codeofconduct/ > ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/N5AEUON35RJXJJ5ENXCXVXYAJXM5TIMO/ Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-Dev] Can Python Be Built Without Distutils
I tried a Makefile based build of python (+ some module) in the past for Android (and macos): https://bitbucket.org/cavallo71/android There was no particular problem in dropping autoconfigure+setup.py in the process: the only real problem was the pgen must be compiled on the host machine (but that could have changed since). In general python was a plain compile and link stuff with not much magic involved and that is (very) good in my opinion: magic detection tends to fail, autoconfigure need to execute stuff to detect things (reason why is quite useless in cross-compile mode) etc. I would've tried also cmake and qmake builds as well, but simply I didn't have time to spend on it. I hope this helps, Antonio Zachary Ware wrote: On Saturday, January 24, 2015, Brett Cannon br...@python.org mailto:br...@python.org wrote: On Fri Jan 23 2015 at 5:45:28 PM Gregory P. Smith g...@krypto.org javascript:_e(%7B%7D,'cvml','g...@krypto.org'); wrote: On Fri Jan 23 2015 at 11:20:02 AM M.-A. Lemburg m...@egenix.com javascript:_e(%7B%7D,'cvml','m...@egenix.com'); wrote: On 23.01.2015 19:48, Matthias Klose wrote: On 01/23/2015 06:30 PM, Cyd Haselton wrote: Related to my earlier question regarding building Python on Android and an undefined reference to dlopen error...I have the following question: Is it possible to build and install Python without having to build and install...or use...distutils? Some background: I can build the python interpreter on my device, and I can build a bunch of modules. The problem appears when make reaches the part where setup.py is used to build and import modules...specifically when setup.py attempts to import distutils.core. you can do this using Setup.local. This works for me building additional extensions as builtins. It might require some tweaking to build everything. You may want to have a look at the Setup files we're using in eGenix PyRun, which uses them to force static builds of the various built-in extensions. Look for these files: PyRun/Runtime/Setup.PyRun-2.7 PyRun/Runtime/Setup.PyRun-3.4 in the source archives: http://www.egenix.com/__products/python/PyRun/ http://www.egenix.com/products/python/PyRun/ Otoh, I would like to get rid off the setup.py altogether (/me ducks ...). Why ? It's great for finding stuff on your system and configuring everything without user intervention (well, most of the time :-)). Because our setup.py is a nightmare of arbitrary code run in a linear fashion with ad-hoc checks for things that are unlike how any other project on the planet determines what is available on your system. It may have seemed great when it was created in 2001. It really shows its age now. It defeats build parallelism and dependency declaration. It also prevents cross compilation. Building an interpreter with a limited standard library on your build host so that you can run said interpreter to have it drive the remainder of your build is way more self hosting that we rightfully need to be for CPython. So are you suggesting to add the build rules to configure and the Makefile -- and Windows build file -- in order to drop setup.py? Windows already doesn't use setup.py. There are a lot more modules built-in on Windows, and others have their own project files; distutils isn't used at all. -- Sent from Gmail Mobile ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/a.cavallo%40cavallinux.eu ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Sad status of Python 3.x buildbots
Ok, I've put everything under the tools directory of the build server for the project home:cavallo71:opt-python-interpreters: https://build.opensuse.org/package/show/home:cavallo71:opt-python-interpreters/tools To get started you need using the osc client (it uses the same subversion syntax, so is very simple): $ osc co home:cavallo71:opt-python-interpreters $ make -f tools/Makefile update (for the 2.7 branch) $ make -f tools/Makefile BRANCH=3 update (for the 3.x branch, I haven't fixed few issue with the build) I don't think you'd bee able to use as it is because is still linked to my account (few hardcoded paths). But I can help in bootstrapping it if there's a general consensus on that (maybe having the python project official release build on susebuild?). Then there's the testing bit (it's separate from the python binaries build to insulate the two). It's very similar and I'll post more details when ready. I hope this helps, Thnaks Antonio Cavallo wrote: Setting up a repo with the code and cleaning a bit here and there. Over the weekend I can put something useable. Nick Coghlan wrote: On 4 September 2014 18:50, A. Cavalloa.cava...@cavallinux.eu wrote: Yes there are details indeed. But not show stoppers. A prototype can be seen here: http://cclimited.webfactional.com The nice bit is testing will be executed in a total fresh instance, an added bonus Oh, very nice! What is needed to get that up and running? Regards, Nick. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/a.cavallo%40cavallinux.eu ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Sad status of Python 3.x buildbots
Setting up a repo with the code and cleaning a bit here and there. Over the weekend I can put something useable. Nick Coghlan wrote: On 4 September 2014 18:50, A. Cavalloa.cava...@cavallinux.eu wrote: Yes there are details indeed. But not show stoppers. A prototype can be seen here: http://cclimited.webfactional.com The nice bit is testing will be executed in a total fresh instance, an added bonus Oh, very nice! What is needed to get that up and running? Regards, Nick. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Sad status of Python 3.x buildbots
I wonder if there is any interest in starting to use the opensuse build servers for continuous build and testing on redhat, fedora suse and (I think) debian: that will solve once for all the maintenance issues on those platforms (and provide a reliable build). Regards, Antonio ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Binary CPython distribution for Linux
I have a little pet project for building rpm of python 2.7 (it should be trivial to port to 3.x): https://build.opensuse.org/project/show/home:cavallo71:opt-python-modules If there's enough interest I can help to integrate with python.org. I understand there may be technical challenges with doing this for some distributions and with producing a universal binary distribution. Opensuse provides the vm to build binaries for multiple platforms already since a very long time. Of course Anaconda is oriented towards scientific applications but it is a proof that a pre-build binary installer works and can be simple to use. Rpm are the blessed way to instal software on linux: it supports what most sysadmin expect (easy to list the installed packages, easy to validate if tampering to a package occurred, which file belongs to a package? etc..). Anaconda might appeal some group of user, but for deployment company-wide rpm is the best technical solution given its integration in linux. I hope this helps, Antonio ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Accepting PEP 445
Yes that is indeed a great news. Having to debug some binary only extension modules that will make my (rather selfish) life so much easier ;) Thanks On 7 Jul 2013, at 00:04, Victor Stinner victor.stin...@gmail.com wrote: 2013/7/6 Antonio Cavallo a.cava...@cavallinux.eu: Could that remove the need for the --with-pydebug flag? With the PEP 445, you still have to recompile Python with --with-debug, but you don't have to recompile Python extensions anymore. Debug checks on memory allocators are now implemented as hooks, instead of using C macros. I see this as an improvment ;-) Victor ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Accepting PEP 445
Could that remove the need for the --with-pydebug flag? On 6 Jul 2013, at 21:54, Antoine Pitrou solip...@pitrou.net wrote: Hello, I'm accepting PEP 445 (A C API to customize memory allocators) by Victor. There is probably some grammar to correct here and there (neither Victor nor I are native English speakers), but I don't want this to hold back acceptance. The PEP is an obvious improvement in flexibility for everyone embedding Python in some external application, or even recompiling their own Python runtime for specific uses. The implementation is tracked in http://bugs.python.org/issue3329 Thanks to Victor for his patience :-) Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/a.cavallo%40cavallinux.eu ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Compiling Python with Python
What's the advantage in writing a new build tool? I'm asking this because I'm
doing the same using scons:
https://bitbucket.org/cavallo71/fatpython
At the moment I'm very interested into this problem: the main advantages I see
so far are (in scons) are node dependencies and the fact it is plain python
syntax.
Thanks
On 5 Jun 2013, at 20:21, Jussi Pakkanen jpakk...@gmail.com wrote:
Hello all
I'd like to start this email by saying this is not a proposal to change
Python's build system. This is just the results of some experimentation you
might be interested it.
I have been working on a cross-platform build system called Meson, which is
implemented in Python 3. For symmetry I wanted to see if it could be used to
build Python itself. After about an evening worth of work, I got the basic C
parts (that is, the build targets in the main Makefile such as core Python,
pgen etc) built.
Main highlights:
- pyconfig.h generation in a fully cross-platform way without Autoconf (not
tested with Visual Studio but should work as Meson has unit tests for this,
tests for functions in header files and some others not done)
- builds in a separate build directory, can have arbitrarily many build dirs
with different configurations (for gcc/clang/static
analysis/debug/release/etc)
- configure time 5s, build time 8s on an i5 Macbook running Ubuntu
(Autotool-configure takes 37 s but it's not directly comparable because it
does a lot more)
- the file describing the build is 433 lines, most of which look like this:
if cc.has_header('io.h')
pyconf.set('HAVE_IO_H', 1)
endif
- implementation of Meson is 100% Python 3, it does not have a dependency on
the shell and in fact already works on Windows
If you want to try it yourself, here are the steps (only 64 bit Linux tested
thus far):
- install python3-ply and Ninja (Ubuntu package ninja-build)
- get Meson git head: https://sourceforge.net/p/meson/code/
- get Python3 trunk
- download and extract the build files into trunk:
https://dl.dropboxusercontent.com/u/37517477/python-meson.tar.gz
- cd into Python trunk and do the following commands:
mkdir build
cd build
path/to/meson.py ..
ninja
And there you have it. You can't do much with it, though (except run pgen to
ensure that it actually did something ;) ).
If you have any questions that are not directly related to Python, feel free
to email me or the Meson mailing list.
Enjoy,
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/a.cavallo%40cavallinux.eu
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] HAVE_FSTAT?
I've had a quick look with grep -R HAVE_ * | egrep '[.]c:'. Modules/posixmodule.c has HAVE_UTIME_H and it might be standard libc on all posix platforms. Objects/obmalloc.c has HAVE_MMAP… but I guess that's fine given other platforms might not have such facility. Depending on the granularity (on a per platform or per feature) probably yes, there aren't many left. I hope this helps On 17 May 2013, at 16:56, Antoine Pitrou solip...@pitrou.net wrote: On Fri, 17 May 2013 09:15:29 -0500 Skip Montanaro s...@pobox.com wrote: Some pieces of code are still guarded by: #ifdef HAVE_FSTAT ... #endif Are there other guards for similarly common libc functions? I don't think so. Someone should take a look though :-) Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/a.cavallo%40cavallinux.eu ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] The end of 2.7
It almost always comes down to bytes vs. strings, IME. Cool, next time I have to port an extension written in C/C++ I'll be looking only for bytes vs. strings problems. I knew it was easy. Thanks ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] question about packaging
Correct if I'm wrong but distlib isn't targeting resources managent? distutils is targeted to distribute python modules/packages instead; small differences but on the field they really mean different things. distlib is under http://hg.python.org/distlib too :O ..my first guess would be that's the released branch. Antoine Pitrou wrote: On Fri, 28 Dec 2012 17:59:05 -0500 Doug Hellmanndoug.hellm...@gmail.com wrote: A couple of us from the OpenStack project are interested in getting involved in the packaging rewrite/update project. I was following that work for a while, but have lost track of its current state. Can someone point me to the right mailing list, and maybe a status page or something so I can start figuring out where we might be able to help? The current effort seems to be distlib, Vinay's project to gather the good parts of packaging and distutils as a library API: http://packages.python.org/distlib/ (there's an active bitbucket repo) Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/a.cavallo%40cavallinux.eu ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] question about packaging
There's the distutil mailing list: http://mail.python.org/mailman/listinfo/distutils-sig Doug Hellmann wrote: A couple of us from the OpenStack project are interested in getting involved in the packaging rewrite/update project. I was following that work for a while, but have lost track of its current state. Can someone point me to the right mailing list, and maybe a status page or something so I can start figuring out where we might be able to help? Thanks, Doug ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/a.cavallo%40cavallinux.eu ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Distutils] Is is worth disentangling distutils?
Mmm, so the question would be distutils2 or distlib? I think tarek made a graph of the different packages systems... seen on reddit some time ago. My requirements would quite simple: 1. support DESTDIR approach where a package can be installed in an intermediate directory before its final destination) 2. cross compiling 3. install even if a dependecy package is not installed so decoupling installation from configuration point 1 is needed for system integrators (eg. people working in rpm builds). point 2 is entirely mine :) point 3 is the same philosophical difference between build, install and configuration steps: its part of good practices. In short it shouldn't replace the system wide dependency manager (in rpm it would be yum/zypp and in debian is much more confused, in window.. it doesn't exists as well in macos having the approach to pack it up everything in one place). Funny enough distutils (the old dead horse) does it all except point 2: that is my reason to clean up the code. I've just seen py3k distutils but it would be worth a back port to py2k. Thanks Nick Coghlan wrote: On Fri, Dec 14, 2012 at 10:10 AM, Antonio Cavallo a.cava...@cavallinux.eu mailto:a.cava...@cavallinux.eu wrote: I'll have a look into distutils2, I tough it was (another) dead end. I every case my target is py2k (2.7.x) and I've no case for transitioning to py3k (too much risk). distutils2 started as a copy of distutils, so it's hard to tell the difference between the parts which have been fixed and the parts which are still just distutils legacy components (this is why the merge back was dropped from 3.3 - too many pieces simply weren't ready and simply would have perpetuated problems inherited from distutils). distlib (https://distlib.readthedocs.org/en/latest/overview.html) is a successor project that takes a different view of building up the low level pieces without inheriting the bad parts of the distutils legacy (a problem suffered by both setuptools/distribute and distutils2). distlib also runs natively on both 2.x and 3.x, as the idea is that these interoperability standards should be well supported in *current* Python versions, not just those where the stdlib has caught up (i.e. now 3.4 at the earliest) The aim is to get to a situation more like that with wsgiref, where the stdlib defines the foundation and key APIs and data formats needed for interoperability, while allowing a flourishing ecosystem of user-oriented tools (like pip, bento, zc.buildout, etc) that still solve the key problems addressed by setuptools/distribute without the opaque and hard to extend distutils core that can make the existing tools impossible to debug when they go wrong. Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com mailto:ncogh...@gmail.com | Brisbane, Australia ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Distutils] Is is worth disentangling distutils?
It is not that complex... What's ahead is even more complex. Lennart Regebro wrote: On Fri, Dec 14, 2012 at 9:49 AM, Antonio Cavallo a.cava...@cavallinux.eu wrote: My requirements would quite simple: 2. cross compiling That is *not* a simple requirement. //Lennart ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Distutils] Is is worth disentangling distutils?
I'll have a look into distutils2, I tough it was (another) dead end. I every case my target is py2k (2.7.x) and I've no case for transitioning to py3k (too much risk). Lennart Regebro wrote: On Mon, Dec 10, 2012 at 8:22 AM, Antonio Cavallo a.cava...@cavallinux.eu wrote: Hi, I wonder if is it worth/if there is any interest in trying to clean up distutils: nothing in terms to add new features, just a *major* cleanup retaining the exact same interface. I'm not planning anything like *adding features* or rewriting rpm/rpmbuild here, simply cleaning up that un-holy code mess. Yes it served well, don't get me wrong, and I think it did work much better than anything it was meant to replace it. I'm not into the py3 at all so I wonder how possibly it could fit/collide into the big plan. Or I'll be wasting my time? The effort of making something that replaces distutils is, as far as I can understand, currently on the level of taking the best bits out of distutils2 and putting it into Python 3.4 under the name packaging. I'm sure that effort can need more help. //Lennart ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Is is worth disentangling distutils?
Hi, I wonder if is it worth/if there is any interest in trying to clean up distutils: nothing in terms to add new features, just a *major* cleanup retaining the exact same interface. I'm not planning anything like *adding features* or rewriting rpm/rpmbuild here, simply cleaning up that un-holy code mess. Yes it served well, don't get me wrong, and I think it did work much better than anything it was meant to replace it. I'm not into the py3 at all so I wonder how possibly it could fit/collide into the big plan. Or I'll be wasting my time? Thanks ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
