Re: Packaging pypy
On Mon, Nov 28, 2011 at 10:25 PM, Stefano Rivera wrote: > Hi Gustavo (2011.11.28_18:32:52_+0200) >> Would someone here be able to give a hand to Maciej on pushing that >> integration forward? > > I'm interested in this, and happy to help. It's probably time to get > PyPy back into Debian, I think all of our amd64 and i386 buildds are big > enough to handle it these days. How do people feel about the other > concerns raised by lamby (CCed, don't know if he still follows this > list) when he removed it? > > http://bugs.debian.org/538858: >> One day we can look at packaging this again, but it's not helpful to >> Debian in its current state - it is too premature to have in unstable >> and has some issues that would be difficult to fix. Development is also >> difficult due to the long build times and buildd specifications >> required. >> >> It has a very low popcon - anyone with a non-trivial interest in this >> project would be using upstream's HEAD anyway. > > Of course, it would have to be packaged as a separate Python stack, > again. Although it would be interesting to allow modules to be built for > alternate Python implementations, but that's not a trivial project... > > SR > > -- > Stefano Rivera > http://tumbleweed.org.za/ > H: +27 21 465 6908 C: +27 72 419 8559 UCT: x3127 For what is worth, the .py files (but not the .pyc files) can be shared among pypy and cpython. However some packages have different installation process for pypy and not pypy build (for example building optional C extensions or not). For what is worth, pypy also follows 3149 (http://www.python.org/dev/peps/pep-3149/) so it won't automatically try to load CPython compiled .so files (but it'll try and fail to load the .pyc files). -- To UNSUBSCRIBE, email to debian-python-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/CAK5idxSpMVdr3H6=kp-qytjdufttz-5hwp3cuuxjtiapfcr...@mail.gmail.com
Bug #649904: running install_data
[Please CC me] Hi all, I am trying to fix bug #649904. I have been talking with upstream, and apparently the install process on debian is doing something weird after the line "running install_data", see for instance : https://buildd.debian.org/status/fetch.php?pkg=pylibtiff&arch=i386&ver=0.3.0~svn78-1&stamp=1321830785 Upstream author cannot reproduce this behavior on ubuntu apparently. The current guess is that there is something going on wrong with python-numpy ? Has anyone seen this before ? Thanks much ! -- Mathieu -- To UNSUBSCRIBE, email to debian-python-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/CA+7wUsz=+whfkj_fcvzehv_fzag7wom8pss-+gb27hg05rg...@mail.gmail.com
Re: Bug #649904: running install_data
[Mathieu Malaterre, 2011-11-29] > [Please CC me] done > I am trying to fix bug #649904. I have been talking with upstream, > and apparently the install process on debian is doing something weird > after the line "running install_data", see for instance : > > https://buildd.debian.org/status/fetch.php?pkg=pylibtiff&arch=i386&ver=0.3.0~svn78-1&stamp=1321830785 the weird part is that it installs directly into dist-packages and not to dist-packages/libtiff/, right? if that's the case, then removeextralicense.patch is why (you're missing a comma after 'libtiff') -- Piotr Ożarowski Debian GNU/Linux Developer www.ozarowski.pl www.griffith.cc www.debian.org GPG Fingerprint: 1D2F A898 58DA AF62 1786 2DF7 AEF6 F1A2 A745 7645 -- To UNSUBSCRIBE, email to debian-python-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/2029094440.gb10...@piotro.eu
Re: Bug #649904: running install_data
On Tue, Nov 29, 2011 at 10:44 AM, Piotr Ożarowski wrote: > [Mathieu Malaterre, 2011-11-29] >> I am trying to fix bug #649904. I have been talking with upstream, >> and apparently the install process on debian is doing something weird >> after the line "running install_data", see for instance : >> >> https://buildd.debian.org/status/fetch.php?pkg=pylibtiff&arch=i386&ver=0.3.0~svn78-1&stamp=1321830785 > > the weird part is that it installs directly into dist-packages and > not to dist-packages/libtiff/, right? > > if that's the case, then removeextralicense.patch is why (you're missing > a comma after 'libtiff') Thanks so much ! That was it ! -- Mathieu -- To UNSUBSCRIBE, email to debian-python-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/ca+7wusygh5ymwtnckmjp1ntqvaocnx5qgnbu+zddrkzykbu...@mail.gmail.com
Re: Packaging pypy
On 11/28/2011 09:25 PM, Stefano Rivera wrote: > Of course, it would have to be packaged as a separate Python stack, > again. Although it would be interesting to allow modules to be built for > alternate Python implementations, but that's not a trivial project... maybe for binary packages, but there is no reason why a pypy extension couldn't be built from the same source packages. Could you summarize why it needs to be a separate stack? -- To UNSUBSCRIBE, email to debian-python-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/4ed4ce3e.1070...@debian.org
Re: Packaging pypy
> I'm interested in this, and happy to help. It's probably time to get > PyPy back into Debian, I think all of our amd64 and i386 buildds are big > enough to handle it these days. How do people feel about the other > concerns raised by lamby (CCed, don't know if he still follows this > list) when he removed it? That's great to hear, thanks for stepping up Stefano. PyPy indeed seems most ready for having some wider experimentation with, and we'd also appreciate having it in Ubuntu. -- Gustavo Niemeyer http://niemeyer.net http://niemeyer.net/plus http://niemeyer.net/twitter http://niemeyer.net/blog -- I'm not absolutely sure of anything. -- To UNSUBSCRIBE, email to debian-python-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/CANySw1=wfrmjq+dnyracdyt+9uhrd0e8dznanctd83h2ose...@mail.gmail.com
Re: Packaging pypy
On 11/29/2011 09:56 AM, Maciej Fijalkowski wrote: > For what is worth, the .py files (but not the .pyc files) can be > shared among pypy and cpython. IMO, patching pypy to lookup e.g. .pycp files before .pyc files would be appropriate for Debian (already doing something like this for .so files in 2.x). Not sure if backporting pep 3147 would be worth it. > However some packages have different > installation process for pypy and not pypy build (for example building > optional C extensions or not). that should be handled in the packaging. Matthias -- To UNSUBSCRIBE, email to debian-python-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/4ed4dbf6.2070...@debian.org
Re: Packaging pypy
Hi Matthias (2011.11.29_14:21:18_+0200) > maybe for binary packages, but there is no reason why a pypy extension > couldn't > be built from the same source packages. Could you summarize why it needs to > be > a separate stack? One question is: How broken we want to allow modules to be. If it's opt-in, then only modules that are known to work will be importable (but getting adoption from maintainers would probably be slow). I haven't tested if PyPy can successfully byte-compile all .py files (that claim 2.7 support), but that's easy enough to test. We do need separate byte-compiled files, and separate C extensions. (Also discussed elsewhere in this thread). And some packages are presumably going to install differently under PyPy (e.g. skip some C-extensions, even add some pure-python? I have no numbers on this, does anyone?) All of that makes me think that we at least need to treat it as a separate Python version. It could be added as another symlink farm for dh_python2 to manage (although probably with moderate pain, dh_python2 thinks all python versions are sequential). It doesn't necessarily have to be opt-in (although that's one way to ensure that only the things that we know work are exposed). It's plausible that some maintainers wouldn't want to deal with PyPy related failures (they know their upstream code doesn't support PyPy), so opt-out may be necessary. Maciej: Remind me: How stable is PyPy's bytecode format? Do we expect it to change for 2.x-compatible PyPy? SR -- Stefano Rivera http://tumbleweed.org.za/ H: +27 21 465 6908 C: +27 72 419 8559 UCT: x3127 -- To UNSUBSCRIBE, email to debian-python-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/2029141340.ga29...@bach.rivera.co.za
Re: Packaging pypy
Hi Gustavo (2011.11.29_14:43:30_+0200) > That's great to hear, thanks for stepping up Stefano. PyPy indeed > seems most ready for having some wider experimentation with, and we'd > also appreciate having it in Ubuntu. It'd be my most ambitious package. But that's where the fun is, right? :) SR -- Stefano Rivera http://tumbleweed.org.za/ H: +27 21 465 6908 C: +27 72 419 8559 UCT: x3127 -- To UNSUBSCRIBE, email to debian-python-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/2029141416.gb29...@bach.rivera.co.za
Re: Packaging pypy
On Tue, Nov 29, 2011 at 4:13 PM, Stefano Rivera wrote: > Hi Matthias (2011.11.29_14:21:18_+0200) >> maybe for binary packages, but there is no reason why a pypy extension >> couldn't >> be built from the same source packages. Could you summarize why it needs to >> be >> a separate stack? > > One question is: How broken we want to allow modules to be. > If it's opt-in, then only modules that are known to work will be > importable (but getting adoption from maintainers would probably be > slow). > > I haven't tested if PyPy can successfully byte-compile all .py files > (that claim 2.7 support), but that's easy enough to test. > > We do need separate byte-compiled files, and separate C extensions. > (Also discussed elsewhere in this thread). > And some packages are presumably going to install differently under > PyPy (e.g. skip some C-extensions, even add some pure-python? I have no > numbers on this, does anyone?) > > All of that makes me think that we at least need to treat it as a > separate Python version. Sounds like a good plan > > It could be added as another symlink farm for dh_python2 to manage > (although probably with moderate pain, dh_python2 thinks all python > versions are sequential). > > It doesn't necessarily have to be opt-in (although that's one way to > ensure that only the things that we know work are exposed). > It's plausible that some maintainers wouldn't want to deal with PyPy > related failures (they know their upstream code doesn't support PyPy), > so opt-out may be necessary. For pure-python code, pretty much 100% code work. There are failures here and there, but I never encountered a problem with fixing it upstream. One way to check would be to run respective packages test suite to have some idea. > > Maciej: Remind me: How stable is PyPy's bytecode format? Do we expect it > to change for 2.x-compatible PyPy? Bytecode format is an internal detail of a VM. For all I know it might completely disappear. CPython likes to change it's bytecode format every release and we usually follow changes, but we also have quite a few our own bytecodes. The thing is they're a bit unnecessary when it comes to performance - this is not the way we approach optimizing python, but indeed there are no promises this won't change between versions. An ideal scenario would be where you have a separate stack of packages for pypy and CPython and that can reuse files from each other in case they're both installed and have the same binary version, but that's hard I know :) Cheers, fijal > > SR > > -- > Stefano Rivera > http://tumbleweed.org.za/ > H: +27 21 465 6908 C: +27 72 419 8559 UCT: x3127 -- To UNSUBSCRIBE, email to debian-python-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/cak5idxtybpyjoo5afslgcfmq7rknav2c1pikvp6+bk8wujt...@mail.gmail.com
Re: Packaging pypy
On Nov 29, 2011, at 04:20 PM, Maciej Fijalkowski wrote: >Bytecode format is an internal detail of a VM. For all I know it might >completely disappear. CPython likes to change it's bytecode format >every release and we usually follow changes, but we also have quite a >few our own bytecodes. The thing is they're a bit unnecessary when it >comes to performance - this is not the way we approach optimizing >python, but indeed there are no promises this won't change between >versions. While chatting about this in irc (#debian-python on oftc), I mistakenly thought that PyPy supported PEP 3147, but I think it's only PEP 3149. 3147 shouldn't be that difficult to support - what is your thought on adding that to PyPy? It would mean one less symlink farm. -Barry -- To UNSUBSCRIBE, email to debian-python-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/2029110007.61b2b...@resist.wooz.org
Re: Packaging pypy
On Nov 29, 2011, at 02:19 PM, Matthias Klose wrote: >On 11/29/2011 09:56 AM, Maciej Fijalkowski wrote: >> For what is worth, the .py files (but not the .pyc files) can be >> shared among pypy and cpython. > >IMO, patching pypy to lookup e.g. .pycp files before .pyc files would be >appropriate for Debian (already doing something like this for .so files in >2.x). Not sure if backporting pep 3147 would be worth it. I think supporting PEP 3147 should not be difficult, though I haven't looked at the PyPy code, and it seems more fruitful than carrying a delta to look for another implementation specific bytecode file. -Barry -- To UNSUBSCRIBE, email to debian-python-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/2029110214.15723...@resist.wooz.org
Re: Packaging pypy
> I suppose it's not really that much work, but that would mean waiting > for another pypy release (which is probably 2-3 months away) The package may include a patch to enable that specifically, if necessary. -- Gustavo Niemeyer http://niemeyer.net http://niemeyer.net/plus http://niemeyer.net/twitter http://niemeyer.net/blog -- I'm not absolutely sure of anything. -- To UNSUBSCRIBE, email to debian-python-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/canysw1mu905q7yoc39v+l9qq-yaqktb_fveig97pm4syrnq...@mail.gmail.com
Re: Packaging pypy
On Nov 29, 2011, at 02:11 PM, Gustavo Niemeyer wrote: >> I suppose it's not really that much work, but that would mean waiting >> for another pypy release (which is probably 2-3 months away) > >The package may include a patch to enable that specifically, if necessary. Right. We could cherrypick the patch temporarily. -Barry signature.asc Description: PGP signature
Re: Packaging pypy
On Tue, Nov 29, 2011 at 6:02 PM, Barry Warsaw wrote: > On Nov 29, 2011, at 02:19 PM, Matthias Klose wrote: > >>On 11/29/2011 09:56 AM, Maciej Fijalkowski wrote: >>> For what is worth, the .py files (but not the .pyc files) can be >>> shared among pypy and cpython. >> >>IMO, patching pypy to lookup e.g. .pycp files before .pyc files would be >>appropriate for Debian (already doing something like this for .so files in >>2.x). Not sure if backporting pep 3147 would be worth it. > > I think supporting PEP 3147 should not be difficult, though I haven't looked > at the PyPy code, and it seems more fruitful than carrying a delta to look for > another implementation specific bytecode file. > > -Barry I suppose it's not really that much work, but that would mean waiting for another pypy release (which is probably 2-3 months away) -- To UNSUBSCRIBE, email to debian-python-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/CAK5idxRMaRMFyp81UiQ-pqL8oVBGiRrUX1k4kfDFrQG3Z=j...@mail.gmail.com
