Re: Seeking a small group to package Apache Arrow (was: Bug#970021: RFP: apache-arrow -- cross-language development platform for in-memory analytics)
Julian, Arrow is a complicated and large package. We use it at work (where there is a fair amount of Python, also to Conda etc) and do have issues with more complex builds especially because it is 'data infrastructure' and can come in from different parts. I would recommend against packaging at old one -- we also have seen issues with different (py)arrow version biting. Have you seen https://github.com/apache/arrow-nanoarrow ? It works via the C API to Arrow which interchanges data via two void* to the the two structs for arrow array and schema -- and avoids linkage issue. (In user space the pyarrow or R arrow packages can still be used also interfacing via these.) I have been using it for R package bindings for some time and we plan to expand that (again, at work) -- as do others. It is already use by duckdb, by the Arrow 'ADBC' interfaces (which are generic in the ODBC/JDBC sense but for Arrow, and also by a python interface to snowflake. Dirk -- dirk.eddelbuettel.com | @eddelbuettel | e...@debian.org
Re: Need a hand with tiledb-python
I have now orphaned the package [1] as this one needs more attention that I can currently give it due to 180+ other packages I have, and other work and open source commitments. Dirk [1] https://bugs.debian.org/1010953 -- dirk.eddelbuettel.com | @eddelbuettel | e...@debian.org
Need a hand with tiledb-python
Hi all, I look after the 'tiledb' (a C++ library) package and the 'tiledb-r' package. I also effectively maintain 'tiledb-python' but I am running into some issues there. I do know a little bit of Python and have (co-)maintained two or three packages there for decades but this one stumps me. Larger context is that that upstream is fairly heavy user of Conda and used to a lot of explicit version pinning. This creates issues for us here as the build wants to leave the chroot/pbuilder to match those pins. Adam (CC'ed) who set this up initially already patched some calls out (for example sphinx docs are now local). The 'delicate' bit is that I actually work at TileDB ;-) and don't want to have a fight over the upstream choice (so I didn't bug my colleague yet). Upstream use is upstream use, but Debian packaging also has its rules. The packaging is at https://salsa.debian.org/python-team/packages/tiledb-py (and for historical reasons I also have one at https://salsa.debian.org/edd/tiledb-py which it is behind) and I would *really* appreciate it if someone could take a look. I tried patching requirements_dev.txt and misc/requirements_wheel.txt but am apparently out of my depth here. I reached out to Adam (CC'ed) but he is busy too. We all know how it goes. Please CC me on follow-ups as I am not subscribed to debian-python. Thanks, Dirk -- dirk.eddelbuettel.com | @eddelbuettel | e...@debian.org
Re: Salsa python-team write access for tiledb-py ?
On 18 February 2022 at 21:41, Nilesh Patra wrote: | On 2/18/22 9:33 PM, Dirk Eddelbuettel wrote: | > | > Hi debian-python, | > [...] | > I cannot write to https://salsa.debian.org/python-team/packages/tiledb-py so | > I created a (temporary ?) copy at https://salsa.debian.org/edd/tiledb-py | > If you're ok with me updating the team repo, I can reset origin and push to | > existing one. | | I granted you access to that repo; I however do not have enough powers to add you into | the team. | But I hope that helps you, anyway. Sure does, I pushed. So the 'original' repo is no longer an orphan relative to the changes I made. Now to see what ftp-masters / NEW queue have to say. We'll know more in a few weeks. Thanks for the prompt help! Dirk -- https://dirk.eddelbuettel.com | @eddelbuettel | e...@debian.org
Salsa python-team write access for tiledb-py ?
Hi debian-python, Recently I had packaged tiledb for Debian, based on earlier work by Adam (CC'ed), and we now followed up with tiledb-py which I just sent to NEW following up on an older ITP/WNPP from almost two years ago. (TileDB is a 'universal' data storage engine: https://github.com/TileDB-Inc/, the Python bindings are the most widely-used API bindings, I may follow up with packaging the R bindings which I am working on for TileDB.) I cannot write to https://salsa.debian.org/python-team/packages/tiledb-py so I created a (temporary ?) copy at https://salsa.debian.org/edd/tiledb-py If you're ok with me updating the team repo, I can reset origin and push to existing one. Cheers, Dirk -- https://dirk.eddelbuettel.com | @eddelbuettel | e...@debian.org
Re: Bug#967209: rgtk2: Unversioned Python removal in sid/bullseye
On 4 August 2020 at 15:06, Matthias Klose wrote: | block 967209 by 967157 | thanks | | sorry, that's due to uninstallability of libglade2-dev. Thanks, that makes sense. (And thanks also to Julien, that was a good idea too but it does come up negative: edd@rob:~/deb/rgtk2(master)$ grep -ri Python inst/doc/tutorial.sgml:bindings for many other languages including C, C++, Guile, Perl, Python, edd@rob:~/deb/rgtk2(master)$ Dirk | On 8/4/20 2:49 PM, Dirk Eddelbuettel wrote: | > | > Hi doko and Python folks, | > | > On 4 August 2020 at 09:29, Matthias Klose wrote: | > | Package: src:rgtk2 | > | Version: 2.20.36-2 | > | Severity: serious | > | Tags: sid bullseye | > | User: debian-python@lists.debian.org | > | Usertags: py2unversioned | > | | > | Python2 becomes end-of-live upstream, and Debian aims to remove | > | Python2 from the distribution, as discussed in | > | https://lists.debian.org/debian-python/2019/07/msg00080.html | > | | > | We will keep some Python2 package as discussed in | > | https://lists.debian.org/debian-python/2020/07/msg00039.html | > | but removing the unversioned python packages python-minimal, python, | > | python-dev, python-dbg, python-doc. | > | | > | Your package either build-depends, depends on one of those packages. | > | Please either convert these packages to Python3, or if that is not | > | possible, replaces the dependencies on the unversioned Python | > | packages with one of the python2 dependencies (python2, python2-dev, | > | python2-dbg, python2-doc). | > | | > | Please check for dependencies, build dependencies AND autopkg tests. | > | > I am lost. | > | > Build-Depends: | > Source: rgtk2 | > Section: gnu-r | > Priority: optional | > Maintainer: Dirk Eddelbuettel | > Build-Depends: debhelper (>= 10), dh-r (>= 20180615), r-base-dev (>= 3.6.1), libgtk2.0-dev (>= 2.10.12), libglade2-dev, libpango1.0-dev, libcairo2-dev | > Standards-Version: 4.4.0 | > Vcs-Browser: https://salsa.debian.org/edd/r-cran-rgtk2 | > Vcs-Git: https://salsa.debian.org/edd/r-cran-rgtk2.git | > Homepage: https://cran.r-project.org/package=RGtk2 | > | > Build-Depends: debhelper (>= 10), dh-r (>= 20180615), r-base-dev (>= 3.6.1), libgtk2.0-dev (>= 2.10.12), libglade2-dev, libpango1.0-dev, libcairo2-dev | > | > Depends: | > Package: r-cran-rgtk2 | > Source: rgtk2 (2.20.36-2) | > Version: 2.20.36-2+b1 | > Installed-Size: 18722 | > Maintainer: Dirk Eddelbuettel | > Architecture: amd64 | > Depends: r-base-core (>= 4.0.0-3), r-api-4.0, libatk1.0-0 (>= 1.12.4), libc6 (>= 2.14), libcairo2 (>= 1.8.0), libgdk-pixbuf2.0-0 (>= 2.22.0), libglib2.0-0 (>= 2.41.1), libgtk2.0-0 (>= 2.20.0), libpango-1.0-0 (>= 1.25.5), libpangocairo-1.0-0 (>= 1.22.0) | > Description-en: GNU R binding for Gtk2 | > [...] | > | > Where is python coming in from, and what could I change? Is it libglade2 ? | > | > If we cannot change and must remove it that would be, I guess, defensible. | > The package and the related ones are a little end-of-life-ish, but have not | > yet been removed from CRAN either so there is need for imminent action for | > removal. We could remove it but that is likely to annoy a user or two, and | > "as users are our priority" I suggest to keep it ... but I need a helping | > hand here. | > | > | If there are questions, please refer to the wiki page for the removal: | > | https://wiki.debian.org/Python/2Removal, or ask for help on IRC | > | #debian-python, or the debian-python@lists.debian.org mailing list. | > | > d-p CC'ed. | > | > Thanks, Dirk | > | -- https://dirk.eddelbuettel.com | @eddelbuettel | e...@debian.org
Re: Bug#967209: rgtk2: Unversioned Python removal in sid/bullseye
Hi doko and Python folks, On 4 August 2020 at 09:29, Matthias Klose wrote: | Package: src:rgtk2 | Version: 2.20.36-2 | Severity: serious | Tags: sid bullseye | User: debian-python@lists.debian.org | Usertags: py2unversioned | | Python2 becomes end-of-live upstream, and Debian aims to remove | Python2 from the distribution, as discussed in | https://lists.debian.org/debian-python/2019/07/msg00080.html | | We will keep some Python2 package as discussed in | https://lists.debian.org/debian-python/2020/07/msg00039.html | but removing the unversioned python packages python-minimal, python, | python-dev, python-dbg, python-doc. | | Your package either build-depends, depends on one of those packages. | Please either convert these packages to Python3, or if that is not | possible, replaces the dependencies on the unversioned Python | packages with one of the python2 dependencies (python2, python2-dev, | python2-dbg, python2-doc). | | Please check for dependencies, build dependencies AND autopkg tests. I am lost. Build-Depends: Source: rgtk2 Section: gnu-r Priority: optional Maintainer: Dirk Eddelbuettel Build-Depends: debhelper (>= 10), dh-r (>= 20180615), r-base-dev (>= 3.6.1), libgtk2.0-dev (>= 2.10.12), libglade2-dev, libpango1.0-dev, libcairo2-dev Standards-Version: 4.4.0 Vcs-Browser: https://salsa.debian.org/edd/r-cran-rgtk2 Vcs-Git: https://salsa.debian.org/edd/r-cran-rgtk2.git Homepage: https://cran.r-project.org/package=RGtk2 Build-Depends: debhelper (>= 10), dh-r (>= 20180615), r-base-dev (>= 3.6.1), libgtk2.0-dev (>= 2.10.12), libglade2-dev, libpango1.0-dev, libcairo2-dev Depends: Package: r-cran-rgtk2 Source: rgtk2 (2.20.36-2) Version: 2.20.36-2+b1 Installed-Size: 18722 Maintainer: Dirk Eddelbuettel Architecture: amd64 Depends: r-base-core (>= 4.0.0-3), r-api-4.0, libatk1.0-0 (>= 1.12.4), libc6 (>= 2.14), libcairo2 (>= 1.8.0), libgdk-pixbuf2.0-0 (>= 2.22.0), libglib2.0-0 (>= 2.41.1), libgtk2.0-0 (>= 2.20.0), libpango-1.0-0 (>= 1.25.5), libpangocairo-1.0-0 (>= 1.22.0) Description-en: GNU R binding for Gtk2 [...] Where is python coming in from, and what could I change? Is it libglade2 ? If we cannot change and must remove it that would be, I guess, defensible. The package and the related ones are a little end-of-life-ish, but have not yet been removed from CRAN either so there is need for imminent action for removal. We could remove it but that is likely to annoy a user or two, and "as users are our priority" I suggest to keep it ... but I need a helping hand here. | If there are questions, please refer to the wiki page for the removal: | https://wiki.debian.org/Python/2Removal, or ask for help on IRC | #debian-python, or the debian-python@lists.debian.org mailing list. d-p CC'ed. Thanks, Dirk -- https://dirk.eddelbuettel.com | @eddelbuettel | e...@debian.org
Re: Bug#556148: installs files into /usr/local for Python = 2.6
(Resending with _corrected_ 7-bit mail headers)
On 14 November 2009 at 17:48, Bernd Zeimetz wrote:
| Dirk Eddelbuettel wrote:
|
| | Starting from Python 2.6, the installation paths for distutils have
| | changed. /usr/local is now used by default.
| |
| | When rebuilt against python-all{,-dev,-dbg} (and thus python2.6) from
Debian
| | experimental, your package contained these files:
|
| I don't have access to experimental in my pbuilder. So how should I test
this?
|
| Create a pbuilder basetgz for experimental? Thats plain easy. If you really
have
| no clue how to do it, use my .pbuilderrc:
|
http://git.recluse.de/?p=users/bzed/dotfiles.git;a=blob;f=.pbuilderrc;h=b1b9517bc31b3a3e36de87b9c1a2474f709e0b4b;hb=HEAD
| Then all you need to do is pbuilder --create if your debian/changelog says
| experimental.
Sure, I can either create a new (temp.) pbuilder (by modifying my existing
one) or create a new one. What I meant to say was
I am swamped.
I will not be able to get it soon.
You guys have working 'experimental' pbuilders.
Can you help?
as the original bug report suggested that I may want to ask for help on
debian-pyt...@l.d.o.
So, with that out of the way:
I anybody able to quickly review and test this with me?
I am available via email/irc/chat/... if I get a heads up and get work
with one of you guys on this, but I may not be able to foreground this task
otherwise for a little as I have travel and a presentation coming up.
Thanks, Dirk
--
Three out of two people have difficulties with fractions.
--
To UNSUBSCRIBE, email to debian-python-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Re: Bug#556148: installs files into /usr/local for Python = 2.6
On 14 November 2009 at 12:57, Kumar Appaiah wrote: | On Sat, Nov 14, 2009 at 11:25:02AM -0600, Dirk Eddelbuettel wrote: |I am swamped. | |I will not be able to get it soon. | |You guys have working 'experimental' pbuilders. | |Can you help? | | as the original bug report suggested that I may want to ask for help on | debian-pyt...@l.d.o. | | So, with that out of the way: | | I anybody able to quickly review and test this with me? | | I am available via email/irc/chat/... if I get a heads up and get work | with one of you guys on this, but I may not be able to foreground this task | otherwise for a little as I have travel and a presentation coming up. | | I have sent a patch, which should work. I've tested it in my chroot, | and it does seem to work; others can verify and let you know. Two words: You rock! The bug repotr only went to -submitter, so I never saw it. The result set (three files in /usr/bin) looks good and is what we had before the 2.6 transition. I will apply your patch and upload. Many thanks! Dirk -- Three out of two people have difficulties with fractions. -- To UNSUBSCRIBE, email to debian-python-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Removing old .pyo/.pyc files in preinst ?
Howdy, I am about to upload a brown-bag bug fix for python-rpy (dh_installdeb called before dh_pycentral leading to no postinst code to link the new modules in, doh; thanks to Keith, who is CCed as a courtesy, for spotting this). However, there may well be old rpy files hanging around from prior python-rpy version . Shall I remove these? I have the draft debian/python-rpy.preinst that is included below -- could one you of you good Debian Python folks wave a cluebat at me indicating whether this is ok or whether it is rather frowned upon? Shall I remove the .pyo files only? Or shall I go back into my cave and continue coding in Perl as a punishment? CCs encouraged as I am not a debian-python subscriber. Cheers, and thanks in advance for any hints, Dirk #!/bin/sh # # prerinst script for the Debian GNU/Linux python-rpy package # # Copyright 2006 by Dirk Eddelbuettel [EMAIL PROTECTED] set -e case $1 in install|upgrade) for f in rpy rpy_io rpy_options rpy_tools rpy_versions rpy_wintools; do for e in pyc pyo; do for v in 2.3 2.4; do if [ -f /usr/lib/python$v/site-packages/$f.$e ]; then rm -fv /usr/lib/python$v/site-packages/$f.$e fi done done done ;; abort-upgrade|disappear) ;; *) echo prerm called with unknown argument \`$1' 2 exit 0 ;; esac exit 0 -- Hell, there are no rules here - we're trying to accomplish something. -- Thomas A. Edison -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Help needed with #159208
Wajig, once upon a time, compiled its .py files. That wasn't really useful as the performance bottlenecks come from the externally called programs, not wajig itself. So I removed that a while ago. However, the versioned Depends and Build-Depends persisted ... and so I just got a bug report, #159208. In response, I am about to suggest the following: wajig (0.3.13-2) unstable; urgency=low * debian/control: Depends changed to python -- we do not need versioned Depends as .py[co] file are not (anymore) shipped (Closes: #159208) * debian/control: Similarly, Build-Depends simply on python-dev -- Dirk Eddelbuettel [EMAIL PROTECTED] Thu, 5 Sep 2002 06:41:10 -0500 Is that ok with you Python Meisters? Wajig will only ever ship interpreted Python files and does run with all recent Python versions. Lintian and Linda are fine with it, but these robots might not have perused the Python Policy lately. Please CC me as I'm not subscribed here. Dirk -- Good judgement comes from experience; experience comes from bad judgement. -- Fred Brooks
Re: Help needed with #159208
You should depend on exactly the Python versions you support, not on python. For example Depends: python1.5 | python2.1 | python2.2 I see. But why not simply Depends: python (= 1.5) And if you dont compile any Python packages, why do you still have to build-depend on python-dev? Quite right, good point -- thanks. Tschoe mit oe, Dirk -- According to the latest figures, 43% of all signatures are totally worthless.
Re: Help needed with #159208
On Thu, Sep 05, 2002 at 07:48:38AM -0500, Dirk Eddelbuettel wrote: You should depend on exactly the Python versions you support, not on python. For example Depends: python1.5 | python2.1 | python2.2 I see. But why not simply Depends: python (= 1.5) I think this is wrong. If this is a package with python scripts, then they will probably have '#!/usr/bin/python', in which case the dependancy should be; Depends: python (=1.5), python (2.3) You could make a package of scripts that could work with python1.5 | python2.1 | python2.2, but it would either require seperate foo-python1.5, foo-python2.1, and foo-python2.2 scripts, or some sort of smart dispatcher that could find and run the right installed /usr/bin/pythonX.Y. Remember, /usr/bin/python is a symlink installed by the python package, and without a dependancy on this package there is no gaurentee that it will be there. You can't just use #!/usr/bin/python without depending on the python package. Aren't we now fully circular? At first I suggested Depends: python, and now you recommend the same to me. I guess I need more coffee... And what about the #!/usr/bin/env trick? Can't that be used to call whatever suitable python interpreter is found in the $PATH? And if you dont compile any Python packages, why do you still have to build-depend on python-dev? Quite right, good point -- thanks. I _think_ you are saying that this package used to include compiled *.py[oc] files and now it doesn't. That is good. Correct. However, if it still includes *.py modules that are imported, then you still need to be very careful. Python will create and save *.pyc files when these modules are imported. However, typicaly only root has write access to the directories where these *.py files are installed. This means these *.pyc files are only created when root runs the scripts that import them. Worse, if the default python is upgraded, then the old *.pyc files cannot by updated by python, and hang around until root happens to import them again. I'm pretty sure python silently doesn't save *.pyc files if it can't, making them be re-compiled every import. It also re-compiles them if the *.pyc's are from a different version of python. This means worst case, python silently checks existing *.pyc's, recompiles them, then throws them away for every run. It is somewhat of a non-issue by design as wajig is written to be a higher-level wrapper around tasks requiring root. But wajig is only ever called as a normal user, and if and when su powers are needed, sudo is invoked /from inside the python script/ so that no python code should ever run as root. The proper way of handling this is for the postinst scripts to compile the *.pyc modules, and the postinst script to remove them. The really complicated stuff starts to happen when these script+module packages support multiple versions of python... If you need more info on this just ask. I sort-of-know as I once went that route, but it turned out to create only headaches. A simple Depends: python still looks terribly appealing to me. Dirk (who quite likes the kiss principle) -- According to the latest figures, 43% of all signatures are totally worthless.
package building / distutils question
madison and buildd.debian.org show that my quantlib-python package builds only on a few architectures (i386, powerpc, s390, sparc) and fails on others that could be expected to build (alpha, m68k, ...). This is C++ code, and a lot of it. I would like to try less optimisation as per the default, but cannot manage to turn -g -O2 off. In the upstream setup.py, the following is used (where else refers here to everything but Win32) else: from distutils import sysconfig include_dirs = [/usr/local/include] library_dirs = None libraries = [QuantLib] extra_compile_args = None extra_link_args = None define_macros = None # changes the compiler from gcc to g++ save_init_posix = sysconfig._init_posix def my_init_posix(): print 'my_init_posix: changing gcc to g++' save_init_posix() g = sysconfig._config_vars g['CC'] = 'g++' g['LDSHARED'] = 'g++ -shared' sysconfig._init_posix = my_init_posix Is there a way for me to plug other options in? I tried CFLAGS and CXXFLAGS, but no luck. Advice from the python gurus would be appreciated. Cheers, Dirk -- Better to have an approximate answer to the right question than a precise answer to the wrong question. -- John Tukey as quoted by John Chambers
Bug#118916: wajig: should depend on python, not python-base
Ben == Ben Burton [EMAIL PROTECTED] writes: Ben Package: wajig Version: N/A; reported 2001-11-09 Severity: serious Ben Ben Hi. Package wajig cannot be installed because it depends on the Ben non-existant python-base. Python packages have recently been Ben changed according to the new python policy. Wajig should depend on Ben python instead. Yes, I goofed. The only lame excuse is that it worked on my testing box. Now, as I read the python-policy.txt, I should Depends / Build-Depends as in Source: wajig [...] Build-Depends: debhelper ( 3.0.0), python2.1-dev [...] Package: wajig Depends: python (= 2.1), python ( 2.2), wget, apt However wajig is a mere Python app, and a package providing modules. I have verified that my 1.5.2 build works with /usr/bin/python2.1 -- so could I not use Source: wajig [...] Build-Depends: debhelper ( 3.0.0), python-dev [...] Package: wajig Depends: python, wget, apt instead? Please advise as I am unable to make up my mind here. Python Policy seems clear enough, but overly restrictive for this case. My current changelog draft is: wajig (0.2.8-2) unstable; urgency=low * Adapted Python Policy (as of version 0.3.7) * debian/control: Depends python (= 2.1), python ( 2.2) (Closes: #118916) * debian/control: Added Build-Depends on python2.1-dev (Closes: #19009) -- Dirk Eddelbuettel [EMAIL PROTECTED] Sat, 10 Nov 2001 20:18:38 -0600 Thanks, Dirk -- Better to have an approximate answer to the right question than a precise answer to the wrong question. -- John Tukey
